Hive state judgment method and system based on multi-source data fusion

Abstract

The application provides a kind of based on multi-source data fusion's beehive state judgment method and system, it is related to intelligent beehive monitoring technical field.The method comprises: collecting temperature, weight, sound spectrum features, nest door area image and nest door in and out bee count and other multi-source sensor data in beehive, after pre-processing, input correlation rule engine, by cross-dimension correlation rule, output confidence of bee colony abnormal event;With simultaneously activated multiple rule confidence as basic probability distribution, D-S evidence theory fusion is used, and fusion confidence distribution is obtained;Using LSTM time series prediction network, the future state trend is predicted according to historical time series data;The current state level is determined by comprehensively fusing confidence distribution and prediction trend, including normal, early warning, abnormal and emergency.The existing beehive monitoring depends on single parameter independent threshold judgment, lacks multi-source data deep fusion and correlation analysis, which leads to high false alarm and missed alarm, cannot grade evaluation and trend prediction.

Description

Technical Field

[0001] This invention relates to the field of intelligent beehive monitoring technology, and more specifically, to a method and system for judging the status of beehives based on multi-source data fusion. Background Technology

[0002] In beekeeping, timely and accurate assessment of colony status is crucial for maintaining colony strength, controlling diseases, and ensuring bee product yield. Traditional manual hive inspection methods are labor-intensive, disruptive to the colony's normal order, and highly dependent on experience, making them unsuitable for large-scale beekeeping. With the development of IoT technology, intelligent beehive systems that collect environmental and behavioral data from multiple sensors deployed within the hive are gradually being adopted. However, most current intelligent beehive monitoring solutions use independent thresholds for each sensor parameter to determine status; that is, when a parameter's measurement exceeds a preset range, it is considered abnormal and an alarm is triggered. This single-parameter independent threshold method does not utilize the inherent correlation between multi-source sensor data to identify the complex state of the colony. It is prone to false alarms due to short-term changes in the external environment (such as temperature increases caused by summer sunlight) or missed abnormal signs requiring multi-parameter joint confirmation (such as the coordinated changes in temperature and weight before absconding). At the same time, such solutions can usually only provide a binary classification result of normal or abnormal, and cannot conduct a graded and detailed assessment of the bee colony status. This makes it impossible for beekeepers to efficiently arrange disposal work according to the urgency level, and still cannot meet the urgent needs of precision and predictive beekeeping.

[0003] Patent document CN109145032A discloses an intelligent monitoring method and system for beekeeping. This scheme deploys multiple sensors inside the beehive, presets independent thresholds for each sensor parameter, and assigns an abnormal label when any parameter exceeds the limit. The multi-parameter feature vector is then input into a support vector machine classifier for binary classification as normal / abnormal, ultimately pushing intervention prompts to beekeepers. While this scheme collects multi-dimensional sensor data, its judgment mechanism essentially still relies on comparing and concatenating independent thresholds from each sensor, without deep fusion and correlation analysis of multi-source data. For example, when a combined scenario occurs where the beehive weight suddenly drops while the temperature simultaneously rises, this scheme can only respond to individual parameter exceedances, failing to comprehensively determine this symptom as a high-probability precursor to bee absconding, resulting in a high risk of missed detections. Conversely, when the hive temperature exceeds the limit solely due to an increase in ambient temperature, it is prone to false alarms. Furthermore, the output of this scheme is only a binary classification of normal and abnormal, with coarse granularity, which is not conducive to prioritizing and implementing precise management. Meanwhile, this scheme only makes a state cross-section judgment at the current moment and does not have the ability to analyze the temporal evolution of multidimensional data of bee colony, so it cannot predict trends and provide early warnings before anomalies actually occur. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for judging the status of beehives based on multi-source data fusion, which aims to solve the problems of existing beehive monitoring relying on single-parameter independent threshold judgment, lacking deep fusion and correlation analysis of multi-source data, resulting in high false alarms and false negatives, and inability to perform hierarchical assessment and trend prediction.

[0005] This invention is achieved through the following technical solution: A method for determining the state of a beehive based on multi-source data fusion includes the following steps: Collect multi-source sensor data inside the hive, including temperature, weight, sound spectrum characteristics, hive entrance area image, and hive entrance bee count; The multi-source sensor data is preprocessed, including outlier removal, missing value imputation, and normalization, to obtain normalized multi-source data. The normalized multi-source data is input into a pre-built association rule engine, which outputs the confidence level of the corresponding bee colony anomaly event based on multiple cross-dimensional association rules. The association rule engine stores multiple cross-dimensional association rules, and the conditions of each cross-dimensional association rule are composed of parameters of at least two different dimensions of sensor data and corresponding thresholds. When the conditions are met, the corresponding cross-dimensional association rule is triggered and the confidence level of the corresponding bee colony anomaly event is output. Using the confidence scores of multiple simultaneously activated cross-dimensional association rules as the basic probability allocation, the DS evidence theory is used for fusion to obtain the fused confidence score distribution characterizing the beehive state level. The normalized multi-source data within a preset historical time window is input into an LSTM time-series prediction network to obtain the predicted trend of the bee colony state level within a preset future time period. By combining the fusion confidence distribution and the predicted trend, the current status level of the beehive is determined; wherein, the status level includes normal, warning, abnormal and emergency.

[0006] Optionally, the cross-dimensional association rules include: flight absconding warning rules, swarming heat warning rules, robbing bee warning rules, and bee mite anomaly rules; The conditions for the flight absconding warning rule are that the temperature is between 35 and 40 degrees Celsius, the daily weight change rate is less than -5%, and the ratio of the number of bees entering and leaving the hive is greater than 2.0. When the conditions are met simultaneously, the flight absconding warning event is triggered and a confidence level of 0.85 is output. The conditions for the swarming heat warning rule are: the shift of the main sound frequency to the low frequency is greater than 15%, the confidence level of queen cell identification in the hive entrance area image is greater than 0.7, and the daily weight increase is greater than 3%; when the above conditions are met simultaneously, the swarming heat warning event is triggered and a confidence level of 0.80 is output. The conditions for the bee robbing warning rule are: the proportion of bees with aggressive postures in the hive entrance area image is greater than 30%, the fluctuation range of the ratio of bees entering and leaving the hive entrance is greater than a preset fluctuation threshold, and the energy increase of the screaming frequency band in the sound spectrum is greater than a preset energy threshold; when the above conditions are met simultaneously, a bee robbing warning event is triggered and a confidence level of 0.75 is output. The conditions for the bee mite abnormality rule are that the decrease in bee colony activity determined by the bee count at the hive entrance is greater than 20%, the sound spectrum conforms to the preset attenuation mode, and the increase in the number of mites falling on the bottom plate obtained by the image recognition of the hive entrance area is greater than the preset counting threshold; when the above conditions are met simultaneously, the bee mite abnormality event is triggered and a confidence level of 0.82 is output.

[0007] Optionally, in the association rule engine, each cross-dimensional association rule The conditions are given by The threshold determination formula is composed of sensing parameters of different dimensions; For the first The first cross-dimensional association rule Each parameter defines a threshold that satisfies the indicator function. When the first The current value of each parameter When the preset threshold condition is met, ,otherwise ; The rules When triggered, the confidence level of the corresponding bee colony anomaly event is output. Calculate using the following formula:

[0008] in, For rules The pre-set reliability benchmark value; the rule applies only to all Output confidence level when all conditions are met. Output confidence level when any condition is not met. .

[0009] Optionally, the fusion of evidence using DS theory to obtain the fusion confidence distribution characterizing the beehive state level is specifically as follows: Set up the recognition framework , , , and These correspond to four status levels in sequence: normal, warning, abnormal, and emergency. For the current time, the confidence level is greater than zero. Each of the activated association rules serves as independent evidence; for the first... This evidence maps the anomalous swarm event it points to to a state level. The basic probability assignment function is constructed as shown in the following equation:

[0010]

[0011] in, For the first The basic probability assignment function corresponding to the activated cross-dimensional association rule represents the degree of support that the evidence provides for each proposition; For the first The confidence level of the activation rule output; Using Dempster's combinatorial rules The basic probability assignment functions are fused together, and the fused basic probability assignment function is shown in the following equation:

[0012] in, The basic probability assignment function for the combined result after fusion; For any proposition whose combined confidence level needs to be calculated after fusion; This represents the number of activated association rules with a confidence level greater than zero at the current moment. For the first Jiao Yuan, one piece of evidence Conflict coefficient As shown in the following formula:

[0013] The fusion confidence distribution is composed of the fusion confidence of each state level, as shown below:

[0014]

[0015] in, Beehive status level The fusion confidence level.

[0016] Optionally, the step of inputting the time series of normalized multi-source data within a preset historical time window into an LSTM time series prediction network to obtain the predicted change trend of the bee colony state level within a preset future time period includes: At the current moment Based on this, the window length is taken as... Historical normalized multi-source data sequences , ;in, for Multidimensional normalized sensing vector at any given time; The sequence is processed sequentially using an LSTM temporal prediction network to obtain the final hidden state. And use fully connected layers and softmax functions to output the future The state level prediction probability vector at each time step is shown in the following equation:

[0017]

[0018] in, For the model's prediction of the future The probability vector of the bee colony state level at each time step. , , , and These correspond to the predicted probabilities of normal, warning, abnormal, and emergency status, respectively. This is the weight matrix of the fully connected layer output by the LSTM; For the LSTM network to process the historical time windows in sequence The final hidden state obtained after all time steps within the time step; This is the bias vector of the fully connected layer; Define the predicted trend of change as the predicted probability increment of the emergency level. Calculate using the following formula:

[0019] in, This represents the confidence level of the emergency level in the current fusion confidence distribution.

[0020] Optionally, the current state level of the beehive is determined by combining the fused confidence distribution and the predicted change trend, and the specific determination rule is as follows: make , , hive status level fusion confidence, , , and These represent the fusion confidence levels for normal, warning, abnormal, and emergency conditions, respectively. Preset emergency confidence threshold Anomaly confidence threshold Trend threshold and trend weight , ; Calculate the weighted emergency discrimination index As shown in the following formula:

[0021] Determine the current state level using the following formula :

[0022] in, This indicates that the state level corresponding to the highest confidence level is selected. When multiple confidence levels are equal, the one with the highest severity is selected.

[0023] Optionally, the LSTM time series prediction network uses a weighted cross-entropy loss function during training. This weighted cross-entropy loss function assigns weights positively correlated with the urgency of each state level, ensuring that the penalty for prediction errors in the urgency level is greater than that in other levels. The specific expression is shown in the following formula:

[0024] in, This represents the number of samples in the training batch. The number of steps to predict the future, i.e., the length of the prediction period; Status level The preset weights satisfy Emergency level weight Maximum, Normal Level weight Minimum; For the first training samples at time 1 The first one-hot encoded vector of the true state level Dimensional components, when the actual level is Time to take Otherwise take ; The output of the LSTM time series prediction network is the first A sample at time... State level prediction probability vector The Dimensional components, corresponding to state levels The predicted probability.

[0025] Based on the same inventive concept, this invention also provides a method and system for determining the state of a beehive based on multi-source data fusion, used to implement the aforementioned method for determining the state of a beehive based on multi-source data fusion, comprising: The multi-source sensor data acquisition module is used to collect data on temperature, weight, sound spectrum characteristics, hive entrance area images, and hive entrance bee counts inside the hive. The data preprocessing module is connected to the multi-source sensor data acquisition module and is used to perform outlier removal, missing value imputation and normalization on the acquired multi-source sensor data, and output normalized multi-source data. The association rule engine is connected to the data preprocessing module. The association rule engine stores multiple cross-dimensional association rules. The conditions of each cross-dimensional association rule are composed of parameters and corresponding thresholds of at least two different dimensions of sensor data. The association rule engine is configured to output the preset confidence level of the bee colony anomaly event corresponding to the rule when all the conditions of a rule are met at the same time, otherwise it outputs zero. The evidence fusion module, connected to the association rule engine, is used to fuse the confidence scores of multiple simultaneously activated cross-dimensional association rules as the basic probability allocation, employing DS evidence theory to output a fusion confidence score distribution that characterizes the beehive state level. The LSTM time series prediction module is connected to the data preprocessing module. The LSTM time series prediction module is configured with a pre-trained LSTM time series prediction network, which is used to receive the time series sequence of the normalized multi-source data within a preset historical time window, output the prediction probability of each state level in the future preset period, and calculate the emergency level prediction probability increment as the prediction trend. The comprehensive judgment module is connected to the evidence fusion module and the LSTM time series prediction module respectively. It is used to calculate the weighted emergency discrimination index based on the fusion confidence distribution and the prediction change trend, and compare and judge the current status level of the beehive through a preset threshold and output the status level, which includes normal, warning, abnormal and emergency.

[0026] Based on the same inventive concept, the present invention also provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform the above-described beehive state determination method based on multi-source data fusion.

[0027] Based on the same inventive concept, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-described method for determining the state of a beehive based on multi-source data fusion.

[0028] The technical solution of the present invention has at least the following advantages and beneficial effects: By introducing a cross-dimensional association rule engine, each rule is required to involve sensor data from at least two different dimensions. This enables the capture of the intrinsic correlation between multi-source data and joint identification of complex signs such as "sudden weight drop and temperature rise" in the early stages of bee swarm absconding. This overcomes the shortcomings of single-parameter independent threshold judgment, which is easily affected by short-term environmental changes and prone to false alarms, and the tendency to miss multi-parameter collaborative anomalies, thus improving the accuracy and reliability of bee swarm abnormal event identification.

[0029] By employing the DS evidence theory to fuse the confidence levels of multiple simultaneously activated association rules, the system outputs a fused confidence distribution representing the hive status level. This allows for the subdivision of the bee colony status into multiple levels, such as normal, warning, abnormal, and emergency, enabling tiered and refined assessment. This helps beekeepers prioritize tasks and efficiently arrange their work, solving the problem of excessively coarse granularity in traditional binary classification results.

[0030] By using an LSTM time series prediction network to model and analyze the time series of normalized multi-source data within a historical time window, the predicted change trend of the bee colony status level within a preset future period is obtained, giving the system the ability to perceive and predict the time series evolution. This enables early warning before anomalies actually occur, allowing bee colony management to shift from passive response to proactive and predictive intervention.

[0031] When determining the current state level of a beehive, the system integrates the fusion confidence distribution reflecting the current moment of multi-source evidence with the predicted trend reflecting the future direction of change. This avoids relying solely on a single judgment at the current cross-section, improves the stability and foresight of the state assessment, and reduces the risk of misjudgment caused by instantaneous fluctuations. Attached Figure Description

[0032] Figure 1 This is a flowchart illustrating the beehive status determination method based on multi-source data fusion according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the structure of the beehive status judgment system based on multi-source data fusion according to an embodiment of the present invention. Detailed Implementation

[0033] The following is a detailed description of the embodiments, in conjunction with the accompanying drawings.

[0034] Reference Figure 1 A method for determining the state of a beehive based on multi-source data fusion includes the following steps: Step 1: Collect multi-source sensor data inside the beehive. The multi-source sensor data includes temperature, weight, sound spectrum characteristics, hive entrance area image, and hive entrance bee count.

[0035] In some embodiments, a digital temperature sensor can be fixed on the comb spacer inside the hive to sense the air temperature inside the hive in real time, and a resistance strain gauge weighing sensor can be embedded between the hive bottom plate and the supporting structure to continuously acquire the analog signal of the total weight of the hive. After amplification and analog-to-digital conversion, the signal is used to generate weight data. At the same time, an electret microphone is installed on the inner side wall of the hive to continuously pick up the sound generated by the bees' wing flapping and other activities. The microcontroller performs frame segmentation, windowing, and fast Fourier transform on the audio signal to extract the energy distribution of each frequency band as the sound spectrum features. A miniature wide-angle camera is installed above the hive entrance to acquire images of the hive entrance area at a rate of no less than 10 frames per second. This image data is used directly as the hive entrance area image for subsequent image analysis. On the other hand, by running a lightweight target detection and multi-target tracking model in the embedded processor, the individual bees entering and leaving the hive entrance can be identified and distinguished in real time. The number of bees entering and leaving the hive entrance is accumulated separately to generate a count of bees entering and leaving the hive entrance. The temperature, weight, sound spectrum characteristics, hive entrance area image, and hive entrance bee counts are all collected and sent to an edge computing gateway or remote server via a wireless transmission module, forming the multi-source sensor data acquisition and output.

[0036] Step 2: Preprocess the multi-source sensor data. The preprocessing includes outlier removal, missing value imputation, and normalization to obtain normalized multi-source data.

[0037] In some embodiments, outlier removal is performed on the time-series data acquired by each sensor channel: For numerical data such as temperature, weight, energy of various frequency bands of sound spectrum, and bee count at the hive entrance, the mean and standard deviation within the window are calculated based on the statistical distribution characteristics of the sliding window. Points that deviate from the mean by more than three times the standard deviation are marked as outliers and removed. At the same time, values ​​outside the absolutely reasonable range determined by the physical limitations of the sensors (such as setting the temperature inside the hive to -10℃ to 60℃) are directly regarded as outliers and filtered out.

[0038] Imputation is performed on missing values ​​resulting from outlier removal and missing values ​​caused by wireless packet loss: Linear interpolation is used to perform interpolation calculations using the two nearest valid data points before and after the missing position to restore the continuity and equal interval characteristics of the time series.

[0039] Normalize the data in each dimension after interpolation: For each single parameter sequence, its historical maximum and minimum values ​​are statistically analyzed (or reasonable upper and lower bounds are preset according to the actual working conditions of the beehive). The data is linearly mapped to the [0,1] interval through the minimum-maximum normalization formula, so that the sensor data with different dimensions and amplitudes are unified to the same scale, thereby obtaining normalized multi-source data that can be used for subsequent association rule engine and LSTM time series prediction network.

[0040] Step 3: Input the normalized multi-source data into the pre-built association rule engine. The association rule engine outputs the confidence level of the corresponding bee colony anomaly event according to multiple cross-dimensional association rules. The association rule engine stores multiple cross-dimensional association rules. The conditions of each cross-dimensional association rule are composed of parameters of at least two different dimensions of sensor data and corresponding thresholds. When the conditions are met, the corresponding cross-dimensional association rule is triggered and the confidence level of the corresponding bee colony anomaly event is output.

[0041] In some embodiments, the cross-dimensional association rules include: flight absconding warning rules, swarming heat warning rules, robbing bee warning rules, and bee mite anomaly rules; The conditions for the flight absconding warning rule are that the temperature is between 35 and 40 degrees Celsius, the daily weight change rate is less than -5%, and the ratio of the number of bees entering and leaving the hive entrance is greater than 2.0. When the conditions are met simultaneously, the flight absconding warning event is triggered and a confidence level of 0.85 is output. The specific implementation of the absconding early warning rule is as follows: After outlier removal and missing value imputation of multi-source sensor data such as hive temperature, weight, and hive entrance bee count, the hive temperature is normalized to the minimum-maximum range using historical statistical values ​​of -10℃ and 60℃, linearly mapping the temperature values ​​to the [0,1] interval, where 35℃ and 40℃ correspond to normalization thresholds of 0.643 and 0.714, respectively; the weight data is also normalized to the [0,1] interval, and the edge computing gateway saves the normalized weight value at the same time 24 hours ago to calculate the daily weight change rate, which is the absolute value of the difference between the current normalized weight value and the normalized weight value 24 hours ago divided by the normalized weight value 24 hours ago and multiplied by 100%; the hive entrance bee count is based on a lightweight target detection and multi-target tracking model running on an embedded processor, which counts the total number of bees leaving the hive and the total number of bees entering the hive in real time within the most recent preset time window, and calculates the ratio of the number of hive bees to the number of hive bees entering the hive. After the normalized multi-source data is input into the pre-built association rule engine, the swarm absconding warning rule first extracts the normalized temperature value at the current moment and determines whether it falls within the closed interval [0.643, 0.714]. Simultaneously, it obtains the daily weight change rate at the current moment and determines whether its value is less than 5%. It also obtains the ratio of the number of bees entering and leaving the hive and determines whether it is greater than 2.0. The swarm absconding warning rule is triggered only when all three conditions are met, generating a swarm absconding warning event and directly outputting a confidence level of 0.85 based on a pre-set confidence benchmark value for use in the subsequent DS evidence theory fusion stage.

[0042] The swarming heat warning rule is triggered when the dominant sound frequency shifts to lower frequencies by more than 15%, the confidence level of queen cell identification in the hive entrance area image is greater than 0.7, and the daily weight increase is greater than 3%. A swarming heat warning event is triggered when all conditions are met, and a confidence level of 0.80 is output. The specific implementation of the swarming heat warning rule is as follows: an electret microphone installed on the inner side wall of the hive continuously collects the bee colony's sound. After framing, windowing, and fast Fourier transform by a microcontroller, the energy distribution of each frequency band is extracted as the sound spectrum feature. After outlier removal and missing value interpolation, the edge computing gateway locates the frequency with the highest energy in the current spectrum as the dominant frequency. And compare it with the preset normal state reference frequency of the bee colony. By comparison, the shift from the main frequency to the lower frequency was calculated as follows: The base frequency The data was obtained by statistically averaging the low-frequency peak frequencies of the sound spectrum during historical normal periods. Simultaneously, a miniature wide-angle camera installed above the hive entrance captures images of the hive entrance area at a rate of at least 10 frames per second. A lightweight target detection model runs in the embedded processor, which, in addition to identifying individual bees for entry and exit counting, is specifically trained to recognize queen cells and outputs the queen cell recognition confidence score for each frame. The system takes the highest confidence score among the most recent multiple frames as the queen cell recognition confidence score in the hive entrance area image. Furthermore, the weight data collected by the resistance strain gauge weighing sensor embedded between the hive bottom plate and the support structure, after outlier removal and missing value imputation, is normalized using the historical minimum and maximum weight values, mapped to the [0,1] interval. The edge computing gateway stores the normalized weight value at the same time 24 hours ago, and calculates the daily weight increase by dividing the difference between the current normalized weight value and the normalized weight value 24 hours ago by the normalized weight value 24 hours ago and multiplying by 100%. After the normalized and calculated data is input into the pre-built association rule engine, the swarming heat warning rule first extracts the current dominant frequency offset and determines whether it is greater than 15%; simultaneously, it obtains the queen cell identification confidence level and determines whether it is greater than 0.7; and it obtains the daily weight increase and determines whether it is greater than 3%. The swarming heat warning rule is triggered only when all three conditions are met, generating a swarming heat warning event and directly outputting a confidence level of 0.80 according to the preset confidence benchmark value for use in the subsequent DS evidence theory fusion stage.

[0043] The conditions for the bee-robbing warning rule are: the proportion of bees exhibiting aggressive postures in the hive entrance area image is greater than 30%, the fluctuation range of the ratio of bees entering and leaving the hive entrance is greater than a preset fluctuation threshold, and the energy increase of the screaming frequency band in the sound spectrum is greater than a preset energy threshold; when all the conditions are met, a bee-robbing warning event is triggered, and a confidence level of 0.75 is output. The specific implementation of the bee-robbing warning rule is as follows: a miniature wide-angle camera installed above the hive entrance continuously acquires images of the hive entrance area at a rate of no less than 10 frames per second. A lightweight target detection model running in the embedded processor, in addition to identifying individual bees, is specifically trained to distinguish aggressive postures (such as biting, chasing, entanglement, etc.). For each frame, the number of bees exhibiting aggressive postures is counted against the total number of detected bees, and the average ratio of the most recent consecutive frames is taken as the proportion of bees exhibiting aggressive postures. Simultaneously, based on the bee counts entering and leaving the hive entrance generated by the camera in conjunction with the target detection and multi-target tracking model, the most recent preset time window (e.g., Within a 30-minute period, the ratio of the number of bees leaving the hive to the number of bees entering the hive at each sampling moment is used, and the standard deviation of this ratio within the time window is used as the fluctuation range of the ratio of bees leaving and entering the hive. The preset fluctuation threshold can be set to 1.2. In addition, the electret microphone installed on the inner side wall of the hive continuously picks up the bee colony's sound. After framing, windowing, and fast Fourier transform, the energy distribution of each frequency band is extracted as the sound spectrum feature. The average energy of the screaming frequency band (e.g., 3kHz to 5kHz) is statistically obtained as the baseline energy through long-term observation of the bee colony's normal quiet period. The percentage increase of the energy of the screaming frequency band relative to the baseline energy at the current moment is calculated in real time. The preset energy threshold can be set to 40%. The three calculated values—the proportion of bees recognizing aggressive postures, the fluctuation range of the ratio of bees entering and leaving the hive, and the increase in energy in the screaming frequency band—are input into a pre-built association rule engine. The bee-robbing warning rule then determines whether the proportion of bees recognizing aggressive postures is greater than 30%, whether the fluctuation range of the ratio of bees entering and leaving the hive is greater than a preset fluctuation threshold of 1.2, and whether the increase in energy in the screaming frequency band is greater than a preset energy threshold of 40%. The rule is triggered only when all three conditions are met, generating a bee-robbing warning event and directly outputting a confidence level of 0.75 based on a preset confidence benchmark value for subsequent DS evidence theory fusion.

[0044] The conditions for the bee mite anomaly rule are: a decrease in colony activity greater than 20% as determined by bee counts at the hive entrance, a sound spectrum conforming to a preset attenuation pattern, and an increase in the number of mites on the floor obtained from image recognition of the hive entrance area greater than a preset counting threshold; when all conditions are met, a bee mite anomaly event is triggered, and a confidence level of 0.82 is output. The specific implementation of the bee mite anomaly rule is as follows: a miniature wide-angle camera installed above the hive entrance, combined with a lightweight target detection and multi-target tracking model running in an embedded processor, is used to identify and distinguish individual bees entering and leaving the hive entrance in real time, continuously recording the number of bees entering and leaving in both directions; the system uses the current time as a baseline, backtracks to the most recent preset statistical period (e.g., 30 minutes), and accumulates the total number of bees entering and leaving the hive entrance during this period as the current colony activity level. Simultaneously, the average activity level over multiple days in a healthy colony state is retrieved from the historical database at the same time and day as a normal baseline. The decrease in activity level is calculated as (baseline activity level - current activity level) / baseline activity level × 100%. When this decrease exceeds 20%, the activity level condition is met. Secondly, electret microphones installed on the inner side wall of the enclosure continuously pick up the sounds of bee colony activity, such as wing flapping. The microcontroller performs frame segmentation, windowing, and fast Fourier transform to obtain the energy distribution of each frequency band as the sound spectrum characteristics. Statistical learning is performed on the long-term spectrum of the bee colony during its healthy period to establish a frequency band energy baseline under normal conditions. A preset attenuation mode is defined as follows: the energy integral value of a specific main frequency band (e.g., 100Hz to 800Hz) that can characterize the bee colony's vitality in the real-time spectrum, compared to the energy integral value of the corresponding frequency band of the normal baseline, decreases by a percentage exceeding a preset attenuation threshold (e.g., a decrease of more than 35%). When the calculated percentage decrease in energy meets this condition, the sound spectrum is determined to conform to the preset attenuation mode. Furthermore, still utilizing the image of the nest entrance region, a lightweight target detection model specifically designed for Varroa mites is deployed and run in the embedded processor. The nest entrance floor region is defined as the region of interest within each frame image, and the number of individual mites falling in this region is identified and output in real time. The system accumulates the number of mites detected in all frames within a preset statistical period (e.g., 30 minutes) prior to the current moment to obtain the floor mite count for the current period. Then, the floor mite count for the previous day at the same time is retrieved as the baseline, and the increase in the floor mite count is calculated as the current period mite count minus the baseline mite count. It is then determined whether this increase is greater than a preset counting threshold (e.g., the increase is greater than 2). If it is greater, the floor mite counting condition is met.After the three types of cross-dimensional conditional data that have been preprocessed (outlier removal and missing value imputation while maintaining temporal alignment) are input into the pre-built association rule engine, the bee mite anomaly rule obtains the decrease in bee colony activity, the result of the sound spectrum attenuation pattern, and the increase in the number of mites falling on the substrate. The rule is triggered when all three conditions are met simultaneously: the decrease in activity is greater than 20%, the sound spectrum matches the preset attenuation pattern, and the increase in the number of mites falling on the substrate is greater than the preset counting threshold. This generates a bee mite anomaly event, and the confidence level of the event is directly output according to the preset rule confidence benchmark value of 0.82 for use in the subsequent DS evidence theory fusion stage.

[0045] In some embodiments, each cross-dimensional association rule in the association rule engine The conditions are given by The threshold determination formula is composed of sensing parameters of different dimensions; For the first The first cross-dimensional association rule Each parameter defines a threshold that satisfies the indicator function. When the first The current value of each parameter When the preset threshold condition is met, ,otherwise ; The rules When triggered, the confidence level of the corresponding bee colony anomaly event is output. Calculate using the following formula:

[0046] in, For rules The pre-set reliability benchmark value; the rule applies only to all Output confidence level when all conditions are met. Output confidence level when any condition is not met. .

[0047] Step 4: Using the confidence scores of multiple simultaneously activated cross-dimensional association rules as the basic probability allocation, the DS evidence theory is used to fuse them to obtain the fused confidence score distribution representing the beehive state level.

[0048] In some embodiments, the fusion using DS evidence theory to obtain a fusion confidence distribution characterizing the beehive state level specifically involves: Set up the recognition framework , , , and These correspond to four status levels in sequence: normal, warning, abnormal, and emergency. For the current time, the confidence level is greater than zero. Each of the activated association rules serves as independent evidence; for the first... This evidence maps the anomalous swarm event it points to to a state level. The basic probability assignment function is constructed as shown in the following equation:

[0049]

[0050] in, For the first The basic probability assignment function corresponding to the activated cross-dimensional association rule represents the degree of support that the evidence provides for each proposition; For the first The confidence level of the activation rule output; Using Dempster's combinatorial rules The basic probability assignment functions are fused together, and the fused basic probability assignment function is shown in the following equation:

[0051] in, The basic probability assignment function for the combined result after fusion; For any proposition whose combined confidence level needs to be calculated after fusion; This represents the number of activated association rules with a confidence level greater than zero at the current moment. For the first Jiao Yuan, one piece of evidence Conflict coefficient As shown in the following formula:

[0052] The fusion confidence distribution is composed of the fusion confidence of each state level, as shown below:

[0053]

[0054] in, Beehive status level The fusion confidence level.

[0055] Step 5: Input the time series of the normalized multi-source data within a preset historical time window into the LSTM time series prediction network to obtain the predicted change trend of the bee colony state level within a preset future time period.

[0056] In some embodiments, the step of inputting the time series of normalized multi-source data within a preset historical time window into an LSTM time series prediction network to obtain the predicted change trend of the bee colony state level within a preset future time period includes: At the current moment Based on this, the window length is taken as... Historical normalized multi-source data sequences , ;in, for Multidimensional normalized sensing vector at any given time; The sequence is processed sequentially using an LSTM temporal prediction network to obtain the final hidden state. And use fully connected layers and softmax functions to output the future The state level prediction probability vector at each time step is shown in the following equation:

[0057]

[0058] in, For the model's prediction of the future The probability vector of the bee colony state level at each time step. , , , and These correspond to the predicted probabilities of normal, warning, abnormal, and emergency status, respectively. This is the weight matrix of the fully connected layer output by the LSTM; For the LSTM network to process the historical time windows in sequence The final hidden state obtained after all time steps within the time step; This is the bias vector of the fully connected layer; Define the predicted trend of change as the predicted probability increment of the emergency level. Calculate using the following formula:

[0059] in, This represents the confidence level of the emergency level in the current fusion confidence distribution.

[0060] In some embodiments, the LSTM time-series prediction network uses a weighted cross-entropy loss function during training. This weighted cross-entropy loss function assigns weights positively correlated with urgency to each state level, such that the penalty for prediction errors in the urgency level is greater than that in other levels. The specific expression is shown in the following formula:

[0061] in, This represents the number of samples in the training batch. The number of steps to predict the future, i.e., the length of the prediction period; Status level The preset weights satisfy Emergency level weight Maximum, Normal Level weight Minimum; For the first training samples at time 1 The first one-hot encoded vector of the true state level Dimensional components, when the actual level is Time to take Otherwise take ; The output of the LSTM time series prediction network is the first A sample at time... State level prediction probability vector The Dimensional components, corresponding to state levels The predicted probability.

[0062] Step 6: Based on the fusion confidence distribution and the predicted trend, determine the current status level of the beehive; wherein the status level includes normal, warning, abnormal and emergency.

[0063] In some embodiments, the determination of the current state level of the beehive by combining the fused confidence distribution and the predicted change trend is specifically determined by the following rules: make , , hive status level fusion confidence, , , and These represent the fusion confidence levels for normal, warning, abnormal, and emergency conditions, respectively. Preset emergency confidence threshold Anomaly confidence threshold Trend threshold and trend weight , ; Calculate the weighted emergency discrimination index As shown in the following formula:

[0064] Determine the current state level using the following formula :

[0065] in, This indicates that the state level corresponding to the highest confidence level is selected. When multiple confidence levels are equal, the one with the highest severity is selected.

[0066] Based on the same inventive concept, and corresponding to any of the above embodiments, refer to... Figure 2 This invention provides a beehive status determination system based on multi-source data fusion, used to implement the aforementioned beehive status determination method based on multi-source data fusion, comprising: The multi-source sensor data acquisition module is used to collect data on temperature, weight, sound spectrum characteristics, hive entrance area images, and hive entrance bee counts inside the hive. The data preprocessing module is connected to the multi-source sensor data acquisition module and is used to perform outlier removal, missing value imputation and normalization on the acquired multi-source sensor data, and output normalized multi-source data. The association rule engine is connected to the data preprocessing module. The association rule engine stores multiple cross-dimensional association rules. The conditions of each cross-dimensional association rule are composed of parameters and corresponding thresholds of at least two different dimensions of sensor data. The association rule engine is configured to output the preset confidence level of the bee colony anomaly event corresponding to the rule when all the conditions of a rule are met at the same time, otherwise it outputs zero. The evidence fusion module, connected to the association rule engine, is used to fuse the confidence scores of multiple simultaneously activated cross-dimensional association rules as the basic probability allocation, employing DS evidence theory to output a fusion confidence score distribution that characterizes the beehive state level. The LSTM time series prediction module is connected to the data preprocessing module. The LSTM time series prediction module is configured with a pre-trained LSTM time series prediction network, which is used to receive the time series sequence of the normalized multi-source data within a preset historical time window, output the prediction probability of each state level in the future preset period, and calculate the emergency level prediction probability increment as the prediction trend. The comprehensive judgment module is connected to the evidence fusion module and the LSTM time series prediction module respectively. It is used to calculate the weighted emergency discrimination index based on the fusion confidence distribution and the prediction change trend, and compare and judge the current status level of the beehive through a preset threshold and output the status level, which includes normal, warning, abnormal and emergency.

[0067] Based on the same inventive concept, corresponding to any of the above embodiments, the present invention provides an electronic device, including a memory and a processor. The memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the beehive state determination method based on multi-source data fusion of the embodiments.

[0068] Alternatively, the aforementioned electronic device may be a server.

[0069] In addition, this embodiment also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the beehive state determination method based on multi-source data fusion of the embodiment.

[0070] It is understood that the processor in the embodiments of the present invention may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (D-SPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0071] The method steps in the embodiments of the present invention can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0072] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of the present invention is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a storage medium or transmitted through a storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (D-SL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive (SSD)).

Claims

1. A method for determining the state of a beehive based on multi-source data fusion, characterized in that, Includes the following steps: Collect multi-source sensor data inside the hive, including temperature, weight, sound spectrum characteristics, hive entrance area image, and hive entrance bee count; The multi-source sensor data is preprocessed, including outlier removal, missing value imputation, and normalization, to obtain normalized multi-source data. The normalized multi-source data is input into a pre-built association rule engine, which outputs the confidence level of the corresponding bee colony anomaly event based on multiple cross-dimensional association rules. The association rule engine stores multiple cross-dimensional association rules, and the conditions of each cross-dimensional association rule are composed of parameters of at least two different dimensions of sensor data and corresponding thresholds. When the conditions are met, the corresponding cross-dimensional association rule is triggered and the confidence level of the corresponding bee colony anomaly event is output. Using the confidence scores of multiple simultaneously activated cross-dimensional association rules as the basic probability allocation, the DS evidence theory is used for fusion to obtain the fused confidence score distribution characterizing the beehive state level. The normalized multi-source data within a preset historical time window is input into an LSTM time-series prediction network to obtain the predicted trend of the bee colony state level within a preset future time period. By combining the fusion confidence distribution and the predicted trend, the current status level of the beehive is determined; wherein, the status level includes normal, warning, abnormal and emergency.

2. The beehive status determination method based on multi-source data fusion as described in claim 1, characterized in that, The cross-dimensional association rules include: flight absconding early warning rules, swarming fever early warning rules, robbing bee early warning rules, and bee mite anomaly rules; The conditions for the flight absconding warning rule are that the temperature is between 35 and 40 degrees Celsius, the daily weight change rate is less than -5%, and the ratio of the number of bees entering and leaving the hive is greater than 2.

0. When the conditions are met simultaneously, the flight absconding warning event is triggered and a confidence level of 0.85 is output. The conditions for the swarming heat warning rule are: the shift of the main sound frequency to the low frequency is greater than 15%, the confidence level of queen cell identification in the hive entrance area image is greater than 0.7, and the daily weight increase is greater than 3%; when the above conditions are met simultaneously, the swarming heat warning event is triggered and a confidence level of 0.80 is output. The conditions for the bee robbing warning rule are: the proportion of bees with aggressive postures in the hive entrance area image is greater than 30%, the fluctuation range of the ratio of bees entering and leaving the hive entrance is greater than a preset fluctuation threshold, and the energy increase of the screaming frequency band in the sound spectrum is greater than a preset energy threshold; when the above conditions are met simultaneously, a bee robbing warning event is triggered and a confidence level of 0.75 is output. The conditions for the bee mite abnormality rule are that the decrease in bee colony activity determined by the bee count at the hive entrance is greater than 20%, the sound spectrum conforms to the preset attenuation mode, and the increase in the number of mites falling on the bottom plate obtained by the image recognition of the hive entrance area is greater than the preset counting threshold; when the above conditions are met simultaneously, the bee mite abnormality event is triggered and a confidence level of 0.82 is output.

3. The beehive status determination method based on multi-source data fusion as described in claim 1, characterized in that, In the association rule engine, each cross-dimensional association rule The conditions are given by The threshold determination formula is composed of sensing parameters of different dimensions; For the first The first cross-dimensional association rule Each parameter defines a threshold that satisfies the indicator function. When the first The current value of each parameter When the preset threshold condition is met, ,otherwise ; The rules When triggered, the confidence level of the corresponding bee colony anomaly event is output. Calculate using the following formula: in, For rules The pre-set reliability benchmark value; the rule applies only to all Output confidence level when all conditions are met. Output confidence level when any condition is not met. .

4. The beehive status determination method based on multi-source data fusion as described in claim 1, characterized in that, The method employs DS evidence theory for fusion to obtain the fusion confidence distribution characterizing the beehive state level, specifically as follows: Set up the recognition framework , , , and These correspond to four status levels in sequence: normal, warning, abnormal, and emergency. For the current time, the confidence level is greater than zero. Each of the activated association rules serves as independent evidence; for the first... This evidence maps the anomalous swarm event it points to to a state level. The basic probability assignment function is constructed as shown in the following equation: in, For the first The basic probability assignment function corresponding to the activated cross-dimensional association rule represents the degree of support that the evidence provides for each proposition; For the first The confidence level of the activation rule output; Using Dempster's combinatorial rules The basic probability assignment functions are fused together, and the fused basic probability assignment function is shown in the following equation: in, The basic probability assignment function for the combined result after fusion; For any proposition whose combined confidence level needs to be calculated after fusion; This represents the number of activated association rules with a confidence level greater than zero at the current moment. For the first Jiao Yuan, one piece of evidence Conflict coefficient As shown in the following formula: The fusion confidence distribution is composed of the fusion confidence of each state level, as shown below: in, Beehive status level The fusion confidence level.

5. The beehive status determination method based on multi-source data fusion as described in claim 4, characterized in that, The step of inputting the time series of normalized multi-source data within a preset historical time window into an LSTM time series prediction network to obtain the predicted change trend of the bee colony state level within a preset future time period includes: At the current moment Based on this, the window length is taken as... Historical normalized multi-source data sequences , ;in, for Multidimensional normalized sensing vector at any given time; The sequence is processed sequentially using an LSTM temporal prediction network to obtain the final hidden state. And use fully connected layers and softmax functions to output the future The state level prediction probability vector at each time step is shown in the following equation: in, For the model's prediction of the future The probability vector of the bee colony state level at each time step. , , , and These correspond to the predicted probabilities of normal, warning, abnormal, and emergency status, respectively. This is the weight matrix of the fully connected layer output by the LSTM; For the LSTM network to process the historical time windows in sequence The final hidden state obtained after all time steps within the time step; This is the bias vector of the fully connected layer; Define the predicted trend of change as the predicted probability increment of the emergency level. Calculate using the following formula: in, This represents the confidence level of the emergency level in the current fusion confidence distribution.

6. The beehive status determination method based on multi-source data fusion as described in claim 5, characterized in that, The current state level of the beehive is determined by combining the fused confidence distribution and the predicted trend. The specific determination rule is as follows: make , , hive status level fusion confidence, , , and These represent the fusion confidence levels for normal, warning, abnormal, and emergency conditions, respectively. Preset emergency confidence threshold Anomaly confidence threshold Trend threshold and trend weight , ; Calculate the weighted emergency discrimination index As shown in the following formula: Determine the current state level using the following formula : in, This indicates that the state level corresponding to the highest confidence level is selected. When multiple confidence levels are equal, the one with the highest severity is selected.

7. The beehive status determination method based on multi-source data fusion as described in claim 5, characterized in that, The LSTM time series prediction network uses a weighted cross-entropy loss function during training. The weighted cross-entropy loss function assigns weights that are positively correlated with the urgency of each state level, so that the penalty for prediction deviation of the urgency level is greater than that of other levels. The weighted cross-entropy loss function The specific expression is shown in the following formula: in, This represents the number of samples in the training batch. The number of steps to predict the future, i.e., the length of the prediction period; Status level The preset weights satisfy Emergency level weight Maximum, Normal Level weight Minimum; For the first training samples at time 1 The first one-hot encoded vector of the true state level Dimensional components, when the actual level is Time to take Otherwise take ; The output of the LSTM time series prediction network is the first A sample at time... State level prediction probability vector The Dimensional components, corresponding to state levels The predicted probability.

8. A beehive state determination system based on multi-source data fusion, used to implement the beehive state determination method based on multi-source data fusion as described in any one of claims 1-7, characterized in that, include: The multi-source sensor data acquisition module is used to collect data on temperature, weight, sound spectrum characteristics, hive entrance area images, and hive entrance bee counts inside the hive. The data preprocessing module is connected to the multi-source sensor data acquisition module and is used to perform outlier removal, missing value imputation and normalization on the acquired multi-source sensor data, and output normalized multi-source data. The association rule engine is connected to the data preprocessing module. The association rule engine stores multiple cross-dimensional association rules. The conditions of each cross-dimensional association rule are composed of parameters and corresponding thresholds of at least two different dimensions of sensor data. The association rule engine is configured to output the preset confidence level of the bee colony anomaly event corresponding to the rule when all the conditions of a rule are met at the same time, otherwise it outputs zero. The evidence fusion module, connected to the association rule engine, is used to fuse the confidence scores of multiple simultaneously activated cross-dimensional association rules as the basic probability allocation, employing DS evidence theory to output a fusion confidence score distribution that characterizes the beehive state level. The LSTM time series prediction module is connected to the data preprocessing module. The LSTM time series prediction module is configured with a pre-trained LSTM time series prediction network, which is used to receive the time series sequence of the normalized multi-source data within a preset historical time window, output the prediction probability of each state level in the future preset period, and calculate the emergency level prediction probability increment as the prediction trend. The comprehensive judgment module is connected to the evidence fusion module and the LSTM time series prediction module respectively. It is used to calculate the weighted emergency discrimination index based on the fusion confidence distribution and the prediction change trend, and compare and judge the current status level of the beehive through a preset threshold and output the status level, which includes normal, warning, abnormal and emergency.

9. An electronic device, characterized in that, The device includes a memory and a processor, wherein the memory stores a computer program and the processor runs the computer program to enable the electronic device to perform the beehive state determination method based on multi-source data fusion as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the beehive state determination method based on multi-source data fusion as described in any one of claims 1-7.

Images