Dynamic bee colony regulation method and system based on multi-modal biometric recognition

Abstract

The application provides a dynamic bee colony regulation method and system based on multi-modal biological feature recognition, and relates to the technical field of intelligent bee industry. The method comprises the following steps: collecting images of a nest door, a brood nest, sound, gas concentration, vibration, temperature and the weight of a beehive; positioning a queen bee from the image of the brood nest, determining the egg-laying state in combination with the temperature difference in the queen bee area and the abdominal movement characteristics; extracting the larva shape by instance segmentation of the image of the brood nest, obtaining the larva age and health by using a development model; extracting the worker bee body surface features from the image of the nest door, clustering the worker bee age structure in combination with the frequency of entry and exit and the pollen carrying ratio; inputting the emotional features extracted based on sound, smell, vibration and image into a model to output the emotional state of the bee colony; generating pheromone release and mechanical regulation instructions, releasing pheromone analogs at a speed rate according to the number of the bee colony, adjusting the opening degree of the nest door, the distance between the bee paths and the temperature of the brood nest area. The problems that the prior art cannot accurately identify the biological features inside the bee colony and actively close-loop regulate are solved.

Description

Technical Field

[0001] This invention relates to the field of intelligent beekeeping technology, and more specifically, to a dynamic bee colony control method and system based on multimodal biometric recognition. Background Technology

[0002] In intelligent beekeeping, the behavior and health of bee colonies depend on biological characteristics such as queen bee egg-laying, larval development, worker bee age structure, and colony mood. Traditional beekeeping relies on beekeepers opening hives for observation, which is highly subjective, disruptive to the colony, and difficult to quantify. To reduce human intervention, intelligent monitoring systems are increasingly being used. Patent document CN121301816A discloses an AI-based intelligent beekeeping monitoring, analysis, and management system and method. This system collects environmental parameters such as temperature, humidity, weight, and sound frequency within the beehive, cleans and normalizes the data, inputs it into an activity assessment model, outputs a colony activity score and status label, and finally pushes management suggestions to the beekeeper. However, this solution can only make a rough judgment on the overall activity of the bee colony based on environmental parameters. It cannot directly identify biological characteristics such as queen bee egg-laying status, larval development stage, and worker bee age structure, nor can it actively execute any control actions at the beehive level, and it does not involve coordinated management between multiple beehives.

[0003] The existing technology has the following main problems: First, there is a many-to-many mapping relationship between the collected environmental parameters and the actual biological state of the bee colony. For example, both a decrease in temperature and queen bee aging can cause a decrease in activity. Environmental parameters alone cannot distinguish the root cause, resulting in management recommendations that are general and lack specificity. Second, the beehive itself does not have the ability to actively control and execute. There is a significant time lag between the system detecting a problem and manual intervention, which can easily lead to missing the best intervention window. Third, the lack of means to perceive the emotional state of bee colonies (such as anxiety, swarming fever, and robbing alertness) makes it difficult to provide early warnings before swarming, robbing, and other group behavior events occur. Fourth, it does not consider the coordination between multiple hive units at the apiary level, and cannot achieve the balance between strong and weak colonies, isolation of diseased colonies, and dynamic division of labor between foraging and breeding colonies based on the biological characteristics of each colony.

[0004] It is evident that existing technologies cannot meet the needs of deeply intelligent beekeeping, from monitoring environmental parameters to accurately identifying biological characteristics, and then to proactive closed-loop control and farm-level collaborative management. Summary of the Invention

[0005] The purpose of this invention is to provide a dynamic bee colony control method and system based on multimodal biometric identification, which aims to solve the problem that existing beekeeping monitoring systems cannot accurately identify the internal biological characteristics of bee colonies and carry out active closed-loop control by relying solely on environmental parameters.

[0006] This invention is achieved through the following technical solution: A dynamic bee colony control method based on multimodal biometric recognition includes the following steps: Collect multimodal data of the beehive, including images of the hive entrance area, images of the brood comb area, bee colony sounds, distribution of volatile organic compound concentration inside the beehive, beehive vibration, multi-point temperature data, and beehive weight; The queen bee is located from the image of the brood comb area, and the abdominal and movement features of the queen bee are extracted. Combined with the temperature difference between the area where the queen bee is located and the surrounding area in the multi-point temperature data, the queen bee's egg-laying status is determined. The image of the brood region is segmented into instances to extract the morphological features of the larvae in the nest cells. The age and developmental health status of the larvae are determined based on the larval development model. The body surface features of returning worker bees are extracted from the images of the hive entrance area. Combined with the frequency of worker bees entering and leaving the hive and the proportion of pollen carried, the proportion of worker bees of different ages is estimated through cluster analysis to obtain the age structure of worker bees. Emotional features are extracted from the bee colony sounds, odor emotional features are extracted from the volatile organic compound concentration distribution, vibration emotional features are extracted from the hive vibrations, and image emotional features are extracted from the hive entrance area image. The extracted features are input into an emotion classification model to output the bee colony's emotional state. Based on the queen bee's egg-laying status, the age and health of the larvae, the age structure of the worker bees, and the emotional state of the bee colony, control instructions are generated; wherein, the control instructions include pheromone release instructions and mechanical control instructions; The pheromone release command is executed to release pheromone analogs in a pulse manner, and the release rate is dynamically adjusted based on the number of bees estimated from the hive weight and the hive entrance area image; the mechanical control command is executed to adjust at least one of the hive entrance opening, bee passage spacing, and brood comb area temperature.

[0007] Optionally, determining the queen bee's egg-laying status specifically includes: The image of the brood comb region is input into the queen bee localization model to obtain the queen bee bounding box; the abdominal swelling of the queen bee is extracted within the bounding box. Oviposition posture probability and mobile activity The average temperature difference between the queen bee's location and the background area of ​​the brood comb was obtained from the multi-point temperature data. ; Calculate the queen bee's egg-laying status index using the following formula. As shown in the following formula:

[0008] in, , , and For the corresponding weight coefficients, and ; when and At that time, it is determined that spawning is vigorous; when and If oviposition occurs, oviposition is considered to have stopped; otherwise, oviposition is considered to have weakened. The threshold for peak egg production, The threshold for ovulation arrest, and .

[0009] Optionally, determining the larval age and developmental health status specifically includes: An instance segmentation network was used to process the sponge region image to segment out each complete nest cell region; the body length of the larva was extracted within each nest cell region. Body color saturation curvature and feeding status indicators ,in This indicates that the cell is being fed by worker bees. This indicates that the food was not fed. The estimated age of larvae was calculated using a multiple linear regression model. As shown in the following formula:

[0010] in, The intercept is... , , and These are the regression coefficients corresponding to each feature; Will Compared with the preset age range: If Then classify them as 1-3 days old. Then classify them as 4-6 days old. Then classify them as 7-8 days old; When the body length Less than the lower limit of the standard body length for the preset age range, and When the duration exceeds the time threshold, the larvae are considered to be developing slowly.

[0011] Optionally, estimating the proportion of worker bees of different ages through cluster analysis specifically includes: For each returning worker bee detected from the image of the hive entrance region, a three-dimensional feature vector is constructed. ,in The average grayscale value of the body color. The wing edge wear index. The density index of body hair on the chest and back; A Gaussian mixture model is used to cluster the feature vectors of all returning worker bees. The probability density function of the Gaussian mixture model is shown in the following equation:

[0012] in, For the first The mixture weights of Gaussian components, and ; It is the mean vector; It is the covariance matrix; Solving for parameters using the expectation-maximization algorithm , The four cluster centers were assigned to four age ranges: childhood, adolescence, adulthood, and old age. By the mixed weight of each component The worker bee age structure is obtained by estimating the proportion of worker bees in the colony within that age range.

[0013] Optionally, the step of inputting the extracted features into the emotion classification model and outputting the swarm's emotion state specifically includes: Extract high-frequency screaming energy from the bee swarm sound. and low-frequency humming energy Calculate the emotional index of the voice As shown in the following formula:

[0014] Extract the response vector of the electronic nose sensor array from the volatile organic compound concentration distribution. and its rate of change vector in adjacent acquisition cycles ; Extract the main frequency of the vibration spectrum from the vibration of the beehive. Dominant frequency amplitude as well as The ratio of frequency band energy to total energy ; Extract bee colony density from the image of the hive entrance area. Percentage of worker bees in an aggressive posture and the average wingbeat frequency of worker bees ; Constructing emotional feature vectors Inputting a pre-trained lightweight gradient boosting machine model yields a five-dimensional probability vector. , , , , , and These correspond to the probabilities of being quiet, normal, agitated, swarming, and robbing bee alert, respectively. The category corresponding to the highest probability is taken as the bee colony's emotional state.

[0015] Optionally, executing the pheromone release instruction specifically includes: Based on the target emotion probability corresponding to the bee colony's emotional state ,when At the following release rates The micropump is controlled as shown in the following formula:

[0016] in, Based on the release rate; The number of bee individuals in the colony is estimated from the weight of the hive and the image of the hive entrance area; The baseline number of bee colonies; For gain coefficient; when hour, ; Employing a pulsed release mode: within each decision cycle, with rate of release Each time, lasting Seconds, the interval between two consecutive releases Second; Real-time acquisition of total volatile organic compound concentration inside the beehive ,when season Linearly reduced to 50% of the original value; when Immediately stop releasing and send an alarm to the beekeeper's terminal; among them, To set a warning concentration, The concentration is at the absolute safety upper limit, and .

[0017] Optionally, executing the mechanical control command specifically includes: Nest entrance opening adjustment: Define the nest entrance opening angle Its adjustable range is When the bee colony's emotional state is swarming fever, control... When the external environment is in the honey flow period and the bee colony is not exhibiting swarming behavior, control... Control of other situations ; Cellular spacing adjustment: Define the cell spacing When the bee colony is identified as being in its breeding season, control... When the bee colony is identified as being in the honey flow period, control... ; Spleen zone temperature control: Set the target temperature for the spleen zone. Based on the measured temperature of the spleen region in the multi-point temperature data. A closed-loop control strategy is used to drive the heating or cooling unit, so that... Maintain at Within the range.

[0018] Based on the same inventive concept, this invention also provides a dynamic bee colony control system based on multimodal biometric recognition, used to implement the aforementioned dynamic bee colony control method based on multimodal biometric recognition, comprising: The multimodal data acquisition module is used to collect images of the hive entrance area, brood comb area, bee colony sounds, volatile organic compound concentration distribution inside the hive, hive vibration, multi-point temperature data, and hive weight. The queen bee egg-laying status determination module is connected to the multimodal data acquisition module. It receives the brood comb area image and the multi-point temperature data. It is used to locate the queen bee from the brood comb area image, extract the queen bee's abdominal features and movement features, and determine the queen bee's egg-laying status by combining the temperature difference between the area where the queen bee is located and the surrounding area in the multi-point temperature data. The larval development status determination module is connected to the multimodal data acquisition module, receives the image of the brood region, performs instance segmentation on the image of the brood region, extracts the morphological features of the larvae in the nest, and determines the larval age and developmental health status according to the larval development model. The worker bee age structure estimation module is connected to the multimodal data acquisition module. It receives the image of the hive entrance area and extracts the body surface features of returning worker bees from the image of the hive entrance area. Combining the frequency of worker bees entering and leaving the hive and the proportion of pollen carried, it estimates the proportion of worker bees of different ages through cluster analysis and outputs the worker bee age structure. The bee colony emotion recognition module is connected to the multimodal data acquisition module. It receives the bee colony sounds, the volatile organic compound concentration distribution, the beehive vibration, and the hive entrance area image. It is used to extract sound emotion features, odor emotion features, vibration emotion features, and image emotion features. The extracted features are input into the emotion classification model to output the bee colony emotion state. The regulation command generation module is connected to the queen bee egg-laying status determination module, the larvae development status determination module, the worker bee age structure estimation module, and the bee colony emotion recognition module, respectively. It is used to generate regulation commands based on the queen bee egg-laying status, larvae age and developmental health status, worker bee age structure, and bee colony emotion status. The regulation commands include pheromone release commands and mechanical regulation commands. The pheromone release execution module is connected to the control command generation module and the multimodal data acquisition module. It receives the pheromone release command, the weight of the beehive, the image of the hive entrance area, and the concentration distribution of volatile organic compounds. It is used to release pheromone analogs in a pulse manner and dynamically adjust the release rate according to the number of bees estimated from the beehive weight and the image of the hive entrance area, as well as the total concentration of volatile organic compounds. The mechanical control execution module is connected to the control command generation module and the multimodal data acquisition module. It receives the mechanical control command and the multi-point temperature data and is used to perform hive entrance opening adjustment, bee spacing adjustment, and closed-loop control of the brood comb temperature based on the multi-point temperature data.

[0019] Based on the same inventive concept, the present invention also provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform the above-described dynamic bee colony control method based on multimodal biometric recognition.

[0020] Based on the same inventive concept, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-described dynamic bee colony control method based on multimodal biometric recognition.

[0021] The technical solution of the present invention has at least the following advantages and beneficial effects: By locating the queen bee from brood comb images and integrating the temperature difference between the queen bee's location and the surrounding area to determine the egg-laying status, segmenting brood comb image instances to extract larval morphology to determine larval age and developmental health, and clustering worker bee age structure from hive entrance images combined with entry and exit frequency and pollen-carrying ratio, this approach overcomes the shortcomings of traditional methods that rely solely on environmental parameters and cannot distinguish the root cause. It makes the biological characteristics within the bee colony quantifiable and distinguishable, providing an accurate basis for targeted regulation.

[0022] Simultaneously extracting the emotional characteristics of bee colony sounds, the emotional characteristics of odor concentration distribution of volatile organic compounds, the emotional characteristics of hive vibration, and the emotional characteristics of images of hive entrance area, and inputting them into an emotion classification model, it can perceive emotional states such as anxiety, swarming fever, and robbing alertness in real time, filling the technical gap in bee colony emotion perception. It can provide early warnings before swarming, robbing, and other group behavior events occur, improving risk prediction capabilities.

[0023] Based on the queen bee's egg-laying status, larval age and health, worker bee age structure, and colony mood, the system automatically generates control commands to release pheromone analogs in a pulsed manner. It also dynamically adjusts the release rate using the hive weight and the colony size estimated by the hive entrance area image. Simultaneously, it automatically adjusts at least one of the following: hive entrance opening, bee passage spacing, and brood comb temperature. This gives the hive autonomous control capabilities, enabling timely and precise control during the optimal intervention window and avoiding missed opportunities.

[0024] By outputting high-granular biological characteristics such as the queen bee's egg-laying status, the health status of larvae development, the age structure of worker bees, and the mood of the colony, the strength and development trend of the colony can be accurately reflected. This provides a basis for decision-making in collaborative operations such as balancing strong and weak colonies, isolating diseased colonies, and the dynamic division of labor between foraging and breeding colonies, which is conducive to achieving efficient and healthy production in the apiary as a whole. Attached Figure Description

[0025] Figure 1 This is a flowchart illustrating the dynamic bee colony control method based on multimodal biometric recognition, according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the structure of a dynamic bee colony control system based on multimodal biometric recognition according to an embodiment of the present invention. Detailed Implementation

[0026] The following is a detailed description of the embodiments, in conjunction with the accompanying drawings.

[0027] Reference Figure 1 A dynamic bee colony control method based on multimodal biometric recognition includes the following steps: Step 1: Collect multimodal data of the beehive, including images of the hive entrance area, images of the brood comb area, bee colony sounds, distribution of volatile organic compound concentration inside the beehive, beehive vibration, multi-point temperature data, and beehive weight.

[0028] In some embodiments, at least one high-definition industrial camera can be installed in front of the hive entrance to continuously capture images of the entrance area at a frequency of 10-30 frames per second. A high-resolution macro camera is also installed inside the hive, perpendicular to the brood comb, and, with supplemental lighting, captures images of the brood comb area during each hive inspection or at a preset cycle (e.g., every 10 minutes). A wideband microphone is installed on the inner wall of the hive near the center of the brood comb, with a sampling frequency of at least 44.1 kHz, to continuously collect the bee colony's sound signals. Multiple semiconductor or electrochemical volatile organic compound (VOC) sensors are evenly arranged around the brood comb inside the hive, forming an electronic nose sensor array to acquire VOC concentration distribution data at different locations with a sampling cycle of once per minute. Piezoelectric accelerometers are fixed to the bottom plate and side walls of the hive, with a sampling frequency of 500-1000 Hz, to collect data on the hive's three-dimensional movement. The system collects vibration signals; multiple digital temperature sensors are arranged vertically in layers inside the hive (e.g., 3-5 temperature measurement points are set in the gaps between each layer of combs) to collect multi-point temperature data at a frequency of once every 10 seconds; a weighing sensor (e.g., a cantilever beam pressure sensor) is installed at the bottom of the hive to continuously monitor the total weight of the hive, with a data sampling frequency of once per minute; all sensors and cameras are synchronously collected through a data acquisition card or embedded edge computing node, and the timestamps of the collected data are uniformly calibrated with the Network Time Protocol (NTP) to ensure that the multimodal data is aligned in the time dimension. The raw data is then converted from analog to digital, filtered and denoised (e.g., median filtering is used to remove image impulse noise, and bandpass filtering is used to retain the effective frequency bands of sound and vibration), and packaged in a standardized format. It is then temporarily stored in the local storage unit and simultaneously uploaded to the cloud or edge server through a wireless communication module for subsequent steps. During the above data acquisition process, RGB color images were preferentially used for the hive entrance and brood comb areas. Exposure time and gain parameters were automatically recorded during acquisition to ensure image consistency under different lighting conditions. Bee colony sound was simultaneously recorded during acquisition for subsequent noise reduction processing. Volatile organic compound (VOC) concentration distribution data underwent zero-point calibration and standard gas calibration before acquisition. Multi-point temperature data was simultaneously recorded with ambient temperature for compensation and correction. Beehive weight data was automatically zeroed before each acquisition to eliminate zero-point drift. All multimodal data were stored using a unified time-series index to form a structured dataset for use in steps two through seven.

[0029] Step 2: Locate the queen bee from the brood comb area image, extract the queen bee's abdominal features and movement characteristics, and combine the temperature difference between the queen bee's location and the surrounding area in the multi-point temperature data to determine the queen bee's egg-laying status.

[0030] In some embodiments, determining the queen bee's egg-laying status specifically includes: The image of the brood comb region is input into the queen bee localization model to obtain the queen bee bounding box; the abdominal swelling of the queen bee is extracted within the bounding box. Oviposition posture probability and mobile activity The average temperature difference between the queen bee's location and the background area of ​​the brood comb was obtained from the multi-point temperature data. ; Calculate the queen bee's egg-laying status index using the following formula. As shown in the following formula:

[0031] in, , , and For the corresponding weight coefficients, and ; when and At that time, it is determined that spawning is vigorous; when and If oviposition occurs, oviposition is considered to have stopped; otherwise, oviposition is considered to have weakened. The threshold for peak egg production, The threshold for ovulation arrest, and .

[0032] Further, the RGB color images of the brood region collected in step one are input into a pre-trained queen bee localization model. This model uses a deep learning-based object detection network (such as YOLOv5 or Faster R-CNN), and outputs a rectangular bounding box containing the queen bee after feature extraction and bounding box regression. After obtaining this bounding box, the abdominal swelling degree A of the queen bee is further extracted from the image region within the box. Specifically, the degree of abdominal swelling is quantified by calculating the ratio of the major axis to the minor axis of the ellipse fitting of the abdominal contour. At the same time, a pose classification network (such as ResNet-18) is used to output the probability that the queen bee is in an egg-laying posture. (i.e., the movement characteristics of the queen bee bending her abdomen and extending her ovipositor into the brood cell), and the pixel displacement of the queen bee between adjacent image frames is calculated by cross-frame tracking, and then divided by the frame interval time to obtain the movement activity level. Subsequently, from the multi-point temperature data collected in step one, using the projection position of the queen bee's bounding box center on the brood comb plane as a reference, the average temperature of 5×5 temperature measurement points around this position was selected as the temperature of the queen bee's area, and the average temperature of the brood comb background area far from the queen bee's position (more than 10cm away) was selected as the background temperature. The difference between the two was calculated. To ensure the stability of the judgment results, three consecutive decision cycles (each cycle being 10 minutes) were performed. and A moving average filter is applied before threshold comparison to avoid misjudgments caused by instantaneous fluctuations in single image or temperature data. The final judgment result is output as the queen bee's egg-laying status to subsequent control steps. Throughout the process, the queen bee positioning model and posture classification network are trained using a dataset of brood comb images labeled with the queen bee's bounding box, abdominal swelling level, and posture category. Mobility calculation is based on multi-target tracking algorithms (such as SORT or DeepSORT). Temperature data is selected through a calibration mapping relationship between image coordinates and the physical coordinates of the temperature sensor, ensuring accurate spatial alignment of multimodal information and comprehensively and objectively reflecting the queen bee's current reproductive physiological state.

[0033] Step 3: Perform instance segmentation on the image of the brood region, extract the morphological features of the larvae in the nest, and determine the age and developmental health status of the larvae based on the larval development model.

[0034] In some embodiments, determining the larval age and developmental health status specifically includes: An instance segmentation network was used to process the sponge region image to segment out each complete nest cell region; the body length of the larva was extracted within each nest cell region. Body color saturation curvature and feeding status indicators ,in This indicates that the cell is being fed by worker bees. This indicates that the food was not fed. The estimated age of larvae was calculated using a multiple linear regression model. As shown in the following formula:

[0035] in, The intercept is... , , and These are the regression coefficients corresponding to each feature; Will Compared with the preset age range: If Then classify them as 1-3 days old. Then classify them as 4-6 days old. Then classify them as 7-8 days old; When the body length Less than the lower limit of the standard body length for the preset age range, and When the duration exceeds the time threshold, the larvae are considered to be developing slowly.

[0036] Further, the RGB color images of the brood region acquired in step one are first input into a pre-trained instance segmentation network (e.g., Mask R-CNN or YOLACT). After feature pyramid extraction and pixel-level mask regression, the network performs instance-level segmentation on each cell in the image, outputting the contour mask of each cell and its corresponding cell center coordinates. Subsequently, for each segmented cell region, morphological features related to the larva are further extracted within the mask. Specifically, this includes obtaining the larval body length by calculating the minimum bounding rectangle or ellipse fitting of the larval connected components within the mask region. (In millimeters, converted using the calibration relationship between image pixel size and physical size), body color saturation. (i.e., the average saturation component of the larval body color in the HSV color space), curvature (Defined as the maximum curvature of the larval body's centerline, calculated through skeleton extraction and polynomial fitting) and feeding status indicators (By detecting whether worker bees are inserting their heads into or touching the larvae with their mouthparts inside the cell, a lightweight attitude estimation network is used to determine whether the cell is being fed by worker bees at any given moment. If feeding is observed, then...) ,otherwise Based on this, the estimated age of the larvae was calculated. Then Compare with the preset age range, if Then classify them as 1-3 days old. Then classify them as 4-6 days old. Then classify them as 7-8 days old. If the age exceeds the above range, it is marked as an abnormal age; at the same time, based on the estimated age range to which the larva belongs, a preset standard lower limit threshold for body length is retrieved (this threshold is determined by the lower quartile or mean of the statistical distribution of body length of healthy larvae within the same age range minus two standard deviations). When the extracted body length... If the temperature is below the lower threshold and the feeding status indicator of the nest cell is continuously monitored, then the temperature is below the lower threshold. If the duration of continuous occurrence exceeds a time threshold (e.g., two consecutive decision cycles, each lasting 10 minutes, totaling over 20 minutes), the larva is determined to be underdeveloped, and the location information and degree of underdevelopment of the nest are recorded. To ensure the robustness of segmentation and feature extraction, the segmentation results of each nest are screened for confidence, and only samples with a mask confidence score higher than 0.85 are retained for subsequent calculations. For regions where segmentation fails due to image blurring or uneven lighting, a super-resolution reconstruction based on adjacent frames and a multi-frame fusion strategy are used to supplement the segmentation. Finally, the age distribution of larvae and the proportion of underdeveloped individuals in all nests are output in structured data form for use in subsequent control steps.

[0037] Step 4: Extract the body surface features of returning worker bees from the image of the hive entrance area, and combine the frequency of worker bees entering and leaving the hive and the proportion of pollen carried, and estimate the proportion of worker bees of different ages through cluster analysis to obtain the age structure of worker bees.

[0038] In some embodiments, estimating the proportion of worker bees of different ages through cluster analysis specifically includes: For each returning worker bee detected from the image of the hive entrance region, a three-dimensional feature vector is constructed. ,in The average grayscale value of the body color. The wing edge wear index. The density index of body hair on the chest and back; A Gaussian mixture model is used to cluster the feature vectors of all returning worker bees. The probability density function of the Gaussian mixture model is shown in the following equation:

[0039] in, For the first The mixture weights of Gaussian components, and ; It is the mean vector; It is the covariance matrix; Solving for parameters using the expectation-maximization algorithm , The four cluster centers were assigned to four age ranges: childhood, adolescence, adulthood, and old age. By the mixed weight of each component The worker bee age structure is obtained by estimating the proportion of worker bees in the colony within that age range.

[0040] Furthermore, firstly, high-definition industrial cameras are set up in the hive entrance area to continuously acquire images of returning worker bees. A deep learning object detection model (such as YOLOv5) is used to detect individual worker bees frame by frame, and a multi-object tracking algorithm (such as DeepSORT) is employed to assign a unique identifier to each worker bee, recording its hive entry and exit events and timestamps, thereby statistically determining the hive entry and exit frequency of each worker bee within a preset observation period. Simultaneously, a pollen cluster recognition network is used to classify the hind leg image patches of each tracked worker bee to determine whether it carries pollen, and the pollen-carrying ratio of each individual is calculated accordingly. Then, for returning worker bees with stable identifiers, their body regions are cropped from the images, and three body surface aging features are extracted: the average grayscale value of the pixels on the thorax, abdomen, and dorsal plate is calculated as the average grayscale value of the body color. Curvature analysis and notch counting were performed on the wing edge profile, and the normalized result yielded the wing edge wear index. The degree of trichome pubescence coverage on the chest and back is measured by local binary pattern texture analysis, and the trichome pubescence density index is output. Thus, a three-dimensional feature vector is constructed. Next, the feature vectors of all samples are input into a Gaussian mixture model, with four Gaussian components corresponding to the four age intervals of infancy, adolescence, adulthood, and old age. The expectation-maximization algorithm is used to iteratively solve for the mixture weights of each component. Mean vector Covariance Matrix After convergence, the parameters of each Gaussian component are obtained; the mixture weights of each component are then used. This is directly used as the initial estimate of the proportion of worker bees in the corresponding age range. Finally, the frequency of entering and leaving the hive and the proportion of pollen carried are introduced to verify the biological consistency of the clustering results: the average frequency of entering and leaving the hive and the average proportion of pollen carried by worker bee samples falling into each Gaussian component are statistically analyzed and compared with known behavioral patterns of worker bee age (e.g., older worker bees have higher foraging flight frequency and higher pollen carrying proportion). If the behavioral index of a certain component deviates from the preset range, the corresponding mixed weight is adaptively corrected according to the degree of deviation. Finally, the age structure of worker bees after behavioral feature calibration is output, that is, the precise proportion of juvenile, young, older and middle-aged worker bees in the bee colony, providing an accurate basis for subsequent regulation.

[0041] Step 5: Extract sound emotion features from the bee colony sounds, extract odor emotion features from the volatile organic compound concentration distribution, extract vibration emotion features from the hive vibration, and extract image emotion features from the hive entrance area image. Input the extracted features into the emotion classification model and output the bee colony's emotional state.

[0042] In some embodiments, inputting the extracted features into the emotion classification model and outputting the swarm's emotion state specifically includes: Extract high-frequency screaming energy from the bee swarm sound. and low-frequency humming energy Calculate the emotional index of the voice As shown in the following formula:

[0043] Extract the response vector of the electronic nose sensor array from the volatile organic compound concentration distribution. and its rate of change vector in adjacent acquisition cycles ; Extract the main frequency of the vibration spectrum from the vibration of the beehive. Dominant frequency amplitude as well as The ratio of frequency band energy to total energy ; Extract bee colony density from the image of the hive entrance area. percentage of worker bees in an aggressive posture and the average wingbeat frequency of worker bees ; Constructing emotional feature vectors Inputting a pre-trained lightweight gradient boosting machine model yields a five-dimensional probability vector. , , , , , and These correspond to the probabilities of being quiet, normal, agitated, swarming, and robber bee alert, respectively. The category corresponding to the highest probability is taken as the bee colony's emotional state.

[0044] Furthermore, the sound signals of the bee colony inside the hive are continuously collected by a wideband microphone, and the time spectrum is obtained by short-time Fourier transform. The energy is integrated in the high-frequency range of 2kHz to 8kHz to obtain the high-frequency screaming energy that characterizes the bee colony's aggression and agitation. The low-frequency buzzing energy, characterizing the quiet and low-activity state of the bee colony, is obtained by integrating the energy in the low-frequency range of 100Hz to 500Hz. Then calculate the voice emotion index. Secondly, using an electronic nose array composed of multiple semiconductor or electrochemical sensors, the concentration of volatile organic compounds at different locations within the beehive is collected once per minute. The combination of the response values ​​of each sensor within the current sampling period is used to construct an odor spatial response vector. The difference between the response vector and the previous cycle is calculated and divided by the cycle interval to obtain the dynamic rate of change vector reflecting the rate of change of odor pheromone concentration. Simultaneously, three-dimensional vibration signals were acquired at a frequency of 500–1000 Hz using piezoelectric accelerometers fixed to the bottom plate and side walls of the beehive. After fast Fourier transform, the dominant frequency corresponding to the maximum amplitude in the spectrum was extracted. and its amplitude And calculate the ratio of energy in the low-frequency band (0.5Hz to 2Hz) to the total energy of the entire frequency band. This ratio reflects the intensity of collective behaviors such as swaying and shaking of the bee colony. For continuous images captured by high-definition industrial cameras in the hive entrance area, YOLOv5 and other object detection networks are used to identify individual worker bees, and DeepSORT and other multi-object tracking algorithms are used for cross-frame correlation. The number of worker bees per unit area within the hive entrance is counted as the aggregation density. Simultaneously, an attack posture such as an upturned abdomen and exposed stinger is detected using a posture classification network, and the proportion of worker bees in an attack posture relative to the total number of worker bees in the current hive entrance area is calculated. Optical flow analysis or frequency domain wing vibration detection was performed on continuous images of the body of each tracked worker bee to determine the average wing vibration frequency of all returning worker bees. After completing the multimodal feature extraction described above, a unified emotion feature vector is constructed. The data is then fed into a pre-trained lightweight gradient boosting machine (LTM) model. The LTM model is trained on samples labeled with five emotion tags: calm, normal, anxious, swarming fever, and robbing bee alert. Based on ensemble learning, the LTM model performs non-linear mapping on high-dimensional heterogeneous features and outputs a five-dimensional probability vector. Finally, the emotion category corresponding to the maximum probability is selected as the output swarm emotion state, thereby achieving accurate swarm emotion identification based on the fusion of multi-dimensional perception of hearing, smell, touch and vision.

[0045] Furthermore, the specific implementation of the lightweight gradient booster model is as follows: Multimodal data samples were pre-collected and labeled with five bee colony emotion tags: quiet, normal, anxious, swarming fever, and robbing bee alert. For each sample, an audio emotion index was simultaneously extracted as described in step five. Electronic nose sensor array response vector and its rate of change vector Vibration spectrum dominant frequency With amplitude and low-frequency energy ratio Nest entrance aggregation density Percentage of worker bees in an aggressive posture and average wingbeat frequency Vector features and After being flattened into multiple scalar features, these features are concatenated with other scalar features to form a unified high-dimensional heterogeneous input feature vector. During training, this feature vector is input into a lightweight gradient booster, which uses gradient boosting decision trees as base learners. Employing a histogram-based feature discretization and leaf node-first splitting strategy, and targeting a multi-class log loss function, multiple regression decision trees are iteratively generated. Each new tree is fitted along the negative gradient direction of the current ensemble model residuals, thus gradually constructing a nonlinear mapping of the discrimination boundaries for the five emotion categories. During inference, the lightweight gradient booster distributes the real-time extracted high-dimensional heterogeneous emotion feature vectors to each decision tree. Each tree routes the feature splitting condition on its internal nodes to the corresponding leaf node, outputting the original predicted scores for the five categories from the leaf node. After summing the scores from all decision trees, the results are normalized using a softmax function, outputting a five-dimensional probability vector with a sum of 1. This enables an integrated nonlinear mapping from multimodal and heterogeneous features to the probability distribution of swarm emotions.

[0046] Step Six: Generate control instructions based on the queen bee's egg-laying status, larval age and developmental health, worker bee age structure, and colony mood; wherein, the control instructions include pheromone release instructions and mechanical control instructions.

[0047] In some embodiments, the regulation instruction generation module receives the queen bee's egg-laying status (vigorous egg-laying, weakened egg-laying, or cessation of egg-laying) from step two, the larval age distribution and the proportion of individuals with delayed development output from step three, the precise proportions of juvenile, young, adult, and old worker bees in the colony after behavioral characteristic calibration output from step four, and the colony's emotional state category and its corresponding five-dimensional probability vector output from step five. This module has a pre-built multi-condition combination regulation rule library constructed based on bee colony biological laws and beekeeping management experience. When the queen bee's egg-laying status is determined to be cessation of egg-laying or weakened egg-laying and the proportion of larvae with delayed development exceeds 10%, a first pheromone release instruction is automatically generated. The instruction includes releasing a queen bee mandibular gland pheromone (QMP) analogue and specifying a base release rate. Simultaneously, the first mechanical control command is generated to set the target temperature of the sub-spleen region under closed-loop control. The temperature was temporarily increased from 34.5℃ to 35.0℃ to promote the development of eggs and larvae; when the proportion of mature worker bees in the age structure of worker bees exceeded 50% and the pollen carrying ratio extracted in step four was higher than 35%, it was determined that the external environment had entered the nectar flow period, and a second mechanical control command was immediately generated to adjust the bee spacing. Set to 12mm and set the nest entrance angle Reaching the maximum To enhance ventilation and facilitate collection; if the bee colony's emotional state is identified as swarming fever and its probability is... Then, a second pheromone release command is generated, specifying the release of a queen pheromone analog to inhibit the construction of swarm queen cells, and a third mechanical control command is simultaneously generated to forcibly adjust the hive entrance angle. Restricted to Bee spacing Set to 8mm to reduce the space inside the hive, encouraging worker bees to secrete wax and build comb, thus stabilizing the colony's mood; when the mood is that robbing bees are alert and the probability is high... At the same time, a third pheromone release command is generated, releasing an alarm pheromone analogue to disrupt the orientation of the robbing bees. Simultaneously, a fourth mechanical control command is generated to adjust the hive entrance opening angle. Reduced to Strengthen hive entrance defenses; in addition, if the queen bee is laying eggs vigorously and the proportion of juvenile and young worker bees in the reproductive period is within the characteristic range, a fifth mechanical control command is generated to maintain the bee flight distance. 8mm and maintain the target temperature of the daughter spleen area The temperature is 34.5℃; all generated pheromone release commands are encapsulated with pheromone type identifiers and base release rates. Number of releases within a pulse cycle Duration of each session and the interval between two adjacent releases The execution parameters and mechanical control commands explicitly include the target value of the nest entrance opening angle. Target value of bee spacing Target temperature of spleen region After the above instructions are assigned a timestamp corresponding to the current decision-making cycle and a priority tag reflecting the urgency of the regulation, they are sent in parallel to the pheromone release execution module and the mechanical regulation execution module, providing a direct basis for subsequent pulse release, mechanical action and closed-loop temperature control.

[0048] Step 7: Execute the pheromone release command to release pheromone analogs in a pulse manner, and dynamically adjust the release rate based on the number of bees estimated from the hive weight and the image of the hive entrance area; execute the mechanical control command to adjust at least one of the hive entrance opening, bee passage spacing, and brood comb temperature.

[0049] In some embodiments, executing the pheromone release instruction specifically includes: Based on the target emotion probability corresponding to the bee colony's emotional state ,when At the following release rates The micropump is controlled as shown in the following formula:

[0050] in, The base release rate, the specific value of which needs to be determined based on the type of pheromone and the volume of the beehive; The number of bee individuals in the colony is estimated from the weight of the hive and the image of the hive entrance area; The baseline number of bee colonies is 30,000 to 50,000, preferably 40,000. For gain coefficient; when hour, ; Employing a pulsed release mode: within each decision cycle (ranging from 5 to 30 minutes, preferably 10 minutes), a pulsed release is applied. rate of release Each time, lasting Seconds, the interval between two consecutive releases seconds; preferably, each duration The value ranges from 0.1 seconds to 5 seconds, and the interval between two consecutive releases is... The value ranges from 10 seconds to 60 seconds.

[0051] Real-time acquisition of total volatile organic compound concentration inside the beehive ,when season Linearly reduced to 50% of the original value; when Immediately stop releasing and send an alarm to the beekeeper's terminal; among them, To set a warning concentration, The concentration is at the absolute safety upper limit, and Alert concentration The value range is 0.5 mg / m³ to 2 mg / m³, with the absolute safe upper limit concentration. The value range is 3 mg / m³ to 10 mg / m³.

[0052] Furthermore, the implementation of estimating the number of bee individuals in the colony from the weight of the beehive and the image of the hive entrance area is as follows: The total weight of the beehive is measured once per minute using a weighing sensor deployed at the bottom of the hive. Simultaneously, from the continuous video stream of the hive entrance area image, a deep object detection network and a multi-object tracking algorithm are used to identify and associate each worker bee entering or leaving the hive entrance frame by frame, record the outgoing and returning events of each worker bee, and accumulate the total number of outgoing worker bees at the current moment. Compared with the total number of returning worker bees The number of worker bees outside the hive is calculated using the following formula:

[0053] The baseline weight of the beehive, combs, and fixing accessories is obtained in advance by weighing and calibrating the empty hive. The weight of non-bee biomass such as honey and pollen stored within the bee colony is used as time-varying reserves. Construct a dynamic model of beehive weight, as shown in the following formula:

[0054] in, The total number of bee individuals in the colony to be estimated. The average body weight of a single worker bee; the selection period was late at night when all worker bees returned to the hive. Furthermore, since there is no window for nectar flow and foraging activity outside the hive, the weight change in the hive at this time is mainly due to the consumption of stored food. The rate of stored food consumption is estimated by linear fitting of the weight data during this period, and the hive weight is updated accordingly. During continuous monitoring, and As the system state variable, the weight model and the entry / exit count constraint are used as the observation equations. A recursive least squares estimator with a forgetting factor is used for online solution, and the estimated number of bee colonies is obtained through recursive updates. Simultaneously, adaptive correction is made based on the weight residual. The drift is calculated to output the number of bee colony individuals estimated by fusing weight and visual information, which is then used for calculations to regulate pheromone release rate, etc.

[0055] In some embodiments, executing the mechanical control command specifically includes: Nest entrance opening adjustment: Define the nest entrance opening angle Its adjustable range is When the bee colony's emotional state is swarming fever, control... When the external environment is in the honey flow period and the bee colony is not exhibiting swarming behavior, control... Control of other situations Maximum opening angle of the nest entrance The value range is 60° to 90°, which is determined by the design stroke of the mechanical baffle at the nest entrance.

[0056] Cellular spacing adjustment: Define the cell spacing When the bee colony is identified as being in its breeding season, control... When the bee colony is identified as being in the honey flow period, control... ; Spleen zone temperature control: Set the target temperature for the spleen zone. Based on the measured temperature of the spleen region in the multi-point temperature data. A closed-loop control strategy is used to drive the heating or cooling unit, so that... Maintain at Within the range.

[0057] Based on the same inventive concept, and corresponding to any of the above embodiments, refer to... Figure 2 This invention provides a dynamic bee colony control system based on multimodal biometric recognition, used to implement the aforementioned dynamic bee colony control method based on multimodal biometric recognition, comprising: The multimodal data acquisition module is used to collect images of the hive entrance area, brood comb area, bee colony sounds, volatile organic compound concentration distribution inside the hive, hive vibration, multi-point temperature data, and hive weight. The queen bee egg-laying status determination module is connected to the multimodal data acquisition module. It receives the brood comb area image and the multi-point temperature data. It is used to locate the queen bee from the brood comb area image, extract the queen bee's abdominal features and movement features, and determine the queen bee's egg-laying status by combining the temperature difference between the area where the queen bee is located and the surrounding area in the multi-point temperature data. The larval development status determination module is connected to the multimodal data acquisition module, receives the image of the brood region, performs instance segmentation on the image of the brood region, extracts the morphological features of the larvae in the nest, and determines the larval age and developmental health status according to the larval development model. The worker bee age structure estimation module is connected to the multimodal data acquisition module. It receives the image of the hive entrance area and extracts the body surface features of returning worker bees from the image of the hive entrance area. Combining the frequency of worker bees entering and leaving the hive and the proportion of pollen carried, it estimates the proportion of worker bees of different ages through cluster analysis and outputs the worker bee age structure. The bee colony emotion recognition module is connected to the multimodal data acquisition module. It receives the bee colony sounds, the volatile organic compound concentration distribution, the beehive vibration, and the hive entrance area image. It is used to extract sound emotion features, odor emotion features, vibration emotion features, and image emotion features. The extracted features are input into the emotion classification model to output the bee colony emotion state. The regulation command generation module is connected to the queen bee egg-laying status determination module, the larvae development status determination module, the worker bee age structure estimation module, and the bee colony emotion recognition module, respectively. It is used to generate regulation commands based on the queen bee egg-laying status, larvae age and developmental health status, worker bee age structure, and bee colony emotion status. The regulation commands include pheromone release commands and mechanical regulation commands. The pheromone release execution module is connected to the control command generation module and the multimodal data acquisition module. It receives the pheromone release command, the weight of the beehive, the image of the hive entrance area, and the concentration distribution of volatile organic compounds. It is used to release pheromone analogs in a pulse manner and dynamically adjust the release rate according to the number of bees estimated from the beehive weight and the image of the hive entrance area, as well as the total concentration of volatile organic compounds. The mechanical control execution module is connected to the control command generation module and the multimodal data acquisition module. It receives the mechanical control command and the multi-point temperature data and is used to perform hive entrance opening adjustment, bee spacing adjustment, and closed-loop control of the brood comb temperature based on the multi-point temperature data.

[0058] Based on the same inventive concept, corresponding to any of the above embodiments, the present invention provides an electronic device, including a memory and a processor. The memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the dynamic bee colony control method based on multimodal biometric recognition of the embodiments.

[0059] Alternatively, the aforementioned electronic device may be a server.

[0060] In addition, this embodiment also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the dynamic bee colony control method based on multimodal biometric recognition of the embodiment.

[0061] It is understood that the processor in the embodiments of the present invention may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0062] The method steps in the embodiments of the present invention can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0063] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of the present invention is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a storage medium or transmitted through a storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive (SSD)).

Claims

1. A dynamic bee colony control method based on multimodal biometric recognition, characterized in that, Includes the following steps: Collect multimodal data of the beehive, including images of the hive entrance area, images of the brood comb area, bee colony sounds, distribution of volatile organic compound concentration inside the beehive, beehive vibration, multi-point temperature data, and beehive weight; The queen bee is located from the image of the brood comb area, and the abdominal and movement features of the queen bee are extracted. Combined with the temperature difference between the area where the queen bee is located and the surrounding area in the multi-point temperature data, the queen bee's egg-laying status is determined. The image of the brood region is segmented into instances to extract the morphological features of the larvae in the nest cells. The age and developmental health status of the larvae are determined based on the larval development model. The body surface features of returning worker bees are extracted from the images of the hive entrance area. Combined with the frequency of worker bees entering and leaving the hive and the proportion of pollen carried, the proportion of worker bees of different ages is estimated through cluster analysis to obtain the age structure of worker bees. Emotional features are extracted from the bee colony sounds, odor emotional features are extracted from the volatile organic compound concentration distribution, vibration emotional features are extracted from the hive vibrations, and image emotional features are extracted from the hive entrance area image. The extracted features are input into an emotion classification model to output the bee colony's emotional state. Based on the queen bee's egg-laying status, the age and health of the larvae, the age structure of the worker bees, and the emotional state of the bee colony, control instructions are generated; wherein, the control instructions include pheromone release instructions and mechanical control instructions; The pheromone release command is executed to release pheromone analogs in a pulse manner, and the release rate is dynamically adjusted based on the number of bees estimated from the hive weight and the image of the hive entrance area; the mechanical control command is executed to adjust at least one of the hive entrance opening, bee passage spacing, and brood comb temperature.

2. The dynamic bee colony control method based on multimodal biometric recognition as described in claim 1, characterized in that, Determining the queen bee's egg-laying status specifically includes: The image of the brood comb region is input into the queen bee localization model to obtain the queen bee bounding box; the abdominal swelling of the queen bee is extracted within the bounding box. Oviposition posture probability and mobile activity The average temperature difference between the queen bee's location and the background area of ​​the brood comb was obtained from the multi-point temperature data. ; Calculate the queen bee's egg-laying status index using the following formula. As shown in the following formula: in, , , and For the corresponding weight coefficients, and ; when and At that time, it is determined that spawning is vigorous; when and If oviposition occurs, oviposition is considered to have stopped; otherwise, oviposition is considered to have weakened. The threshold for peak egg production, The threshold for ovulation arrest, and .

3. The dynamic bee colony control method based on multimodal biometric recognition as described in claim 1, characterized in that, Determining the larval age and developmental health status specifically includes: An instance segmentation network was used to process the sponge region image to segment out each complete nest cell region; the body length of the larva was extracted within each nest cell region. Body color saturation curvature and feeding status indicators ,in This indicates that the cell is being fed by worker bees. This indicates that the food was not fed. The estimated age of larvae was calculated using a multiple linear regression model. As shown in the following formula: in, The intercept is... , , and These are the regression coefficients corresponding to each feature; Will Compared with the preset age range: If Then classify them as 1-3 days old. Then classify them as 4-6 days old. Then classify them as 7-8 days old; When the body length Less than the lower limit of the standard body length for the preset age range, and When the duration exceeds the time threshold, the larvae are considered to be developing slowly.

4. The dynamic bee colony control method based on multimodal biometric recognition as described in claim 1, characterized in that, The estimation of the proportion of worker bees of different ages through cluster analysis specifically includes: For each returning worker bee detected from the image of the hive entrance region, a three-dimensional feature vector is constructed. ,in The average grayscale value of the body color. The wing edge wear index. The density index of body hair on the chest and back; A Gaussian mixture model is used to cluster the feature vectors of all returning worker bees. The probability density function of the Gaussian mixture model is shown in the following equation: in, For the first The mixture weights of Gaussian components, and ; It is the mean vector; It is the covariance matrix; Solving for parameters using the expectation-maximization algorithm , The four cluster centers were assigned to four age ranges: childhood, adolescence, adulthood, and old age. By the mixed weight of each component The worker bee age structure is obtained by estimating the proportion of worker bees in the colony within that age range.

5. The dynamic bee colony control method based on multimodal biometric recognition as described in claim 1, characterized in that, The step of inputting the extracted features into the emotion classification model and outputting the swarm's emotion state specifically includes: Extract high-frequency screaming energy from the bee swarm sound. and low-frequency humming energy Calculate the emotional index of the voice As shown in the following formula: Extract the response vector of the electronic nose sensor array from the volatile organic compound concentration distribution. and its rate of change vector in adjacent acquisition cycles ; Extract the main frequency of the vibration spectrum from the vibration of the beehive. Dominant frequency amplitude as well as The ratio of frequency band energy to total energy ; Extract bee colony density from the image of the hive entrance area. percentage of worker bees in an aggressive posture and the average wingbeat frequency of worker bees ; Constructing emotional feature vectors Inputting a pre-trained lightweight gradient boosting machine model yields a five-dimensional probability vector. , , , , , and These correspond to the probabilities of being quiet, normal, agitated, swarming, and robber bee alert, respectively. The category corresponding to the highest probability is taken as the bee colony's emotional state.

6. The dynamic bee colony control method based on multimodal biometric recognition as described in claim 1, characterized in that, The execution of the pheromone release instruction specifically includes: Based on the target emotion probability corresponding to the bee colony's emotional state ,when At the following release rates The micropump is controlled as shown in the following formula: in, Based on the release rate; The number of bee individuals in the colony is estimated from the weight of the hive and the image of the hive entrance area; The baseline number of bee colonies; For gain coefficient; when hour, ; Employing a pulsed release mode: within each decision cycle, with rate of release Each time, lasting Seconds, the interval between two consecutive releases Second; Real-time acquisition of total volatile organic compound concentration inside the beehive ,when season Linearly reduced to 50% of the original value; when Immediately stop releasing and send an alarm to the beekeeper's terminal; among them, To set a warning concentration, The concentration is at the absolute safety upper limit, and .

7. The dynamic bee colony control method based on multimodal biometric recognition as described in claim 1, characterized in that, The execution of the mechanical control command specifically includes: Nest entrance opening adjustment: Define the nest entrance opening angle Its adjustable range is When the bee colony's emotional state is swarming fever, control... When the external environment is in the honey flow period and the bee colony is not exhibiting swarming behavior, control... Control of other situations ; Cellular spacing adjustment: Define the cell spacing When the bee colony is identified as being in its breeding season, control... When the bee colony is identified as being in the honey flow period, control... ; Spleen zone temperature control: Set the target temperature for the spleen zone. Based on the measured temperature of the spleen region in the multi-point temperature data. A closed-loop control strategy is used to drive the heating or cooling unit, so that... Maintain at Within the range.

8. A dynamic bee colony control system based on multimodal biometric recognition, used to implement the dynamic bee colony control method based on multimodal biometric recognition as described in any one of claims 1-7, characterized in that, include: The multimodal data acquisition module is used to collect images of the hive entrance area, brood comb area, bee colony sounds, volatile organic compound concentration distribution inside the hive, hive vibration, multi-point temperature data, and hive weight. The queen bee egg-laying status determination module is connected to the multimodal data acquisition module. It receives the brood comb area image and the multi-point temperature data. It is used to locate the queen bee from the brood comb area image, extract the queen bee's abdominal features and movement features, and determine the queen bee's egg-laying status by combining the temperature difference between the area where the queen bee is located and the surrounding area in the multi-point temperature data. The larval development status determination module is connected to the multimodal data acquisition module, receives the image of the brood region, performs instance segmentation on the image of the brood region, extracts the morphological features of the larvae in the nest, and determines the larval age and developmental health status according to the larval development model. The worker bee age structure estimation module is connected to the multimodal data acquisition module. It receives the image of the hive entrance area and extracts the body surface features of returning worker bees from the image of the hive entrance area. Combining the frequency of worker bees entering and leaving the hive and the proportion of pollen carried, it estimates the proportion of worker bees of different ages through cluster analysis and outputs the worker bee age structure. The bee colony emotion recognition module is connected to the multimodal data acquisition module. It receives the bee colony sounds, the volatile organic compound concentration distribution, the beehive vibration, and the hive entrance area image. It is used to extract sound emotion features, odor emotion features, vibration emotion features, and image emotion features. The extracted features are input into the emotion classification model to output the bee colony emotion state. The regulation command generation module is connected to the queen bee egg-laying status determination module, the larvae development status determination module, the worker bee age structure estimation module, and the bee colony emotion recognition module, respectively. It is used to generate regulation commands based on the queen bee egg-laying status, larvae age and developmental health status, worker bee age structure, and bee colony emotion status. The regulation commands include pheromone release commands and mechanical regulation commands. The pheromone release execution module is connected to the control command generation module and the multimodal data acquisition module. It receives the pheromone release command, the weight of the beehive, the image of the hive entrance area, and the concentration distribution of volatile organic compounds. It is used to release pheromone analogs in a pulse manner and dynamically adjust the release rate according to the number of bees estimated from the beehive weight and the image of the hive entrance area, as well as the total concentration of volatile organic compounds. The mechanical control execution module is connected to the control command generation module and the multimodal data acquisition module. It receives the mechanical control command and the multi-point temperature data and is used to perform hive entrance opening adjustment, bee spacing adjustment, and closed-loop control of the brood comb temperature based on the multi-point temperature data.

9. An electronic device, characterized in that, The device includes a memory and a processor, the memory being used to store a computer program, and the processor running the computer program to cause the electronic device to perform the dynamic bee colony control method based on multimodal biometric recognition as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the dynamic bee colony control method based on multimodal biometric recognition as described in any one of claims 1-7.

Images