A method and device for predicting bumblebee fighting behavior

Abstract

The present invention discloses a method and device for predicting the fighting behavior of bumblebees. The method comprises: obtaining a training sample set, wherein the training sample set is a plurality of photos of bumblebee colonies marked with label categories; identifying the training samples of the training sample set according to a fighting recognition model, and obtaining a polar coordinate system of the bumblebee outline, wherein the fighting recognition model is a residual convolutional neural network model fused with an attention mechanism in a pyramid structure; determining the training prediction result of the fighting recognition model according to the polar coordinate system, and determining that the fighting recognition model has completed training when the training prediction result reaches a preset target; and identifying bumblebees with fighting behavior according to the fighting recognition model that has completed training. The prediction method uses a primary classification obtained by the fighting recognition model and a secondary classification obtained based on the bumblebee outline to jointly predict the fighting situation of the bumblebees at the location. The prediction method combines the bumblebee picture features and the bumblebee outline features, thereby improving the accuracy of predicting the bumblebee fighting behavior.

Description

Technical Field

[0001] The present application relates to the technical field of insect behavior prediction, and in particular to a method and device for predicting bumblebee fighting behavior. Background Art

[0002] Beekeeping plays an important role in modern agriculture. Not only do bee products have high economic value, bee pollination can also replace human labor to better improve the fruit setting rate of crops. Currently, more than one-third of the world's food production relies on bee pollination. Bees have established a strict and complete labor division system in the long-term evolution process. Bumblebees are a type of bee. Related scientific research has found that when a bumblebee colony frequently shows biological behaviors of fighting between bumblebees, it indicates that the social structure of the colony may have shown signs of collapse, which will cause a large number of bumblebees to leave the current hive, greatly affecting the economic benefits of the bee industry. If this series of bumblebee biological behaviors can be detected in a timely and effective manner, it will not only help beekeepers increase the output of the bee industry, but also help to study the deeper social structure relationship of the hive.

[0003] Therefore, how to accurately predict the fighting behavior of bumblebees is a technical problem that needs to be solved urgently. Summary of the invention

[0004] The present invention provides a method and device for predicting bumblebee fighting behavior, which can accurately predict bumblebee fighting behavior.

[0005] The embodiment of the present invention provides the following solution:

[0006] In a first aspect, an embodiment of the present invention provides a method for predicting bumblebee fighting behavior, the method comprising:

[0007] Obtaining a training sample set, wherein the training sample set is a plurality of bumblebee swarm photos annotated with label categories;

[0008] Identify the training samples of the training sample set according to the fight recognition model to obtain the polar coordinate system of the bumblebee outline, wherein the fight recognition model is a residual convolutional neural network model integrated with an attention mechanism in a pyramid structure;

[0009] Determining a training prediction result of the fight recognition model according to the polar coordinate system, and determining that the fight recognition model has completed training when the training prediction result reaches a preset target;

[0010] Bumblebees with fighting behaviors are identified based on the trained fighting identification model.

[0011] In an optional embodiment, obtaining a training sample set includes:

[0012] Use different types of night vision cameras in low-light environments to capture bumblebee swarms at different time periods to obtain activity videos;

[0013] Splitting the activity video according to a preset number of frames to obtain activity photos of the bumblebee swarm;

[0014] The bumblebee swarm activity photos are screened and the label categories are correspondingly marked to obtain the training sample set.

[0015] In an optional embodiment, after screening the bumblebee swarm activity photos and labeling the label categories accordingly to obtain the training sample set, the method further includes:

[0016] The bumblebee colony activity photos are processed according to a quantity enhancement algorithm and a quality enhancement algorithm to update the training sample set.

[0017] In an optional embodiment, the step of identifying the training samples of the training sample set according to the fighting recognition model to obtain the polar coordinate system of the bumblebee outline includes:

[0018] Processing the training samples according to the fighting recognition model to obtain a plurality of regression coordinate points mapped to the bumblebee outline;

[0019] Determining a central coordinate point among the multiple regression coordinate points;

[0020] The polar coordinate system is obtained according to a preset number of line segments from the central coordinate point to the bumblebee outline.

[0021] In an optional embodiment, the processing of the training sample according to the fighting recognition model to obtain a plurality of regression coordinate points mapped to the bumblebee outline includes:

[0022] Input the training sample into the fight recognition model to perform convolution calculation to obtain a feature map of a preset number of layers;

[0023] Mapping each feature point on the feature map to the training sample;

[0024] All the feature points in the bumblebee outline marked by the training sample are determined as the multiple regression coordinate points.

[0025] In an optional embodiment, determining a central coordinate point from the multiple regression coordinate points includes:

[0026] According to a preset number of radiation points from each regression coordinate point to the bumblebee outline, the preset number of radiation distances are correspondingly obtained, wherein each regression coordinate point is any one of the multiple regression coordinate points;

[0027] Obtaining a regression result of a regression coordinate point according to the product of the preset number of radiation distances;

[0028] According to the maximum value among all the regression results, the corresponding regression coordinate point is determined as the central coordinate point.

[0029] In an optional embodiment, determining the training prediction result of the fight recognition model according to the polar coordinate system includes:

[0030] The training prediction result is determined according to the regression information of the polar coordinate system and the binary cross entropy loss function of the fight recognition model.

[0031] In a second aspect, an embodiment of the present invention further provides a device for predicting bumblebee fighting behavior, the device comprising:

[0032] An acquisition module, used to acquire a training sample set, wherein the training sample set is a plurality of bumblebee swarm photos annotated with label categories;

[0033] An acquisition module is used to identify the training samples of the training sample set according to the fight recognition model to obtain the polar coordinate system of the bumblebee outline, wherein the fight recognition model is a residual convolutional neural network model integrated with an attention mechanism in a pyramid structure;

[0034] A determination module, used to determine the training prediction result of the fight recognition model according to the polar coordinate system, and determine that the training of the fight recognition model is completed when the training prediction result reaches a preset target;

[0035] The identification module is used to identify bumblebees with fighting behavior based on the trained fighting identification model.

[0036] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a processor and a memory, wherein the memory is coupled to the processor, and the memory stores instructions, which, when executed by the processor, enable the electronic device to perform the steps of any one of the methods described in the first aspect.

[0037] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the methods described in the first aspect.

[0038] Compared with the prior art, the method and device for predicting bumblebee fighting behavior provided by the present invention have the following advantages:

[0039] The present invention obtains a plurality of photos of bumblebee colonies marked with label categories as a training sample set, identifies the training samples of the training sample set according to a fighting recognition model, classifies the background, fighting bumblebees and non-fighting bumblebees of the training samples, obtains a polar coordinate system of the bumblebee contour, determines the training prediction result of the fighting recognition model according to the polar coordinate system, establishes a polar coordinate system in the bumblebee contour for calculation, and uses a convolutional layer to extract regression information again for secondary classification, so as to make the training and recognition of the fighting recognition model more accurate. The principle is to use the primary classification obtained by the fighting recognition model and the secondary classification obtained based on the bumblebee contour to jointly predict the fighting situation of the bumblebees at the position. The prediction method combines the bumblebee picture features and the bumblebee contour features, thereby improving the accuracy of the prediction of the bumblebee fighting behavior. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 A flowchart of a method for predicting bumblebee fighting behavior provided by an embodiment of the present invention;

[0042] Figure 2 A schematic diagram of a bumblebee colony provided in an embodiment of the present invention;

[0043] Figure 3 A schematic diagram of the structure of the ResNet-18 backbone network provided in an embodiment of the present invention;

[0044] Figure 4 A schematic diagram of the structure of a BasicBlock module provided in an embodiment of the present invention;

[0045] Figure 5 A flow chart of extracting a characteristic graph provided by an embodiment of the present invention;

[0046] Figure 6 A schematic diagram of the structure of a polar coordinate system provided by an embodiment of the present invention;

[0047] Figure 7 A schematic diagram of a central coordinate point and a bumblebee outline provided by an embodiment of the present invention;

[0048] Figure 8 A training flow chart of a fight recognition model provided by an embodiment of the present invention;

[0049] Fig. 9A schematic diagram of the structure of a device for predicting bumblebee fighting behavior provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention belong to the scope of protection of the embodiments of the present invention.

[0051] With the continuous development of artificial intelligence technology, especially the introduction of deep neural networks, the research and development of computer vision technology has been greatly promoted. Computer vision technology is widely used in all walks of life, and has realized functions such as visual measurement, event monitoring, target detection, target tracking, etc. Among them, the target detection technology has been widely used in the identification and tracking of various animals, biological behavior analysis, etc., which greatly reduces the workload of researchers in related fields. The existing processing method is to detect the overall convolution model under a white background, and the generated anchor frame can roughly detect the position of the bee. Its implementation method is based on the one-stage SSD model for the problem of positive and negative anchor ratio imbalance, and uses a difficult sample mining algorithm, which requires a large amount of calculation to be invested in the model, limiting the system operation performance; the design of the overall model is limited to discovering and roughly marking the position of the bee, which is insufficient in the detection accuracy of the bee, and has limitations on in-depth research on the complex biological habits of the bee, so it can only perform a rough identification of the number of organisms. The prediction method of the present invention can accurately predict the fighting behavior of bumblebees, and the following will specifically explain how to implement the technical solution of the present invention.

[0052] See also Figure 1 , Figure 1 A flowchart of a method for predicting bumblebee fighting behavior provided by an embodiment of the present invention, the method comprising:

[0053] S11. Obtain a training sample set, wherein the training sample set is a plurality of photos of bumblebee colonies annotated with label categories.

[0054] Specifically, the training sample set can be obtained by screening existing bumblebee colony photos; or by taking photos in the bumblebee hive, with each photo corresponding to a label category. Figure 2 ,The label categories include three states: background 0, non-fighting state 1 and fighting state 2.

[0055] In specific implementation, since the training sample set obtained by conventional methods may have a relatively simple morphology of bumblebees, there are differences in the quality of each training sample in the training sample set.

[0056] To solve the above problem, in a specific implementation, obtaining a training sample set includes:

[0057] In a weak-light environment, different types of night-vision cameras are used to shoot bumblebee swarms at different time periods to obtain activity videos; the activity videos are segmented according to a preset number of frames to obtain activity photos of the bumblebee swarm; the activity photos of the bumblebee swarm are screened and labeled with corresponding categories to obtain a training sample set.

[0058] Specifically, bumblebees are more active in biological behavior under weak light, which makes it easier to shoot and observe their behavior, and the applicable scenarios for post-processing are more adaptable. In a beehive in a closed weak light environment, bumblebee culture dishes can be randomly placed in different numbers and positions, and bumblebees in the hive can be randomly captured. Different types of night vision cameras can be used to shoot videos of bumblebee biological activities in multiple culture dishes at different time periods. The collected video data can be uploaded to the server via FTP (File Transfer Protocol). The openCV library can be used to cut the video according to the preset number of frames, and the obtained data images can be screened and annotated. The biological activity status of each bumblebee in the effective cut images and annotated images can be retained to obtain a training sample set.

[0059] It is understandable that in actual situations the source of data may be recorded by various types of shooting equipment and using different types of cameras. Using multiple types of cameras to shoot during data collection can diversify the data and ensure that the model has better stability when facing more complex and diverse scenes.

[0060] In a specific implementation, after screening the bumblebee swarm activity photos and labeling the corresponding label categories to obtain the training sample set, the following steps are also included:

[0061] The photos of bumblebee colony activities are processed according to the quantity enhancement algorithm and the quality enhancement algorithm, and the training sample set is updated.

[0062] Specifically, the quantity enhancement algorithm is used to obtain more photos through photos of bumblebee swarm activities. The GridMask algorithm can be used to increase the amount of data and generate a mask (or mask) with the same resolution as the original image. The mask contains noise boxes with a grayscale value of 0 that are evenly distributed. Multiplying the mask with the original image can produce a new image, i.e., a new photo of bumblebee swarm activities, to achieve the loss of image information in a specific area. The quality enhancement algorithm is used to enhance the data quality of the labeled bumblebee swarm activities based on color and contrast. The Retinex algorithm can be used to achieve image color enhancement. First, the input image P and the ambient light illumination component L are calculated according to log(R)=log(P)-log(L) to obtain the reflection component R of the object; then log(R) is converted according to The color-enhanced image is obtained. By processing the bumblebee swarm activity photos through the quantity enhancement algorithm and the quality enhancement algorithm, more quantity and better quality photos can be obtained to update the training sample set. After obtaining the training sample set, step S12 is entered.

[0063] S12. Identify the training samples of the training sample set according to the fight recognition model to obtain the polar coordinate system of the bumblebee contour, wherein the fight recognition model is a residual convolutional neural network model with an attention mechanism integrated in a pyramid structure.

[0064] Specifically, the polar coordinate system is a coordinate system mapped to the center position of the bumblebee outline, and the mapping point in the bumblebee outline can be obtained through the fight recognition model. The fight recognition model uses a one-stage convolutional neural network model based on the ResNet-18 backbone network to extract the features of the bumblebees in the picture. Of course, it can also be a backbone network with other structures. The feature pyramid (FPN) plus the attention mechanism (Attention) structure is used to improve the matching accuracy of samples in feature maps at different levels, and various optimization algorithms are added to improve the overall performance of the model.

[0065] Take ResNet-18 as the backbone network as an example, see Figure 3 The backbone network includes 8 BasicBlock modules. The BasicBlock module is a combination of a series of computing operations, used for convolution, regression (BN layer) and activation (ReLU layer) calculations. The specific structure of the BasicBlock module is shown in Figure 4 As shown in the figure, each module includes 2 3x3 convolution calculations, 2 BN calculations and 1 ReLU calculation, and finally the convolution result is residually added to the input content. The model as a whole has 17 convolutional layers and 1 fully connected layer. The feature map output after calculation of each convolutional layer (or layer) is added with the attention mechanism to realize the structure of feature pyramid plus attention mechanism.

[0066] Because the volume of training data is small, the use of ResNet-18 backbone network can alleviate the overfitting of the model, and the relatively small amount of calculation also improves the prediction efficiency of the model. The BasicBlock module obtains the residual part after convolution, BN and ReLU calculation, and adds the residual to the direct mapping part, which can solve the problem of gradient vanishing caused by a large number of derivative calculations in the back propagation of complex neural networks, making the fighting recognition model easier to converge, supporting deeper network layers, and improving model accuracy. The judgment of positive and negative samples in the model is based on the feature maps of different levels of the pyramid structure. Among them, when the pixel point (x, y) at a certain position in the feature map falls into the corresponding correct label category (or ground truth), it is considered to be a positive sample, otherwise it is a negative sample. The positive sample is the sample marked as 1 in the training stage, that is, the sample that can bring a positive fitting effect to the model. The negative sample is the sample marked as 0 (which can also be understood as the background part), which cannot bring a positive fitting effect to the model training. Therefore, a loss function (or FocalLoss) is used to reduce the weight of a large number of negative samples in training, so that they have a smaller impact on the convergence of the model. For positive samples, the boundary-to-center distance value is regressed from multiple angles with the pixel point (x, y) as the center during the regression calculation process, completely covering the overall contour of the target. Faced with a large number of negative samples, the model uses the Focal loss loss function to reduce the weight of a large number of simple negative samples in training, and uses Centerness loss to give higher weights to positive samples that are more in line with the characteristics of the target contour (see the formula below). Compared with traditional methods, the combination of the above two methods greatly reduces the amount of model calculation, and the addition of the attention mechanism also helps the model locate high-conformity pixels that fall into reality (or groundtruth) faster and more accurately.

[0067] In a specific implementation, the polar coordinate system of the bumblebee outline is obtained according to the training samples of the training sample set identified by the fighting recognition model, including:

[0068] The training samples are processed according to the fighting recognition model to obtain a plurality of regression coordinate points mapped to the bumblebee contour; a central coordinate point is determined among the plurality of regression coordinate points; and a polar coordinate system is obtained according to a preset number of line segments from the central coordinate point to the bumblebee contour.

[0069] For details, please refer to Figure 5, based on the model concept of FCOS (Fully Convolutional One-Stage Object Detection) algorithm, each point on the feature map is mapped on the original image. When a point falls within the marked mask, it is determined as a positive sample, otherwise it is a negative sample. Therefore, multiple regression coordinate points mapped to the bumblebee outline can be obtained. Here for Figure 2 There are three states in the film: 0, 1, and 2. We balance positive and negative samples in the FocalLoss loss function and reduce the weight of negative samples to improve the convergence efficiency of the model. It can be foreseen that there will be a lot of 0 samples here because there are multiple feature maps. The center coordinate point is the mapping point closest to the center of the bumblebee outline among all the mapping points. The polar coordinate system can be obtained by a preset number of line segments from the center coordinate point to the bumblebee outline. It should be noted that the preset number can be a set value, which can be referred to Figure 6 According to the marked bumblebee outline, we take 18 points on the outline according to a 20° distribution. The algorithm here is to take a point every 20°. If it does not exist, take the mean coordinates of two nearby points, and then calculate the center coordinate point.

[0070] In a specific implementation, the training samples are processed according to the fighting recognition model to obtain a plurality of regression coordinate points mapped to the bumblebee outline, including:

[0071] Input the training sample to the fight recognition model for convolution calculation to obtain a feature map with a preset number of layers; map each feature point on the feature map to the training sample; and determine multiple regression coordinate points based on all feature points in the bumblebee outline marked by the training sample.

[0072] Specifically, take the backbone network of ResNet-34 as an example.

[0073] Step S21, extract the 5-layer feature map through the backbone network ResNet-34, calculate the feature matrix of size C×W×H in each layer using a 3x3 convolution kernel with C channels, then average each feature map in the C channels, and finally calculate through the sigmoid function to obtain a weight vector of size C×1, multiply it with the original input feature matrix, and obtain a feature matrix based on the channel dimension attention mechanism.

[0074] Step S22: For the feature matrix of C×W×H size based on the channel dimension attention mechanism obtained in S21, use D×1x1 convolution to reduce the dimension, and then calculate with the 3x3 convolution kernel with a channel of D / 2, and finally calculate the average of W×H positions in the D / 2 feature maps. Finally, after calculation with the sigmoid function, a weight map of W×H size is obtained, which is multiplied with the original input feature matrix based on the channel dimension attention mechanism to obtain the final feature matrix based on the attention mechanism.

[0075] Step S23, output the feature maps of each layer with the attention mechanism output by S22 using the FPN structure. For each point on the feature map, map it back to the original image through formula 1. If the mapping coordinates are within the annotated bumblebee outline, it is considered to be a true sample point. Starting from 0°, regress a line segment every 20° to obtain 18 predicted value line segment lengths. The H×W×18 matrix obtained by regression is calculated with the convolution layer with 18, 9, and 1 channels and a convolution kernel size of 3×3 to obtain an output matrix of H×W×1, indicating that the classification prediction is obtained based on the regression information. Because only the positive sample points are regressed, that is, only the two categories of fighting / non-fighting are classified, the output value of the secondary classification is between 0 and 1.

[0076] Formula 1 is: Where (x, y) represents the coordinates of the mapping point, and s represents the ratio of the feature map of this layer to the original image.

[0077] Step S24, for the positive sample point described in S23, calculate the distance from the point to the edge of the annotated bumblebee contour, and also start from 0° for the point, calculate a line segment every 20°, and obtain 18 annotated value line segment lengths. Perform smooth1 regression calculation on the predicted line segment in S23 and the annotated line segment described in this step to complete the regression training of the model. Use the classification information obtained according to the regression information in S23 and the actual classification of the target to perform a binary cross entropy loss function to complete the secondary classification calculation.

[0078] Step S25, for the centralness regression model of model regression, according to formula 2, calculate the centralness of the marked line segment in s2.4 and the centralness of the predicted line segment in s2.4, use the binary cross entropy loss function, let the centralness of the predicted line segment fit the centralness of the marked line segment, so as to select the projection point of the regression calculation to be closer to the true center coordinate of the bumblebee, and make the 18 predicted line segments obtained by model regression more evenly distributed, thereby improving the regression performance of the model.

[0079]

[0080] Formula 2: Among them, the distance from the contour coordinates of the data to the mapping point can be calculated d p d q express The shortest and longest line segments in the figure. The subscripts p and q indicate the number of the line segment starting from 0°. and Indicates that p d qThe lengths of two line segments that differ by 180°.

[0081] In a specific implementation, determining a central coordinate point from a plurality of regression coordinate points includes:

[0082] According to the preset number of radiation points from each regression coordinate point to the bumblebee outline, a preset number of radiation distances are obtained, and each regression coordinate point is any regression coordinate point among the multiple regression coordinate points; according to the product of the preset number of radiation distances, the regression result of the regression coordinate point is obtained; according to the maximum value of all regression results, the corresponding regression coordinate point is determined as the center coordinate point.

[0083] Specifically, Figure 7 Take the case of the two mappings of white and black in the figure as an example. These two points are assumed to be the feature maps of 256×256 projected onto the original image and the landing points of the labeled data. The five radiation distances from the white point to the bumblebee outline are quite different, while the radiation distances from the black point to the bumblebee outline are relatively different. The product of the radiation distances in the two cases is different, and the product of the radiation distances of the black point is relatively large. The corresponding regression coordinate point can be determined as the central coordinate point, and then the polar coordinate system is obtained. After obtaining the polar coordinate system, step S13 is entered.

[0084] S13, determining a training prediction result of the fight recognition model according to the polar coordinate system, and determining that the fight recognition model has completed training when the training prediction result reaches a preset target.

[0085] Specifically, the training prediction result is the bumblebee classification information, that is, the fighting or non-fighting state of the bumblebees at the target position on the picture, as well as the confidence level, are directly output; for the bumblebee regression information obtained by the detection model, a secondary classification calculation is performed to obtain a confidence level. When the bumblebee is classified as fighting, the higher the confidence level, the higher the probability that the target is classified as a fighting category based on the regression information; conversely, the lower the confidence level, the lower the probability that the target is classified as a fighting category based on the regression information. When used in actual scenarios, for the judgment of the bumblebee fighting behavior at a certain location, the classification confidence level is multiplied by the regression confidence level to obtain the final prediction confidence level, thereby improving the judgment of the bumblebee biological behavior.

[0086] The preset target can be determined according to the actual training situation. Figure 8, all bumblebee pictures can be divided into training set, validation set and test set in a ratio of 7:2:1. After loading the constructed network model, weights and parameters are initialized, the training set is put into the model for training, and forward and backward propagation calculations are performed according to the calculation method of the loss function. Among them, i represents the current number of training times, which is initially 1, epoch represents the maximum number of training times set, i%10 means that after each ten rounds of model training with the training set data, the validation set is used for verification, and the training stops when i==epoch.

[0087] In a specific implementation, determining the training prediction result of the fight recognition model according to the polar coordinate system includes:

[0088] The training prediction results are determined based on the regression information of the polar coordinate system and the binary cross entropy loss function of the fight recognition model.

[0089] Specifically, the regression information can be the product result of a preset number of line segments in the polar coordinate system. The classification obtained using the regression information and the real classification of the target are used to complete the secondary classification calculation. Through the back propagation calculation of the binary cross entropy loss function, the fight recognition model can be corrected, the convergence degree of the model can be continuously improved, and the overfitting of the model can be reduced to accurately determine the training prediction result. After determining the training prediction result, step S14 is entered.

[0090] S14. Identify bumblebees with fighting behaviors according to the trained fighting identification model.

[0091] Specifically, there are certain differences between the contours of bumblebees in fighting behavior and non-fighting behavior. The contours of bumblebees in non-fighting behavior are relatively regular, usually an elliptical structure; the contours of bumblebees in fighting behavior are usually a special-shaped structure. When performing secondary classification, the fighting behavior of bumblebees can be accurately identified through the regression information of the polar coordinate system. It can be understood that since the prediction method of the present invention abandons the traditional anchor frame and positive and negative sample screening mechanism, it can accurately identify bumblebees in the video under the premise of high efficiency, determine which bumblebees have engaged in fighting behavior, and has strong robustness, and can perform high-accuracy identification of bumblebee activity videos in various complex shooting scenes.

[0092] Based on the same inventive concept as the prediction method, the embodiment of the present invention also provides a prediction device for bumblebee fighting behavior, see Fig. 9 , the device comprises:

[0093] An acquisition module 901 is used to acquire a training sample set, wherein the training sample set is a plurality of bumblebee colony photos marked with label categories;

[0094] An acquisition module 902 is used to identify the training samples of the training sample set according to the fight recognition model to obtain the polar coordinate system of the bumblebee outline, wherein the fight recognition model is a residual convolutional neural network model integrated with an attention mechanism in a pyramid structure;

[0095] A determination module 903 is used to determine the training prediction result of the fight recognition model according to the polar coordinate system, and when the training prediction result reaches a preset target, determine that the fight recognition model has completed training;

[0096] The identification module 904 is used to identify bumblebees with fighting behaviors according to the trained fighting identification model.

[0097] It should be noted that there is a one-to-one correspondence between the prediction device and the prediction method, and the device also includes a specific structure corresponding to the method, which will not be repeated here.

[0098] Based on the same inventive concept as the prediction method, an embodiment of the present invention also provides an electronic device, including a processor and a memory, wherein the memory is coupled to the processor, and the memory stores instructions, which, when executed by the processor, enable the electronic device to perform the steps of any one of the methods described in the prediction method.

[0099] Based on the same inventive concept as the prediction method, an embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of any one of the prediction methods are implemented.

[0100] The technical solution provided in the embodiments of the present invention has at least the following technical effects or advantages:

[0101] By obtaining multiple photos of bumblebee swarms annotated with label categories as a training sample set, the training samples of the training sample set are identified according to the fighting recognition model, the background, fighting bumblebees and non-fighting bumblebees of the training samples are classified once, and the polar coordinate system of the bumblebee outline is obtained. The training prediction result of the fighting recognition model is determined according to the polar coordinate system, and the polar coordinate system is established in the bumblebee outline for calculation. The convolution layer can be used to extract the regression information again for secondary classification, which can make the training and recognition of the fighting recognition model more accurate. The principle is to use the primary classification obtained by the fighting recognition model and the secondary classification based on the bumblebee outline to jointly predict the fighting situation of the bumblebees at that location. This prediction method combines the bumblebee picture features and the bumblebee outline features, thereby improving the accuracy of predicting the bumblebee fighting behavior.

[0102] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (modules, systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0103] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0104] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0105] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0106] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for predicting bumblebee fighting behavior, characterized in that: The method comprises: Obtaining a training sample set, wherein the training sample set is a plurality of bumblebee swarm photos annotated with label categories; Identify the training samples of the training sample set according to the fight recognition model to obtain the polar coordinate system of the bumblebee outline, wherein the fight recognition model is a residual convolutional neural network model integrated with an attention mechanism in a pyramid structure; Determining a training prediction result of the fight recognition model according to the polar coordinate system, and determining that the fight recognition model has completed training when the training prediction result reaches a preset target; Identify bumblebees that have fighting behaviors according to the trained fighting identification model; The step of identifying the training samples of the training sample set according to the fighting recognition model to obtain the polar coordinate system of the bumblebee outline includes: Processing the training samples according to the fighting recognition model to obtain a plurality of regression coordinate points mapped to the bumblebee outline; Determining a central coordinate point among the multiple regression coordinate points; Obtaining the polar coordinate system according to a preset number of line segments from the central coordinate point to the bumblebee outline; The step of processing the training sample according to the fighting recognition model to obtain a plurality of regression coordinate points mapped to the bumblebee outline includes: Input the training sample into the fight recognition model to perform convolution calculation to obtain a feature map of a preset number of layers; Mapping each feature point on the feature map to the training sample; Determine all the feature points in the bumblebee outline marked by the training sample as the multiple regression coordinate points; Determining a central coordinate point from the plurality of regression coordinate points includes: According to a preset number of radiation points from each regression coordinate point to the bumblebee outline, the preset number of radiation distances are correspondingly obtained, wherein each regression coordinate point is any one of the multiple regression coordinate points; Obtaining a regression result of a regression coordinate point according to the product of the preset number of radiation distances; According to the maximum value among all the regression results, the corresponding regression coordinate point is determined as the central coordinate point.

2. The method for predicting bumblebee fighting behavior according to claim 1, characterized in that: The step of obtaining a training sample set includes: Use different types of night vision cameras in low-light environments to capture bumblebee swarms at different time periods to obtain activity videos; Splitting the activity video according to a preset number of frames to obtain activity photos of the bumblebee swarm; The bumblebee swarm activity photos are screened and the label categories are correspondingly marked to obtain the training sample set.

3. The method for predicting bumblebee fighting behavior according to claim 2, characterized in that: After screening the bumblebee swarm activity photos and labeling the label categories accordingly to obtain the training sample set, the method further includes: The bumblebee colony activity photos are processed according to a quantity enhancement algorithm and a quality enhancement algorithm to update the training sample set.

4. The method for predicting bumblebee fighting behavior according to claim 1, characterized in that: Determining the training prediction result of the fight recognition model according to the polar coordinate system includes: The training prediction result is determined according to the regression information of the polar coordinate system and the binary cross entropy loss function of the fight recognition model.

5. A prediction device for bumblebee fighting behavior, the prediction device is used to implement the steps of the method as claimed in any one of claims 1 to 4, characterized in that: The device comprises: An acquisition module, used to acquire a training sample set, wherein the training sample set is a plurality of bumblebee swarm photos annotated with label categories; An acquisition module is used to identify the training samples of the training sample set according to the fight recognition model to obtain the polar coordinate system of the bumblebee outline, wherein the fight recognition model is a residual convolutional neural network model integrated with an attention mechanism in a pyramid structure; A determination module, used to determine the training prediction result of the fight recognition model according to the polar coordinate system, and determine that the training of the fight recognition model is completed when the training prediction result reaches a preset target; The identification module is used to identify bumblebees with fighting behavior based on the trained fighting identification model.

6. An electronic device, characterized in that: The electronic device comprises a processor and a memory, wherein the memory is coupled to the processor and stores instructions, and when the instructions are executed by the processor, the electronic device executes the steps of the method according to any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 4 are implemented.

Images