Bird re-identification method and device, electronic equipment and computer storage medium
Through the combination of the second-stage recognition method and the neural network model, the problem of low bird recognition accuracy is solved, efficient and accurate bird recognition is achieved, the model's sensitivity to images and parameters are reduced, and the recognition efficiency is improved. Accuracy.
Patent Information
- Application Number
- CN202310975844.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-08-03
- Publication Date
- 2024-11-26
AI Technical Summary
The recognition accuracy of existing bird recognition technology is low, making it difficult to distinguish the detailed characteristics of birds, and the recognition model has a high sensitivity to images and is prone to overfitting, resulting in low recognition efficiency and accuracy.
Using the two-stage recognition method, bird images are processed through preprocessing and data augmentation technology, combined with convolutional layer, attention mechanism and full connection layer, a neural network model with inverse residual structure is constructed, and fine-grained recognition is performed, and the In model training, cross-entropy loss function and triple loss function are used to reduce the number of model parameters and improve generalization ability.
Improve the accuracy and efficiency of bird recognition, reduce the sensitivity of the model to images, reduce the amount of parameters, improve the speed of reasoning, save computing resources, realize fine-grained recognition of birds, and enhance the model scalability.
Smart Images

Figure CN119027975A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence technology, and in particular to a bird re-identification method, device, electronic device and computer storage medium. Background Art
[0002] Birds are a common target in natural scenes. By observing, detecting and classifying birds, we can improve the understanding and protection of birds in the fields of environmental monitoring, wildlife protection, bird observation and ecological research.
[0003] However, in the process of classifying and identifying birds, there is usually only one stage of classification technology, which makes it difficult to effectively distinguish the detailed characteristics of bird species, and the number of birds that can be classified is often limited by the size of the data set used during training. In addition, birds also have species characteristics with large differences between classes and small differences within classes, and it is difficult for non-professionals to distinguish between individuals of different species. Therefore, it can be seen that the current bird recognition technology has the problem of low recognition accuracy. Summary of the invention
[0004] The embodiments of the present application provide a bird re-identification method, device, electronic device and computer storage medium, which can solve the current technical problem of low bird identification accuracy.
[0005] The present application embodiment provides a bird re-identification method, comprising:
[0006] Acquire an original bird image, and preprocess the original bird image to obtain a preprocessed image;
[0007] Detecting whether there are birds in the preprocessed image, and when detecting that there are birds in the preprocessed image, performing feature extraction on the preprocessed image to obtain a basic feature map, acquiring a preset two-stage recognition model, and performing fine-grained recognition on the basic feature map based on the two-stage recognition model to obtain bird information in the original bird image;
[0008] When non-bird animals are detected in the pre-processed image, the source bird watching device of the original bird image is obtained, and the non-bird animals in the field of view are driven away based on the source bird watching device.
[0009] Furthermore, the preprocessing of the original bird image to obtain the preprocessed image includes:
[0010] Performing data enhancement processing on the original bird image to obtain a data enhanced image;
[0011] A normalization range is obtained, and the data augmented image is normalized according to the normalization range to obtain the preprocessed image.
[0012] Furthermore, performing data enhancement processing on the original bird image to obtain a data enhanced image includes:
[0013] Get the preset adjustment parameters;
[0014] The original bird image is subjected to contrast enhancement, horizontal flipping, upside down flipping, image rotation, image translation and Gaussian noise processing according to the adjustment parameters to obtain the data enhanced image.
[0015] Furthermore, the two-stage recognition model includes a first convolutional layer, a second convolutional layer, an attention calculation layer, a third convolutional layer and a fully connected layer, and the fine-grained recognition of the basic feature map based on the two-stage recognition model to obtain the bird information in the original bird image includes:
[0016] Performing convolutional dimension-raising calculation on the basic feature map based on the first convolutional layer to obtain a dimension-raising feature map;
[0017] Performing channel-by-channel convolution calculation on the dimension-increased feature map according to the second convolution layer to obtain a first convolution feature;
[0018] Based on the attention calculation layer, extracting features from the first convolutional features through an attention mechanism to obtain attention features;
[0019] Performing dimension reduction and fusion calculation on the attention feature according to the third convolutional layer to obtain an attention feature map;
[0020] The attention feature map is mapped to category labels through the fully connected layer to obtain bird information in the original bird image.
[0021] Furthermore, based on the attention calculation layer, feature extraction is performed on the first convolution feature through an attention mechanism to obtain an attention feature, including:
[0022] Obtain a preset convolution kernel, and perform convolution calculation on the first convolution feature and the convolution kernel according to the attention calculation layer to obtain a second convolution feature;
[0023] Performing global average pooling on the channels of the dimension-increased feature map according to the second convolutional features to obtain a global information statistic;
[0024] Obtain a preset activation function, and calculate the global information statistic based on the activation function to obtain a channel weight;
[0025] The attention feature is obtained by calculating according to the channel weight and the second convolution feature.
[0026] Furthermore, the attention feature is reduced in dimension and fused according to the third convolutional layer to obtain an attention feature map, including:
[0027] Performing point-by-point convolution calculation on the attention feature according to the third convolution layer to obtain a convolution feature map;
[0028] The convolution feature map and the basic feature map are added element by element to obtain the attention feature map.
[0029] Furthermore, before obtaining the preset two-stage recognition model, the method includes:
[0030] Build a basic recognition model and obtain pre-trained bird sample data;
[0031] Inputting the pre-trained bird sample data into the basic recognition model, training the basic recognition model according to the pre-trained bird sample data, and obtaining a predicted probability value of a predicted category corresponding to the pre-trained bird sample data;
[0032] Obtaining a cross entropy loss function, and calculating a loss value of the predicted probability value according to the cross entropy loss function;
[0033] The basic recognition model is adjusted according to the loss value. When the loss value is less than or equal to a preset first threshold, a triple of the pre-trained bird sample data is randomly generated. According to a preset triple loss function, the ternary loss value of the triple is calculated. When the ternary loss value is within a preset second threshold range, it is determined that the training of the basic recognition model is completed, and the two-stage recognition model is obtained.
[0034] Accordingly, an embodiment of the present application provides a bird re-identification device, comprising:
[0035] A preprocessing module, used for acquiring an original bird image, and preprocessing the original bird image to obtain a preprocessed image;
[0036] A detection module, used to detect whether there are birds in the preprocessed image, and when detecting the presence of birds in the preprocessed image, extracting features from the preprocessed image to obtain a basic feature map, acquiring a preset two-stage recognition model, and performing fine-grained recognition on the basic feature map based on the two-stage recognition model to obtain bird information in the original bird image;
[0037] The expelling module is used to obtain the source bird watching device of the original bird image when detecting the presence of non-bird animals in the pre-processed image, and to expel the non-bird animals in the field of view based on the source bird watching device.
[0038] In addition, an embodiment of the present application also provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, and the processor is used to run the computer program in the memory to implement the bird re-identification method provided in the embodiment of the present application.
[0039] In addition, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is suitable for loading by a processor to execute any one of the bird re-identification methods provided in the embodiment of the present application.
[0040] In addition, an embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements any one of the bird re-identification methods provided in the embodiments of the present application.
[0041] In an embodiment of the present application, an original bird image is obtained and preprocessed to obtain a preprocessed image, so that by identifying the preprocessed image, the accuracy and efficiency of identification can be improved; then, it is detected whether there are birds in the preprocessed image. When it is detected that there are birds in the preprocessed image, features are extracted from the preprocessed image to obtain a basic feature map, a preset two-stage recognition model is obtained, and fine-grained recognition is performed on the basic feature map based on the two-stage recognition model to obtain bird information in the original bird image, thereby achieving accurate extraction of features in the bird image and further improving the accuracy and precision of bird information identification; then, when it is detected that there are non-bird animals in the preprocessed image, the source bird watching device of the original bird image is obtained, and non-bird animals in the field of view are driven away based on the source bird watching device, thereby finally achieving fine-grained recognition of birds and improving the accuracy and precision of bird identification. While ensuring the accuracy, the number of parameters of the model is reduced, the model has stronger scalability, improves the reasoning speed, saves computing resources, and further avoids waste of resources. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.
[0043] Figure 1 is a schematic diagram of a flow chart of a bird re-identification method provided in an embodiment of the present application;
[0044] Figure 2 is a schematic diagram of the structure of a bird re-identification device provided in an embodiment of the present application;
[0045] Figure 3 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0046] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.
[0047] The embodiment of the present application provides a bird re-identification method, device, electronic device and computer storage medium. The bird re-identification device can be integrated in an electronic device, which can be a server or a terminal.
[0048] Among them, the server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, network acceleration services (Content Delivery Network, CDN), as well as big data and artificial intelligence platforms.
[0049] The terminal may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto. The terminal and the server may be directly or indirectly connected via wired or wireless communication, and this application does not limit this.
[0050] In addition, the term "plurality" in the embodiments of the present application refers to two or more than two. The terms "first" and "second" in the embodiments of the present application are used to distinguish descriptions and should not be understood as implying relative importance.
[0051] It should be noted that the description order of the following embodiments is not intended to limit the preferred order of the embodiments.
[0052] See also Figure 1 , Figure 1 : is a flow chart of a bird re-identification method provided in an embodiment of the present application. The bird re-identification method may include:
[0053] S101, obtaining an original bird image, and preprocessing the original bird image to obtain a preprocessed image.
[0054] In this embodiment, the original bird image is an initial bird image obtained by taking a bird watching device. When the original bird image is obtained, the original bird image is preprocessed, wherein the preprocessing includes steps such as contrast enhancement, horizontal flipping, upside down flipping, image rotation, image translation, and Gaussian noise processing of the original bird image. The preprocessed image is obtained by sequentially performing contrast enhancement, horizontal flipping, upside down flipping, image rotation, image translation, and Gaussian noise processing on the original bird image.
[0055] S102, detecting whether there are birds in the preprocessed image. When it is detected that there are birds in the preprocessed image, performing feature extraction on the preprocessed image to obtain a basic feature map, obtaining a preset two-stage recognition model, and performing fine-grained recognition on the basic feature map based on the two-stage recognition model to obtain bird information in the original bird image.
[0056] In this embodiment, when a preprocessed image is obtained, it is detected whether there are birds in the preprocessed image. Specifically, it can be determined whether there are birds in the current preprocessed image by using target detection technology. Target detection technology is an automatic recognition technology in computer vision. By extracting features from bird images and training classifiers, it can automatically detect target objects in preprocessed images or videos, and output information such as their position, size, and category.
[0057] When it is determined that there are birds in the preprocessed image, feature extraction is performed on the preprocessed image to obtain a basic feature map. Specifically, the preprocessed image is input into the feature extraction network f, and a two-dimensional basic feature map F is extracted based on the feature extraction network, F∈(H·W)×D, where H and W are the height and width of the basic feature map F, respectively, and D represents the embedding dimension size. When the basic feature map is obtained, a preset two-stage recognition model is obtained, and the basic feature map is fine-grainedly recognized according to the two-stage recognition model, and finally the bird information in the original bird image is obtained.
[0058] Specifically, the two-stage recognition model is a neural network model of an inverted residual structure. Among them, the residual structure is an hourglass-shaped structure that first performs image dimensionality reduction, then performs feature extraction, and finally performs feature dimensionality increase. However, when the high-dimensional space in the information is nonlinearly mapped to the low-dimensional space, information collapse will occur. Therefore, this embodiment proposes a neural network model of an inverted residual structure, in which the basic feature map is first dimensionalized, then features are extracted by channel-by-channel convolution, and finally feature dimensionality reduction is achieved by convolution. In addition, an attention mechanism and feature fusion are added to the inverted residual structure, and convolution calculation is performed on the features in the inverted residual structure by deep separable convolution, and finally a two-stage recognition model is formed. When the basic feature map is obtained, the basic feature map is input into the two-stage recognition model, and the bird information in the original bird image is calculated by the two-stage recognition model. The bird information includes information such as the category, color, and size of the birds in the original bird image.
[0059] S103: When non-bird animals are detected in the pre-processed image, the source bird-watching device of the original bird image is obtained, and the non-bird animals in the field of view are driven away based on the source bird-watching device.
[0060] In this embodiment, when a non-bird animal, such as a squirrel, is detected in the pre-processed image, a source bird watching device of the original bird image is obtained, and the detection result is sent to the source bird watching device, which is a device that photographs the original bird image. When it is detected that the source bird watching device receives the detection result, a repelling device of the source bird watching device, such as an odor repelling device or a mechanical repelling device, is started, and the non-bird animal in the field of view of the camera of the source bird watching device is repelled based on the driving device.
[0061] Furthermore, if it is detected that there is no animal in the pre-processed image, the pre-processed image is re-detected, or a re-shooting instruction is sent to the source bird-watching device. When it is detected that the source bird-watching device receives the re-shooting instruction, the source bird-watching device is controlled to re-shoot the target field of view content to obtain a shot image, and the bird identification is re-performed based on the shot image.
[0062] This embodiment obtains an original bird image, preprocesses the original bird image, and obtains a preprocessed image; then, detects whether there are birds in the preprocessed image. When it is detected that there are birds in the preprocessed image, features are extracted from the preprocessed image to obtain a basic feature map, a preset two-stage recognition model is obtained, and fine-grained recognition is performed on the basic feature map based on the two-stage recognition model to obtain bird information in the original bird image; then, when it is detected that there are non-bird animals in the preprocessed image, the source bird-watching device of the original bird image is obtained, and non-bird animals in the field of vision are driven away based on the source bird-watching device, thereby achieving fine-grained recognition of birds, improving the accuracy and precision of bird recognition, and while ensuring the accuracy, reducing the number of parameters of the model, the model has stronger scalability, improves the reasoning speed, saves computing resources, and further avoids waste of resources.
[0063] In some embodiments of the present application, the above-mentioned preprocessing of the original bird image to obtain the preprocessed image includes:
[0064] Performing data enhancement processing on the original bird image to obtain a data enhanced image;
[0065] A normalization range is obtained, and the data augmented image is normalized according to the normalization range to obtain the preprocessed image.
[0066] In this embodiment, before the original bird image is recognized, data enhancement is performed on the original bird image to reduce the sensitivity of the model to the image, reduce the phenomenon of model overfitting, and improve the recognition efficiency and accuracy of the model. Therefore, when the original bird image is obtained, the original bird image is subjected to data enhancement processing, wherein the data enhancement processing includes the steps of contrast enhancement, horizontal flipping, upside down flipping, image rotation, image translation, and Gaussian noise processing. When the original bird image is obtained, the original bird image is subjected to contrast enhancement, horizontal flipping, upside down flipping, image rotation, image translation, and Gaussian noise processing in sequence to obtain a data enhanced image. After that, the data augmented image is normalized to obtain a preprocessed image. Specifically, when the data augmented image is obtained, the data augmented image is resized and uniformly adjusted to the target size, for example, to a size of 224x224. Then, a preset normalization range is obtained, and the size-regularized data augmented image is normalized according to the normalization range, that is, the pixel value range of the regularized data augmented image is normalized to the normalization range, such as [0,1], to obtain the preprocessed image. The specific calculation formula is as follows:
[0067]
[0068] Among them, y is the pixel of the regularized data enhancement image, x ijis the pixel value of a point in the regularized data enhancement image, min is the minimum pixel value, and max is the maximum pixel value.
[0069] This embodiment performs data enhancement on the original bird image and then normalizes it to obtain a preprocessed image, thereby achieving preprocessing of the original bird image, improving the efficiency of the model in recognizing the image, and further improving the accuracy of the model in recognizing the bird image.
[0070] In some embodiments of the present application, performing data enhancement processing on the original bird image to obtain the data enhanced image includes:
[0071] Get the preset adjustment parameters;
[0072] The original bird image is subjected to contrast enhancement, horizontal flipping, upside down flipping, image rotation, image translation and Gaussian noise processing according to the adjustment parameters to obtain the data enhanced image.
[0073] In this embodiment, when data enhancement processing is performed on the original bird image, preset adjustment parameters are obtained, and the adjustment parameters include contrast parameters, horizontal flipping parameters, upside down flipping parameters, image rotation parameters, image translation parameters, and Gaussian noise parameters corresponding to contrast enhancement, horizontal flipping, upside down flipping, image rotation, image translation, and Gaussian noise processing, respectively. Contrast enhancement, horizontal flipping, upside down flipping, image rotation, image translation, and Gaussian noise processing are performed on the original bird image according to the adjustment parameters to obtain a data enhanced image.
[0074] Among them, contrast enhancement is the process of mapping the grayscale points in the original bird image. Through contrast enhancement, the grayscale of the overall image can be roughly consistent with the uniform distribution, thereby enhancing the contrast of the image. Specifically, the contrast parameter is obtained, and the contrast of the original bird image is enhanced according to the contrast parameter to obtain a contrast-enhanced image; then, the horizontal flip angle, the horizontal flip angle, and the image rotation angle are obtained, and the contrast-enhanced image is horizontally flipped, flipped up and down, and rotated, such as flipping 180 degrees, in which the missing parts of the image rotation are filled with zeros; then, the flipped image is translated up, down, left, and right, and finally, Gaussian noise is added to the translated image to obtain a data-enhanced image. The probability density function of the Gaussian noise is as follows:
[0075]
[0076] Where p(z) is Gaussian noise, σ is the standard deviation of z, is the mean, and z is a random variable.
[0077] In addition, when the two-stage recognition model is trained by using pre-trained bird sample data, different batches of images can be randomly selected from the pre-trained bird sample data during pre-processing, and the original sample images after contrast enhancement can be randomly selected for horizontal flipping, upside down flipping, image rotation, or Gaussian noise processing, so as to increase the richness of the samples and improve the generalization ability of the model.
[0078] This embodiment obtains preset adjustment parameters, processes the original bird image according to the adjustment parameters, and obtains a data enhanced image, so that the accuracy and efficiency of the model in identifying bird information can be further improved through the data enhanced image.
[0079] In some embodiments of the present application, the two-stage recognition model includes a first convolutional layer, a second convolutional layer, an attention calculation layer, a third convolutional layer and a fully connected layer, and the fine-grained recognition of the basic feature map based on the two-stage recognition model to obtain the bird information in the original bird image includes:
[0080] Performing convolutional dimension-raising calculation on the basic feature map based on the first convolutional layer to obtain a dimension-raising feature map;
[0081] Performing channel-by-channel convolution calculation on the dimension-increased feature map according to the second convolution layer to obtain a first convolution feature;
[0082] Based on the attention calculation layer, extracting features from the first convolutional features through an attention mechanism to obtain attention features;
[0083] Performing dimension reduction and fusion calculation on the attention feature according to the third convolutional layer to obtain an attention feature map;
[0084] The attention feature map is mapped to category labels through the fully connected layer to obtain bird information in the original bird image.
[0085] In this embodiment, the two-stage recognition model is a neural network model with an inverted residual structure, including a first convolutional layer, a second convolutional layer, an attention calculation layer, a third convolutional layer and a fully connected layer. The input feature map is calculated in sequence according to the first convolutional layer, the second convolutional layer, the attention calculation layer, the third convolutional layer and the fully connected layer, and finally the bird information in the original bird image is recognized.
[0086] Specifically, the first convolution layer is the first layer of the inverted residual structure of the two-stage recognition model, and its convolution kernel size is 1X1. The first convolution layer performs 1X1 convolution dimension-raising calculation on the basic feature map to obtain the dimension-raising feature map. Then, the dimension-raising feature map is input into the second convolution layer, and the second convolution layer performs channel-wise convolution (Depthwise Convolution, DW) calculation of 3X3 convolution kernel on the dimension-raising feature map to extract the first convolution feature. Among them, channel-wise convolution is a kind of depth-separable convolution. Different from the conventional convolution kernel that operates each channel of the input image at the same time, in channel-wise convolution, a convolution kernel is only responsible for one channel, so that while ensuring the accuracy of the model, the number of parameters and computing costs are reduced, so that the model can be better deployed on mobile devices.
[0087] Afterwards, in order to better capture the spatial correlation between features and further extract the global information of the image, when the first convolution feature is obtained, the first convolution feature is input into the attention calculation layer, and the first convolution feature is adjusted and extracted through the attention mechanism of the attention calculation layer to obtain the attention feature. Finally, the attention feature is reduced in dimension through the third convolution layer to obtain the attention feature map, wherein the third convolution layer is a 1X1 pointwise convolution (PointwiseConvolution, PW) calculation layer, and the attention feature is reduced in dimension and fused through the pointwise convolution to obtain the attention feature map. When the attention feature map is obtained, the attention feature map is input into the fully connected layer, and the category label mapping is performed through the fully connected layer to extract the bird information.
[0088] This embodiment performs fine-grained recognition of the basic feature map through the first convolutional layer, the second convolutional layer, the attention calculation layer, the third convolutional layer and the fully connected layer of the two-stage recognition model, thereby achieving accurate recognition of the bird information in the image. While improving the feature extraction and expression capabilities of the model, the model performance and accuracy are improved, making the extracted bird information more accurate.
[0089] In some embodiments of the present application, the above-mentioned feature extraction of the first convolution feature through the attention mechanism based on the attention calculation layer to obtain the attention feature includes:
[0090] Obtain a preset convolution kernel, and perform convolution calculation on the first convolution feature and the convolution kernel according to the attention calculation layer to obtain a second convolution feature;
[0091] Performing global average pooling on the channels of the dimension-increased feature map according to the second convolutional features to obtain a global information statistic;
[0092] Obtain a preset activation function, and calculate the global information statistic based on the activation function to obtain a channel weight;
[0093] The attention feature is obtained by calculating according to the channel weight and the second convolution feature.
[0094] In this embodiment, in order to further acquire the global information of the image, a combination of channel and spatial attention mechanisms is used to drive the model to learn the discriminative features of the image, such as the bird's beak, bird feathers, and color. Specifically, when the first convolution feature is obtained, a preset convolution kernel is obtained, and the convolution of the first convolution feature and the convolution kernel is calculated according to the attention calculation layer to obtain the second convolution feature. The convolution calculation formula is as follows:
[0095]
[0096] Among them, V c is the convolution kernel, and X is the first convolution feature.
[0097] When the second convolution feature is calculated, the channels of the dimension-increased feature map are globally pooled according to the second convolution feature to obtain global information statistics, which can express the entire image. Then, a preset activation function is obtained, and the global information statistics are calculated according to the activation function to obtain the channel weight; finally, the channel weight is multiplied with the second convolution feature to obtain the attention feature.
[0098] X c =F scale (U c ,S c )
[0099] Among them, X c is the attention feature, u c is the second convolution feature, S c is the channel weight.
[0100] This embodiment calculates the first convolution feature through the channel and attention mechanism, thereby achieving accurate extraction of global image information and improving the accuracy of the model in extracting bird features.
[0101] In some embodiments of the present application, the above-mentioned performing dimensionality reduction and fusion calculation on the attention feature according to the third convolutional layer to obtain an attention feature map includes:
[0102] Performing point-by-point convolution dimensionality reduction calculation on the attention feature according to the third convolution layer to obtain a convolution feature map;
[0103] The convolution feature map and the basic feature map are added element by element to obtain the attention feature map.
[0104] In this embodiment, in order to further improve the accuracy of the model in re-identifying birds, when obtaining the attention feature, the attention feature is subjected to 1X1 point-by-point convolution dimensionality reduction calculation through the third convolution layer to obtain a convolution feature map; then, the convolution feature map is element-by-element added to the basic feature map to obtain an attention feature map that integrates the attention mechanism.
[0105] This embodiment performs dimensionality reduction and fusion calculation on the attention features through the third convolutional layer to obtain an attention feature map, thereby realizing the extraction of global information of the image and improving the precision and accuracy of the model in re-identifying birds.
[0106] In some embodiments of the present application, before obtaining the preset two-stage recognition model, the following steps are included:
[0107] Build a basic recognition model and obtain pre-trained bird sample data;
[0108] Inputting the pre-trained bird sample data into the basic recognition model, training the basic recognition model according to the pre-trained bird sample data, and obtaining a predicted probability value of a predicted category corresponding to the pre-trained bird sample data;
[0109] Obtaining a cross entropy loss function, and calculating a loss value of the predicted probability value according to the cross entropy loss function;
[0110] The basic recognition model is adjusted according to the loss value. When the loss value is less than or equal to a preset first threshold, a triple of the pre-trained bird sample data is randomly generated. According to a preset triple loss function, the ternary loss value of the triple is calculated. When the ternary loss value is within a preset second threshold range, it is determined that the training of the basic recognition model is completed, and the two-stage recognition model is obtained.
[0111] In this embodiment, before obtaining the preset two-stage recognition model, the model needs to be trained to obtain a two-stage recognition model with efficient recognition ability. Specifically, a basic recognition model is constructed, and the structure of the basic recognition model is the same as that of the two-stage recognition model, that is, both are inverse residual neural network structures. Pre-trained bird sample data is obtained, wherein, due to the limited bird samples, in order to increase the sample diversity and improve the generalization ability and robustness of the model, the pre-trained bird sample data is pre-processed when the pre-trained bird sample data is obtained. First, the pre-trained bird sample data is contrast enhanced, and then, the pictures of each training batch are randomly selected from the contrast-enhanced image for horizontal flipping, or upside down flipping, or image rotation, or Gaussian noise processing, that is, the pre-processed sample picture is obtained. The pre-processed sample picture is input into the basic recognition model, and the basic recognition model is trained according to the pre-processed sample picture, and the predicted probability value of the corresponding predicted category of the pre-processed sample picture is output. Then, the cross entropy loss function is obtained, and the loss value of the predicted probability value is calculated according to the cross entropy loss function. Specifically, the calculation formula of the cross entropy loss function is:
[0112]
[0113] Where M is the number of categories; y ic is a sign function (0 or 1), if the true category of the preprocessed sample image i is equal to c, it takes 1, otherwise, it takes 0; p ic is the predicted probability value that the preprocessed sample image i belongs to category c.
[0114] Then, in order to further improve the accuracy of bird re-identification, random sampling is used to generate triplets during the training process to avoid problems such as model overfitting and imbalanced training samples. Therefore, the basic recognition model is adjusted according to the loss value calculated based on the cross entropy loss function. When the loss value is less than or equal to the preset first threshold, the preset triple loss function is obtained, and a triple of pre-trained bird sample data is randomly generated, wherein the triple is composed of (sample, positive sample, negative sample). The ternary loss value of the triple is calculated according to the triple loss function. When it is determined that the ternary loss value is within the preset second threshold range, it is determined that the training of the basic recognition model is completed, and a two-stage recognition model is obtained. As a result, the distance between the features of the same bird is as small as possible, while the distance between the features of different birds is as large as possible. The calculation formula of the triple loss function is:
[0115]
[0116] Among them, d a,p Indicates the distance between the sample and the positive sample, d a,nRepresents the distance between the sample and the negative sample, α is a constant, a is the sample, p is the positive sample, and n is the negative sample.
[0117] This embodiment constructs a basic recognition model and trains the basic recognition model through a cross entropy loss function and a triplet loss function, thereby achieving accurate training of the basic recognition model and further improving the accuracy of the model in re-identifying birds.
[0118] In order to facilitate better implementation of the bird re-identification method provided in the embodiment of the present application, the embodiment of the present application also provides a device based on the above bird re-identification method. The meanings of the terms are the same as those in the above bird re-identification method, and the specific implementation details can refer to the description in the method embodiment.
[0119] For example, Figure 2 As shown, the bird re-identification device may include: a pre-processing module 201, a detection module 202, and a driving away module 203.
[0120] A preprocessing module 201 is used to obtain an original bird image and preprocess the original bird image to obtain a preprocessed image;
[0121] In one embodiment of the present application, the preprocessing module 201 includes:
[0122] A first processing unit, configured to perform data enhancement processing on the original bird image to obtain a data enhanced image;
[0123] A normalization unit is used to obtain a normalization range, and normalize the data enhanced image according to the normalization range to obtain the preprocessed image.
[0124] In one embodiment of the present application, the first processing unit includes:
[0125] An acquisition unit, used for acquiring preset adjustment parameters;
[0126] The first processing unit is used to perform contrast enhancement, horizontal flipping, upside down flipping, image rotation, image translation and Gaussian noise processing on the original bird image according to the adjustment parameters to obtain the data enhanced image.
[0127] In this embodiment, the original bird image is an initial bird image obtained by taking a bird watching device. When the original bird image is obtained, the original bird image is preprocessed, wherein the preprocessing includes steps such as contrast enhancement, horizontal flipping, upside down flipping, image rotation, image translation, and Gaussian noise processing of the original bird image. The preprocessed image is obtained by sequentially performing contrast enhancement, horizontal flipping, upside down flipping, image rotation, image translation, and Gaussian noise processing on the original bird image.
[0128] The detection module 202 is used to detect whether there are birds in the pre-processed image. When it is detected that there are birds in the pre-processed image, feature extraction is performed on the pre-processed image to obtain a basic feature map, a preset two-stage recognition model is acquired, and fine-grained recognition is performed on the basic feature map based on the two-stage recognition model to obtain bird information in the original bird image.
[0129] In one embodiment of the present application, the detection module 202 includes:
[0130] A first computing unit, configured to perform a convolution dimension-raising calculation on the basic feature map based on the first convolution layer to obtain a dimension-raising feature map;
[0131] A second calculation unit is used to perform channel-by-channel convolution calculation on the dimension-increased feature map according to the second convolution layer to obtain a first convolution feature;
[0132] A third computing unit, configured to extract features from the first convolutional features through an attention mechanism based on the attention computing layer to obtain attention features;
[0133] a fourth computing unit, configured to perform dimension reduction and fusion calculation on the attention feature according to the third convolutional layer to obtain an attention feature map;
[0134] A mapping unit is used to perform category label mapping on the attention feature map through the fully connected layer to obtain the bird information in the original bird image.
[0135] In one embodiment of the present application, the third calculation unit includes:
[0136] a fifth computing unit, configured to obtain a preset convolution kernel, and perform convolution calculation on the first convolution feature and the convolution kernel according to the attention computing layer to obtain a second convolution feature;
[0137] A pooling unit, used for performing global average pooling on the channels of the dimension-upgraded feature map according to the second convolutional feature to obtain a global information statistic;
[0138] a sixth calculation unit, configured to obtain a preset activation function, and calculate the global information statistic based on the activation function to obtain a channel weight;
[0139] A seventh computing unit is used to perform calculation according to the channel weight and the second convolution feature to obtain the attention feature.
[0140] In one embodiment of the present application, the fourth calculation unit includes:
[0141] A convolution calculation unit, configured to perform point-by-point convolution calculation on the attention feature according to the third convolution layer to obtain a convolution feature map;
[0142] An eighth computing unit is used to perform element-by-element addition of the convolution feature map and the basic feature map to obtain the attention feature map.
[0143] In this embodiment, when a preprocessed image is obtained, it is detected whether there are birds in the preprocessed image. Specifically, it can be determined whether there are birds in the current preprocessed image by using target detection technology. Target detection technology is an automatic recognition technology in computer vision. By extracting features from bird images and training classifiers, it can automatically detect target objects in preprocessed images or videos, and output information such as their position, size, and category.
[0144] When it is determined that there are birds in the preprocessed image, feature extraction is performed on the preprocessed image to obtain a basic feature map. Specifically, the preprocessed image is input into the feature extraction network f, and a two-dimensional basic feature map F is extracted based on the feature extraction network, F∈(H·W)×D, where H and W are the height and width of the basic feature map F, respectively, and D represents the embedding dimension size. When the basic feature map is obtained, a preset two-stage recognition model is obtained, and the basic feature map is fine-grainedly recognized according to the two-stage recognition model, and finally the bird information in the original bird image is obtained.
[0145] Specifically, the two-stage recognition model is a neural network model of an inverted residual structure. Among them, the residual structure is an hourglass-shaped structure that first performs image dimensionality reduction, then performs feature extraction, and finally performs feature dimensionality increase. However, when the high-dimensional space in the information is nonlinearly mapped to the low-dimensional space, information collapse will occur. Therefore, this embodiment proposes a neural network model of an inverted residual structure, in which the basic feature map is first dimensionalized, then features are extracted by channel-by-channel convolution, and finally feature dimensionality reduction is achieved by convolution. In addition, an attention mechanism and feature fusion are added to the inverted residual structure, and convolution calculation is performed on the features in the inverted residual structure by deep separable convolution, and finally a two-stage recognition model is formed. When the basic feature map is obtained, the basic feature map is input into the two-stage recognition model, and the bird information in the original bird image is calculated by the two-stage recognition model. The bird information includes information such as the category, color, and size of the birds in the original bird image.
[0146] The expelling module 203 is used to obtain the source bird watching device of the original bird image when detecting the presence of non-bird animals in the pre-processed image, and expel the non-bird animals in the field of view based on the source bird watching device.
[0147] In this embodiment, when a non-bird animal, such as a squirrel, is detected in the pre-processed image, a source bird watching device of the original bird image is obtained, and the detection result is sent to the source bird watching device, which is a device that photographs the original bird image. When it is detected that the source bird watching device receives the detection result, a repelling device of the source bird watching device, such as an odor repelling device or a mechanical repelling device, is started, and the non-bird animal in the field of view of the camera of the source bird watching device is repelled based on the driving device.
[0148] Furthermore, if it is detected that there is no animal in the pre-processed image, the pre-processed image is re-detected, or a re-shooting instruction is sent to the source bird-watching device. When it is detected that the source bird-watching device receives the re-shooting instruction, the source bird-watching device is controlled to re-shoot the target field of view content to obtain a shot image, and the bird identification is re-performed based on the shot image.
[0149] In one embodiment of the present application, the bird re-identification device further includes:
[0150] Construction module, used to build a basic recognition model and obtain pre-trained bird sample data;
[0151] A training module, used for inputting the pre-trained bird sample data into the basic recognition model, training the basic recognition model according to the pre-trained bird sample data, and obtaining a predicted probability value of a predicted category corresponding to the pre-trained bird sample data;
[0152] A calculation module, used to obtain a cross entropy loss function, and calculate a loss value of the predicted probability value according to the cross entropy loss function;
[0153] An adjustment module is used to adjust the basic recognition model according to the loss value. When the loss value is less than or equal to a preset first threshold, a triple of the pre-trained bird sample data is randomly generated. According to a preset triple loss function, a ternary loss value of the triple is calculated. When the ternary loss value is within a preset second threshold range, it is determined that the training of the basic recognition model is completed, and the two-stage recognition model is obtained.
[0154] The bird re-identification device proposed in this application realizes fine-grained recognition of birds, improves the accuracy and precision of bird recognition, and while ensuring accuracy, reduces the number of model parameters. The model has stronger scalability, improves the reasoning speed, saves computing power resources, and further avoids waste of resources.
[0155] In specific implementation, the above modules can be implemented as independent entities, or can be arbitrarily combined and implemented as the same or several entities. The specific implementation methods and corresponding beneficial effects of the above modules can be found in the previous method embodiments, which will not be repeated here.
[0156] The present application also provides an electronic device, which may be a server or a terminal, etc. Figure 3 As shown, it shows a schematic diagram of the structure of the electronic device involved in the embodiment of the present application, specifically:
[0157] The electronic device may include components such as a processor 601 with one or more processing cores, a memory 602 with one or more computer-readable storage media, a power supply 603, and an input unit 604. Those skilled in the art will appreciate that Figure 3 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.
[0158] The processor 601 is the control center of the electronic device, which uses various interfaces and lines to connect various parts of the entire electronic device, and executes various functions of the electronic device and processes data by running or executing computer programs and / or modules stored in the memory 602, and calling data stored in the memory 602. Optionally, the processor 601 may include one or more processing cores; preferably, the processor 601 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 601.
[0159] The memory 602 can be used to store computer programs and modules. The processor 601 executes various functional applications and data processing by running the computer programs and modules stored in the memory 602. The memory 602 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, a computer program required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 602 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 602 may also include a memory controller to provide the processor 601 with access to the memory 602.
[0160] The electronic device also includes a power supply 603 for supplying power to each component. Preferably, the power supply 603 can be logically connected to the processor 601 through a power management system, so as to manage charging, discharging, power consumption and other functions through the power management system. The power supply 603 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators and other arbitrary components.
[0161] The electronic device may further include an input unit 604, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.
[0162] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 601 in the electronic device will load the executable files corresponding to the processes of one or more computer programs into the memory 602 according to the following instructions, and the processor 601 will run the computer programs stored in the memory 602, thereby realizing various functions, such as:
[0163] Acquire an original bird image, and preprocess the original bird image to obtain a preprocessed image;
[0164] Detecting whether there are birds in the preprocessed image, and when detecting that there are birds in the preprocessed image, performing feature extraction on the preprocessed image to obtain a basic feature map, acquiring a preset two-stage recognition model, and performing fine-grained recognition on the basic feature map based on the two-stage recognition model to obtain bird information in the original bird image;
[0165] When non-bird animals are detected in the pre-processed image, the source bird watching device of the original bird image is obtained, and the non-bird animals in the field of view are driven away based on the source bird watching device.
[0166] The specific implementation methods and corresponding beneficial effects of the above operations can be found in the detailed description of the bird re-identification method above, which will not be elaborated here.
[0167] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by a computer program, or by controlling related hardware through a computer program. The computer program may be stored in a computer-readable storage medium and loaded and executed by a processor.
[0168] To this end, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored, and the computer program can be loaded by a processor to execute the steps in any of the bird re-identification methods provided in the embodiments of the present application. For example, the computer program can execute the following steps:
[0169] Acquire an original bird image, and preprocess the original bird image to obtain a preprocessed image;
[0170] Detecting whether there are birds in the preprocessed image, and when detecting that there are birds in the preprocessed image, performing feature extraction on the preprocessed image to obtain a basic feature map, acquiring a preset two-stage recognition model, and performing fine-grained recognition on the basic feature map based on the two-stage recognition model to obtain bird information in the original bird image;
[0171] When non-bird animals are detected in the pre-processed image, the source bird watching device of the original bird image is obtained, and the non-bird animals in the field of view are driven away based on the source bird watching device.
[0172] The specific implementation methods and corresponding beneficial effects of the above operations can be found in the previous embodiments, which will not be described in detail here.
[0173] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.
[0174] Since the computer program stored in the computer-readable storage medium can execute the steps in any one of the bird re-identification methods provided in the embodiments of the present application, the beneficial effects that can be achieved by any one of the bird re-identification methods provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.
[0175] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the above-mentioned bird re-identification method.
[0176] The above is a detailed introduction to a bird re-identification method, device, electronic device and computer storage medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present application.
Claims
1. A bird re-identification method, characterized in that: include: Acquire an original bird image, and preprocess the original bird image to obtain a preprocessed image; Detecting whether there are birds in the preprocessed image, and when detecting that there are birds in the preprocessed image, performing feature extraction on the preprocessed image to obtain a basic feature map, acquiring a preset two-stage recognition model, and performing fine-grained recognition on the basic feature map based on the two-stage recognition model to obtain bird information in the original bird image; When non-bird animals are detected in the pre-processed image, the source bird watching device of the original bird image is obtained, and the non-bird animals in the field of view are driven away based on the source bird watching device.
2. The bird re-identification method according to claim 1, characterized in that: The preprocessing of the original bird image to obtain a preprocessed image includes: Performing data enhancement processing on the original bird image to obtain a data enhanced image; A normalization range is obtained, and the data augmented image is normalized according to the normalization range to obtain the preprocessed image.
3. The bird re-identification method according to claim 2, characterized in that: The performing data enhancement processing on the original bird image to obtain the data enhanced image comprises: Get the preset adjustment parameters; The original bird image is subjected to contrast enhancement, horizontal flipping, upside down flipping, image rotation, image translation and Gaussian noise processing according to the adjustment parameters to obtain the data enhanced image.
4. The bird re-identification method according to claim 1, characterized in that: The two-stage recognition model includes a first convolution layer, a second convolution layer, an attention calculation layer, a third convolution layer and a fully connected layer. The fine-grained recognition of the basic feature map based on the two-stage recognition model to obtain the bird information in the original bird image includes: Performing convolutional dimension-raising calculation on the basic feature map based on the first convolutional layer to obtain a dimension-raising feature map; Performing channel-by-channel convolution calculation on the dimension-increased feature map according to the second convolution layer to obtain a first convolution feature; Based on the attention calculation layer, extracting features from the first convolutional features through an attention mechanism to obtain attention features; Performing dimension reduction and fusion calculation on the attention feature according to the third convolutional layer to obtain an attention feature map; The attention feature map is mapped to category labels through the fully connected layer to obtain bird information in the original bird image.
5. The bird re-identification method according to claim 4, characterized in that: The step of extracting features of the first convolutional features based on the attention calculation layer through an attention mechanism to obtain attention features includes: Obtain a preset convolution kernel, and perform convolution calculation on the first convolution feature and the convolution kernel according to the attention calculation layer to obtain a second convolution feature; Performing global average pooling on the channels of the dimension-increased feature map according to the second convolutional features to obtain a global information statistic; Obtain a preset activation function, and calculate the global information statistic based on the activation function to obtain a channel weight; The attention feature is obtained by calculating according to the channel weight and the second convolution feature.
6. The bird re-identification method according to claim 4, characterized in that: The step of performing dimension reduction and fusion calculation on the attention feature according to the third convolutional layer to obtain an attention feature map includes: Performing point-by-point convolution calculation on the attention feature according to the third convolution layer to obtain a convolution feature map; The convolution feature map and the basic feature map are added element by element to obtain the attention feature map.
7. The bird re-identification method according to claim 1, characterized in that: Before obtaining the preset two-stage recognition model, it includes: Build a basic recognition model and obtain pre-trained bird sample data; Inputting the pre-trained bird sample data into the basic recognition model, training the basic recognition model according to the pre-trained bird sample data, and obtaining a predicted probability value of a predicted category corresponding to the pre-trained bird sample data; Obtaining a cross entropy loss function, and calculating a loss value of the predicted probability value according to the cross entropy loss function; The basic recognition model is adjusted according to the loss value. When the loss value is less than or equal to a preset first threshold, a triple of the pre-trained bird sample data is randomly generated. According to a preset triple loss function, the ternary loss value of the triple is calculated. When the ternary loss value is within a preset second threshold range, it is determined that the training of the basic recognition model is completed, and the two-stage recognition model is obtained.
8. A bird re-identification device, characterized in that: include: A preprocessing module, used for acquiring an original bird image, and preprocessing the original bird image to obtain a preprocessed image; A detection module, used to detect whether there are birds in the preprocessed image, and when detecting the presence of birds in the preprocessed image, extracting features from the preprocessed image to obtain a basic feature map, acquiring a preset two-stage recognition model, and performing fine-grained recognition on the basic feature map based on the two-stage recognition model to obtain bird information in the original bird image; The expelling module is used to obtain the source bird watching device of the original bird image when detecting the presence of non-bird animals in the pre-processed image, and to expel the non-bird animals in the field of view based on the source bird watching device.
9. An electronic device, characterized in that: The invention comprises a processor and a memory, wherein the memory stores a computer program, and the processor is used to run the computer program in the memory to execute the bird re-identification method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor to execute the bird re-identification method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Bird species recognition method
CN105069817A
Second-order feature-based bird fine granularity identification method
CN113076861A
Bird identification method based on deep neural network and electronic equipment
CN113239965A
Bird fine-grained image recognition method and device based on Transform and component feature fusion
CN114626476A
Bird and non-bird repelling method and system for airport
CN114638975A
Cited By
Bird fine-grained detection algorithm based on snakelike deformable convolution and cavity convolution
CN120451637A
Bird fine-grained detection algorithm of snake-shaped deformable convolution and dilated convolution
CN120451637B
Attention mechanism-based fine-grained bird image recognition method for intelligent bird repelling system in airport
CN120783180A
