Image recognition method for multispectral bird monitoring

Abstract

The invention relates to the technical field of image recognition, and particularly discloses an image recognition method for multispectral bird monitoring, which comprises the following steps of: S1, carrying out multi-angle shooting on a bird monitoring area by utilizing a multispectral camera equipped with a plurality of different spectral filters to obtain image data of a plurality of spectral bands; the collected image data is preprocessed; s2, performing feature extraction according to the spectral band type of the preprocessed image data; feature fusion is carried out; s3, constructing an integrated classifier; training an integrated classifier by using the fused image data set, and optimizing classifier parameters; and S4, acquiring a multispectral bird image in real time, inputting the multispectral bird image into the trained integrated classifier for recognition after preprocessing and feature extraction, and determining a final bird recognition result according to a voting mechanism. According to the invention, the image acquisition, processing and identification process is more accurate and controllable.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to an image recognition method for multispectral bird monitoring. Background Art

[0002] Bird monitoring is of great significance for ecological research and biodiversity conservation. Traditional bird monitoring relies heavily on manual observation, which is inefficient and susceptible to influences from observer experience and environmental factors. With technological advancements, image recognition technology is increasingly being applied to bird monitoring. However, existing image recognition methods based on a single spectrum have numerous drawbacks in bird monitoring.

[0003] A single spectral image can only reflect part of the characteristic information of a bird. For example, in the visible light spectrum, when birds have similar feather colors or are in a complex background environment, it is difficult for the image recognition system to accurately distinguish between different types of birds. Moreover, at night or in low-light environments, the quality of a single visible light spectrum image will drop significantly, resulting in extremely low recognition accuracy. In addition, for some birds with special physiological characteristics, such as their unique markings or structures that can only be clearly presented in the near-infrared spectrum, single spectrum image recognition is even more difficult to play a role. These problems seriously limit the efficient implementation of bird monitoring work and cannot meet the needs of accurate monitoring of bird population numbers, distribution ranges, and behavioral habits. Summary of the Invention

[0004] The purpose of the present invention is to provide an image recognition method for multispectral bird monitoring to solve the problems in the prior art such as low recognition accuracy caused by reliance on a single spectrum and being greatly affected by the environment and bird characteristics.

[0005] The purpose of the present invention can be achieved through the following technical solutions:

[0006] An image recognition method for multispectral bird monitoring comprises the following steps:

[0007] S1: Use a multispectral camera equipped with a variety of spectral filters to shoot bird monitoring areas from multiple angles to obtain image data in multiple spectral bands; and pre-process the collected image data;

[0008] S2: Feature extraction is performed based on the spectral band type of the preprocessed image data; the extracted multispectral features are normalized and feature fusion is performed using a deep belief network;

[0009] S3: Construct an ensemble classifier consisting of a support vector machine, a random forest, and a convolutional neural network. Train the ensemble classifier using the fused image dataset, initialize the convolutional neural network parameters using transfer learning, and optimize the classifier parameters using a stochastic gradient descent algorithm combined with momentum.

[0010] S4: Multispectral bird images are acquired in real time. After preprocessing and feature extraction, they are input into the trained ensemble classifier for recognition. The final bird recognition result is determined based on the voting mechanism.

[0011] As a further solution of the present invention: in S1, the specific steps are as follows:

[0012] The image collected by the camera in the i-th spectral band is I i (x,y), (x,y) represents the coordinate position of the image pixel, i = 1, 2, ..., n, n is the number of spectral bands;

[0013] During the acquisition process, the exposure time t is adjusted according to the ambient light intensity L according to the formula t = k / L, where k is the camera constant;

[0014] The gain g is adjusted according to the average brightness B of the image and the expected average brightness B0 according to the formula g=B0 / B.

[0015] As a further solution of the present invention: in S1, preprocessing the collected image data specifically includes:

[0016] Adaptive median filtering algorithm is used for denoising. For each pixel point in the image I(x,y), a filtering window W is defined with the point as the center, and the median value M and minimum value Z of the pixel values in the window are used to filter the image. min , maximum value Z max and the current pixel value Z xy determining output pixel values;

[0017] The contrast is enhanced by using the histogram normalization method and calculating the cumulative distribution function of the original image. and the cumulative distribution function of the prescribed histogram Map the original pixel value r to s so that |C r (r)-C g (s) | minimum;

[0018] Using the phase correlation-based image registration algorithm, the two-dimensional discrete Fourier transform F1(u,v) and F2(u,v) of two images I1(x,y) and I2(x,y) in different spectral bands are calculated to obtain the cross power spectrum. The correlation function r(x,y) is obtained by performing a two-dimensional discrete Fourier inverse transform on the cross power spectrum, and its peak position (x0,y0) is the translation between the two images.

[0019] As a further solution of the present invention: in S2, the specific steps of feature extraction are as follows:

[0020] Visible light image feature extraction: Using the local binary pattern algorithm, for each pixel in the image I(x,y), a circular neighborhood with a radius of R and a number of neighborhood points P is defined with the pixel as the center, and the LBP code is calculated:

[0021]

[0022] in:

[0023]

[0024] Using the principal component analysis algorithm, the RGB color space data of the image is represented as a matrix X and the covariance matrix is calculated Perform eigenvalue decomposition on C, select the eigenvectors corresponding to the first k largest eigenvalues to form the projection matrix V, and project the original data X into the new feature space Y = XV;

[0025] Near-infrared image feature extraction: Using the wavelet transform algorithm, the near-infrared image I (x, y) is subjected to a two-dimensional wavelet transform and decomposed into the approximate component A, the horizontal detail component H, the vertical detail component V and the diagonal detail component D. The wavelet transform represents s∈{A,H,V,D}; through the grayscale run length matrix, count the number of grayscale runs of different directions and lengths n ij Get the GLRLM and calculate the short-run emphasis:

[0026]

[0027] Short-wave infrared image feature extraction: Using morphological operations, perform corrosion operations on the image I(x,y):

[0028]

[0029] Expansion operation:

[0030]

[0031] The edge contours of the bird are highlighted through opening and closing operations.

[0032] As a further solution of the present invention: in S2, the specific steps of feature fusion are as follows:

[0033] The features of different spectral bands are normalized using the minimum-maximum normalization method. Let a certain feature vector be f, and the normalized feature vector is:

[0034]

[0035] where f min and f maxare the minimum and maximum values in the eigenvector f, respectively;

[0036] A deep belief network is used for feature fusion. The deep belief network is composed of multiple restricted Boltzmann machines stacked together. For the l-th layer RBM, its energy function is:

[0037]

[0038] By training RBM layer by layer, the parameters a, b and w are adjusted to generate a fused feature vector.

[0039] As a further solution of the present invention: in said S3, the specific steps are as follows:

[0040] The support vector machine uses the radial basis function as the kernel function:

[0041]

[0042] The optimization problem is:

[0043] The constraints are: i (w T x i +b)≥1-ξ i ,ξ i ≥0;

[0044] Random forest sets the number of decision trees and feature selection method, and node splitting is based on Gini impurity Select the optimal splitting features and splitting points;

[0045] The convolutional neural network consists of convolutional layers, pooling layers and fully connected layers. The convolution operation Y i，j =∑ m，n X i+m，j+n W m，n , the pooling layer uses the maximum pooling Z i，j =max m，n ∈[0,s-1]Y i×s+m，j×s+n , the fully connected layer outputs y=Wx+b.

[0046] The present invention significantly improves bird identification through multispectral image acquisition and the comprehensive utilization of information from different spectral bands. Whether distinguishing similar species or identifying birds against complex backgrounds, it can more accurately determine the species and individual information of birds, effectively improving the accuracy of bird monitoring. By quantifying each step through mathematical formulas, the image acquisition, processing, and identification processes are more precise and controllable.

[0047] Multispectral technology overcomes the limitations of a single spectrum in specific environments. It enables effective monitoring at night or in low light, as well as for birds with unique spectral characteristics, broadening the temporal and spatial scope of bird monitoring. Quantitative parameter adjustment and feature extraction methods help better capture and analyze bird characteristics in different environments.

[0048] The combination of an integrated classifier and a dynamic feedback optimization mechanism enables the image recognition system to continuously optimize based on actual monitoring conditions, adapting to varying monitoring environments and bird species. This improves the system's adaptability and reliability, providing strong support for long-term, efficient bird monitoring. Quantified accuracy assessment and feedback mechanisms enable timely identification and resolution of issues that arise during the recognition process, continuously improving recognition performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The present invention will be further described below with reference to the accompanying drawings.

[0050] Figure 1 The present invention is a flowchart of an image recognition method for multispectral bird monitoring. DETAILED DESCRIPTION

[0051] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0052] See also Figure 1 As shown, the present invention is an image recognition method for multispectral bird monitoring, comprising the following steps:

[0053] Using a multispectral camera equipped with a variety of spectral filters, we can take full-scale and multi-angle photos of the bird monitoring area to obtain image data covering multiple spectral bands such as visible light, near infrared, and short-wave infrared. Let the image captured by the camera in the i-th spectral band be I i (x, y), where (x, y) represents the coordinate position of the image pixel, and i = 1, 2, ..., n, where n is the number of spectral bands. During the acquisition process, camera parameters such as exposure time t and gain g are dynamically adjusted based on the activity patterns of different birds and the lighting conditions of the monitoring area. Exposure time t can be adjusted based on the ambient light intensity L and can be calculated using the following empirical formula: t = k / L, where k is a constant determined by camera performance and actual shooting requirements. Gain g can be adjusted based on the average brightness B of the image. Assuming the desired average brightness is B0, g = B0 / B.

[0054] Denoising: Adaptive median filtering algorithm is used to remove image noise. For each pixel in the image I(x,y), a filtering window W is defined with the pixel as the center. The window size is adaptively adjusted according to the noise situation. Let the median value of the pixel value in the window W be M and the minimum value be Z. min , the maximum value is Z max , the current pixel value is Z xy If Z min <M<Z max And Z min <Z xy <Z max , then the output pixel value Z out =Z xy Otherwise, increase the window size and recalculate the above values until the condition is met or the maximum window size is reached.

[0055] Contrast enhancement: Use the histogram normalization method to enhance the image contrast. Let the histogram of the original image be H r (k), k=0,1,...,255, the ideal histogram is specified as H g (k). First calculate the cumulative distribution function of the original image Specifies the cumulative distribution function of the histogram For each pixel value r in the original image, find a pixel value s such that |C r (r)-C g (s)|minimum, map the original pixel value r to s.

[0056] Image registration: Use a phase-correlation-based image registration algorithm to accurately align images of different spectral bands. For two images I1(x,y) and I2(x,y) of different spectral bands, first calculate their two-dimensional discrete Fourier transforms F1(u,v) and F2(u,v), where (u,v) are frequency domain coordinates. Then calculate the cross power spectrum. in is the conjugate of F2(u,v). Perform a two-dimensional inverse discrete Fourier transform on the cross power spectrum to obtain the correlation function r(x,y) in the spatial domain. The peak position (x0,y0) of the correlation function is the translation between the two images.

[0057] Visible light image feature extraction:

[0058] Texture feature (Local Binary Pattern - LBP): For each pixel in the image I (x, y), define a circular neighborhood with a radius of R and a number of neighborhood points P with the pixel as the center. Let the central pixel value be I c , the neighborhood pixel value is I p , p=0,1,...,P-1, then the LBP code is in:

[0059]

[0060] Color features (Principal Component Analysis - PCA): The RGB color space data of the image is represented as a matrix X, where each row represents the RGB value of a pixel. Calculate the covariance matrix of the matrix X Where N is the number of pixels. Perform eigenvalue decomposition on the covariance matrix C to obtain the eigenvalue λ i and the corresponding eigenvector v i , i = 1, 2, 3. Select the eigenvectors corresponding to the first k largest eigenvalues to form the projection matrix V, and project the original data X into the new feature space Y = XV.

[0061] Near-infrared image feature extraction:

[0062] Structural features (wavelet transform): The near-infrared image I(x, y) is subjected to a two-dimensional wavelet transform, which can be decomposed into an approximate component A, a horizontal detail component H, a vertical detail component V, and a diagonal detail component D. The wavelet transform can be expressed as where s∈{A,H,V,D}, is the wavelet basis function, j is the scale factor, and k is the translation factor.

[0063] Feather texture features (grayscale run length matrix): Let the grayscale level of the image be G. For each grayscale value i and run length j, count the number of runs n with grayscale value i and run length j in the four directions of horizontal, vertical, diagonal and anti-diagonal ij , and get the grayscale run length matrix GLRLM. Based on GLRLM, various texture features can be calculated, such as short run length emphasis Where N is the maximum run length.

[0064] Shortwave infrared image feature extraction: Use morphological operations to extract the outline features of birds. Let the image be I(x,y), the structural element be B(x,y), and the erosion operation be defined as:

[0065]

[0066] The expansion operation is defined as:

[0067]

[0068] The opening operation of first erosion and then expansion and the closing operation of first expansion and then erosion are used to remove noise and fill holes, thereby highlighting the edge contour of the bird.

[0069] Feature fusion: The extracted multi-spectral features are fused to construct a feature fusion model. First, the features of different spectral bands are normalized. Let a certain feature vector be f, and the minimum-maximum normalization method can be used: the normalized feature vector is where f min and f max are the minimum and maximum values in the feature vector f, respectively. Then, a deep belief network (DBN) is used for feature fusion. DBN is composed of multiple restricted Boltzmann machines (RBMs) stacked together. For the lth layer of RBM, its energy function is:

[0070]

[0071] Where v is the visible layer neuron state, h is the hidden layer neuron state, a and b are the biases of the visible and hidden layers, respectively, and w is the connection weight. By training the RBM layer by layer and adjusting the parameters a, b, and w, a more representative fused feature vector is generated.

[0072] Ensemble classifier construction: Construct an ensemble classifier consisting of support vector machine (SVM), random forest (RF) and convolutional neural network (CNN).

[0073] Support Vector Machine (SVM): For the binary classification problem, let the training sample be (x i ,y i ), i=1,2,...,N, where x i is the eigenvector, y i ∈{-1,1} is the category label. The goal of SVM is to find an optimal hyperplane w T x+b=0, so that the interval between the two types of samples is maximized. The optimization problem can be expressed as The constraint condition is y i (w T x i +b)≥1-ξ i ,ξ i ≥0, where C is the penalty parameter and ξi is the slack variable. Radial basis function (RBF) is used as the kernel function Where σ is the kernel function parameter.

[0074] Random Forest (RF): Random Forest consists of multiple decision trees. For each decision tree, a subset of features and a subset of samples are randomly selected for training. Let the node of the decision tree be t, and the Gini impurity of node t be where p k is the proportion of samples belonging to the kth class in node t, and K is the number of classes. When splitting a node, the feature and split point that maximize the reduction of Gini impurity are selected.

[0075] Convolutional Neural Network (CNN): CNN consists of convolutional layers, pooling layers, and fully connected layers. For the convolutional layer, let the input feature map be X, the convolution kernel be W, and the convolution operation can be expressed as Y i，j =∑ m，n X i+m，j+n W m，n , where Y is the output feature map. The pooling layer usually uses maximum pooling or average pooling. Taking maximum pooling as an example, assuming the pooling window size is s×s, the pooling operation Z i，j =max m，n∈[0，s-1] Y i×s+m，j×s+n The fully connected layer linearly combines the features extracted by the convolutional layer and the pooling layer. Let the input vector be x, the weight matrix be W, and the bias vector be b, then the output vector y = Wx + b.

[0076] Classifier training: The ensemble classifier is trained using a large-scale multispectral bird image dataset. The dataset is divided into training, validation, and test sets according to a certain ratio. During the training process, the transfer learning method is used to initialize the CNN using the model parameters pre-trained on the large-scale image dataset. At the same time, the stochastic gradient descent (SGD) algorithm is combined with the momentum factor to optimize the parameters of the ensemble classifier. Let the loss function be L(θ), where θ is the parameter of the classifier, and the update formula of the SGD algorithm is Where η is the learning rate and μ is the momentum factor.

[0077] Dynamic recognition: In actual monitoring, multispectral bird images are acquired in real time. After preprocessing and feature extraction, they are input into the trained ensemble classifier for recognition. Let the classification result of SVM be ysvm and the classification result of RF be y rf , the classification result of CNN is y cnn , the final bird identification result y is determined through a voting mechanism. If y svm =y rf =y cnn , then y=y svm ; If the results of two of the classifiers are the same, then y is the same result; if the results of the three classifiers are different, the judgment can be made based on the confidence of the classifier.

[0078] Feedback optimization: Establish a feedback optimization mechanism to compare and verify the recognition results with the field manual monitoring data and data obtained by other professional monitoring equipment. Let the recognition result be ypred and the actual result be y true , if y pred =y true, analyze the cause of the deviation. If feature extraction is incomplete, adjust the parameters of the feature extraction algorithm; if the classifier parameters are unreasonable, retrain the classifier. Let the classifier accuracy be Acc = (TP + TN) / (TP + TN + FP + FN), where TP is the number of true positives, TN is the number of true negatives, FP is the number of false positives, and FN is the number of false negatives. When the accuracy falls below the preset threshold, initiate the feedback optimization process.

[0079] The above is a detailed description of an embodiment of the present invention. However, the content described is only a preferred embodiment of the present invention and should not be considered to limit the scope of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. An image recognition method for multispectral bird monitoring, characterized in that: The following steps are involved: S1: Use a multispectral camera equipped with a variety of spectral filters to shoot bird monitoring areas from multiple angles to obtain image data in multiple spectral bands; and pre-process the collected image data; S2: Feature extraction is performed based on the spectral band type of the preprocessed image data; the extracted multispectral features are normalized and feature fusion is performed using a deep belief network; S3: Construct an ensemble classifier consisting of a support vector machine, a random forest, and a convolutional neural network. Train the ensemble classifier using the fused image dataset, initialize the convolutional neural network parameters using transfer learning, and optimize the classifier parameters using a stochastic gradient descent algorithm combined with momentum. S4: Multispectral bird images are acquired in real time. After preprocessing and feature extraction, they are input into the trained ensemble classifier for recognition. The final bird recognition result is determined based on the voting mechanism.

2. The image recognition method for multispectral bird monitoring according to claim 1, characterized in that: In S1, the specific steps are as follows: The image collected by the camera in the i-th spectral band is I i (x,y), (x,y) represents the coordinate position of the image pixel, i = 1, 2, ..., n, n is the number of spectral bands; During the acquisition process, the exposure time t is adjusted according to the ambient light intensity L according to the formula t = k / L, where k is the camera constant; The gain g is adjusted according to the average brightness B of the image and the expected average brightness B0 according to the formula g=B0 / B.

3. The image recognition method for multispectral bird monitoring according to claim 1, characterized in that: In S1, preprocessing the collected image data specifically includes: Adaptive median filtering algorithm is used for denoising. For each pixel point in the image I(x,y), a filtering window W is defined with the point as the center, and the median value M and minimum value Z of the pixel values in the window are used to filter the image. min , maximum value Z max and the current pixel value Z xy determining output pixel values; The contrast is enhanced by using the histogram normalization method and calculating the cumulative distribution function of the original image. and the cumulative distribution function of the prescribed histogram Map the original pixel value r to s so that |C r (r)-C g (s) | minimum; Using the phase correlation-based image registration algorithm, the two-dimensional discrete Fourier transform F1(u,v) and F2(u,v) of two images I1(x,y) and I2(x,y) in different spectral bands are calculated to obtain the cross power spectrum. The correlation function r(x,y) is obtained by performing a two-dimensional discrete Fourier inverse transform on the cross power spectrum, and its peak position (x0,y0) is the translation between the two images.

4. The image recognition method for multispectral bird monitoring according to claim 1, characterized in that: In S2, the specific steps of feature extraction are as follows: Visible light image feature extraction: Using the local binary pattern algorithm, for each pixel in the image I(x,y), a circular neighborhood with a radius of R and a number of neighborhood points P is defined with the pixel as the center, and the LBP code is calculated: in: Using the principal component analysis algorithm, the RGB color space data of the image is represented as a matrix X and the covariance matrix is calculated Perform eigenvalue decomposition on C, select the eigenvectors corresponding to the first k largest eigenvalues to form the projection matrix V, and project the original data X into the new feature space Y = XV; Near-infrared image feature extraction: Using the wavelet transform algorithm, the near-infrared image I (x, y) is subjected to a two-dimensional wavelet transform and decomposed into the approximate component A, the horizontal detail component H, the vertical detail component V and the diagonal detail component D. The wavelet transform represents s∈{A,H,V,D}; through the grayscale run length matrix, count the number of grayscale runs of different directions and lengths n ij Get the GLRLM and calculate the short-run emphasis: Short-wave infrared image feature extraction: Using morphological operations, perform corrosion operations on the image I(x,y): Expansion operation: The edge contours of the bird are highlighted through opening and closing operations.

5. The image recognition method for multispectral bird monitoring according to claim 4, characterized in that: In S2, the specific steps of feature fusion are as follows: The features of different spectral bands are normalized using the minimum-maximum normalization method. Let a certain feature vector be f, and the normalized feature vector is: where f min and f max are the minimum and maximum values in the eigenvector f, respectively; A deep belief network is used for feature fusion. The deep belief network is composed of multiple restricted Boltzmann machines stacked together. For the l-th layer RBM, its energy function is: By training RBM layer by layer, the parameters a, b and w are adjusted to generate a fused feature vector.

6. The image recognition method for multispectral bird monitoring according to claim 1, characterized in that: In the S3, the specific steps are as follows: The support vector machine uses the radial basis function as the kernel function: The optimization problem is: The constraints are: i (w T x i +b)≥1-ξ i ,ξ i ≥0; Random forest sets the number of decision trees and feature selection method, and node splitting is based on Gini impurity Select the optimal splitting features and splitting points; The convolutional neural network consists of convolutional layers, pooling layers and fully connected layers. The convolution operation Y i,j =Σ m,n X i+m,j+n W m,n The pooling layer uses the maximum pooling Z i,j =max m,n∈[0,s-1] Y i×s+m,j×s+n The fully connected layer outputs y=Wx+b.

Images