Intelligent bird species identification method and system based on image recognition

Abstract

The present invention relates to the technical field of bird species identification, and discloses an intelligent bird species identification method and system based on image recognition. By combining image features, behavioral features, time and geographical location information, the accuracy of bird species identification is improved. By introducing a spatiotemporal information fusion model, the dynamic behavior pattern of birds is organically combined with environmental information, and the species of birds can be accurately determined under different lighting, angles and partial occlusion conditions. In addition, by using dynamically weighted spatiotemporal feature vectors for database matching, the identification process is more adapted to environmental conditions such as bird migration periods and habitats, achieving higher identification accuracy and environmental adaptability.

Description

Technical Field

[0001] The present invention relates to the technical field of bird species identification, and in particular to an intelligent bird species identification method and system based on image recognition. Background Art

[0002] During the operation of power and communication facilities, bird activities often cause unexpected damage, so in some scenarios, birds need to be expelled to protect critical equipment. In the expulsion process, it is crucial to correctly identify the species of birds, because different species of birds react differently to expulsion signals, and some birds may require special protection measures. However, the bird identification system in the prior art has significant deficiencies in accuracy, especially in complex natural environments, where it is impossible to reliably distinguish between rare birds and other common birds, thereby increasing the risk of misidentification and improper expulsion.

[0003] At present, there are relatively mature studies on the technology of identifying bird species based on images, but there are still some technical bottlenecks in the accuracy of the recognition results. First, changes in ambient lighting and differences in shooting angles make it difficult to extract image features consistently, especially in situations where birds are in flight or partially obscured, the recognition accuracy is greatly reduced. Secondly, since there are significant differences in the migration or activity habits of birds at different times and regions, the extraction of a single image feature cannot reflect these dynamic characteristics, resulting in insufficient generalization capabilities of existing systems. Therefore, existing recognition systems also lack effective use of bird behavioral characteristics, which further limits the results of accurately identifying different bird species in actual environments. Summary of the invention

[0004] In view of this, an object of the present invention is to provide an intelligent bird species identification method and system based on image recognition, so as to solve the problem that the current recognition results of bird species identification based on images are not accurate.

[0005] The first aspect of the present invention discloses an intelligent bird species identification method based on image recognition, the method comprising:

[0006] S1, obtaining image data of target birds;

[0007] S2, extracting the behavior characteristics, image characteristics, geographical location information of the target bird and image shooting time information of the target bird based on the acquired image data of the target bird; wherein the behavior characteristics include flight mode characteristics and perching posture characteristics of the bird;

[0008] S3, combining the behavior characteristics of the target bird with the image characteristics to generate a feature vector including the image information and the behavior information of the target bird as a first feature vector;

[0009] S4, inputting the first feature vector, the geographical location information of the target bird and the image shooting time information into the spatiotemporal information fusion model, and encoding them into a spatiotemporal feature vector through the spatiotemporal information fusion model;

[0010] S5. Match the spatiotemporal feature vector with a bird species database, and output a target bird species recognition result.

[0011] Furthermore, the extraction operation of the behavior feature in step S2 includes the following sub-steps:

[0012] S2101, detecting and recording position change information of the target bird in the continuous image frames and timestamp information corresponding to the position based on the continuous image frames in the acquired image data of the target bird;

[0013] S2102, generating a flight trajectory of the target bird according to the position change information and the timestamp information corresponding to the position; wherein the flight trajectory includes the timestamp information corresponding to the position points constituting the flight trajectory;

[0014] S2103, calculating the speed and acceleration of the target bird according to the flight trajectory;

[0015] S2104, analyzing the flight direction of the target bird according to the position points constituting the flight trajectory and the corresponding timestamp information, and determining the flight direction change information of the target bird; the flight direction change information includes the change direction of the flight direction, the corresponding timestamp information when the flight direction changes, the flight direction change frequency, and the flight direction change amplitude;

[0016] S2105: Generate a behavior characteristic time series of the target bird based on the flight trajectory, speed and acceleration, and flight direction change information, and extract flight pattern characteristics of the target bird according to the behavior characteristic time series.

[0017] Furthermore, the extraction operation of the behavior feature in step S2 further includes the following sub-steps:

[0018] S2106, determining a time period when the target bird is at a low speed and / or relatively stationary state based on the behavior characteristic time series;

[0019] S2107, determining a static image corresponding to the time period, and extracting perching posture features of the target bird in the static image; the perching posture features include the relative stillness, standing posture, head direction, and body direction of the bird.

[0020] Furthermore, the image features include shape features, color features, texture features and space features.

[0021] Furthermore, the image feature extraction operation in step S2 includes the following sub-steps:

[0022] S2201, performing a preprocessing operation on the image data of the target bird;

[0023] S2202, using a convolutional neural network (CNN) as a feature extraction network, inputting the preprocessed image into an input layer of the feature extraction network, and extracting low-level features;

[0024] S2203. Introduce a multi-scale pyramid structure in CNN to extract fine-grained edge features and texture features from the shallow layer, medium-scale features from the middle layer, and global shape features from the deep layer;

[0025] S2204, fusing the features extracted from different levels to obtain image features.

[0026] Furthermore, the process of encoding into a spatiotemporal feature vector by the spatiotemporal information fusion model in step S4 includes the following sub-steps:

[0027] S401, converting the geographical location information of the target bird into a coordinate vector;

[0028] S402, classify and encode the image shooting time information, divide the time information into discrete time periods based on time categories, and convert the discrete time periods including the time category information into one-hot encoding; wherein the time categories include early morning, daytime, evening, spring, and autumn;

[0029] S403, combining the geographical location information and time information processed in step S401 and step S402 with the first feature vector to form a spatiotemporal input vector;

[0030] S404, inputting the spatiotemporal input vector into the spatiotemporal coding layer of the spatiotemporal information fusion model; the spatiotemporal coding layer includes a multi-layer perceptron and a self-attention mechanism;

[0031] S405. Output the spatiotemporal feature vector through the spatiotemporal coding layer.

[0032] Furthermore, after the space-time input vector is input into the space-time coding layer of the space-time information fusion model in step S404, the following sub-steps are also included:

[0033] S4051, performing nonlinear mapping of the relationship between geographic location information, time information, image features, and behavior features through a multi-layer perceptron;

[0034] S4052. Learn the interactive relationship between geographic location information, time information, image features, and behavioral features through the self-attention mechanism.

[0035] Furthermore, before matching the spatiotemporal feature vector with the bird species database, the method further includes:

[0036] Determine the current environmental conditions of the target birds based on the time information and geographic location information in the spatiotemporal feature vector; the environmental conditions include migration period and suitable habitat;

[0037] The weights of the time and geographic location information in the spatiotemporal feature vector are adjusted according to the determined environmental conditions to obtain a weighted spatiotemporal feature vector.

[0038] Furthermore, the step S5 includes the following sub-steps:

[0039] S501, calculating the similarity value between the weighted spatiotemporal feature vector and each bird in the bird species database, and sorting the similarity values;

[0040] S502: Calculate the confidence based on the highest similarity value, and when the confidence is greater than or equal to a preset threshold, output the bird corresponding to the highest similarity value as the recognition result.

[0041] The second aspect of the present invention discloses an intelligent bird species identification system based on image recognition, which is implemented based on the method disclosed in the first aspect. The system includes an image acquisition module, an information extraction module, an information processing module, and an identification module; wherein:

[0042] The image acquisition module is used to obtain image data of target birds;

[0043] The information extraction module is used to extract the behavior characteristics, image characteristics, geographical location information of the target bird and image shooting time information of the target bird based on the acquired image data of the target bird; wherein the behavior characteristics include the flight mode characteristics and the perching posture characteristics of the bird;

[0044] The information processing module is used to combine the behavior characteristics of the target bird with the image characteristics to generate a feature vector including the image information and the behavior information of the target bird as a first feature vector;

[0045] The information processing module is also used to input the first feature vector, the geographical location information of the target bird and the image shooting time information into the spatiotemporal information fusion model, and encode them into a spatiotemporal feature vector through the spatiotemporal information fusion model;

[0046] The recognition module is used to match the spatiotemporal feature vector with a bird species database and output a target bird species recognition result.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] The present invention improves the accuracy of bird species recognition by combining image features, behavioral features, time and geographic location information. By introducing a spatiotemporal information fusion model, the dynamic behavior patterns of birds are organically combined with environmental information, and the species of birds can be accurately determined under different lighting, angles and partial occlusion conditions. In addition, the use of dynamically weighted spatiotemporal feature vectors for database matching makes the recognition process more adaptable to environmental conditions such as bird migration periods and habitats, achieving higher recognition accuracy and environmental adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of the economic application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:

[0050] Figure 1 A schematic diagram of a flow chart of an intelligent bird species identification method based on image recognition disclosed in an embodiment of the present invention;

[0051] Figure 2 This is a schematic diagram of the structure of an intelligent bird species identification system based on image recognition disclosed in yet another embodiment of the present invention. DETAILED DESCRIPTION

[0052] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Example 1

[0053] The first aspect of the present invention discloses an intelligent bird species identification method based on image recognition, see Figure 1 , Figure 1 The present invention discloses an intelligent bird species identification method based on image recognition, which includes the following steps:

[0054] S1, obtaining image data of target birds;

[0055] S2. Extracting the behavior characteristics, image characteristics, geographical location information of the target bird and image shooting time information of the target bird based on the acquired image data of the target bird. The behavior characteristics include but are not limited to the flight mode characteristics and perching posture characteristics of the bird.

[0056] S3, combining the behavior characteristics of the target bird with the image characteristics to generate a feature vector including the image information and the behavior information of the target bird as a first feature vector;

[0057] S4, inputting the first feature vector, the geographical location information of the target bird and the image shooting time information into the spatiotemporal information fusion model, and encoding them into a spatiotemporal feature vector through the spatiotemporal information fusion model;

[0058] S5. Match the spatiotemporal feature vector with a bird species database, and output a target bird species recognition result.

[0059] In the present invention, the process of obtaining the image data of the target bird includes but is not limited to photographing the bird by using an unmanned aerial vehicle camera or a fixed camera. Preferably, the camera in the embodiment of the present invention is arranged in an area where the birds need to be driven away.

[0060] Furthermore, the extraction operation of the behavior feature in step S2 includes the following sub-steps:

[0061] S2101, detecting and recording position change information of the target bird in the continuous image frames and timestamp information corresponding to the position based on the continuous image frames in the acquired image data of the target bird;

[0062] S2102, generating a flight trajectory of the target bird according to the position change information and the timestamp information corresponding to the position; wherein the flight trajectory includes the timestamp information corresponding to the position points constituting the flight trajectory;

[0063] S2103, calculating the speed and acceleration of the target bird according to the flight trajectory;

[0064] S2104, analyzing the flight direction of the target bird according to the position points constituting the flight trajectory and the corresponding timestamp information, and determining the flight direction change information of the target bird; wherein the flight direction change information includes the change direction of the flight direction, the timestamp information corresponding to the change of the flight direction, the frequency of the flight direction change, and the magnitude of the flight direction change;

[0065] S2105: Generate a behavior characteristic time series of the target bird based on the flight trajectory, speed and acceleration, and flight direction change information, and extract the flight mode characteristics of the target bird according to the behavior characteristic time series.

[0066] Specifically, in the embodiment of the present invention, the flight mode feature is used to represent the movement habits of birds in the air, including speed, acceleration, flight direction, trajectory changes, etc. Among them, speed and acceleration can reflect the rhythm and stability of the bird's flight, and the main purpose of analyzing the change of the bird's flight direction is to analyze the current specific behavior of the bird, such as some birds will frequently change direction when hunting, but will maintain a relatively stable direction when migrating; the flight trajectory is mainly used to reflect the bird's movement path and flight style, such as straight line, wave type, etc.

[0067] By extracting the flight pattern characteristics of birds, the current specific behaviors of birds, such as flying, chasing with companions, foraging, migration, etc., can be determined based on the information in the flight pattern characteristics, and the movement habits of birds can be analyzed based on these behaviors, which is conducive to the subsequent identification of bird species.

[0068] Furthermore, the behavior feature extraction operation in step S2 further includes the following sub-steps:

[0069] S2106, determining a time period when the target bird is at a low speed and / or relatively stationary state based on the behavioral characteristic time series;

[0070] S2107, determining a static image corresponding to a time period, and extracting perching posture features of the target bird in the static image; the perching posture features include but are not limited to the relative stillness, standing posture, head direction, and body direction of the bird.

[0071] In an embodiment of the present invention, the perching posture feature is used to represent the posture of a bird at rest or at a low speed, including the bird's relative resting state, standing posture, head and body direction, etc. Therefore, it can be understood that when determining the perching posture features of a target bird, the present invention first determines the time period when the target bird is at a low speed and / or relatively still state, and then determines the image in this time period, and extracts the perching posture features of the target bird by detecting the posture of the bird in these images. Among them, the relative still state refers to the action of the bird at a low speed or resting state relative to the bird's relatively active and fast flying speed such as continuous flight or predation, such as spreading wings, standing on one foot, etc. Analyzing the perching posture features of birds is also conducive to the identification of bird species.

[0072] Furthermore, image features include shape features, color features, texture features and space features.

[0073] Specifically, shape features can help distinguish the outline and overall posture of birds, and are suitable for identifying birds of similar size and general outline; color features can use differences in feather color to distinguish species; texture features use detailed texture to distinguish subtle differences in feathers or other features; spatial features capture the local and overall structure of birds through information at different scales.

[0074] Furthermore, the image feature extraction operation in step S2 includes the following sub-steps:

[0075] S2201, performing a preprocessing operation on the image data of the target bird;

[0076] S2202, using a convolutional neural network (CNN) as a feature extraction network, inputting the preprocessed image into an input layer of the feature extraction network, and extracting low-level features;

[0077] S2203. Introduce a multi-scale pyramid structure in CNN to extract fine-grained edge features and texture features from the shallow layer, medium-scale features from the middle layer, and global shape features from the deep layer;

[0078] S2204, fusing the features extracted from different levels to obtain image features.

[0079] Since bird images are complex and diverse, and are affected by different environmental conditions (such as lighting, angles, and partial occlusion) during the recognition process, more comprehensive multi-level processing is required when extracting image features. In an embodiment of the present invention, a multi-scale pyramid structure is introduced into CNN to obtain multi-dimensional information of the image from different levels: the shallow layer focuses on edge and texture features to help identify bird feathers and fine textures; the middle layer focuses on mesoscale information, such as the partial shape of the bird's body structure; the deep layer extracts global shape features to reflect the overall outline and posture of the bird. Through this hierarchical feature extraction structure, the details and overall features of the bird can be captured even in a complex environment, ensuring accurate judgment of the species. In addition, by fusing the features of each level to form a comprehensive feature vector containing multiple information such as shape, color, and texture, the model can comprehensively analyze the characteristics of bird images from details to the whole and from multiple dimensions. Not only does it improve the accuracy and environmental adaptability of recognition, but it also enhances the ability to distinguish the subtle differences between different bird species.

[0080] Furthermore, the process of encoding into a spatiotemporal feature vector by the spatiotemporal information fusion model in step S4 includes the following sub-steps:

[0081] S401, converting the geographical location information of the target bird into a coordinate vector;

[0082] S402, classify and encode the image shooting time information, divide the time information into discrete time periods based on time categories, and convert the discrete time periods including the time category information into one-hot encoding; wherein the time categories include early morning, daytime, evening, spring, and autumn;

[0083] S403, combining the geographical location information and time information processed in step S401 and step S402 with the first feature vector to form a spatiotemporal input vector;

[0084] S404, inputting the spatiotemporal input vector into the spatiotemporal coding layer of the spatiotemporal information fusion model; the spatiotemporal coding layer includes a multi-layer perceptron and a self-attention mechanism;

[0085] S405. Output the spatiotemporal feature vector through the spatiotemporal coding layer.

[0086] Furthermore, after the spatiotemporal input vector is input into the spatiotemporal coding layer of the spatiotemporal information fusion model in step S404, the following sub-steps are also included:

[0087] S4051, performing nonlinear mapping of the relationship between geographic location information, time information, image features, and behavior features through a multi-layer perceptron;

[0088] S4052. Learn the interactive relationship between geographic location information, time information, image features, and behavioral features through the self-attention mechanism.

[0089] In an embodiment of the present invention, the purpose of adding spatiotemporal information is to more accurately identify the species of birds, because the activities and appearance characteristics of birds will show obvious regularity due to changes in time and geographical location. By fusing image features and behavioral features with spatiotemporal information, the characteristic differences of birds in different environments can be captured. For example, some birds only appear in specific seasons or geographical areas, or show unique behavioral patterns in specific time periods. The multi-layer perceptron can identify the complex associations between different features during the fusion process, and the self-attention mechanism helps the model automatically focus on the most relevant information in the current recognition scenario. The final generated spatiotemporal feature vector combines the visual, behavioral, temporal and geographical characteristics of birds, providing multi-dimensional data support for accurate species judgment, ensuring that the system can more accurately identify and distinguish different bird species based on the regularity of spatiotemporal features in complex natural environments.

[0090] Preferably, the construction process of the spatiotemporal information fusion model includes: first, setting the first feature vector including image information and behavior information as , where R dimg is the dimension of the first eigenvector; set the geographic location information to a two-dimensional coordinate vector ,in, Indicates latitude, Represents longitude; converts time information into a vector through one-hot encoding ,in, The dimension of the time category.

[0091] Then the space-time input vector is expressed as:

[0092] ;

[0093] In order to capture the more complex relationship between features, the spatiotemporal input vector is mapped to a high-dimensional space through a multi-layer perceptron MLP. Assuming that the MLP contains L layers, the mapping formula for each layer is:

[0094] ;

[0095] ;

[0096] in, is the input vector of MLP, i.e., the spatiotemporal input vector; For MLP The nonlinearly transformed output vector of the layer; For the The weight matrix of the layer; For the The bias vector of the layer; is a nonlinear activation function. Finally, the vector obtained after MLP Represents spatiotemporal features that have been mapped to a high-dimensional space.

[0097] right Generate query, key, and value vectors:

[0098] ;

[0099] ;

[0100] V=W v .f MLP ;

[0101] in, is the query vector; K is the key vector; is a value vector; , , Generate weight matrices for queries, keys, and values, respectively.

[0102] Calculate the attention score:

[0103] ;

[0104] ;

[0105] ;

[0106] in, is the dot product of the query vector and the key. The dot product operation here is used to calculate the similarity between the query vector and the key vector. The higher the similarity, the stronger the correlation between the features.

[0107] is the dimension of query vector and key vector; is the attention weight matrix; is the output of the self-attention mechanism, that is, the spatiotemporal feature vector.

[0108] Furthermore, before matching the spatiotemporal feature vector with the bird species database, the method further includes:

[0109] Determine the current environmental conditions of the target birds based on the time information and geographic location information in the spatiotemporal feature vector; the environmental conditions include migration period and suitable habitat;

[0110] The weights of the time and geographic location information in the spatiotemporal feature vector are adjusted according to the determined environmental conditions to obtain a weighted spatiotemporal feature vector.

[0111] Specifically, time and geographic location information is extracted from the spatiotemporal feature vector to determine the environmental conditions of the current target bird, such as whether it is in the migration period or a suitable habitat. This environmental judgment allows for better adaptation to the behavioral patterns of birds during the identification process. For example, during the migration period, the weight of time information is increased to reflect the impact of seasonality on bird activities; in a suitable habitat, the weight of geographic location information is enhanced to emphasize the specific distribution characteristics of birds in that geographic area. By dynamically adjusting the weight of time and geographic information in the spatiotemporal feature vector and generating a weighted spatiotemporal feature vector, the impact of environmental factors can be more accurately considered when identifying bird species, effectively improving recognition accuracy, which is especially important in environments with strong seasonal migration or habitat specificity.

[0112] Furthermore, step S5 includes the following sub-steps:

[0113] S501, calculating the similarity value between the weighted spatiotemporal feature vector and each bird in the bird species database, and sorting the similarity values;

[0114] S502: Calculate the confidence based on the highest similarity value, and when the confidence is greater than or equal to a preset threshold, output the bird corresponding to the highest similarity value as the recognition result.

[0115] Specifically, in order to identify the species of the target bird, in the embodiment of the present invention, the weighted spatiotemporal feature vector is firstly similarly calculated with the feature vector of each bird in the bird species database. It can be understood that the bird species database contains a large amount of feature information of different birds, including visual features, behavior patterns, geographical distribution and time regularity. These feature vectors in the database are obtained by pre-training and optimizing a large amount of annotated data, and can fully reflect the unique characteristics of different types of birds.

[0116] Example 2

[0117] The second aspect of the present invention discloses an intelligent bird species identification system based on image recognition, see Figure 2 , Figure 2 : is a structural diagram of an intelligent bird species identification system based on image recognition disclosed in another embodiment of the present invention, the system includes an image acquisition module, an information extraction module, an information processing module, and an identification module; wherein,

[0118] The image acquisition module is used to obtain image data of target birds;

[0119] The information extraction module is used to extract the behavior characteristics, image characteristics, geographical location information of the target bird and image shooting time information of the target bird based on the acquired image data of the target bird; wherein the behavior characteristics include the flight mode characteristics and the perching posture characteristics of the bird;

[0120] The information processing module is used to combine the behavior characteristics of the target bird with the image characteristics to generate a feature vector including the image information and the behavior information of the target bird as a first feature vector;

[0121] The information processing module is also used to input the first feature vector, the geographical location information of the target bird and the image shooting time information into the spatiotemporal information fusion model, and encode them into a spatiotemporal feature vector through the spatiotemporal information fusion model;

[0122] The recognition module is used to match the spatiotemporal feature vector with a bird species database and output a target bird species recognition result.

[0123] Furthermore, the information extraction module includes a behavior feature extraction submodule and an image feature extraction submodule.

[0124] Furthermore, the behavior feature extraction submodule is used to perform the following operations:

[0125] Based on the continuous image frames in the acquired image data of the target bird, the position change information of the target bird in the continuous image frames and the time stamp information corresponding to the position are detected and recorded;

[0126] Generate a flight trajectory of the target bird according to the position change information and the timestamp information corresponding to the position; wherein the flight trajectory includes the timestamp information corresponding to the position points constituting the flight trajectory;

[0127] calculating the speed and acceleration of the target bird according to the flight trajectory;

[0128] Analyze the flight direction of the target bird according to the position points constituting the flight trajectory and the corresponding timestamp information, and determine the flight direction change information of the target bird; the flight direction change information includes the change direction of the flight direction, the corresponding timestamp information when the flight direction changes, the flight direction change frequency and the flight direction change amplitude;

[0129] A behavior characteristic time series of the target bird is generated based on the flight trajectory, speed and acceleration, and flight direction change information, and flight mode characteristics of the target bird are extracted according to the behavior characteristic time series.

[0130] Determine the time period when the target bird is at a low speed and / or relatively still state based on the behavioral characteristic time series;

[0131] A static image corresponding to the time period is determined, and the perching posture features of the target bird in the static image are extracted; the perching posture features include the relative stillness, standing posture, head direction, and body direction of the bird.

[0132] Furthermore, the image feature extraction submodule is used to perform the following operations:

[0133] Performing preprocessing operations on the image data of the target bird;

[0134] Convolutional neural network (CNN) is used as the feature extraction network. The preprocessed image is input into the input layer of the feature extraction network to extract low-level features.

[0135] A multi-scale pyramid structure is introduced into CNN to extract fine-grained edge features and texture features from the shallow layer, medium-scale features from the middle layer, and global shape features from the deep layer.

[0136] The features extracted from different levels are fused to obtain image features, which include shape features, color features, texture features, and space features.

[0137] Furthermore, the information processing module includes a feature fusion submodule, which is used to perform the following operations:

[0138] Convert the geographical location information of the target bird into a coordinate vector;

[0139] Classify and encode the image shooting time information, divide the time information into discrete time periods based on time categories, and convert the discrete time periods including the time category information into one-hot encoding; wherein the time categories include early morning, daytime, evening, spring, and autumn;

[0140] The processed geographic location information and time information are combined with the first feature vector to form a spatiotemporal input vector;

[0141] Inputting the spatiotemporal input vector into the spatiotemporal coding layer of the spatiotemporal information fusion model; the spatiotemporal coding layer includes a multi-layer perceptron and a self-attention mechanism;

[0142] The spatiotemporal feature vector is output through the spatiotemporal encoding layer.

[0143] Furthermore, after the spatiotemporal input vector is input into the spatiotemporal coding layer of the spatiotemporal information fusion model, the following operations are also included:

[0144] The relationship between geographic location information, time information, image features, and behavioral features is nonlinearly mapped through a multi-layer perceptron;

[0145] The interactive relationship between geographic location information, time information, image features, and behavioral features is learned through the self-attention mechanism.

[0146] Furthermore, the information processing module also includes a weight adjustment submodule. The weight adjustment submodule is used to perform the following operations:

[0147] Determine the current environmental conditions of the target birds based on the time information and geographic location information in the spatiotemporal feature vector; the environmental conditions include migration period and suitable habitat;

[0148] The weights of the time and geographic location information in the spatiotemporal feature vector are adjusted according to the determined environmental conditions to obtain a weighted spatiotemporal feature vector.

[0149] Furthermore, the recognition module also includes a calculation submodule and a comparison submodule.

[0150] The calculation submodule is used to calculate the similarity value between the weighted spatiotemporal feature vector and each bird in the bird species database, and to sort the similarity values.

[0151] In addition, the calculation submodule is also used to calculate the confidence based on the highest similarity value. After the calculation submodule calculates the confidence, the comparison submodule outputs the bird corresponding to the highest similarity value as the recognition result when the confidence is greater than or equal to a preset threshold.

[0152] It should be noted that the specific implementation process of the second embodiment is similar to that of the first embodiment and will not be repeated in the second embodiment.

[0153] Finally, it should be noted that the intelligent bird species identification method and system based on image recognition disclosed in the embodiments of the present invention only discloses the preferred embodiments of the present invention and is only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, it should be understood by those skilled in the art that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features therein may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An intelligent bird species identification method based on image recognition, characterized in that: The method comprises the following steps: S1, obtaining image data of target birds; S2, extracting the behavior characteristics, image characteristics, geographical location information of the target bird and image shooting time information of the target bird based on the acquired image data of the target bird; wherein the behavior characteristics include flight mode characteristics and perching posture characteristics of the bird; S3, combining the behavior characteristics of the target bird with the image characteristics to generate a feature vector including the image information and the behavior information of the target bird as a first feature vector; S4, inputting the first feature vector, the geographical location information of the target bird and the image shooting time information into the spatiotemporal information fusion model, and encoding them into a spatiotemporal feature vector through the spatiotemporal information fusion model; S5, matching the spatiotemporal feature vector with a bird species database, and outputting a target bird species recognition result; The extraction operation of the behavior feature in step S2 includes the following sub-steps: S2101, detecting and recording position change information of the target bird in the continuous image frames and timestamp information corresponding to the position based on the continuous image frames in the acquired image data of the target bird; S2102, generating a flight trajectory of the target bird according to the position change information and the timestamp information corresponding to the position; wherein the flight trajectory includes the timestamp information corresponding to the position points constituting the flight trajectory; S2103, calculating the speed and acceleration of the target bird according to the flight trajectory; S2104, analyzing the flight direction of the target bird according to the position points constituting the flight trajectory and the corresponding timestamp information, and determining the flight direction change information of the target bird; the flight direction change information includes the change direction of the flight direction, the corresponding timestamp information when the flight direction changes, the flight direction change frequency, and the flight direction change amplitude; S2105, generating a behavior characteristic time series of the target bird based on the flight trajectory, speed and acceleration, and flight direction change information, and extracting flight mode characteristics of the target bird according to the behavior characteristic time series; The extraction operation of the behavior feature in step S2 further includes the following sub-steps: S2106, determining a time period when the target bird is at a low speed and / or relatively stationary state based on the behavior characteristic time series; S2107, determining a static image corresponding to the time period, and extracting perching posture features of the target bird in the static image; the perching posture features include the relative stillness, standing posture, head direction, and body direction of the bird.

2. The intelligent bird species identification method based on image recognition according to claim 1 is characterized in that: The image features include shape features, color features, texture features and space features.

3. The intelligent bird species identification method based on image recognition according to claim 2 is characterized in that: The image feature extraction operation in step S2 includes the following sub-steps: S2201, performing a preprocessing operation on the image data of the target bird; S2202, using a convolutional neural network (CNN) as a feature extraction network, inputting the preprocessed image into an input layer of the feature extraction network, and extracting low-level features; S2203. Introduce a multi-scale pyramid structure in CNN to extract fine-grained edge features and texture features from the shallow layer, medium-scale features from the middle layer, and global shape features from the deep layer; S2204, fusing the features extracted from different levels to obtain image features.

4. The intelligent bird species identification method based on image recognition according to claim 1 is characterized in that: The process of encoding into a spatiotemporal feature vector by the spatiotemporal information fusion model in step S4 includes the following sub-steps: S401, converting the geographical location information of the target bird into a coordinate vector; S402, classify and encode the image shooting time information, divide the time information into discrete time periods based on time categories, and convert the discrete time periods including the time category information into one-hot encoding; wherein the time categories include early morning, daytime, evening, spring, and autumn; S403, combining the geographical location information and time information processed in step S401 and step S402 with the first feature vector to form a spatiotemporal input vector; S404, inputting the spatiotemporal input vector into the spatiotemporal coding layer of the spatiotemporal information fusion model; the spatiotemporal coding layer includes a multi-layer perceptron and a self-attention mechanism; S405. Output the spatiotemporal feature vector through the spatiotemporal coding layer.

5. The intelligent bird species identification method based on image recognition according to claim 4 is characterized in that: After the spatiotemporal input vector is input into the spatiotemporal coding layer of the spatiotemporal information fusion model in step S404, the following sub-steps are further included: S4051, performing nonlinear mapping of the relationship between geographic location information, time information, image features, and behavior features through a multi-layer perceptron; S4052. Learn the interactive relationship between geographic location information, time information, image features, and behavioral features through the self-attention mechanism.

6. The intelligent bird species identification method based on image recognition according to claim 5 is characterized in that: Before matching the spatiotemporal feature vector with the bird species database, the method further includes: Determine the current environmental conditions of the target birds based on the time information and geographic location information in the spatiotemporal feature vector; the environmental conditions include migration period and suitable habitat; The weights of the time and geographic location information in the spatiotemporal feature vector are adjusted according to the determined environmental conditions to obtain a weighted spatiotemporal feature vector.

7. The intelligent bird species identification method based on image recognition according to claim 6 is characterized in that: The step S5 comprises the following sub-steps: S501, calculating the similarity value between the weighted spatiotemporal feature vector and each bird in the bird species database, and sorting the similarity values; S502: Calculate the confidence based on the highest similarity value, and when the confidence is greater than or equal to a preset threshold, output the bird corresponding to the highest similarity value as the recognition result.

8. An intelligent bird species identification system based on image recognition, the system is implemented based on the method described in any one of claims 1 to 7, characterized in that: The system includes an image acquisition module, an information extraction module, an information processing module, and a recognition module; wherein, The image acquisition module is used to obtain image data of target birds; The information extraction module is used to extract the behavior characteristics, image characteristics, geographical location information of the target bird and image shooting time information of the target bird based on the acquired image data of the target bird; wherein the behavior characteristics include the flight mode characteristics and the perching posture characteristics of the bird; The information processing module is used to combine the behavior characteristics of the target bird with the image characteristics to generate a feature vector including the image information and the behavior information of the target bird as a first feature vector; The information processing module is also used to input the first feature vector, the geographical location information of the target bird and the image shooting time information into the spatiotemporal information fusion model, and encode them into a spatiotemporal feature vector through the spatiotemporal information fusion model; The recognition module is used to match the spatiotemporal feature vector with a bird species database and output a target bird species recognition result.

Images