Bird identification method and device based on key features

Abstract

The invention provides a bird recognition method and device based on key features, belongs to the technical field of bird recognition, and can improve the recognition accuracy of various similar or same-family birds with similar features. According to the method, a plurality of feature groups including block features and key features of a plurality of blocks of birds are extracted from a bird image to be recognized according to different feature detection sizes, and then the recognition result of each feature in the bird image to be recognized is obtained according to the recognition result of each feature in each feature group. And finally, determining a bird species identification result in the bird image to be identified according to the identification result of each feature in the bird image to be identified.

Description

Technical Field

[0001] The present invention relates to the technical field of bird identification, and in particular to a method and device for bird identification based on key features. Background Art

[0002] Intelligent bird recognition is the most classic fine-grained visual categorization (FGVC) task. Currently, existing intelligent bird recognition methods can identify birds based on the characteristics of local areas to improve the accuracy of bird recognition.

[0003] However, most of the current methods only consider partial areas within the image and ignore the details composed of multiple features in the bird image. Therefore, for multiple birds of similar or the same family with similar features, the recognition accuracy of existing recognition methods is low. Summary of the invention

[0004] The present invention proposes a method and device for bird identification based on key features, which identifies the block features and key features (features unique to birds) of birds, and determines the bird identification results based on the identification results of the block features and the key features, thereby improving the identification accuracy of multiple birds of similar or same families and genera with similar features.

[0005] In order to achieve the above object, the present invention adopts the following technical solution:

[0006] In the first aspect, the present invention provides a method for bird identification based on key features, comprising: extracting features from a bird image to be identified according to a plurality of different feature detection sizes, and obtaining a plurality of feature groups of the bird in the bird image to be identified. Among them, one feature detection size corresponds to one feature group, and one feature group includes a plurality of features, and the plurality of features include block features of a plurality of blocks of the bird and at least one key feature of the bird, and the key feature is a feature unique to the bird; the key feature is at least one of the following: eye, cheek, neck, chest or wing; the feature detection size of feature extraction includes any of the following: 4×4, 8×8, 16×16, 32×32. And according to the features of the same kind in the plurality of feature groups, the identification result corresponding to each feature of the bird is obtained; the identification result is the probability that the bird belongs to each species. Then, according to the identification results of the block features of the plurality of blocks of the bird, the first species identification result of the bird is determined; based on the first species identification result of the bird and the identification results of the key features of the bird, the second species identification result of the bird is determined.

[0007] In the method for bird identification based on key features provided by the present invention, first, according to different feature detection sizes, multiple block features including multiple blocks of the bird and feature groups of key features (eye, cheek, neck, chest or wing) are extracted from the bird image to be identified, and then the recognition result of each feature in each feature group is obtained to obtain the recognition result of each feature in the bird image to be identified. Finally, according to the recognition result of each feature in the bird image to be identified, the species recognition result of the bird in the bird image to be identified is determined. It can be seen that not only the blocks of the bird in the bird image to be identified can be feature extracted and identified, but also the key features unique to the bird can be feature extracted and identified, and the final recognition result is obtained by combining the recognition results of the two, thereby improving the accuracy of bird identification, especially improving the recognition accuracy of multiple birds of similar or same family with similar features.

[0008] In an implementation of the first aspect, the plurality of regions include legs, tail, crown, beak, and back.

[0009] In an implementation of the first aspect, obtaining a recognition result corresponding to each feature of a bird according to features of the same type in a plurality of feature groups includes: for each feature group in the plurality of feature groups, identifying each feature of the plurality of features in the feature group to obtain a recognition result of each feature in the feature group. Performing weighted processing on the recognition results corresponding to features of the same type in the plurality of feature groups, and using the weighted processing result as the recognition result corresponding to each feature of the bird.

[0010] In an implementation of the first aspect, a bird image recognition model is trained based on a bird image training set; the bird image recognition model is used to perform species recognition on bird images to be recognized, and the bird image training set includes multiple bird images annotated with multiple features.

[0011] In an implementation manner of the first aspect, the bird image recognition model includes a feature extraction unit, a feature recognition unit, and a recognition result determination unit that are connected in sequence.

[0012] The feature extraction unit for extracting features of the bird image to be identified includes: an input layer, a first convolution layer, a second convolution layer, a first feature extraction layer, a third convolution layer, a second feature extraction layer, a fourth convolution layer, a third feature extraction layer, a fifth convolution layer, a fourth feature extraction layer, a spatial pyramid pooling layer and a squeeze-excitation attention module connected in sequence.

[0013] The feature recognition unit for determining a first species recognition result of a bird and a second species recognition result of a bird comprises: a first feature recognition subunit and a second feature recognition subunit connected in sequence.

[0014] The first feature recognition subunit includes: a first upsampling layer, a first feature fusion layer, a fifth feature extraction layer, a second upsampling layer, a second feature fusion layer, a sixth feature extraction layer, a third upsampling layer and a third feature fusion layer connected in sequence; the input end of the first upsampling layer is connected to the output end of the SEattention module, the input end of the first feature fusion layer is connected to the output end of the third feature extraction layer, the input end of the second feature fusion layer is connected to the output end of the third convolutional layer, and the input end of the third feature fusion layer is connected to the output end of the first feature extraction layer.

[0015] The second feature recognition subunit includes: a seventh feature extraction layer, a sixth convolutional layer, a fourth feature fusion layer, an eighth feature extraction layer, a seventh convolutional layer, a fifth feature fusion layer, a ninth feature extraction layer, an eighth convolutional layer, a sixth feature fusion layer and a tenth feature extraction layer connected in sequence; the input end of the seventh feature extraction layer is connected to the output end of the third feature fusion layer, the input end of the fourth feature fusion layer is connected to the output end of the sixth feature extraction layer, the input end of the fifth feature fusion layer is connected to the output end of the fifth feature extraction layer, and the input end of the sixth feature fusion layer is connected to the output end of the squeeze-stimulation attention module.

[0016] An identification result determination unit for determining a species identification result of a bird according to multiple identification results corresponding to multiple features of the bird includes: a first detection module, a second detection module, a third detection module, a fourth detection module and an output layer connected in sequence; an input end of the first detection module is connected to an output end of the seventh feature extraction layer, an input end of the second detection module is connected to an output end of the eighth feature extraction layer, an input end of the third detection module is connected to an output end of the ninth feature extraction layer, and an input end of the fourth detection module is connected to an output end of the tenth feature extraction layer; an input end of the output layer is connected to an output end of the first detection module, an output end of the second detection module, an output end of the third detection module and an output end of the fourth detection module.

[0017] In an implementation of the first aspect, the second feature fusion layer, the sixth feature extraction layer, the third feature fusion layer, the seventh feature extraction layer, the sixth convolution layer and the fourth feature fusion layer constitute a small feature detection unit; the small feature detection unit is used to extract features of the bird image to be identified, and the feature detection size of the feature extraction is 4×4.

[0018] In a second aspect, the present invention provides a device for bird recognition based on key features, comprising a feature extraction module, a feature recognition module and a recognition result determination module. The feature extraction module is used to extract features of a bird image to be recognized according to a plurality of different feature detection sizes, and obtain a plurality of feature groups of the bird in the bird image to be recognized; wherein one feature detection size corresponds to one feature group, and one feature group includes a plurality of features, and the plurality of features include block features of a plurality of blocks of the bird and at least one key feature of the bird, and the key feature is a feature unique to the bird; the key feature is at least one of the following: eye, cheek, neck, chest or wing; the feature detection size of feature extraction includes any of the following: 4×4, 8×8, 16×16, 32×32. The feature recognition module is used to obtain the recognition result corresponding to each feature of the bird according to the features of the same kind in the plurality of feature groups; the recognition result is the probability that the bird belongs to each species. The recognition result determination module is used to determine the first species recognition result of the bird according to the recognition results of the block features of the plurality of blocks of the bird; and determine the second species recognition result of the bird based on the first species recognition result of the bird and the recognition results of the key features of the bird.

[0019] In an implementation of the second aspect, the feature recognition module is specifically used to, for each feature group in multiple feature groups, identify each of the multiple features in the feature group to obtain an identification result for each feature in the feature group; perform weighted processing on the identification results corresponding to the same type of features in the multiple feature groups, and use the result of the weighted processing as the identification result corresponding to each feature of the bird.

[0020] In a third aspect, the present invention provides an electronic device, comprising a processor and a memory coupled to the processor; the memory is used to store computer instructions, and when the electronic device is running, the processor executes the computer instructions stored in the memory, so that the electronic device executes the method as described in the first aspect or any one of its implementations.

[0021] In a fourth aspect, the present invention provides a computer-readable storage medium, comprising computer program instructions, which, when executed by a computer, enable the computer to execute the method in the first aspect or any one of its implementations.

[0022] In a fifth aspect, the present invention provides a computer program product, comprising computer program instructions, which, when executed on a computer, enable the computer to execute the method in the first aspect or any one of its implementations.

[0023] The technical effects corresponding to the above-mentioned second to fifth aspects and their possible implementations can refer to the above-mentioned description of the technical effects of the first aspect and its possible implementations, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is one of the schematic diagrams of the method for bird identification based on key features provided in the embodiments of the present application;

[0025] Figure 2 This is the second schematic diagram of the method for bird identification based on key features provided in the embodiment of the present application;

[0026] Figure 3 This is the third schematic diagram of the method for bird identification based on key features provided in the embodiment of the present application;

[0027] Figure 4 This is a bird image recognition model architecture diagram provided in an embodiment of the present application;

[0028] Figure 5 It is a schematic diagram of the structure of a device for bird identification based on key features provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] The terms "first" and "second" etc. in the description and claims of the present invention are used to distinguish different objects rather than to describe a specific order of the objects.

[0030] The “and / or” in the embodiments of the present application indicates the relationship between objects. For example, A and / or B may indicate the following three situations: A exists alone, B exists alone, and A and B exist at the same time.

[0031] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0032] In the description of the present invention, unless otherwise specified, the meaning of "plurality" refers to two or more than two. For example, a plurality of feature groups refers to two or more feature groups.

[0033] It should be understood that the blocks of the bird in the embodiments of the present application refer to parts of the bird's body, such as the bird's legs, tail, crown, beak, back, etc.

[0034] In the embodiment of the present application, the key feature of a bird is a unique feature of a bird, that is, a feature that distinguishes a certain bird from other birds. The exclusive feature of a bird can be used as the key feature of a bird. The key feature of a bird includes, for example, the posture, outline, living environment, etc. of the bird, which is not limited in the embodiment of the present application.

[0035] The method for bird recognition based on key features provided in the embodiments of the present application aims to identify the category of birds in an image (i.e., determine the species of the bird) based on the analysis and processing of the image. In the following embodiments, the method is illustrated by taking the identification of different species of birds included in the Ardeidae and Graeco-Warbleridae as an example.

[0036] It can be understood that the Ardeidae belongs to the Pelecaniformes in animal taxonomy, and the Ardeidae includes multiple bird species. For example, the Ardeidae includes but is not limited to egrets, great egrets, herons, cattle egrets, pond herons, and night herons.

[0037] In animal taxonomy, the Leucodactylidae belongs to the order Passeriformes, and the Leucodactylidae includes multiple bird species. For example, the Leucodactylidae includes but is not limited to the Brown Leucodactyl, Yellow-browed Leucodactyl, Black-billed Leucodactyl, Yellow-rumped Leucodactyl, Yellow-rumped Leucodactyl, and Emei Leucodactyl.

[0038] The method and device provided in the embodiments of the present application relate to bird recognition, and can be used to identify the species of a bird in a bird image to be identified. Specifically, by identifying the block features of the bird in the bird image to be identified and the key features unique to the bird, an identification result of each feature is obtained, and then based on the identification result of each feature, the species of the bird in the bird image to be identified is determined.

[0039] For multiple birds of similar families with similar features, the recognition accuracy of existing bird recognition methods is low. For example, most of the physical features of birds in the family of warblers are the same. Specifically, the difference in features between the yellow-rumped willow warbler and the yellow-browed willow warbler is only reflected in the difference in the color of the crown and cheeks of the two. In addition, birds of the family of ardea, such as egrets, intermediate egrets and great egrets, have similar appearance features. In order to solve the above problems, the embodiments of the present application provide a method and device for bird recognition based on key features, which can not only extract and recognize features of the blocks of birds in the bird image to be recognized, but also extract and recognize key features unique to the bird, and combine the recognition results of the two to obtain the final recognition result, thereby improving the accuracy of bird recognition, especially improving the recognition accuracy of multiple birds of similar or the same family with similar features.

[0040] Exemplarily, a method for bird identification based on key features provided in an embodiment of the present invention may be performed by an electronic device having a processing function, such as a computer, a server, etc. Taking the electronic device as a computer as an example, the hardware part of the computer may include: a processor, a memory, a network interface, a user interface, a communication bus, etc.

[0041] The processor is used to control the electronic device to perform related processing and computing tasks, for example, for feature extraction, feature recognition, and determining recognition results. The processor may include a central processing unit (CPU) or other processors, and the processor may be single-core or multi-core, for example, the processor may include multiple CPUs.

[0042] The memory is used to store computer instructions and related data, for example, to store feature detection dimensions, feature extraction units, feature recognition units, and recognition result determination units. The memory can be a random access memory (RAM), a read only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, or an optical memory, a disk storage medium or other magnetic storage device, or any other medium that can be used to store program code or data that can be accessed by a computer. Optionally, the memory can be integrated into the processor, and the memory can also be independent of the processor.

[0043] The network interface is used for the computer to communicate with other devices or communication networks. The network interface can be a transceiver with transceiver functions. Optionally, the network interface can include a standard wired interface, a wireless interface (such as a WI-FI interface, a Bluetooth interface, a 5G interface).

[0044] The communication bus is used to realize the connection and communication between different components. For example, the processor, memory, network interface and user interface mentioned above can be interconnected through the communication bus.

[0045] The user interface may include a display screen and an input unit (such as a keyboard). Optionally, the user interface may also include a standard wired interface and a wireless interface.

[0046] Those skilled in the art will appreciate that the above-mentioned computer may also include more or fewer components, or a combination of certain components, or different arrangements of components, which is not limited in the embodiments of the present application.

[0047] The following describes in detail the method for bird identification based on key features provided in the embodiment of the present application, taking the identification of bird species in the Ardeidae and the Warbleridae as an example. Figure 1 As shown, a method for bird identification based on key features provided in an embodiment of the present application includes S101-S103.

[0048] S101 , extracting features of a bird image to be identified according to a plurality of different feature detection sizes, and obtaining a plurality of feature groups of the bird in the bird image to be identified.

[0049] Among them, one feature detection size corresponds to a feature group, and a feature group includes multiple features. It should be understood that in the embodiment of the present application, the above-mentioned feature detection size refers to the size of the feature obtained by feature extraction. The multiple features include block features of multiple blocks of the bird and at least one key feature of the bird. The key feature is a feature unique to the bird. The multiple blocks include: legs, tail, crown, beak and back, and the key features include at least one of the following: eye head, cheek, neck, chest or wing; the feature detection size of feature extraction includes any of the following: 4×4, 8×8, 16×16, 32×32.

[0050] Optionally, the block features of the above blocks may include color features, structural features, body shape features and body color structural features of the blocks. The structural features of the block refer to whether the bird in the bird image to be identified has the body part corresponding to the block. For example, the color feature of the block refers to the color of the block of the bird. If the block is a beak, the color feature of the block refers to the color of the block of the bird. If the body part of the bird in the bird image to be identified is blocked, then when the block is a tail, the content of the structural feature of the block is whether the tail of the bird is included in the bird image to be identified. The body shape feature of the block refers to the shape of the body part corresponding to the block of the bird. For example, if the block is a leg, the body shape feature of the block refers to the size and shape of the block of the bird. The body color structural feature of the block refers to whether the block of the bird is a pure color. For example, if the block is a back, the content of the structural body color feature is whether the color of the back is a pure color or a non-pure color.

[0051] In one implementation, when the bird to be identified is an ardeidae, and the bird's blocks include legs, tail, crown, beak and back, the block features of the legs include body shape features of the legs, such as longer legs; the block features of the tail include color features of the tail, such as a black tail; the features of the crown include structural features of the crown, such as the absence of a crown; the block features of the beak include color features of the beak, such as the color of the beak being yellowish brown; and the block features of the back include body color and structural features of the back, such as the color of the back being a solid color.

[0052] The above key features may include color features, structural features, body shape features, and body color and structure features. For example, the key features of some birds in the Ardeidae and Willow Warbleridae are shown in Table 1.

[0053] Table 1

[0054]

[0055]

[0056] As shown in Table 1, in the Ardeidae family, the key feature of the egret is the eye (yellow); the key feature of the great egret is the tail (with a hooded feather); the key feature of the heron is the neck (long and black neck); the key feature of the cattle egret is the beak (short and thick); the key feature of the pond heron is the wings (gray), and the key feature of the night heron is the crest (gray); in the Lymantriaidae family, the key feature of the brown willow warbler is the cheek (dark brown stripes), the key feature of the yellow-browed willow warbler is the cheek (light yellow-green stripes), the key feature of the black-billed willow warbler is the beak (gray), the key feature of the yellow-rumped willow warbler is the abdomen (light yellow), the key feature of the yellow-breasted willow warbler is the chest (yellow), and the key feature of the Emei willow warbler is the wings (two light yellow stripes).

[0057] It should be understood that key classification features can be used to distinguish similar birds. When the key classification feature parts of two birds are the same, they can be further distinguished through the block features of the birds to finally get the result. For example, the white-tailed willow warbler and the crowned willow warbler are difficult to distinguish both in the wild and in pictures. The key classification feature of both is the crown, and the crown of the white-tailed willow warbler is yellow, and the crown of the crowned willow warbler is green. Therefore, the two can be further distinguished through the block features of the birds to determine their species.

[0058] S102: Obtain recognition results corresponding to each feature of the bird according to the features of the same type in multiple feature groups.

[0059] Among them, the recognition result is the probability that the bird belongs to each species.

[0060] Exemplary, combined Figure 1 ,like Figure 2 As shown, S102 includes S1021 - S1022 .

[0061] S1021. For each feature group in the plurality of feature groups, identify each of the plurality of features in the feature group to obtain an identification result of each feature in the feature group.

[0062] In the embodiment of the present application, the recognition result of each feature refers to the probability that the bird in the bird image to be identified belongs to each species after identifying each feature in each feature group.

[0063] In one application scenario, four feature groups are obtained in S101. When the key feature obtained by feature extraction in the bird image to be identified is cheek, the first feature group includes block features a11 of the legs, a12 of the tail, a13 of the crown, a14 of the beak, a15 of the back, and related features a16 of the cheek (including color features, structural features, body shape features, and body color and structural features of the beak). Then, after identifying each of the above features, the identification results p11 of the legs, p21 of the tail, p31 of the crown, p41 of the beak, p51 of the back, and p61 of the cheek are obtained.

[0064] Then, the recognition results of each feature in the second feature group are: recognition result p12 for the leg, recognition result p22 for the tail, recognition result p32 for the crown, recognition result p42 for the beak, recognition result p52 for the back, and recognition result p62 for the cheek.

[0065] The recognition results of each feature in the third feature group are: the recognition result of the leg is p13, the recognition result of the tail is p23, the recognition result of the crown is p33, the recognition result of the beak is p43, the recognition result of the back is p53, and the recognition result of the cheek is p63.

[0066] The recognition results of each feature in the fourth feature group are: the recognition result of the leg is p14, the recognition result of the tail is p24, the recognition result of the crown is p34, the recognition result of the beak is p44, the recognition result of the back is p54, and the recognition result of the cheek is p64.

[0067] S1022, weighting the recognition results corresponding to the same type of features in multiple feature groups, and using the weighted processing result as the recognition result corresponding to each feature of the bird.

[0068] Continuing with the above application scenario as an example, after obtaining the recognition results of each feature of the four feature groups, the process of weighting the recognition results corresponding to the same type of features in multiple feature groups is as follows: weighting the recognition results of the legs in the four feature groups (p11, p12, p13, p14), and obtaining the recognition result of the legs of the bird in the bird to be identified as p1; weighting the recognition results of the tail in the four feature groups (p21, p22, p23, p24), and obtaining the recognition result of the tail of the bird in the bird to be identified as p2; weighting the recognition results of the crown in the four feature groups (p31, p32, p33, p34), and obtaining the recognition result of the crown in the four feature groups (p35, p36, p37, p38, p39, p40, p41, p42, p43, p44, p45, p46, p47, p48, p49, p50, p51, p52, p53, p54, p55, p56, p57, p58, p59, p60, p61, p62, p63, p64, p65, p66, p67, p68, p69, p70, p71, p72, p73, p74, p75 p34) are weighted, and the recognition result of the crown of the bird in the bird to be identified is p3; the recognition results of the beak in the four feature groups (p41, p42, p43, p44) are weighted, and the recognition result of the beak of the bird in the bird to be identified is p4; the recognition results of the back in the four feature groups (p51, p52, p53, p54) are weighted, and the recognition result of the back of the bird in the bird to be identified is p5; the recognition results of the cheek in the four feature groups (p61, p62, p63, p64) are weighted, and the recognition result of the cheek of the bird in the bird to be identified is p6. It can be seen that in this application scenario, the recognition result of the bird's legs is p1, the recognition result of the tail is p2, the recognition result of the crown is p3, the recognition result of the beak is p4, the recognition result of the back is p5, and the recognition result of the cheek is p6.

[0069] S103, determining a first species identification result of the bird according to the identification results of the block features of the multiple blocks of the bird; and determining a second species identification result of the bird based on the first species identification result of the bird and the identification results of the key features of the bird.

[0070] Among them, the second species recognition result of the bird is the final bird recognition result.

[0071] Continuing with the above application scenario as an example, after obtaining multiple recognition results corresponding to multiple features of the bird, the recognition results of the block features of multiple blocks of the bird (p1, p2, p3, p4 and p5) are weighted and summed to obtain the first species recognition result of the bird (i.e., P'). The first species recognition result (P') and the recognition result of the key feature of the bird (i.e., p6) are weighted and summed again to obtain the second species recognition result of the bird (P"), and P" is the final bird recognition result.

[0072] It can be understood that the weight distribution of the above weighted summation can be obtained through the loss function.

[0073] Optionally, combined Figure 2 ,like Figure 3 As shown, before S101, the above method further includes S104.

[0074] S104. Training a bird image recognition model based on the bird image training set.

[0075] The bird image recognition model is used to perform species recognition on bird images to be recognized, and the bird image training set includes multiple bird images annotated with multiple features and bird species.

[0076] In one implementation, the bird image recognition model includes a feature extraction unit, a feature recognition unit, and a recognition result determination unit.

[0077] Specifically, Figure 4 As shown, the feature extraction unit for extracting features of the bird image to be identified includes: an input layer, a first convolution layer, a second convolution layer, a first feature extraction layer, a third convolution layer, a second feature extraction layer, a fourth convolution layer, a third feature extraction layer, a fifth convolution layer, a fourth feature extraction layer, a spatial pyramid pooling layer, and a squeeze-excitation attention module connected in sequence;

[0078] The feature recognition unit for obtaining the recognition result corresponding to each feature of the bird according to the features of the same kind in the plurality of feature groups comprises: a first feature recognition subunit and a second feature recognition subunit connected in sequence;

[0079] The first feature recognition subunit includes: a first upsampling layer, a first feature fusion layer, a fifth feature extraction layer, a second upsampling layer, a second feature fusion layer, a sixth feature extraction layer, a third upsampling layer, and a third feature fusion layer connected in sequence; the input end of the first upsampling layer is connected to the output end of the SEattention module, the input end of the first feature fusion layer is connected to the output end of the third feature extraction layer, the input end of the second feature fusion layer is connected to the output end of the third convolution layer, and the input end of the third feature fusion layer is connected to the output end of the first feature extraction layer;

[0080] The second feature recognition subunit includes: a seventh feature extraction layer, a sixth convolution layer, a fourth feature fusion layer, an eighth feature extraction layer, a seventh convolution layer, a fifth feature fusion layer, a ninth feature extraction layer, an eighth convolution layer, a sixth feature fusion layer and a tenth feature extraction layer connected in sequence; the input end of the seventh feature extraction layer is connected to the output end of the third feature fusion layer, the input end of the fourth feature fusion layer is connected to the output end of the sixth feature extraction layer, the input end of the fifth feature fusion layer is connected to the output end of the fifth feature extraction layer, and the input end of the sixth feature fusion layer is connected to the output end of the squeeze incentive attention module;

[0081] The identification result determination unit for determining the first species identification result of the bird and the second species identification result of the bird includes: a first detection module, a second detection module, a third detection module, a fourth detection module and an output layer connected in sequence; the input end of the first detection module is connected to the output end of the seventh feature extraction layer, the input end of the second detection module is connected to the output end of the eighth feature extraction layer, the input end of the third detection module is connected to the output end of the ninth feature extraction layer, and the input end of the fourth detection module is connected to the output end of the tenth feature extraction layer; the input end of the output layer is connected to the output end of the first detection module, the output end of the second detection module, the output end of the third detection module and the output end of the fourth detection module.

[0082] The second feature fusion layer, the sixth feature extraction layer, the third feature fusion layer, the seventh feature extraction layer, the sixth convolution layer and the fourth feature fusion layer constitute a tiny feature detection unit; the tiny feature detection unit is used to extract features of the bird images to be identified, and the feature detection size of the feature extraction is 4×4.

[0083] From the above content, it can be seen that the above-mentioned tiny feature detection unit can extract features in the bird image to be identified with a feature detection size of 4×4, thereby realizing the identification of smaller features and further being able to extract features more effectively.

[0084] The training and evaluation process of the above bird image recognition model includes:

[0085] The first step is model training.

[0086] The bird images in the bird image training set are input into the bird image recognition model, the recognition results of the birds in the bird images are calculated, and the loss function value between the recognition results and the real bird species is calculated, and gradient back propagation is performed, and the parameters of the model are updated according to the set optimization algorithm.

[0087] It should be understood that model training is a process of multiple rounds of iterations. Each round of iteration traverses the training set once, and each time a small batch of samples is obtained from it and sent to the model to perform forward calculations to obtain predicted values. The effect of model training is judged by observing the decreasing trend of the loss value in each round of iteration. For example, when the decreasing trend of the loss value remains unchanged, it means that the model tends to converge.

[0088] Taking egrets as an example, after obtaining a picture marked with the egret's eye shape (key feature) and egret species information, the picture is input into the model. First, the feature input is determined and extracted through the feature pyramid. At the same time, the bird species information is extracted, and the eye shape (key feature information belonging to egrets) and egrets (species information) are input into the feature fusion module. The final output result is determined according to the loss function.

[0089] The second step is model evaluation.

[0090] The bird image test set is input into the trained bird image recognition model for evaluation to obtain the bird recognition result, calculate the loss function value between the bird recognition result and the real bird species, and calculate the evaluation index value to facilitate the evaluation of the model effect.

[0091] It can be understood that the bird image test set also includes multiple bird images annotated with multiple features and bird species. The bird image test set can be selected from the bird image training set, or other bird images annotated with multiple features and bird species can be selected. This embodiment of the present application is not limited to this.

[0092] The third step is model reasoning.

[0093] Input the bird data to be verified into the trained bird image recognition model to perform reasoning, and observe and verify whether the reasoning results meet expectations.

[0094] Compared with the existing models that adaptively determine important features of birds (such as contours, postures, etc.), since the embodiments of the present application target the block features and key features of bird blocks in the bird image training set, label the block features and key features of the blocks, and use the labeled bird image training set to train a bird image recognition model that can extract the block features and key features of the blocks, the bird features extracted by the bird image recognition model can not only extract the common features of most birds, but also extract the features that distinguish the birds from other birds. Therefore, even for multiple birds with similar features of similar or same families and genera, relatively accurate recognition can be achieved through key features, thereby further improving the recognition accuracy of the bird image recognition model.

[0095] In summary, in the method for bird identification based on key features provided in the embodiment of the present application, first, according to different feature detection sizes, multiple block features including multiple blocks of the bird and feature groups of key features (eye, cheek, neck, chest or wings) are extracted from the bird image to be identified, and then the recognition result of each feature in each feature group is obtained to obtain the recognition result of each feature in the bird image to be identified. Finally, according to the recognition result of each feature in the bird image to be identified, the species recognition result of the bird in the bird image to be identified is determined. It can be seen that not only can the blocks of the bird in the bird image to be identified be feature extracted and identified, but also the key features unique to the bird can be feature extracted and identified, and the final recognition result can be obtained by combining the recognition results of the two, thereby improving the accuracy of bird recognition, especially the recognition accuracy of multiple birds of similar or the same family with similar features.

[0096] Accordingly, the embodiment of the present application provides a device for bird identification based on key features, such as Figure 5As shown, it includes a feature extraction module 501, a feature recognition module 502 and a recognition result determination module 503.

[0097] The feature extraction module 501 is used to extract features from the bird image to be identified according to a plurality of different feature detection sizes, and obtain a plurality of feature groups of the bird in the bird image to be identified; wherein one feature detection size corresponds to one feature group, and one feature group includes a plurality of features, and the plurality of features include block features of a plurality of blocks of the bird and at least one key feature of the bird, and the key feature is a feature unique to the bird; the key feature is at least one of the following: eye, cheek, neck, chest or wing; the feature extraction feature detection size includes any of the following: 4×4, 8×8, 16×16, 32×32. For example, the feature extraction module 501 is used to implement S101 of the above-mentioned method for bird identification based on key features.

[0098] The feature recognition module 502 is used to obtain the recognition result corresponding to each feature of the bird according to the features of the same type in the plurality of feature groups; the recognition result is the probability that the bird belongs to each species. For example, the feature recognition module 502 is used to implement S102 of the above-mentioned method for bird recognition based on key features.

[0099] The recognition result determination module 503 is used to determine the first species recognition result of the bird according to the recognition results of the block features of the multiple blocks of the bird; and determine the second species recognition result of the bird based on the first species recognition result of the bird and the recognition results of the key features of the bird. For example, the recognition result determination module 503 is used to implement S103 of the above-mentioned method for bird recognition based on key features.

[0100] Optionally, the feature recognition module 502 is specifically used to: for each feature group in the multiple feature groups, recognize each feature in the multiple features of the feature group to obtain a recognition result for each feature in the feature group; perform weighted processing on the recognition results corresponding to the features of the same type in the multiple feature groups, and use the weighted processing result as the recognition result corresponding to each feature of the bird. For example, the feature recognition module 502 is specifically used to implement S1021-S1022 of the above-mentioned method for bird recognition based on key features.

[0101] In one application scenario, the above device also includes a model training module 504.

[0102] The model training module 504 is used to train a bird image recognition model based on a bird image training set. For example, the model training module 504 is used to implement S104 of the above-mentioned method for bird recognition based on key features.

[0103] Each module of the above-mentioned device for bird identification based on key features can also be used to execute other steps in the above-mentioned method embodiment. All relevant contents involved in the above-mentioned method embodiment can be referred to the functional description of the corresponding functional module, which will not be repeated here.

[0104] The embodiment of the present application also provides an electronic device, including: a processor and a memory coupled to the processor; the memory is used to store computer instructions, and when the electronic device is running, the processor executes the computer instructions stored in the memory, so that the electronic device executes the method described in the above embodiment. Among them, the processor can implement the above feature extraction module 501, feature recognition module 502 and recognition result determination module 503; the above memory can also be used to store feature detection size, feature extraction unit, feature recognition unit and recognition result determination unit, etc.

[0105] An embodiment of the present application further provides a computer-readable storage medium, which includes a computer program. When the computer program runs on a computer, the method described in the above embodiment is executed.

[0106] An embodiment of the present application further provides a computer program product, which includes computer program instructions. When the computer program instructions are run on a computer, the method described in the above embodiment is executed.

[0107] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0108] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for bird identification based on key features, characterized in that: include: According to a plurality of different feature detection sizes, feature extraction is performed on the bird image to be identified to obtain a plurality of feature groups of the bird in the bird image to be identified; wherein one feature detection size corresponds to one feature group, and one feature group includes a plurality of features, and the plurality of features include block features of a plurality of blocks of the bird and at least one key feature of the bird, and the key feature is a feature unique to the bird; the key feature is at least one of the following: eye, cheek, neck, chest or wing; the feature detection size of the feature extraction includes any one of the following: 4×4, 8×8, 16×16, 32×32; Obtaining, based on the features of the same type in the plurality of feature groups, an identification result corresponding to each feature of the bird; the identification result being the probability that the bird belongs to each species; According to the recognition results of the block features of the multiple blocks of the bird, a first species recognition result of the bird is determined; based on the first species recognition result of the bird and the recognition results of the key features of the bird, a second species recognition result of the bird is determined.

2. The method according to claim 1, characterized in that The multiple sections include: legs, tail, crown, beak and back.

3. The method according to claim 1 or 2, characterized in that Obtaining, based on the features of the same type in the plurality of feature groups, a recognition result corresponding to each feature of the bird, including: For each feature group in the plurality of feature groups, identifying each of the plurality of features in the feature group to obtain an identification result of each feature in the feature group; The recognition results corresponding to the same type of features in the multiple feature groups are weighted, and the result of the weighted processing is used as the recognition result corresponding to each feature of the bird.

4. The method according to claim 1, characterized in that The method further comprises: A bird image recognition model is trained based on a bird image training set; the bird image recognition model is used to perform species recognition on the bird images to be recognized, and the bird image training set includes multiple bird images annotated with the multiple features.

5. The method according to claim 4, characterized in that The bird image recognition model comprises a feature extraction unit, a feature recognition unit and a recognition result determination unit connected in sequence; The feature extraction unit for extracting features from the bird image to be identified comprises: an input layer, a first convolutional layer, a second convolutional layer, a first feature extraction layer, a third convolutional layer, a second feature extraction layer, a fourth convolutional layer, a third feature extraction layer, a fifth convolutional layer, a fourth feature extraction layer, a spatial pyramid pooling layer, and a squeeze-stimulated attention module connected in sequence; The feature recognition unit for obtaining the recognition result corresponding to each feature of the bird according to the features of the same kind in the plurality of feature groups comprises: a first feature recognition subunit and a second feature recognition subunit connected in sequence; The first feature recognition subunit includes: a first upsampling layer, a first feature fusion layer, a fifth feature extraction layer, a second upsampling layer, a second feature fusion layer, a sixth feature extraction layer, a third upsampling layer, and a third feature fusion layer connected in sequence; the input end of the first upsampling layer is connected to the output end of the SEattention module, the input end of the first feature fusion layer is connected to the output end of the third feature extraction layer, the input end of the second feature fusion layer is connected to the output end of the third convolutional layer, and the input end of the third feature fusion layer is connected to the output end of the first feature extraction layer; The second feature recognition subunit includes: a seventh feature extraction layer, a sixth convolution layer, a fourth feature fusion layer, an eighth feature extraction layer, a seventh convolution layer, a fifth feature fusion layer, a ninth feature extraction layer, an eighth convolution layer, a sixth feature fusion layer and a tenth feature extraction layer connected in sequence; the input end of the seventh feature extraction layer is connected to the output end of the third feature fusion layer, the input end of the fourth feature fusion layer is connected to the output end of the sixth feature extraction layer, the input end of the fifth feature fusion layer is connected to the output end of the fifth feature extraction layer, and the input end of the sixth feature fusion layer is connected to the output end of the squeeze-stimulation attention module; The identification result determination unit for determining the first species identification result of the bird and the second species identification result of the bird includes: a first detection module, a second detection module, a third detection module, a fourth detection module and an output layer connected in sequence; the input end of the first detection module is connected to the output end of the seventh feature extraction layer, the input end of the second detection module is connected to the output end of the eighth feature extraction layer, the input end of the third detection module is connected to the output end of the ninth feature extraction layer, and the input end of the fourth detection module is connected to the output end of the tenth feature extraction layer; the input end of the output layer is connected to the output end of the first detection module, the output end of the second detection module, the output end of the third detection module and the output end of the fourth detection module.

6. The method according to claim 5, characterized in that The second feature fusion layer, the sixth feature extraction layer, the third feature fusion layer, the seventh feature extraction layer, the sixth convolution layer and the fourth feature fusion layer constitute a small feature detection unit; the small feature detection unit is used to extract features of the bird image to be identified, and the feature detection size of the feature extraction is 4×4.

7. A device for bird identification based on key features, characterized in that: It includes a feature extraction module, a feature recognition module and a recognition result determination module; The feature extraction module is used to extract features from the bird image to be identified according to a plurality of different feature detection sizes, and obtain a plurality of feature groups of the bird in the bird image to be identified; wherein one feature detection size corresponds to one feature group, and one feature group includes a plurality of features, and the plurality of features include block features of a plurality of blocks of the bird and at least one key feature of the bird, and the key feature is a feature unique to the bird; the key feature is at least one of the following: eye, cheek, neck, chest or wing; the feature detection size of the feature extraction includes any one of the following: 4×4, 8×8, 16×16, 32×32; The feature recognition module is used to obtain a recognition result corresponding to each feature of the bird according to the features of the same type in the multiple feature groups; the recognition result is the probability that the bird belongs to each species; The recognition result determination module is used to determine a first species recognition result of the bird according to the recognition results of the block features of the multiple blocks of the bird; and determine a second species recognition result of the bird based on the first species recognition result of the bird and the recognition results of the key features of the bird.

8. The device according to claim 7, characterized in that The feature recognition module is specifically used to, for each feature group in the multiple feature groups, recognize each of the multiple features in the feature group to obtain a recognition result of each feature in the feature group; The recognition results corresponding to the same type of features in the multiple feature groups are weighted, and the result of the weighted processing is used as the recognition result corresponding to each feature of the bird.

9. An electronic device, characterized in that: It comprises a processor and a memory coupled to the processor; the memory is used to store computer instructions, and when the electronic device is running, the processor executes the computer instructions stored in the memory, so that the electronic device executes the method as described in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The method comprises computer program instructions, which, when executed by a computer, cause the computer to perform the method according to any one of claims 1 to 6.

Images