Method and device for identifying species
By constructing a feature database and using distance judgment to correct the erroneous identification results of the AI classification model, the problems of low accuracy and low efficiency in species identification were solved, and efficient and accurate species identification was achieved.
Patent Information
- Application Number
- CN202410705826.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-03
- Publication Date
- 2024-11-08
AI Technical Summary
Existing species identification methods have low accuracy and low efficiency, requiring a large amount of image data and multiple training sessions to improve the identification effect, resulting in time-consuming and labor-intensive processes.
A feature database is constructed, containing feature vectors of incorrectly identified animal images and their true species labels. The feature vectors of the images to be identified are processed by an AI classification model, and the species labels are determined based on distance judgments from the feature database to correct the incorrect identification results.
It improves the accuracy and efficiency of species identification, reduces computing power and labor costs, and enables rapid species identification.
Smart Images

Figure CN118918352A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of species identification and artificial intelligence (AI), and in particular to a species identification method and device. Background Art
[0002] Intelligent image recognition is based on a computer system. It converts the collected images into digital information, uses a comprehensive physical algorithm to pre-process the image data, and then identifies the objects in the image based on a classification model to obtain the species.
[0003] However, on the one hand, in the actual recognition process, the classification model may not be applicable to a variety of image data, resulting in low recognition accuracy. On the other hand, in order to improve the recognition accuracy, it is often necessary to collect more image data and annotate them, and then continue to train the classification model. After obtaining a better classification model, it is used for recognition to improve the recognition effect. However, in this method, training the classification model multiple times is very time-consuming, resulting in low species recognition efficiency. Summary of the invention
[0004] The present invention provides a species identification method and device, which can improve the accuracy of species identification and also improve the efficiency of species identification.
[0005] In order to achieve the above object, the present invention adopts the following technical solution:
[0006] In a first aspect, the present invention provides a species identification method, comprising: constructing a feature database based on an AI classification model and a first image data set; the AI classification model comprises a first classification network and a second classification network, and the feature database comprises feature vectors of multiple target images and real species labels of animals in the multiple target images; wherein the multiple target images include animal images in the first image data set that are incorrectly identified after being identified by the first classification network, and the feature vectors of the multiple target images are obtained by processing the first classification network and the second classification network of the AI classification model in sequence; and processing the animal image to be identified based on the AI classification model to obtain the feature vector of the animal image to be identified; when the distance between the multiple feature vectors in the feature database and the feature vector of the animal image to be identified is less than or equal to a first distance threshold, the species label corresponding to the feature vector with the smallest distance to the feature vector of the animal image to be identified among the multiple feature vectors in the feature database is determined as the species label of the animal image to be identified; when the distance between the multiple feature vectors in the feature database and the feature vector of the animal image to be identified is greater than the first distance threshold, the species label of the animal image to be identified is determined according to the classification result of the first classification network in the AI classification model.
[0007] The species identification method provided by the present invention can be used to address the situation where species identification is incorrectly performed using an AI classification model. A feature database is constructed based on feature vectors of multiple animal images with incorrect species identification. A distance judgment is performed between the feature vector of the animal image to be identified obtained by the AI classification model and multiple feature vectors in the feature database. Whether the identification result of the AI classification model is correct is determined based on the judgment result, thereby obtaining the correct species of the animal image to be identified and improving the accuracy of species identification.
[0008] Furthermore, when the recognition result obtained by the AI classification model is inaccurate, the species of the animal in the animal image to be identified is determined according to the feature vector in the feature database, without spending more computing power resources and time resources to retrain or continue to train the AI classification model, which can save computing power and labor costs. Therefore, the species identification method provided in the embodiment of the present application can quickly complete species identification, which helps to improve the efficiency of species identification.
[0009] In an implementation of the first aspect, the animal image to be identified is processed based on the AI classification model to obtain a feature vector of the animal image to be identified, including: processing the animal image to be identified based on the first classification network in the AI classification model to obtain a classification result of the animal image to be identified; and processing the classification result of the animal image to be identified based on the second classification network in the AI classification model to obtain a feature vector of the animal image to be identified.
[0010] In an implementation manner of the first aspect, the first classification network is an EfficientNetV2 network.
[0011] In an implementation manner of the first aspect, the second classification network includes a fully connected layer.
[0012] In an implementation manner of the first aspect, the first distance threshold is an average value of distances between feature vectors of multiple pairs of animal images of the same species.
[0013] Optionally, the distance between the feature vectors may be a cosine distance or a Euclidean distance.
[0014] In an implementation of the first aspect, when the distance between the feature vectors is a cosine distance, the cosine distance is calculated by cosine similarity, wherein cosine distance=1-cosine similarity. The calculation formula of cosine similarity is:
[0015]
[0016] In the above formula, cossim(x,y) is the cosine similarity, x iis the ith eigenvector value of the eigenvector x of the animal image to be identified, y i is the i-th eigenvector value of a eigenvector y in the feature database. The dimensions of the feature vector of the animal image to be identified and the feature vector in the feature database are both M, 1≤i≤M, and M is the same as the number of channels of the second classification network.
[0017] In an implementation of the first aspect, in the process of determining the first distance threshold, the distance between the feature vectors of each pair of animal images of the same species is calculated. After obtaining multiple distances, abnormal distances among the multiple distances can be eliminated, and then the first distance threshold is determined based on the average value of the remaining distances among the multiple distances.
[0018] In an implementation of the first aspect, the species identification method provided by the present invention further includes: calculating the distance between the feature vectors of each pair of animal images of the same species, and among the multiple distances obtained, determining a distance greater than a second distance threshold as an abnormal distance.
[0019] In an implementation of the first aspect, the method for determining the second distance threshold is: calculating the distance between the feature vectors of each pair of animal images of the same species, and averaging all distances to obtain an average distance value; and determining N (N≥1) times the average distance value as the second distance threshold.
[0020] Optionally, the value of N is 2 or 3.
[0021] In an implementation manner of the first aspect, the method for determining the second distance threshold is: determining the average distance value of the distances between feature vectors of multiple pairs of animal images of different species as the second distance threshold.
[0022] In a second aspect, the present invention provides a species identification device, including a construction module, a processing module and a determination module. The construction module is used to construct a feature database based on an AI classification model and a first image data set; the AI classification model includes a first classification network and a second classification network, and the feature database includes feature vectors of multiple target images and real species labels of animals in multiple target images; the species labels of the multiple target images include animal images in the first image data set that are incorrectly identified after being identified by the first classification network, and the feature vectors of the multiple target images are processed by the first classification network and the second classification network of the AI classification model in turn. The processing module is used to process the animal images to be identified based on the AI classification model to obtain the feature vectors of the animal images to be identified. The determination module is used to determine the species label corresponding to the feature vector with the smallest distance from the feature vector of the animal image to be identified among the multiple feature vectors in the feature database as the species label of the animal image to be identified when the distances between the multiple feature vectors in the feature database and the feature vector of the animal image to be identified are less than or equal to a first distance threshold; and determine the species label of the animal image to be identified according to the classification result of the first classification network in the AI classification model when the distances between the multiple feature vectors in the feature database and the feature vector of the animal image to be identified are all greater than the first distance threshold.
[0023] In an implementation of the second aspect, the construction module is specifically used to process the animal image to be identified based on the first classification network in the AI classification model to obtain a classification result of the animal image to be identified; and process the classification result of the animal image to be identified based on the second classification network in the AI classification model to obtain a feature vector of the animal image to be identified.
[0024] In a third aspect, the present invention provides an electronic device, comprising a processor and a memory coupled to the processor; the memory is used to store computer instructions, and when the electronic device is running, the processor executes the computer instructions stored in the memory, so that the electronic device performs the method described in the first aspect or any one of its implementations.
[0025] In a fourth aspect, the present invention provides a computer-readable storage medium, comprising computer program instructions, which, when executed by a computer, enable the computer to execute the method as described in the first aspect or any one of its implementations.
[0026] In a fifth aspect, the present invention provides a computer program product, comprising computer program instructions, which, when executed on a computer, enable the computer to execute the method as described in the first aspect or any one of its implementations.
[0027] The technical effects corresponding to the above-mentioned second to fifth aspects and their possible implementations can refer to the above-mentioned description of the technical effects of the first aspect and its possible implementations, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] Figure 1 This is one of the schematic diagrams of the species identification method provided in the embodiments of the present application;
[0029] Figure 2 It is a hierarchical structure diagram of the AI classification model in the embodiment of the present application;
[0030] Figure 3 This is the second schematic diagram of the species identification method provided in the embodiment of the present application;
[0031] Figure 4 is a flowchart of the training process of the AI classification model provided in an embodiment of the present application;
[0032] Figure 5 It is a schematic diagram of the structure of a species identification device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0033] The terms "first" and "second" etc. in the description and claims of the present invention are used to distinguish different objects rather than to describe a specific order of the objects.
[0034] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.
[0035] In the description of the present invention, unless otherwise specified, the meaning of "plurality" refers to two or more than two. For example, a plurality of animal images to be identified refers to two or more animal images to be identified.
[0036] The method and device provided in the embodiments of the present application relate to species identification, and can be used to identify the species of animals. Specifically, a classification model is used to process captured images containing animals to identify the species of the animals in the images.
[0037] In order to solve the problems of low recognition accuracy and low recognition efficiency in the species identification method described in the background technology, the embodiments of the present application provide a species identification method and device, which can construct a feature database including feature vectors of incorrectly identified animal images and real species labels of incorrectly identified animal images; then, the animal image to be identified is processed based on the AI classification model to obtain the feature vector of the animal image to be identified, and then the feature vector of the animal image to be identified is compared with multiple feature vectors in the feature database to determine the species of the animal in the animal image to be identified, which can improve the accuracy of species identification and the efficiency of species identification.
[0038] Illustratively, the species identification method provided in the embodiments of the present application can be used to identify species of multiple types of animals, such as birds, mammals, amphibians, fish, reptiles and other species.
[0039] Taking the different types of birds included in the order Passeriformes (in animal taxonomy, Passeriformes is an order in the class Aves) among birds as an example, the order Passeriformes includes multiple bird species. For example, the bird species included in the order Passeriformes include but are not limited to blue-throated robin, starling, raven, chestnut-backed shrike, mynah bird, silk-tailed starling, large-billed crow, red-tailed water robin, northern redstart, red-flanked white-eye, red-throated robin, red-billed blue magpie, black-necked oriole, bald-nosed crow and chestnut-bellied sandthrush.
[0040] The species identification method and device provided in the embodiments of the present application are described in detail below.
[0041] like Figure 1 As shown, the species identification method provided in the embodiment of the present application includes S101-S104.
[0042] S101. Build a feature database based on the AI classification model and the first image data set.
[0043] In the embodiment of the present application, the AI classification model includes a first classification network and a second classification network, wherein the first classification network and the second classification network are connected in sequence.
[0044] In one implementation, the first classification network is an EfficientNetV2 network. Figure 2 As shown, the EfficientNetV2 network includes a convolutional layer with a convolution kernel size of 3×3, a first fused inverted residual convolutional layer, a second fused inverted residual convolutional layer, a third fused inverted residual convolutional layer, a first depth-separable convolutional layer, a second depth-separable convolutional layer, a third depth-separable convolutional layer and a first fully connected layer.
[0045] Optionally, the first fused inverted residual convolution layer is a 2-layer network structure of Fused-MBConv1, k3×3 containing 24 channels; the second fused inverted residual convolution layer is a 4-layer network structure of Fused-MBConv4, k3×3 containing 48 channels; the third fused inverted residual convolution layer is a 4-layer network structure of Fused-MBConv4, k3×3 containing 64 channels; the first depth-separable convolution layer is a 6-layer network structure of MBConv4, k3×3, SE0.25 containing 128 channels; the second depth-separable convolution layer is a 9-layer network structure of MBConv6, k3×3, SE0.25 containing 160 channels; the third depth-separable convolution layer is a 15-layer network structure of MBConv6, k3×3, SE0.25 containing 256 channels; the first fully connected layer is a 1-layer network structure of Conv1×1&Pooling&FC containing 1280 channels.
[0046] Optionally, the above-mentioned first classification network can also be AlexNet, ConvNeXt, DenseNet, EfficientNet, GoogLeNet, Inception3, MNASNet, MobileNetV2, MobileNetV3, RegNet, ShuffleNetV2, SqueezeNet, SwinTransformer, VGG and VisionTransformer. The embodiment of the present application does not limit the first classification network.
[0047] In one implementation, the second classification network is a second fully connected layer, the number of layers of the second fully connected layer is 1, including 1280 channels, and the network structure of the second fully connected layer is Conv1×1&Pooling&FC, that is, a joint fully connected layer.
[0048] Optionally, the second classification network may be a second fully connected layer of a multi-layer network structure including multiple channels.
[0049] In an embodiment of the present application, a camera is used to collect images of animals in different environments and in different action forms (the images may also be referred to as animal images), and the real species of the animals in the images are annotated to obtain species labels, thereby obtaining an image collection including animal images and real species labels (the image collection may also be referred to as an image data set).
[0050] Optionally, the first image data set comes from the image collection.
[0051] Exemplarily, different environments may include environments with different light conditions and / or different natural conditions, and light conditions include but are not limited to morning, noon, evening, and night, etc. Natural conditions include but are not limited to under a tree, on a tree, on a road, on the side of a road, in the air, and by a river, etc. Action forms include but are not limited to standing forms and flying forms, etc., and flying includes forms such as flapping wings, gliding, and soaring.
[0052] In the embodiment of the present application, the visible part of the animal in the animal image includes all body parts of the animal, and may also include partial body parts of the animal.
[0053] In combination with the above, after the animal images are collected, in one implementation, the real species label of the animal can be obtained by manual annotation, that is, the real species of the animal in the animal image is manually judged and annotated, and the annotation result is the real species label of the animal. In another implementation, the animal image can be annotated by a data annotation tool (such as Labelmg), and the annotation result (i.e., species label) can be obtained, and the species label obtained by the annotation tool is used as the real species label of the animal.
[0054] Optionally, before S101, the species identification method provided in the embodiment of the present application further includes: training the AI classification model based on the second image data set. Optionally, the second image data set may also come from the above image set.
[0055] Optionally, the first image data set and the second image data set may be the same or different.
[0056] In some cases, the animal images in the first image dataset and / or the second image dataset may be pre-processed (e.g., resizing, filtering, etc.) before being input into the AI classification model.
[0057] In one implementation, the feature database constructed by S101 includes feature vectors of multiple target images and real species labels of animals in the multiple target images. Optionally, the multiple feature vectors in the feature database are stored in a corresponding relationship with the multiple real species labels. Among them, the species labels of the multiple target images include animal images that are misidentified after being identified by the first classification network in the first image data set, and the feature vectors of the multiple target images are processed by the first classification network and the second classification network of the AI classification model in turn.
[0058] In an embodiment of the present application, the first classification network is used to identify (or classify) the species of the animal in the animal image to obtain a classification result, which includes the probability that the animal in the animal image belongs to all animal species, and selects the animal species with the highest probability as the species of the animal according to the probability distribution. In the above-mentioned use of the first classification network to identify the species of the animal, if the determined species label is consistent with the real species label in the animal image, the identification is correct; if the determined species label is inconsistent with the real species label in the animal image, the identification is wrong, that is, the species identification results include correct identification and wrong identification.
[0059] In an embodiment of the present application, the classification result of the animal image that was incorrectly identified after being identified by the first classification network is input into the second classification network (i.e., the second fully connected layer) to obtain a feature vector of the animal image, and the classification result is converted into a feature vector containing animal feature information.
[0060] Optionally, the above-mentioned feature vectors, i.e., embeddings (embeddings are a dense numerical representation of real-world objects and relationships as vectors), can be called space vectors or embedded vectors. In the embodiment of the present application, the feature vector of the animal image is an M-dimensional vector (for example, M can be 1280), and the feature vector contains all the animal features in the animal image. Therefore, the feature vector can indicate the animal in the animal image.
[0061] S102. Process the image of the animal to be identified based on the AI classification model to obtain a feature vector of the image of the animal to be identified.
[0062] As an implementation method, the classification result obtained by the first classification network and the feature vector of the animal image to be identified obtained by the second classification network can be obtained through the AI classification model. The classification result includes the probability that the animal in the animal image to be identified belongs to all animal species.
[0063] In the embodiment of the present application, the animal image to be identified may include images of animals in different action forms in different environments captured by a camera (also referred to as animal images).
[0064] Optionally, combined Figure 1 ,like Figure 3 As shown, the above S102 can be implemented through S1021-S1022.
[0065] S1021. Process the image of the animal to be identified based on the first classification network in the AI classification model to obtain a classification result of the image of the animal to be identified.
[0066] S1022. Process the classification result of the animal image to be identified based on the second classification network in the AI classification model to obtain a feature vector of the animal image to be identified.
[0067] In an embodiment of the present application, the classification result output by the first classification network is input into the second classification network, so that the second classification network outputs a feature vector of the animal image to be identified.
[0068] In an embodiment of the present application, after obtaining the feature vector of the animal image to be identified through S102, the feature vector of the animal image to be identified is compared with multiple feature vectors in the constructed feature database to determine the species label of the animal in the animal image to be identified, which is specifically achieved through S103-S104.
[0069] S103. When a distance between multiple feature vectors in the feature database and a feature vector of the animal image to be identified is less than or equal to a first distance threshold, the species label corresponding to the feature vector with the shortest distance to the feature vector of the animal image to be identified among the multiple feature vectors in the feature database is determined as the species label of the animal image to be identified.
[0070] In an embodiment of the present application, if the distance between the feature vectors of two animal images is less than or equal to the first distance threshold, it means that the similarity between the two feature vectors is very high, and it also means that the similarity between the animal images corresponding to the two feature vectors is very high. In this way, it can be considered that the species similarity of the animals in the two images is very high.
[0071] In the embodiment of the present application, the distance between each feature vector in the multiple feature vectors and the feature vector of the animal image to be identified is calculated respectively to obtain multiple distances. When there is a distance less than or equal to the first distance threshold among the multiple distances, the feature vector with the smallest distance is selected from the distances less than or equal to the first distance threshold. It can be seen that the feature vector with the smallest distance from the feature vector of the animal image to be identified among the multiple feature vectors is the feature vector with the highest similarity to the feature vector of the animal image to be identified.
[0072] Based on the above content, it can be known that multiple feature vectors in the feature database are feature vectors of animal images that are incorrectly identified after being recognized by the AI classification model. When there are feature vectors in the multiple feature vectors in the feature database that are highly similar to the feature vectors of the animal image to be identified obtained by the AI classification model, it can be determined that the recognition result of the AI classification model for the animal image to be identified is wrong, that is, the recognition result obtained by the first classification network is wrong. In this case, the true species label corresponding to the feature vector in the feature database with the smallest distance from the feature vector of the animal image to be identified is determined as the species label of the animal image to be identified. It can be seen that the correct species of the animal in the animal image to be identified can be determined by the above method, and the erroneous recognition results obtained by the AI classification model can be corrected, thereby improving the accuracy of species identification.
[0073] Optionally, the distance between the above-mentioned feature vectors may be a cosine distance, a Euclidean distance, or other distances, which is not limited in the embodiment of the present application.
[0074] As an implementation method, when the distance between the above feature vectors is a cosine distance, the cosine distance is obtained by cosine similarity, where cosine distance = 1-cosine similarity. Cosine similarity can be obtained by the following calculation formula:
[0075]
[0076] In the above formula, cossim(x,y) is the cosine similarity, x i is the ith eigenvector value of the eigenvector x of the animal image to be identified, y i is the i-th eigenvector value of a eigenvector y in the feature database. The dimensions of the feature vector of the animal image to be identified and the feature vector in the feature database are both M, 1≤i≤M, and M is the same as the number of channels of the second fully connected layer.
[0077] As an implementation method, the first distance threshold is the average value of the distances between the feature vectors of multiple pairs of animal images of the same species. In the embodiment of the present application, multiple pairs of animal images of the same species are obtained, and the distance between the feature vectors of each pair of animal images is calculated respectively (for example, it can be calculated by the above-mentioned cosine distance calculation method), and multiple distances can be obtained, and then the average value of the multiple distances is used as the first distance threshold.
[0078] Optionally, the number of the multiple pairs of animal images of the same species is 5 to 50 pairs.
[0079] Optionally, in the process of determining the first distance threshold, the distance of the feature vector of each pair of animal images of the same species is calculated. After obtaining multiple distances, the abnormal distances in the multiple distances can be eliminated, and then the first distance threshold is determined based on the average value of the remaining distances in the multiple distances. It can be understood that the abnormal distance may be caused by the quality of the image (for example, the animal features of the image are not obvious, the image is blurred due to the shooting angle, shooting distance and / or lighting conditions), etc., which causes the distance between a pair of animal images of the same species to be abnormal, indicating that the animal image corresponding to the abnormal distance does not have good species attributes and / or characteristics, and the species image should not be used as a basis for distance comparison. It can be seen that using the abnormal distance to calculate the above-mentioned first distance threshold will result in a lower accuracy of the calculated first distance threshold. Therefore, the accuracy of the first distance threshold calculated after eliminating the abnormal distances in multiple distances is higher.
[0080] In the embodiment of the present application, the distance between the feature vectors of each pair of animal images of the same species is calculated, and among the multiple distances obtained, the distance greater than the second distance threshold is determined as an abnormal distance.
[0081] In one implementation, the second distance threshold is determined by calculating the distance between feature vectors of each pair of animal images of the same species, averaging all distances to obtain an average distance value, and determining N (N≥1) times the average distance value as the second distance threshold.
[0082] Optionally, the value of N may be 2 or 3.
[0083] In another implementation, the second distance threshold is determined by: determining an average distance value of distances between feature vectors of multiple pairs of animal images of different species as the second distance threshold.
[0084] Optionally, the plurality of pairs of animal images of the same species may be selected from: animal images of the same individual (referring to the same animal) in the same or different conditions, wherein the conditions include at least one of light conditions, natural conditions and motion forms. For example, animal images of the same individual in the same light conditions, the same natural conditions and different motion forms; or, animal images of the same individual in the same light conditions, different natural conditions and the same motion forms; or, animal images of the same individual in different light conditions, the same natural conditions and the same motion forms;
[0085] For example, taking the red-billed blue magpie of the order Passeriformes as an example, images of the red-billed blue magpie are captured at noon (light conditions) by the water (natural conditions), with the animal in standing and flying states, respectively, to obtain a pair of bird images of the same species.
[0086] Optionally, the plurality of pairs of animal images of the same species may also be selected from: animal images of different individuals of the same species under the same or different conditions.
[0087] For example, taking the Red-flanked White-eye of the Passeriformes as an example, an image of a female Red-flanked White-eye flying by the water at noon is captured, and an image of a male Red-flanked White-eye flying by the water at noon is captured, to obtain a pair of images of birds of the same species.
[0088] S104. When the distances between multiple feature vectors in the feature database and the feature vector of the animal image to be identified are all greater than a first distance threshold, determine the species label of the animal image to be identified according to the classification result of the first classification network in the AI classification model.
[0089] In an embodiment of the present application, if the distance between the feature vectors of two animal images is greater than a first distance threshold, it means that the similarity between the two feature vectors is very low, and it also means that the similarity between the animal images corresponding to the two feature vectors is very low. In this way, it can be considered that the species similarity of the animals in the two images is very low.
[0090] When there is no feature vector similar to the feature vector of the animal image to be identified obtained by the AI classification model among the multiple feature vectors in the feature database (that is, the similarity between the multiple feature vectors and the feature vector of the animal image to be identified obtained by the AI classification model is very low), it can be determined that the recognition result of the AI classification model for the animal image to be identified is correct, that is, the classification result obtained by the first classification network is correct. In this case, the animal species label obtained by the first classification network is directly used as the species label of the animal in the animal image to be identified. Specifically, the classification result of the first classification network includes the probability that the animal in the animal image to be identified belongs to each species among all animal species in the feature database.
[0091] In summary, the animal species identification method provided in the embodiment of the present application establishes a feature database based on the incorrectly identified animal image. After the feature vector of the animal image to be identified is obtained through the AI classification model, the feature vector of the animal image to be identified is judged by distance with multiple feature vectors in the feature database, and whether the recognition result of the AI classification model is correct is determined based on the distance judgment result. In the case of an identification error, the true species label corresponding to the feature vector in the feature database with the smallest distance to the feature vector of the animal image to be identified is determined as the species label of the animal in the animal image to be identified. In this way, the correct species of the animal in the animal image to be identified can be determined. It can be seen that the species identification method can correct the erroneous recognition results obtained by the AI classification model, thereby improving the accuracy of species identification.
[0092] Furthermore, in the embodiment of the present application, when the recognition result obtained by the AI classification model is inaccurate, the species of the animal in the animal image to be identified is determined according to the feature vector in the feature database, without spending more computing power resources and time resources to retrain or continue to train the AI classification model, which can save computing power and labor costs. Therefore, the species identification method provided in the embodiment of the present application can quickly complete species identification, which helps to improve the efficiency of species identification.
[0093] Combine the following Figure 4 The training process of the AI classification model in the embodiment of the present application is briefly described. Figure 4 As shown, the training process of the AI classification model includes the following A1-A3.
[0094] A1. Model training.
[0095] The animal images in the second image data set are input into the AI classification model, the AI classification model outputs the classification results of the animal images, and determines the recognition results of the animal images based on the classification results (the recognition results include the species labels of the animals), and then compares the recognition results with the actual species labels of the animals in the animal images, adjusts the loss function value based on the comparison results, performs gradient back propagation, and updates the parameters of the AI classification model based on the set optimization algorithm.
[0096] Optionally, the situation where the recognition result of the above-mentioned animal image is inconsistent with the real species label of the animal in the animal image (i.e., recognition error) may include inter-species misidentification and environmental misidentification. Among them, inter-species misidentification refers to misidentifying one animal species as another animal species. For example, a large-billed jay in the order Passeriformes is misidentified as a bald crow in the order Passeriformes; environmental misidentification refers to misidentifying environmental objects in an animal image as animals of a certain species.
[0097] Therefore, in order to avoid the above-mentioned problem of misidentification of the environment, when the AI classification model identifies interference / obstructions / macro environment (such as white clouds) in an animal image as an animal of a certain species, the feature vector of the animal image obtained by the AI classification model is added to the feature database, and the species label is marked as the environment.
[0098] It should be understood that the model training of the AI classification model is carried out through multiple rounds of iterations. Each round of iteration traverses the training set once, and each time a small batch of samples is obtained from it and sent to the AI classification model to perform forward calculations to obtain predicted values. The training effect of the AI classification model is judged by observing the decreasing trend of the loss value in each round of iteration. For example, when the decreasing trend of the loss value remains unchanged, it indicates that the AI classification model tends to converge.
[0099] A2. Model evaluation.
[0100] The animal images in the image test data set are input into the trained AI classification model for evaluation to obtain the classification results. According to the correctness of the classification results, the loss function value is adjusted and the evaluation index value is calculated to facilitate the evaluation of the effect of the AI classification model.
[0101] Optionally, the image test data set may be selected from the second image data set, or may be selected from the above-mentioned image set.
[0102] A3. Model reasoning.
[0103] The animal images in the image verification data set are input into the evaluated AI classification model to perform reasoning, and the reasoning results are observed and verified to see whether they meet expectations. The AI classification model that can meet expectations is used as the final classification model, and then used to identify the species of animals in the animal images (for example, used in S101 and S102 in the above embodiments).
[0104] It is understandable that when the above reasoning results do not meet expectations, the AI classification model continues to be trained until the reasoning results of the AI classification model meet expectations.
[0105] Optionally, the image verification data set may be selected from the second image data set, or may be selected from the above-mentioned image set.
[0106] Accordingly, the present application embodiment provides a species identification device, such as Figure 5 As shown, it includes a construction module 501, a processing module 502 and a determination module 503.
[0107] The construction module 501 is used to construct a feature database based on the AI classification model and the first image data set; the AI classification model includes a first classification network and a second classification network, and the feature database includes feature vectors of multiple target images and real species labels of animals in multiple target images; the species labels of multiple target images include animal images in the first image data set that are incorrectly identified after being identified by the first classification network, and the feature vectors of the multiple target images are processed by the first classification network and the second classification network of the AI classification model in turn. For example, the construction module 501 is used to implement S101 of the above-mentioned species identification method.
[0108] The processing module 502 is used to process the to-be-identified animal image based on the AI classification model to obtain a feature vector of the to-be-identified animal image. For example, the processing module 502 is used to implement S102 of the above-mentioned species identification method.
[0109] The determination module 503 is used to, when there is a distance less than or equal to a first distance threshold between the multiple feature vectors in the feature database and the feature vector of the animal image to be identified, determine the species label corresponding to the feature vector with the smallest distance from the feature vector of the animal image to be identified among the multiple feature vectors in the feature database as the species label of the animal image to be identified; when the distances between the multiple feature vectors in the feature database and the feature vector of the animal image to be identified are all greater than the first distance threshold, determine the species label of the animal image to be identified according to the classification result of the first classification network in the AI classification model. For example, the determination module 503 is used to implement S103-S104 of the above-mentioned species identification method.
[0110] Optionally, the processing module 502 is specifically used to: process the image of the animal to be identified based on the first classification network in the AI classification model to obtain a classification result of the image of the animal to be identified; and process the classification result of the image of the animal to be identified based on the second classification network in the AI classification model to obtain a feature vector of the image of the animal to be identified. For example, the processing module 502 is specifically used to implement S1021-S1022 of the above-mentioned species identification method.
[0111] The modules of the above-mentioned species identification device can also be used to execute other steps in the above-mentioned method embodiment. All relevant contents involved in the above-mentioned method embodiment can be referred to the functional description of the corresponding functional module, which will not be repeated here.
[0112] The embodiment of the present application also provides an electronic device, including: a processor and a memory coupled to the processor; the memory is used to store computer instructions, and when the electronic device is running, the processor executes the computer instructions stored in the memory, so that the electronic device executes the method described in the above embodiment. Among them, the processor can implement the above-mentioned construction module 501, processing module 502 and determination module 503; the above-mentioned memory can also be used for the above-mentioned storage image set, first image data set, second image data set, image test data set, image verification data set, feature database and AI classification model, etc.
[0113] An embodiment of the present application further provides a computer-readable storage medium, which includes a computer program. When the computer program runs on a computer, the method described in the above embodiment is executed.
[0114] An embodiment of the present application further provides a computer program product, which includes computer program instructions. When the computer program instructions are run on a computer, the method described in the above embodiment is executed.
[0115] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.
[0116] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. A species identification method, characterized in that: include: Building a feature database based on the AI classification model and the first image dataset; The AI classification model includes a first classification network and a second classification network, and the feature database includes feature vectors of multiple target images and real species labels of animals in the multiple target images; wherein the multiple target images include animal images in the first image data set that are incorrectly identified after being identified by the first classification network, and the feature vectors of the multiple target images are processed by the first classification network and the second classification network of the AI classification model in sequence; Processing the animal image to be identified based on the AI classification model to obtain a feature vector of the animal image to be identified; In the case where a distance between a plurality of feature vectors in the feature database and the feature vector of the to-be-identified animal image is less than or equal to a first distance threshold, the species label corresponding to the feature vector having the smallest distance to the feature vector of the to-be-identified animal image among the plurality of feature vectors in the feature database is determined as the species label of the to-be-identified animal image; When the distances between multiple feature vectors in the feature database and the feature vector of the animal image to be identified are all greater than a first distance threshold, the species label of the animal image to be identified is determined according to the classification result of the first classification network in the AI classification model.
2. The method according to claim 1, characterized in that The processing of the animal image to be identified based on the AI classification model to obtain a feature vector of the animal image to be identified includes: Processing the to-be-identified animal image based on the first classification network in the AI classification model to obtain a classification result of the to-be-identified animal image; The classification result of the animal image to be identified is processed based on the second classification network in the AI classification model to obtain a feature vector of the animal image to be identified.
3. The method according to claim 1 or 2, characterized in that The first classification network is the EfficientNetV2 network.
4. The method according to claim 1 or 2, characterized in that: The second classification network includes a fully connected layer.
5. The method according to claim 1 or 2, characterized in that: The first distance threshold is an average value of distances between feature vectors of multiple pairs of animal images of the same species.
6. A species identification device, characterized in that: It includes building module, processing module and determining module; The construction module is used to construct a feature database based on the AI classification model and the first image data set; the AI classification model includes a first classification network and a second classification network, and the feature database includes feature vectors of multiple target images and real species labels of animals in the multiple target images; wherein the species labels of the multiple target images include animal images in the first image data set that are misidentified after being identified by the first classification network, and the feature vectors of the multiple target images are obtained by being processed by the first classification network and the second classification network of the AI classification model in sequence; The processing module is used to process the animal image to be identified based on the AI classification model to obtain a feature vector of the animal image to be identified; The determination module is used to determine the species label corresponding to the feature vector with the smallest distance from the feature vector of the animal image to be identified among the multiple feature vectors in the feature database as the species label of the animal image to be identified when the distances between the multiple feature vectors in the feature database and the feature vector of the animal image to be identified are less than or equal to a first distance threshold; and determine the species label of the animal image to be identified according to the classification result of the first classification network in the AI classification model when the distances between the multiple feature vectors in the feature database and the feature vector of the animal image to be identified are all greater than the first distance threshold.
7. The device according to claim 6, characterized in that The construction module is specifically used to process the animal image to be identified based on the first classification network in the AI classification model to obtain the classification result of the animal image to be identified; and process the classification result of the animal image to be identified based on the second classification network in the AI classification model to obtain the feature vector of the animal image to be identified.
8. An electronic device, characterized in that: It comprises a processor and a memory coupled to the processor; the memory is used to store computer instructions, and when the electronic device is running, the processor executes the computer instructions stored in the memory, so that the electronic device executes the method as described in any one of claims 1 to 5.
9. A computer-readable storage medium, characterized in that: The method comprises computer program instructions, which, when executed by a computer, cause the computer to perform the method according to any one of claims 1 to 5.
10. A computer program product, characterized in that The method comprises computer program instructions, which, when executed on a computer, cause the computer to execute the method according to any one of claims 1 to 5.
