Animal identification method and device, equipment and storage medium
By training the first classification model and the second classification model for wild animal recognition, identifying the animal's families and performing fine-grained classification, the problem of high misidentification rate of wild animal recognition in the prior art is solved, and the accuracy of recognition is improved and efficiency.
Patent Information
- Application Number
- CN202411425249.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-12
- Publication Date
- 2025-01-07
AI Technical Summary
In the prior art, there is a problem that there is a high misidentification rate for different families with high similarity and different genera under the same family when identifying wild animals.
By obtaining sample images, position labeling information, first labeling information and second labeling information, training the first classification model and the second classification model, first identify the animal's family, and then classify it in fine-grained manner based on the family information to reduce misidentification Rate.
It effectively reduces the misidentification rate of interfaith and intra-family animals, improves the accuracy of identification, and reduces the workload of manually labeling data sets.
Smart Images
Figure CN119274211A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to an animal identification method, device, equipment and storage medium. Background Art
[0002] There are many kinds of wild animals, and many of them are very similar in appearance and color. In addition, due to the complex wild environment and the random movement trajectories of wild animals, it is difficult to accurately identify wild animals from pictures of wild animals taken by cameras.
[0003] The method used in the prior art is to identify wild animals based on deep learning. Specifically, a neural network model is pre-trained through a large number of sample images, and then images containing animals are identified based on the neural network model to obtain the names of the animals contained in the images.
[0004] However, when using the current neural network model for image recognition, there is a high misidentification rate for animals of different families with high similarity and animals of different genera and species under the same family. Summary of the invention
[0005] The purpose of the present application is to provide an animal identification method, device, equipment and storage medium to address the deficiencies in the above-mentioned prior art, so as to solve the problem in the prior art of high misidentification rate for animals of different families with high similarity and animals of different genera and species under the same family.
[0006] To achieve the above objectives, the technical solutions adopted in this application are as follows:
[0007] In a first aspect, the present application provides an animal identification method, the method comprising:
[0008] Acquire a plurality of sample data, wherein the sample data includes: a sample image, position annotation information, first annotation information, and second annotation information, wherein the first annotation information is used to indicate the category information of the animal in the sample image, the second annotation information is used to indicate the name of the animal in the sample image, and the position annotation information is used to indicate the position of the animal in the sample image;
[0009] Inputting each of the sample images into a first initial classification model, and training the first initial classification model based on the first annotation information and position annotation information of each of the sample images to obtain a first classification model;
[0010] Inputting each of the sample images into a corresponding second initial classification model, and training the second initial classification model based on the second annotation information of each of the sample images to obtain a plurality of second classification models;
[0011] Inputting an image to be identified containing an animal to be identified into the first classification model, and using the first classification model to identify and obtain target family and location information of the animal to be identified in the image to be identified;
[0012] According to the target family and location information of the animal to be identified, the image to be identified is input into the target second classification model corresponding to the target family to obtain the animal name of the animal to be identified, wherein the target second classification model is one of multiple second classification models, each of the second classification models corresponds to a family, and each of the second classification models is used to identify the name of the animal within the corresponding family.
[0013] Optionally, inputting each of the sample images into a first initial classification model, and training the first initial classification model based on first annotation information and position annotation information of each of the sample images to obtain a first classification model includes:
[0014] Input each of the sample images into the first initial classification model, and use the first initial classification model to predict the predicted position and predicted category of each of the sample images;
[0015] The first initial classification model is iteratively trained according to the predicted position, the predicted category, the position annotation information and the first annotation information to obtain the first classification model.
[0016] Optionally, before inputting each of the sample images into a first initial classification model and training the first initial classification model based on the first annotation information and position annotation information of each of the sample images to obtain the first classification model, the method further includes:
[0017] The family information of the animals in each of the sample images is obtained, and the format of the animal family in each of the sample images is converted to obtain each of the first labeling information.
[0018] Optionally, the inputting each of the sample images into a corresponding second initial classification model, and training the second initial classification model based on the second annotation information of each of the sample images to obtain a plurality of second classification models, including:
[0019] Based on the first annotation information, the second annotation information and the position annotation information, each of the sample images is divided into a plurality of target sample image sets, each of the target sample image sets respectively including sample images of an animal;
[0020] The target sample image set is input into a corresponding second initial classification model, and the second initial classification model is trained based on the second annotation information of each sample image in the target sample image set to obtain a second classification model.
[0021] Optionally, the dividing each of the sample images into a plurality of target sample image sets based on the first annotation information, the second annotation information and the position annotation information includes:
[0022] Based on the position annotation information of each of the sample images, cropping each of the sample images to obtain a cropped sample image;
[0023] Classifying each of the cropped sample images based on the first annotation information of each of the cropped sample images to obtain a plurality of subject sample image sets;
[0024] Based on the second annotation information of each of the cropped sample images, the sample images in each of the subject sample image sets are classified to obtain a plurality of target sample image sets.
[0025] Optionally, inputting the target sample image set into a corresponding second initial classification model, and training the second initial classification model based on second annotation information of each sample image in the target sample image set to obtain a second classification model, comprises:
[0026] Inputting the sample images in the target sample image set into the second initial classification model in sequence, and obtaining the predicted name of each of the sample images by prediction by the second initial classification model;
[0027] The second initial classification model is iteratively trained based on the predicted name of each of the sample images and the second annotation information of each of the sample images to obtain the second classification model.
[0028] Optionally, according to the target family and location information of the animal to be identified, the image to be identified is input into a target second classification model corresponding to the target family to obtain the animal name of the animal to be identified, including:
[0029] Cropping the image to be identified according to the position information to obtain a cropped image;
[0030] Determine the target second classification model corresponding to the target family according to the target family of the animal to be identified;
[0031] The cropped image is input into the target second classification model, and the second classification model recognizes the animal name of the animal to be identified.
[0032] In a second aspect, the present application provides an animal identification device, the device comprising:
[0033] an acquisition module, used to acquire a plurality of sample data, wherein the sample data includes: a sample image, position annotation information, first annotation information, and second annotation information, wherein the first annotation information is used to indicate the category information of the animal in the sample image, the second annotation information is used to indicate the name of the animal in the sample image, and the position annotation information is used to indicate the position of the animal in the sample image;
[0034] A first training module, used for inputting each of the sample images into a first initial classification model, and training the first initial classification model based on the first annotation information and position annotation information of each of the sample images to obtain a first classification model;
[0035] A second training module, used for inputting each of the sample images into a corresponding second initial classification model, and training the second initial classification model based on the second annotation information of each of the sample images to obtain a plurality of second classification models;
[0036] A first recognition module, used for inputting an image to be recognized containing an animal to be recognized into the first classification model, and obtaining target family and location information of the animal to be recognized in the image to be recognized by the first classification model;
[0037] The second recognition module is used to input the image to be identified into the target second classification model corresponding to the target family according to the target family and location information of the animal to be identified, so as to obtain the animal name of the animal to be identified, wherein the target second classification model is one of multiple second classification models, each of the second classification models corresponds to a family, and each of the second classification models is used to identify the name of the animal within the corresponding family.
[0038] Optionally, the first training module is specifically used for:
[0039] Input each of the sample images into the first initial classification model, and use the first initial classification model to predict the predicted position and predicted category of each of the sample images;
[0040] The first initial classification model is iteratively trained according to the predicted position, the predicted category, the position annotation information and the first annotation information to obtain the first classification model.
[0041] Optionally, the first training module is further specifically used for:
[0042] The family information of the animals in each of the sample images is obtained, and the format of the animal family in each of the sample images is converted to obtain each of the first labeling information.
[0043] Optionally, the second training module is specifically used for:
[0044] Based on the first annotation information, the second annotation information and the position annotation information, each of the sample images is divided into a plurality of target sample image sets, each of the target sample image sets respectively including sample images of an animal;
[0045] The target sample image set is input into a corresponding second initial classification model, and the second initial classification model is trained based on the second annotation information of each sample image in the target sample image set to obtain a second classification model.
[0046] Optionally, the second training module is specifically used for:
[0047] Based on the position annotation information of each of the sample images, cropping each of the sample images to obtain a cropped sample image;
[0048] Classifying each of the cropped sample images based on the first annotation information of each of the cropped sample images to obtain a plurality of subject sample image sets;
[0049] Based on the second annotation information of each of the cropped sample images, the sample images in each of the subject sample image sets are classified to obtain a plurality of target sample image sets.
[0050] Optionally, the second training module is specifically used for:
[0051] Inputting the sample images in the target sample image set into the second initial classification model in sequence, and obtaining the predicted name of each of the sample images by prediction by the second initial classification model;
[0052] The second initial classification model is iteratively trained based on the predicted name of each of the sample images and the second annotation information of each of the sample images to obtain the second classification model.
[0053] Optionally, the second identification module is specifically used to:
[0054] Cropping the image to be identified according to the position information to obtain a cropped image;
[0055] Determine the target second classification model corresponding to the target family according to the target family of the animal to be identified;
[0056] The cropped image is input into the target second classification model, and the second classification model recognizes the animal name of the animal to be identified.
[0057] In a third aspect, an embodiment of the present application further provides an electronic device comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the animal identification method as described in any one of the first aspects.
[0058] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the animal identification method as described in any one of the first aspects are executed.
[0059] The beneficial effect of the present application is that by adding first annotation information representing the family information and second annotation information representing the animal name to the sample image, the sample image can meet the training of the first classification model and the second classification model, reducing the workload of manually annotating the data set. The accuracy of identifying animals between families can be improved by the first classification model, and the accuracy of identifying animals within a family can be improved by the second classification model. The family information is first identified, and then the image is input into the corresponding second classification model for fine-grained classification and identification based on the identified family, making the identification process more refined and effectively reducing the misidentification rate of animals between and within families.
[0060] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0061] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0062] Figure 1 A schematic diagram of the architecture of a model provided in an embodiment of the present application is shown;
[0063] Figure 2 A flow chart of an animal identification method provided by an embodiment of the present application is shown;
[0064] Figure 3 A schematic diagram of an image to be recognized provided by an embodiment of the present application is shown;
[0065] Figure 4 A schematic diagram showing an image recognition result of a first classification model provided in an embodiment of the present application;
[0066] Figure 5 A training flow chart of a first classification model provided in an embodiment of the present application is shown;
[0067] Figure 6 A training flow chart of a second classification model provided in an embodiment of the present application is shown;
[0068] Figure 7 A flow chart of generating a target sample image set provided by an embodiment of the present application is shown;
[0069] Figure 8 A training flow chart of another second classification model provided in an embodiment of the present application is shown;
[0070] Fig. 9 A flowchart of performing image recognition using a second classification model provided in an embodiment of the present application is shown;
[0071] Fig.10 A schematic diagram of the structure of an animal identification device provided in an embodiment of the present application is shown;
[0072] Fig.11 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION
[0073] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.
[0074] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.
[0075] The method used in the prior art is to identify wild animals based on deep learning. Specifically, a neural network model is pre-trained through a large number of sample images, and then images containing animals are identified based on the neural network model to obtain the names of the animals contained in the images.
[0076] Specifically, the first method is to identify based on the target classification algorithm. The specific process is to put the data images into the corresponding category folders according to the animal categories, and then train the classification model based on the images in the folder to obtain the classification recognition model, and then identify the images containing animals based on the classification recognition model to obtain the name of the animal.
[0077] However, this method has high requirements for the data set, and it can only identify different animal categories, and cannot accurately identify animals with high similarity between families or within families. For example, there is a high similarity between the Felidae and the Civetidae, the Cervidae and the Musk Deer. When using the first method for identification, if you want to achieve accurate category identification, you need to create folders for different animals and train classification models, which undoubtedly increases the workload of model training and increases the complexity of data set establishment.
[0078] The second method is the target detection algorithm. First, the animals in the image data are marked to generate labels, and then the images and labels are sent to the target detection algorithm for model training. The test images are sent to the trained target detection model to predict the location and category of the animals in the image.
[0079] However, since the appearance differences among animals within a family are very small, the algorithm cannot accurately identify animals within the same family, and even some animals between different families are prone to misidentification.
[0080] Based on this, the present application proposes an animal identification method, which first labels the sample images containing animals with their families and specific names, creates a data set according to the families and trains to obtain a first classification model, creates a data set according to the specific names and trains to obtain a second classification model, detects the family of the animal and the position of the animal in the image through the first classification model, and identifies the specific name of the animal through the second classification model. The accuracy of inter-family and intra-family animal identification is improved, and compared with the existing technology, the cost of making the data set is also greatly reduced.
[0081] Figure 1 This is a basic architecture diagram of a model given in this application, refer to Figure 1 The models used in the animal recognition method of the present application include a first classification model and multiple second classification models. The first classification model can be a target detection model, which is used to detect the position of the animal in the image and identify the category information to which the animal belongs. After the category to which the animal belongs is identified based on the first classification model, the image can be input into the corresponding second classification model to identify the specific name of the animal. The second classification models are used to identify the specific names of animals under a family. By performing rough recognition of the category information of the image and specific recognition of the name within the family, the problem of easy misidentification of animals between and within families can be solved.
[0082] Next, combine Figure 2 The animal identification method of the present application is described. The execution subject of the method can be an electronic device, such as Figure 2 As shown, the method includes:
[0083] S201. Acquire multiple sample data, the sample data including: a sample image, position annotation information, first annotation information, and second annotation information, the first annotation information is used to indicate the family information of the animal in the sample image, the second annotation information is used to indicate the name of the animal in the sample image, and the position annotation information is used to represent the position of the animal in the sample image.
[0084] The sample images may be images of animals of various families that need to be identified. As an example, images of wild animals may be obtained by setting up cameras in wild areas or from animal databases. After a large number of images are collected, the collected data may be screened, and data sets with unclear image quality, blurred animals or severe occlusion may be deleted to complete data set cleaning, and then annotation information may be added to the cleaned images.
[0085] The annotation information includes position annotation information, first annotation information and second annotation information. The position annotation information is used to indicate the position of the animal in the image. As an example, the animal in the image can be selected by a rectangular selection box to mark the position of the animal in the image.
[0086] The first labeling information may indicate the family to which the animal in the sample image belongs, for example, the animal belongs to the family of cats, the family of dogs, the family of deer, the family of musk deer, etc. The second labeling information may indicate the specific name of the animal contained in the sample image.
[0087] The first labeling information and the second labeling information may be pre-labeled in the sample image in the form of a character string, for example, labeling the sample image with "Phasianidae_Pheasant", wherein "Phasianidae" is the first labeling information and "Pheasant" is the second labeling information.
[0088] It is worth noting that the problem solved in the present application is that animals of different families are difficult to identify. Therefore, the family information to which the animals belong is marked. As a possible implementation method, when it is necessary to identify animals of different genera more accurately, the genus to which the animals belong can also be marked. For example, pictures of cats can be further marked with genus as lynx, feline, cheetah, clouded leopard, etc.
[0089] S202: Input each sample image into a first initial classification model, and train the first initial classification model based on first annotation information and position annotation information of each sample image to obtain a first classification model.
[0090] The first classification model may be a target detection model, which is used to perform target detection on the sample image to obtain the position of the animal contained in the sample image and the family to which the animal belongs. Exemplarily, the first classification model may be a YOLOv8 (You only look once version 8) target detection model.
[0091] After the sample image is input into the first initial classification model, the first initial classification model can predict the sample image to obtain the predicted animal position and the predicted animal family, and then the first initial classification model can be trained based on the predicted animal position, the predicted animal family, the position annotation information and the first annotation information to obtain the first classification model.
[0092] It should be noted that when training the first initial classification model, the sample images can be divided into multiple data sets, for example, the sample images can be divided into a training set and a verification set. After the first initial classification model is trained based on the sample images of the training set, the model performance can be verified based on the sample images of the verification set. After multiple rounds of training and optimization, if the model obtained in the current round meets the preset iteration end requirements, the model of the current round can be used as the first classification model.
[0093] S203: Input each sample image into a corresponding second initial classification model, and train the second initial classification model based on the second annotation information of each sample image to obtain a plurality of second classification models.
[0094] The second classification model may be a fine-grained classification model for identifying the name of the animal contained in the sample image. Each second classification model is used to identify a sample image of a family. Exemplarily, the second classification model may be a CMAL_Net model.
[0095] Optionally, each sample image can be classified into multiple data sets based on the second annotation information of the sample image, for example, the sample images of the Phasianidae can be further subdivided into multiple data sets such as chicken, pheasant, peacock, etc., and then the second initial classification model is trained based on each data set to obtain a second classification model. In this way, after the sample image of the Phasianidae animal is input into the second classification model, the second classification model can identify which specific category of the Phasianidae animal the sample image belongs to, and obtain the name of the animal in the sample image.
[0096] S204, inputting the image to be identified containing the animal to be identified into the first classification model, and using the first classification model to identify and obtain the target family and location information of the animal to be identified in the image to be identified.
[0097] The image to be identified may include at least one animal to be identified. After the image to be identified is input into the first classification model, the first classification model can identify the location information of all animals contained in the image to be identified and the target family to which the animals belong.
[0098] The position information is used to represent the position of the animal to be identified in the image to be identified, and the position information may be the coordinate positions of the four vertices of the minimum enclosing rectangle of the animal to be identified. The target family may represent the family to which the animal to be identified belongs, such as Phasianidae, Felidae, Canidae, Cervidae, Musk Deer, Bovidae, etc.
[0099] For example, assuming that the image to be recognized is Figure 3 As shown, the figure includes three animals, animal 1, animal 2 and animal 3. After the image to be identified is input into the first classification model, the first classification model can identify the positions of animal 1 and animal 2 in the image, as well as the family "Phasianidae" to which animal 1 and animal 2 belong, and identify the position of animal 3 and the family "Felidae" to which animal 3 belongs, and obtain Figure 4 The recognition results are shown.
[0100] S205. According to the target family and location information of the animal to be identified, the image to be identified is input into the target second classification model corresponding to the target family to obtain the animal name of the animal to be identified, wherein the target second classification model is one of multiple second classification models, each second classification model corresponds to a family, and each second classification model is used to identify the name of the animal within the corresponding family.
[0101] Reference Figure 1 In the embodiment of the present application, the second classification model can be used to further identify the animals of each family. The target second classification model can be a second classification model for identifying the target family, which is used to identify all animals under the target family and obtain the name of the animal.
[0102] As a possible implementation, the animals included in the image to be identified can be determined based on the position information. When the image to be identified includes multiple animals, the image area where each animal is located can be determined based on the position information, and each image area can be input into the target second classification model for identification to obtain the name of the animal in each image area.
[0103] If the target families of multiple animals in the image to be identified are different, the family of the animal in each position information can be determined separately, and the target second classification model corresponding to each family can be determined, and then the image area can be input into the target second classification model corresponding to the target family.
[0104] Reference Figure 3, the image to be identified includes three animals. After obtaining the target family of each animal, the image areas of animal 1 and animal 2 can be input into the target second classification model corresponding to "Phasianidae" to obtain the animal names "gray jungle fowl" and "mountain peacock", and the image area of animal 3 can be input into the target second classification model corresponding to "Felidae" to obtain the animal name "leopard cat" corresponding to the image area.
[0105] In an embodiment of the present application, a plurality of sample data are obtained, the sample data including: a sample image, position annotation information, a first annotation information, and a second annotation information, the first annotation information being used to indicate the family information of the animal in the sample image, the second annotation information being used to indicate the name of the animal in the sample image, the position annotation information being used to characterize the position of the animal in the sample image, each sample image being input into a first initial classification model, the first initial classification model being trained based on the first annotation information and the position annotation information of each sample image to obtain a first classification model, each sample image being input into a corresponding second initial classification model, the second initial classification model being trained based on the second annotation information of each sample image to obtain a plurality of second classification models, an image to be identified containing an animal to be identified being input into the first classification model, the first classification model being used to identify the target family and location information of the animal to be identified in the image to be identified, according to the target family and location information of the animal to be identified, the image to be identified being input into a target second classification model corresponding to the target family to obtain the name of the animal to be identified, wherein the target second classification model is one of a plurality of second classification models, each second classification model corresponds to a family, and each second classification model is used to identify the name of the animal within the corresponding family.
[0106] By adding first annotation information representing the family information and second annotation information representing the animal name to the sample image, the sample image can meet the training of the first classification model and the second classification model, reducing the workload of manually annotating the data set. The first classification model can improve the accuracy of identifying animals between families, and the second classification model can improve the accuracy of identifying animals within a family. The family information is first identified, and then the image is input into the corresponding second classification model for fine-grained classification and identification based on the identified family, making the identification process more refined and effectively reducing the misidentification rate of animals between and within families.
[0107] The following is a further description of inputting each sample image into the first initial classification model, training the first initial classification model based on the first annotation information and position annotation information of each sample image, and obtaining the first classification model. Figure 5 As shown, the above step S202 includes:
[0108] S501. Input each sample image into a first initial classification model, and use the first initial classification model to predict the predicted position and predicted category of each sample image.
[0109] The structure of the first initial classification model includes a backbone network layer and a detection layer. The backbone network layer is used to extract multi-scale features of the sample image, obtain image features of multiple resolutions and dimensions, and transmit the image features to the detection layer. The detection layer is used to fuse and connect multiple image features and predict the location and family of the animals contained in the sample image. The predicted location of the sample image can be obtained by Figure 4 The detection box shown represents.
[0110] S502. Iteratively train the first initial classification model according to the predicted position, predicted category, position annotation information, and first annotation information to obtain a first classification model.
[0111] As a possible implementation manner, the sample images may be divided into multiple data sets, and the first initial classification model may be trained for multiple rounds based on the multiple data sets.
[0112] After executing the above S501 step to obtain the prediction result, the loss value of the location can be calculated based on the predicted location and the location annotation information, and the loss value of the predicted category can be calculated based on the predicted category and the first annotation information. By backpropagating the loss value of the location and the loss value of the predicted category, it is possible to achieve iterative optimization of the first initial classification model.
[0113] During multiple rounds of training optimization, if the number of iterations meets the preset iteration threshold, or the loss value of the current model converges to a preset value, the model obtained in the current round can be used as the first classification model.
[0114] Before inputting each sample image into the first initial classification model and training the first initial classification model based on the first annotation information and the position annotation information of each sample image to obtain the first classification model, the method of the present application further includes:
[0115] The family information of the animals in each sample image is obtained, and the format of the animal family in each sample image is converted to obtain each first labeling information.
[0116] As a possible implementation method, researchers can add annotation information to the animals in each sample image in advance to mark the family to which the animal in the sample image belongs. The annotation information can be in the form of a string, such as Chinese characters.
[0117] After completing the labeling of the class information of all sample images, the electronic device can convert the label information into a form that is easy for the device to understand, and based on a preset mapping relationship table, convert the label information into numbers, specifically converting the tags in the form of an XML file into a TXT format to obtain the first label information. For example: Chinchilla (xml) into 0 (txt), Ursidae (xml) into 1 (txt), etc.
[0118] It should be noted that the format conversion of the animal family of each sample image can be to convert the animal family information into a data format that can be recognized by the first classification model. The above example only uses the txt data format required by the yolo series as an example for explanation and should not be limited to this.
[0119] Next, the process of inputting each sample image into the corresponding second initial classification model, training the second initial classification model based on the second annotation information of each sample image, and obtaining multiple second classification models is described. Figure 6 As shown, the above step S203 includes:
[0120] S601 . Divide each sample image into a plurality of target sample image sets based on first annotation information, second annotation information, and position annotation information. Each target sample image set includes sample images of one animal.
[0121] The category to which the sample image belongs can be determined based on the first annotation information, so sample images with the same first annotation information can be added to a unified sample image set to obtain sample image sets of various categories.
[0122] Based on the second annotation information, the name of the animal contained in the sample image can be determined. Therefore, each sample image set can be further subdivided based on the second annotation information, and the sample images in the sample image set can be divided into different target sample image sets according to the animal names, and each target sample image set includes sample images of one animal.
[0123] The position annotation information may indicate the position of the animal in the sample image, and based on the position annotation information, the position of at least one animal in the sample image may be determined, and the image region where the animal is located may be obtained. In the process of further classification based on the second annotation information, the region where the animal is located may be obtained from the sample image based on the position annotation information, and the image of this region may be added as a sample image to the target sample image set.
[0124] S602: Input the target sample image set into the corresponding second initial classification model, and train the second initial classification model based on the second annotation information of each sample image in the target sample image set to obtain a second classification model.
[0125] Optionally, multiple target sample image sets belonging to the same category can be input into a second initial classification model of the category, the second initial classification model can be trained based on the sample images in each target sample image set, and the second initial classification model can be iteratively optimized based on the prediction results of the second initial classification model and the second annotation information to obtain a second classification model.
[0126] The second initial classification model can be implemented based on a fine-grained image classification algorithm, for example, through a general DCNN (Deep Convolutional Neural Network), a positioning-recognition based method, a network integration based method, or a high-order coding method based on convolutional features for fine-grained classification. The specific method is not limited in this application.
[0127] Furthermore, the process of dividing each sample image into a plurality of target sample image sets based on the first annotation information, the second annotation information and the position annotation information is as follows: Figure 7 As shown, including:
[0128] S701 . Crop each sample image based on the position annotation information of each sample image to obtain a cropped sample image.
[0129] As a possible implementation, the location annotation information may be a minimum bounding box of the animal. By cropping the sample image based on the minimum bounding box, the area where the animal is located, ie, the cropped sample image, may be obtained.
[0130] By cropping the sample images of the area where the animals are located based on the location annotation information, the influence of other content in the image can be reduced, thereby improving the accuracy of subsequent training of the second classification model.
[0131] S702: Classify each cropped sample image based on the first annotation information of each cropped sample image to obtain a plurality of subject sample image sets.
[0132] The sample image may contain multiple animals of the same family or different families. If the sample image contains animals of different families, after the sample image is cropped based on the position annotation information, each cropped sample image can be added to the sample image set of the corresponding family based on the first annotation information.
[0133] The sample images in the family sample image set are sample images of animals of the same family.
[0134] S703 , classifying the sample images in the sample image sets of each subject based on the second annotation information of each cropped sample image, to obtain a plurality of target sample image sets.
[0135] The sample images in each target sample image set may be sample images containing animals of the same type.
[0136] Exemplarily, the sample images can first be divided into "Bovidae", "Anatidae", "Phasianidae" and "Legionidae" based on the first annotation information. For all sample images of "Bovidae", after the image region where the animal is located and the name of the animal are determined based on the position annotation information of each sample image and the second annotation information, the sample images can be further divided into three target sample image sets of "cattle", "antelope" and "rhinoceros". Next, by executing the above step S602, the three target sample image sets of "cattle", "antelope" and "rhinoceros" can be used as the input of the second initial classification model corresponding to "Bovidae", and the second classification model of "Bovidae" can be trained.
[0137] In an embodiment of the present application, by dividing the sample image set based on position annotation information, first annotation information and second annotation information, and using the same set of sample images to annotate category information and animal names, the sample image set can simultaneously meet the training requirements of the subsequent first classification model and the second classification model, thereby reducing the labor cost of data set production.
[0138] The following is a further description of the above-mentioned inputting the target sample image set into the corresponding second initial classification model, training the second initial classification model based on the second annotation information of each sample image in the target sample image set, and obtaining the second classification model. Figure 8 As shown, the above step S602 includes:
[0139] S801. Input sample images in the target sample image set into a second initial classification model in sequence, and obtain a predicted name of each sample image by prediction by the second initial classification model.
[0140] S802: Iteratively train the second initial classification model based on the predicted name of each sample image and the second annotation information of each sample image to obtain a second classification model.
[0141] Optionally, the target sample image set can be divided into a training set and a validation set, and the second initial classification model can be trained for multiple rounds based on the training set and the validation set. After each round of training, the loss value can be calculated based on the predicted name predicted by the second initial classification model and the second annotation information, and then the second initial classification model of the current round can be optimized based on the loss value to obtain the second classification model of the current round.
[0142] If the number of iterations reaches a preset iteration threshold, or the loss value converges to a preset value, the second classification model of the current round can be used as the final second classification model.
[0143] After obtaining the first classification model and the second classification model corresponding to each family, the family of the animal to be identified in the image to be identified can be first identified based on the first classification model, and then the image to be identified is input into the target second classification model corresponding to the target family according to the target family and location information of the animal to be identified, so as to obtain the animal name of the animal to be identified, such as Fig. 9 As shown, the above step S205 includes:
[0144] S901 , cropping the image to be recognized according to the position information to obtain a cropped image.
[0145] After the detection frame based on the position information is used to crop the image to be identified, the cropped image region and the category of the image region can be used as a cropped image.
[0146] S902. Determine a target second classification model corresponding to the target family according to the target family of the animal to be identified.
[0147] If the image to be identified includes a plurality of animals to be identified, the target second classification model corresponding to each target family can be determined respectively according to the target family of each animal to be identified.
[0148] Exemplarily, the image to be identified includes Figure 3 For the three animals to be identified, namely, the gray jungle fowl, the mountain peacock and the leopard cat, after obtaining the family to which the animals belong through the first classification model, the target second classification model corresponding to "Felidae" and "Phasianidae" can be determined based on the target family to which the animals belong.
[0149] S903: Input the cropped image into the target second classification model, and use the second classification model to identify the animal name of the animal to be identified.
[0150] Continuing with the above example, the cropped image of the cat family can be input into the target second classification model of "cat family" to obtain the animal name of the animal to be identified, "leopard cat". The cropped image of the pheasant family can be input into the target second classification model of "pheasant family" to obtain the animal names of the animals to be identified, "gray jungle fowl" and "mountain peacock".
[0151] It is worth noting that the solution of the present application can improve the accuracy of intra-family and inter-family identification of animals. It should be understood that when identifying plants, the accuracy of intra-family and inter-family identification of plants can also be improved based on the method of the present application. Specifically, the first classification model can be obtained by training based on plant images of different families, and the second classification model can be obtained by training based on plant images of the same family. The specific training method can be the same as the above-mentioned S201-S203 steps, and the present application will not go into details here. When performing image recognition, the image containing the plant to be identified is input into the first classification model. After obtaining the family and position of the plant to be identified, the image can be input into the corresponding second classification model based on the family to obtain the name of the plant to be identified.
[0152] Based on the same inventive concept, an animal identification device corresponding to the animal identification method is also provided in the embodiment of the present application. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the above-mentioned animal identification method in the embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.
[0153] Fig.10 A schematic structural diagram of an animal identification device provided in an embodiment of the present application is shown, comprising: an acquisition module 1001, a first training module 1002, a second training module 1003, a first identification module 1004 and a second identification module 1005.
[0154] The acquisition module 1001 is used to acquire a plurality of sample data, wherein the sample data includes: a sample image, position annotation information, first annotation information, and second annotation information, wherein the first annotation information is used to indicate the family information of the animal in the sample image, the second annotation information is used to indicate the name of the animal in the sample image, and the position annotation information is used to indicate the position of the animal in the sample image;
[0155] A first training module 1002 is used to input each sample image into a first initial classification model, and train the first initial classification model based on the first annotation information and position annotation information of each sample image to obtain a first classification model;
[0156] A second training module 1003 is used to input each sample image into a corresponding second initial classification model, and train the second initial classification model based on the second annotation information of each sample image to obtain multiple second classification models;
[0157] The first recognition module 1004 is used to input the image to be recognized containing the animal to be recognized into the first classification model, and the first classification model recognizes the target family and location information of the animal to be recognized in the image to be recognized;
[0158] The second recognition module 1005 is used to input the image to be identified into the target second classification model corresponding to the target family according to the target family and location information of the animal to be identified, so as to obtain the animal name of the animal to be identified, wherein the target second classification model is one of multiple second classification models, each second classification model corresponds to a family, and each second classification model is used to identify the name of the animal within the corresponding family.
[0159] Optionally, the first training module 1002 is specifically used for:
[0160] Input each sample image into the first initial classification model, and use the first initial classification model to predict the predicted position and predicted category of each sample image;
[0161] The first initial classification model is iteratively trained according to the predicted position, predicted category, position annotation information and the first annotation information to obtain a first classification model.
[0162] Optionally, the first training module 1002 is further specifically configured to:
[0163] The family information of the animals in each sample image is obtained, and the format of the animal family in each sample image is converted to obtain each first labeling information.
[0164] Optionally, the second training module 1003 is specifically used for:
[0165] Based on the first annotation information, the second annotation information and the position annotation information, each sample image is divided into a plurality of target sample image sets, each target sample image set respectively including sample images of an animal;
[0166] The target sample image set is input into the corresponding second initial classification model, and the second initial classification model is trained based on the second annotation information of each sample image in the target sample image set to obtain a second classification model.
[0167] Optionally, the second training module 1003 is specifically used for:
[0168] Based on the position annotation information of each sample image, each sample image is cropped to obtain a cropped sample image;
[0169] Classifying each cropped sample image based on the first annotation information of each cropped sample image to obtain a plurality of subject sample image sets;
[0170] Based on the second annotation information of each cropped sample image, the sample images in the sample image sets of each subject are classified to obtain a plurality of target sample image sets.
[0171] Optionally, the second training module 1003 is specifically used for:
[0172] Inputting sample images in the target sample image set into the second initial classification model in sequence, and obtaining a predicted name of each sample image by prediction by the second initial classification model;
[0173] The second initial classification model is iteratively trained based on the predicted name of each sample image and the second annotation information of each sample image to obtain a second classification model.
[0174] Optionally, the second identification module 1005 is specifically used for:
[0175] The image to be identified is cropped according to the position information to obtain a cropped image;
[0176] Determine a target second classification model corresponding to the target family according to the target family of the animal to be identified;
[0177] The cropped image is input into the target second classification model, and the second classification model recognizes the animal name of the animal to be identified.
[0178] The embodiment of the present application adds first annotation information representing the family information and second annotation information representing the animal name to the sample image, so that the sample image can meet the training of the first classification model and the second classification model, reducing the workload of manually annotating the data set. The first classification model can improve the accuracy of identifying animals between families, and the second classification model can improve the accuracy of identifying animals within a family. The family information is first identified, and then the image is input into the corresponding second classification model for fine-grained classification and identification based on the identified family, making the identification process more refined and effectively reducing the misidentification rate of animals between and within families.
[0179] Fig.11 A structural schematic diagram of an electronic device provided in an embodiment of the present application is shown, including: a processor 1101, a storage medium 1102 and a bus 1103, wherein the storage medium 1102 stores machine-readable instructions executable by the processor 1101. When the electronic device runs the animal identification method in the embodiment, the processor 1101 communicates with the storage medium 1102 through the bus 1103, and the processor 1101 executes the machine-readable instructions and the preamble of the method item of the processor 1101 to execute the steps in the above-mentioned animal identification method.
[0180] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. The computer program is executed when a processor is running, and the processor executes the steps in the above-mentioned animal identification method.
[0181] In the embodiment of the present application, the computer program can also execute other machine-readable instructions when run by the processor to execute other methods described in the embodiment. For the specific execution method steps and principles, please refer to the description of the embodiment, which will not be repeated here.
[0182] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.
[0183] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0184] In addition, each functional unit in the embodiments provided in the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0185] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0186] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.
[0187] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application. They should all be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.
Claims
1. A method for animal identification, characterized in that: include: Acquire a plurality of sample data, wherein the sample data includes: a sample image, position annotation information, first annotation information, and second annotation information, wherein the first annotation information is used to indicate the category information of the animal in the sample image, the second annotation information is used to indicate the name of the animal in the sample image, and the position annotation information is used to indicate the position of the animal in the sample image; Inputting each of the sample images into a first initial classification model, and training the first initial classification model based on the first annotation information and position annotation information of each of the sample images to obtain a first classification model; Inputting each of the sample images into a corresponding second initial classification model, and training the second initial classification model based on the second annotation information of each of the sample images to obtain a plurality of second classification models; Inputting an image to be identified containing an animal to be identified into the first classification model, and using the first classification model to identify and obtain target family and location information of the animal to be identified in the image to be identified; According to the target family and location information of the animal to be identified, the image to be identified is input into the target second classification model corresponding to the target family to obtain the animal name of the animal to be identified, wherein the target second classification model is one of multiple second classification models, each of the second classification models corresponds to a family, and each of the second classification models is used to identify the name of the animal within the corresponding family.
2. The method according to claim 1, characterized in that The step of inputting each of the sample images into a first initial classification model, and training the first initial classification model based on the first annotation information and position annotation information of each of the sample images to obtain a first classification model comprises: Input each of the sample images into the first initial classification model, and use the first initial classification model to predict the predicted position and predicted category of each of the sample images; The first initial classification model is iteratively trained according to the predicted position, the predicted category, the position annotation information and the first annotation information to obtain the first classification model.
3. The method according to claim 1, characterized in that Before inputting each of the sample images into the first initial classification model and training the first initial classification model based on the first annotation information and position annotation information of each of the sample images to obtain the first classification model, the method further includes: The family information of the animals in each of the sample images is obtained, and the format of the animal family in each of the sample images is converted to obtain each of the first labeling information.
4. The method according to claim 1, characterized in that: The step of inputting each of the sample images into a corresponding second initial classification model, and training the second initial classification model based on the second annotation information of each of the sample images to obtain a plurality of second classification models comprises: Based on the first annotation information, the second annotation information and the position annotation information, each of the sample images is divided into a plurality of target sample image sets, each of the target sample image sets respectively including sample images of an animal; The target sample image set is input into a corresponding second initial classification model, and the second initial classification model is trained based on the second annotation information of each sample image in the target sample image set to obtain a second classification model.
5. The method according to claim 4, characterized in that The dividing each of the sample images into a plurality of target sample image sets based on the first annotation information, the second annotation information and the position annotation information comprises: Based on the position annotation information of each of the sample images, cropping each of the sample images to obtain a cropped sample image; Classifying each of the cropped sample images based on the first annotation information of each of the cropped sample images to obtain a plurality of subject sample image sets; Based on the second annotation information of each of the cropped sample images, the sample images in each of the subject sample image sets are classified to obtain a plurality of target sample image sets.
6. The method according to claim 4, characterized in that The step of inputting the target sample image set into a corresponding second initial classification model, and training the second initial classification model based on second annotation information of each sample image in the target sample image set to obtain a second classification model comprises: Inputting the sample images in the target sample image set into the second initial classification model in sequence, and obtaining the predicted name of each of the sample images by prediction by the second initial classification model; The second initial classification model is iteratively trained based on the predicted name of each of the sample images and the second annotation information of each of the sample images to obtain the second classification model.
7. The method according to claim 1, characterized in that According to the target category and location information of the animal to be identified, the image to be identified is input into a target second classification model corresponding to the target category to obtain the animal name of the animal to be identified, including: Cropping the image to be identified according to the position information to obtain a cropped image; Determine the target second classification model corresponding to the target family according to the target family of the animal to be identified; The cropped image is input into the target second classification model, and the second classification model recognizes the animal name of the animal to be identified.
8. An animal identification device, characterized in that: include: an acquisition module, used to acquire a plurality of sample data, wherein the sample data includes: a sample image, position annotation information, first annotation information, and second annotation information, wherein the first annotation information is used to indicate the category information of the animal in the sample image, the second annotation information is used to indicate the name of the animal in the sample image, and the position annotation information is used to indicate the position of the animal in the sample image; A first training module, used for inputting each of the sample images into a first initial classification model, and training the first initial classification model based on the first annotation information and position annotation information of each of the sample images to obtain a first classification model; A second training module, used for inputting each of the sample images into a corresponding second initial classification model, and training the second initial classification model based on the second annotation information of each of the sample images to obtain a plurality of second classification models; A first recognition module, used for inputting an image to be recognized containing an animal to be recognized into the first classification model, and obtaining target family and location information of the animal to be recognized in the image to be recognized by the first classification model; The second recognition module is used to input the image to be identified into the target second classification model corresponding to the target family according to the target family and location information of the animal to be identified, so as to obtain the animal name of the animal to be identified, wherein the target second classification model is one of multiple second classification models, each of the second classification models corresponds to a family, and each of the second classification models is used to identify the name of the animal within the corresponding family.
9. An electronic device, characterized in that: include: A processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the storage medium communicate via the bus, and the processor executes the machine-readable instructions to perform the steps of the animal identification method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the animal identification method according to any one of claims 1 to 7 are executed.