Methods and apparatus for determining animal species

Abstract

This application provides a method and apparatus for determining animal species, relating to the field of animal monitoring technology. The method includes: extracting feature data from an acquired image of an animal to be identified, obtaining first feature data, second feature data, and third feature data. The first feature data includes animal surface data, the second feature data includes animal posture data, and the third feature data includes predation information. The multi-dimensional feature data is input into different models to obtain multi-dimensional identification results. The animal species is determined based on these multi-dimensional identification results. Even if feature data in one dimension of the image of the animal to be identified is missing, which may lead to inaccurate identification results in that dimension, a relatively accurate animal species identification result can be obtained based on feature data from other dimensions. Therefore, the animal species obtained through this comprehensive approach is relatively accurate, thereby improving the accuracy of animal species identification.

Description

Technical Field

[0001] This application belongs to the field of animal monitoring technology, and in particular relates to methods and devices for determining animal species. Background Technology

[0002] To better monitor animals in complex environments, it is necessary to identify and monitor animal species to help zoologists or animal protection departments study and understand animal resources. Current animal species identification generally involves inputting collected animal images into a specific model and outputting the animal species in the images. When the collected animal images are incomplete or incomplete, it can lead to inaccurate animal species output, or even failure to output animal species, thus affecting the accuracy of animal species identification. Summary of the Invention

[0003] This application provides a method and apparatus for determining animal species, which can improve the accuracy of animal species identification.

[0004] To achieve the above objectives, in a first aspect, a method for determining animal species is provided, the method being applied to a first device, comprising:

[0005] Acquire a video stream of the animal to be identified, the video stream including a first image;

[0006] Extract first feature data, second feature data and third feature data from the first image. The first feature data includes at least one of the outline of the animal's body surface, the color of the animal's body surface or the texture of the animal's body surface; the second feature data includes the animal's posture; and the third feature data includes predation information of the animal's body surface.

[0007] Input the first feature data into the first model and output the first recognition result;

[0008] The second feature data is input into the second model, and the second recognition result is output.

[0009] Input the third feature data into the third model and output the third recognition result;

[0010] The target animal species to be identified is determined based on the first identification result, the second identification result, and the third identification result.

[0011] Optionally, the method further includes:

[0012] Obtain the audio data corresponding to the video stream;

[0013] The sound data is input into the fourth model, which outputs the fourth recognition result, which indicates the species of the animal to be identified.

[0014] If the target animal species matches the species of the animal to be identified indicated by the fourth identification result, the fourth model is updated based on the fourth identification result and the sound data.

[0015] If the target animal species does not match the species of the animal to be identified indicated by the fourth identification result, the first instruction is output, which is used to instruct the fourth identification result to be manually calibrated.

[0016] Receive the fourth identification result after calibration;

[0017] The fourth model is updated based on the calibrated fourth recognition results and sound data.

[0018] Optionally, the method further includes:

[0019] Obtain the audio data corresponding to the video stream;

[0020] Input the sound data into the fifth model, output the fifth recognition result, which indicates the species of the animal to be identified;

[0021] The process of determining the target animal species based on the first identification result, the second identification result, and the third identification result includes:

[0022] The target animal species to be identified is determined based on the first, second, third, and fifth identification results.

[0023] Optionally, the method further includes:

[0024] Obtain the audio data corresponding to the video stream;

[0025] Based on the sound data, determine the health status of the target animal species.

[0026] Optionally, after determining the target animal species of the animal to be identified based on the first identification result, the second identification result, and the third identification result, the method further includes:

[0027] Determine the primary health status of the target animal species based on the primary characteristic data;

[0028] Determine the second health status of the target animal species based on the second characteristic data;

[0029] The third health status of the target animal species is determined based on the third characteristic data;

[0030] Among these measures, determining the health status of the target animal species based on sound data includes:

[0031] The health status of the target animal species is determined based on the first, second, and third health statuses and sound data.

[0032] Optionally, the method further includes:

[0033] Animals of the same species in the first image are labeled with the same tag, and animals of different species are labeled with different tags, thus obtaining the labeled first image;

[0034] After determining the target animal species based on the first, second, and third identification results, the process also includes:

[0035] The number of tags for the target animal species in the first image after statistical labeling is counted to determine the number of target animal species.

[0036] Optionally, the first identification result is used to indicate the species of the animal to be identified, the second identification result is used to indicate the species of the animal to be identified, and the third identification result is used to indicate the species of the animal to be identified; after determining the target animal species of the animal to be identified based on the first identification result, the second identification result, and the third identification result, the method further includes:

[0037] Identify the second image with the highest similarity to the first image from the animal information database;

[0038] The second image is input into the YOLO algorithm, which only requires browsing the image once to obtain the first animal species.

[0039] If the first animal species is the same as the target animal species, update the first model based on the target animal species and the first feature data; update the second model based on the target animal species and the second feature data; update the third model based on the target animal species and the third feature data; or...

[0040] If the first animal species is different from the target animal species, output the second instruction, which is used to instruct the target animal species to be manually calibrated.

[0041] Receive the calibrated target animal species;

[0042] The first model is updated based on the calibrated target animal species and first characteristic data; the second model is updated based on the calibrated target animal species and second characteristic data; and the third model is updated based on the calibrated target animal species and third characteristic data.

[0043] Optionally, the animal information database includes an animal body surface database, an animal posture database, and an animal predation information database. The second image with the highest similarity to the first image is determined from the animal information database, including:

[0044] Obtain the first body surface data from the animal body surface database that has the highest similarity to the first feature data;

[0045] Obtain the first posture data that has the highest similarity to the second feature data from the animal movement state database;

[0046] Obtain the first predation information that has the highest similarity to the third feature data from the animal predation status database;

[0047] The second image is obtained based on the first body surface data, the first posture data, and the first predation information.

[0048] Secondly, embodiments of this application provide an apparatus for determining animal species, the apparatus comprising:

[0049] The acquisition unit is configured to acquire a video stream of an animal to be identified, the video stream including a first image; and to acquire first feature data, second feature data and third feature data of the first image, the first feature data including at least one of the outline of the animal's body surface, the color of the animal's body surface or the texture of the animal's body surface, the second feature data including the animal's posture, and the third feature data including predation information of the animal to be identified.

[0050] Processing unit, used for:

[0051] Input the first feature data into the first model and output the first recognition result;

[0052] The second feature data is input into the second model, and the second recognition result is output.

[0053] Input the third feature data into the third model and output the third recognition result;

[0054] The target animal species to be identified is determined based on the first identification result, the second identification result, and the third identification result.

[0055] Thirdly, embodiments of this application provide an apparatus for determining animal species, the apparatus including a processor coupled to a memory, the processor being configured to execute a computer program or instructions stored in the memory to implement the method described in the first aspect or any embodiment of the first aspect.

[0056] Fourthly, embodiments of this application provide a computer storage medium storing a computer program, which, when executed by a processor, implements the method described in the first aspect or any of the embodiments of the first aspect.

[0057] The beneficial effects of this application embodiment compared with the prior art are as follows: The first device of this application extracts feature data from the acquired image of the animal to be identified, obtaining first feature data, second feature data, and third feature data. The first feature data includes at least one of the following: the outline of the animal's body surface, the color of the animal's body surface, or the texture of the animal's body surface. The second feature data includes the animal's posture. The third feature data includes the predation information of the animal. The first, second, and third feature data are multi-dimensional features of the animal to be identified. Thus, by using the multi-dimensional features as input to different models, multi-dimensional first, second, and third identification results are obtained. The target of the animal to be identified is determined using the multi-dimensional identification results. Even if a feature in one dimension of the image of the animal to be identified is missing, which may lead to inaccurate identification results in that dimension, the extracted features from other dimensions can be input into the corresponding model to obtain relatively accurate animal species identification results. In this way, the target animal species obtained by combining the results is relatively accurate, thereby improving the accuracy of animal species identification. For example, if the first feature data in the image of the animal to be identified is incomplete or incomplete, it may lead to inaccurate first identification results. However, by inputting the second feature data into the second model and the third feature data into the third model, relatively accurate second and third identification results can be obtained. Thus, by combining the first, second, and third identification results, the target animal species obtained is relatively accurate. Attached Figure Description

[0058] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1 This is a schematic diagram of a method for determining animal species provided in an embodiment of this application;

[0060] Figure 2 This is a flowchart illustrating a method for determining animal species provided in an embodiment of this application;

[0061] Figure 3 This is a schematic block diagram of a device for determining animal species provided in an embodiment of this application;

[0062] Figure 4 This is a schematic block diagram of another device for determining animal species provided in the embodiments of this application. Detailed Implementation

[0063] The technical solutions in the embodiments of this application will be described in detail below with reference to the embodiments of this application.

[0064] It should be understood that the methods, situations, categories, and classifications of embodiments in this application are only for the convenience of description and do not constitute any limitation on this application. Various methods, categories, situations, and features in the embodiments can be combined with each other without contradiction.

[0065] It should also be understood that the terms "first," "second," "third," "fourth," and "fifth" in the embodiments of this application are for distinction only and do not constitute any limitation on this application. It should also be understood that in the various embodiments of this application, the sequence number of each process does not imply the execution order of the steps; the execution order of the steps is determined by their internal logic and does not constitute any limitation on the execution process of the embodiments of this application.

[0066] The current main method for identifying animal species is to input the collected animal images into a specific model and output the animal species in the images. When the collected animal images are incomplete or incomplete, the output animal species will be inaccurate or even unable to be output, thus affecting the accuracy of animal species identification.

[0067] Based on the above problems, this application proposes a method and apparatus for determining animal species. A first device extracts features from a first image of an animal to be identified to obtain first feature data, second feature data, and third feature data. The first feature data includes at least one of the outline of the animal's body surface, the color of the animal's body surface, or the texture of the animal's body surface. The second feature data includes the animal's posture. The third feature data includes predation information of the animal. The first feature data is input into a first model to obtain a first identification result. The second feature data is input into a second model to obtain a second identification result. The third feature data is input into a third model to obtain a third identification result. The animal species is determined based on the first identification result, the second identification result, and the third identification result. The first, second, and third feature data are multi-dimensional features of the animal to be identified. By inputting these multi-dimensional features into different models, multi-dimensional first, second, and third identification results are obtained. The target animal species determined by these multi-dimensional identification results can be determined even if a feature of a certain dimension is missing in the image of the animal to be identified, which may lead to inaccurate identification results in that dimension. However, by inputting the extracted features of other dimensions into the corresponding models, a relatively accurate animal species identification result can be obtained. Thus, the overall target animal species obtained is relatively accurate, thereby improving the accuracy of animal species identification.

[0068] The technical solution of this application will be described in detail below with reference to specific embodiments.

[0069] This application first describes the training process of the first model, second model, third model, fourth model, and fifth model in the embodiments. Any one of the first model, second model, third model, fourth model, and fifth model can be a neural network model.

[0070] The following is a description of the training process for the first model.

[0071] Optionally, the second device acquires N sample data, where N is greater than a preset value. The nth sample data in the N sample data includes input parameters and output parameters. The input parameters include at least one of the following: the outline of the nth animal's body surface, the color of the nth animal's body surface, or the texture of the nth animal's body surface. The output parameters include the animal species of the nth animal corresponding to the nth sample data, or the animal species of the nth animal and the probability of that animal species. The second device trains a first model based on the N sample data, meaning that the output of the first model can be the animal species, or it can be the animal species and the probability of that animal species. Here, n is a positive integer from 1 to N.

[0072] Optionally, the animal species corresponding to the nth animal in the nth sample data can be a major category of animal species, such as mammals, reptiles, birds, etc. Optionally, the animal species corresponding to the nth animal in the nth sample data can also be a minor category of animal species, such as tigers, lions, etc.

[0073] The following is a description of the training process for the second model.

[0074] Optionally, the second device acquires M sample data, where M is greater than a preset value. The m-th sample data in the M sample data includes input parameters and output parameters. The input parameters include the animal posture of the m-th animal corresponding to the m-th sample data, and the output parameters include the animal species of the m-th animal corresponding to the m-th sample data, or the animal species of the m-th animal and the probability of that animal species. The second device trains a second model based on the M sample data, that is, the output of the second model can be the animal species, or it can be the animal species and the probability of that animal species. Here, m is a positive integer from 1 to M.

[0075] Optionally, the animal species corresponding to the m-th sample data can be either a major category or a minor category of animal species.

[0076] The following is a description of the training process for the third model.

[0077] Optionally, the second device acquires X sample data points, where X is greater than a preset value. The x-th sample data point includes input parameters and output parameters. The input parameters include the predation information of the x-th animal corresponding to the x-th sample data point, and the output parameters include the animal species of the x-th animal corresponding to the x-th sample data point, or the animal species and probability of the animal species. The second device trains a third model based on the X sample data points. That is, the output of the third model can be the animal species, or it can be the animal species and the probability of that animal species. Here, x is a positive integer from 1 to X.

[0078] Optionally, the animal species corresponding to the xth sample data can be either a major category or a minor category of animal species.

[0079] The following is a description of the training process for the fourth model.

[0080] Optionally, the second device acquires Y sample data, where Y is greater than a preset value. The y-th sample data in the Y sample data includes input parameters and output parameters. The input parameters include the sound data of the y-th animal corresponding to the y-th sample data, and the output parameters include the animal species of the y-th animal corresponding to the y-th sample data. The second device trains a fourth model based on the Y sample data, meaning that the output of the fourth model is the animal species. Here, y is a positive integer from 1 to Y.

[0081] Optionally, the animal species corresponding to the y-th sample data can be either a major category or a minor category of animal species.

[0082] The following is a description of the training process for the fifth model.

[0083] Optionally, the second device acquires Z sample data points, where Z is greater than a preset value. The z-th sample data point in the Z sample data points includes input parameters and output parameters. The input parameters include the sound data of the z-th animal corresponding to the z-th sample data point, and the output parameters include the animal species of the z-th animal corresponding to the z-th sample data point, or the animal species of the z-th animal and the probability of that animal species. The second device trains a fifth model based on the Z sample data points, meaning that the output of the fifth model can be the animal species, or it can be the animal species and the probability of that animal species. Here, z is a positive integer from 1 to Z.

[0084] Optionally, the animal species corresponding to the z-th animal in the z-th sample data can be either a major category or a minor category of animal species.

[0085] Optionally, the output parameters of the first model, the second model, the third model, and the fifth model should be of the same type during the training process. For example, the outputs of the first model, the second model, the third model, and the fifth model can all be the major categories of animal species, or they can all be the major categories of animal species and the probability of the major category of that animal species, or they can all be the minor categories of animal species, or they can all be the minor categories of animal species and the probability of the minor category of that animal species.

[0086] Optionally, the second device and the first device can be the same device or different devices.

[0087] Figure 1 A schematic diagram illustrating a method for determining animal species provided in an embodiment of this application, as shown below. Figure 1 As shown, the method can be applied to a first device, and the method 100 may include the following steps:

[0088] S110, the first device acquires a video stream of the animal to be identified, the video stream including a first image.

[0089] Optionally, the first image in S110 can be any image in the video stream of the animal to be identified.

[0090] Optionally, the first device may be an edge computing device.

[0091] Optionally, S110 includes: the camera acquiring an original video stream and sending it to a first device; the first device receiving the original video stream from the camera; the first device performing animal detection on the original video stream; if an animal is present, the first device extracts the video stream containing the animal from the original video stream as the video stream of the animal to be identified in S110; if no animal is present, the original video stream is deleted. That is, the original video stream acquired by the camera may contain a video stream with or without an animal. In other words, regardless of whether there is an animal in the camera's view, the camera is always acquiring a video stream. In another possible implementation, the first device preprocesses the video stream containing the animal extracted from the original video stream and uses the resulting video stream as the video stream of the animal to be identified in S110. Optionally, the preprocessing of the video stream containing the animal extracted from the original video stream includes at least one of image enhancement, image denoising, or image deduplication.

[0092] Optionally, the first device performs animal detection on the original video stream, including: the first device performing animal detection on the original video stream captured by the receiving camera can use the You Only Look Once (YOLO) algorithm to perform animal target detection on the original video stream. It should be noted that animal target detection means only detecting whether there are animals in the video stream, without needing to detect the species of animals.

[0093] Optionally, S110 includes: a sound sensor collecting sound data; a first device detecting animal sounds in the sound data collected by the sound sensor; if there are animal sounds, the first device can also determine the direction of the sound source of the animal sounds based on the extracted sound data containing animal sounds, and output a control command to the camera; the control command is used to control the camera to rotate to the direction of the sound source to collect the original video stream; if there are no animal sounds, the first device deletes the sound data.

[0094] S120, the first device extracts first feature data, second feature data and third feature data from the first image, wherein the first feature data includes at least one of the outline of the animal body surface of the animal to be identified, the color of the animal body surface or the texture of the animal body surface, the second feature data includes the animal posture of the animal to be identified, and the third feature data includes the predation information of the animal to be identified.

[0095] Optionally, the texture of the animal's body surface in S120 may include: animal feathers, hair, skin texture, or scales, etc.

[0096] Optionally, the first feature data in S120 including at least one of the outline of the animal body surface of the animal to be identified, the color of the animal body surface, or the texture of the animal body surface can be understood as: the first feature data including any one of the outline of the animal body surface of the animal to be identified, the color of the animal body surface, or the texture of the animal body surface; or the first feature data including any two of the outline of the animal body surface of the animal to be identified, the color of the animal body surface, or the texture of the animal body surface; or the first feature data including the outline of the animal body surface of the animal to be identified, the color of the animal body surface, and the texture of the animal body surface.

[0097] Optionally, extracting the animal's body contour from the first feature data of the first image can be done using the boundary feature method, by describing the boundary features to obtain the animal's contour in the first image.

[0098] Optionally, the color of the animal's body surface in the first feature data of the first image can be extracted using any of the following color feature extraction methods: color histogram, color set, color moment, color aggregation vector, or color correlation graph.

[0099] Optionally, the texture of the animal's body surface can be extracted from the first feature data of the first image using any of the following texture feature extraction methods: statistical methods, geometric methods, model methods, or signal processing methods.

[0100] Optionally, extracting the second feature data of the first image in S120 can be understood as extracting the animal posture of the animal to be identified in the first image.

[0101] Optionally, the animal posture of the animal to be identified can indicate the animal's movement state. For example, the animal's movement state can include running, crawling, jumping, flying, etc.

[0102] Optionally, the animal pose of the animal to be identified in the first image can be extracted using a model-based pose estimation method or a learning-based pose estimation method. For example, a learning-based pose estimation method can be used, employing the OpenPose algorithm to detect key points for different animals. Key points include the animal's left eye, right eye, mouth, left ear, right ear, head, left shoulder girdle, right shoulder girdle, waist girdle, left forelimb, right forelimb, left hindlimb, right hindlimb, and tail. The mapping relationship between key points is calculated, and the animal's pose is determined and output.

[0103] Optionally, extracting the third feature data of the first image in S120 can be understood as extracting the predation information of the animal to be identified in the first image.

[0104] Optionally, the predation information of the animal to be identified can indicate the prey of the animal to be identified, which may include animals, leaves or grass, etc.

[0105] Optionally, the predation information of the animal to be identified in the first image can be extracted by using the YOLO algorithm to identify the prey of the animal. For example, if the YOLO algorithm identifies the prey of the animal to be identified in the image as leaves, the third feature data includes the feature data of the leaves.

[0106] S130, the first device inputs the first feature data into the first model and outputs the first recognition result, inputs the second feature data into the second model and outputs the second recognition result, and inputs the third feature into the third model and outputs the third recognition result.

[0107] Optionally, the first identification result is used to indicate the species of the animal to be identified, the second identification result is used to indicate the species of the animal to be identified, and the third identification result is used to indicate the species of the animal to be identified.

[0108] Optionally, the first identification result, the second identification result, and the third identification result may all indicate the animal species of the animal to be identified, or the first identification result, the second identification result, and the third identification result may all indicate the animal species of the animal to be identified and the probability of that animal species.

[0109] The first, second, and third identification results in S130 are described below in four different scenarios.

[0110] Case 1

[0111] Optionally, the first, second, and third identification results in S130 can all indicate the major category of the animal species. Optionally, any two of the major categories of the animal species indicated by the first, second, and third identification results can be the same or different. For example, the first identification result indicates that the animal species is a mammal, the second identification result indicates that the animal species is a reptile, and the third identification result indicates that the animal species is a mammal.

[0112] In other words, in scenario one, during the aforementioned training of the first, second, and third models, the output parameters of the first, second, and third models are all major categories of animal species.

[0113] Scenario 2

[0114] Optionally, the first, second, and third identification results in S130 can all indicate the major category of the animal species and the probability of that major category. Optionally, any two of the major categories of the animal species indicated by the first, second, and third identification results can indicate the same or different major categories; optionally, any two of the probabilities of the major categories of the animal species indicated by the first, second, and third identification results can indicate the same or different probabilities. For example, the first identification result indicates that the animal species is a mammal with a probability of 80%, the second identification result indicates that the animal species is a reptile with a probability of 70%, and the third identification result indicates that the animal species is a mammal with a probability of 50%.

[0115] In other words, in scenario two, during the aforementioned training of the first, second, and third models, the output parameters of the first, second, and third models are the major categories of animal species and the probability of those major categories.

[0116] Scenario 3

[0117] Optionally, the first, second, and third identification results in S130 can all indicate the subclass of the animal species. Optionally, any two of the subclasses of the animal species indicated by the first, second, and third identification results can indicate the same or different subclasses. For example, the first identification result indicates the animal species as tiger, the second identification result indicates the animal species as lion, and the third identification result indicates the animal species as tiger.

[0118] In other words, in scenario three, during the aforementioned training of the first, second, and third models, the output parameters of the first, second, and third models are all subclasses of animal species.

[0119] Case 4

[0120] Optionally, the first, second, and third identification results in S130 can all indicate the subclass of the animal species and the probability of that subclass. Optionally, any two of the subclasses of the animal species indicated by the first, second, and third identification results can indicate the same or different subclasses; optionally, any two of the probabilities of the subclasses of the animal species indicated by the first, second, and third identification results can indicate the same or different probabilities. For example, the first identification result indicates that the animal species is a tiger and the probability of it being a tiger is 70%, the second identification result indicates that the animal species is a lion and the probability of it being a lion is 70%, and the third identification result indicates that the animal species is a tiger and the probability of it being a tiger is 50%.

[0121] In other words, in scenario four, during the training of the first, second, and third models, the output parameters of the first, second, and third models are the subclasses of the animal species and the probability of those subclasses.

[0122] S140, the first device determines the target animal species of the animal to be identified based on the first identification result, the second identification result and the third identification result.

[0123] Optionally, in step S140, the first device determines the target animal species of the animal to be identified based on the first identification result, the second identification result, and the third identification result, including: if at least two of the first identification result, the second identification result, or the third identification result indicate the same animal species, then the target animal species of the animal to be identified is the same animal species; or, if the first identification result, the second identification result, and the third identification result indicate different animal species, then the target animal species can be determined based on the contribution values ​​of the first feature data, the second feature data, and the third feature data to the target animal species.

[0124] It should be noted that the contribution values ​​of the first, second, and third characteristic data to the target animal species can be determined empirically.

[0125] S140 is described below in four different cases.

[0126] Case 1

[0127] Optionally, the first identification result, the second identification result, and the third identification result all indicate the major category of the animal species.

[0128] If at least two of the first, second, or third identification results indicate the same major category of animal species, then the target animal species of the animal to be identified is the same major category of animal species.

[0129] If the first, second, and third identification results indicate different major animal categories, the target animal category can be determined based on the contribution values ​​of the first, second, and third feature data to the target animal category. Specifically, the feature data with the largest contribution value to the target animal category among the first, second, and third feature data is identified. The animal category indicated by the identification result output by the model corresponding to the feature data with the largest contribution value is the target animal category of the animal to be identified. For example, if the first feature data has the largest contribution value to the target animal category, then the target animal category of the animal to be identified is the major category of the animal category indicated by the first identification result.

[0130] Scenario 2

[0131] Optionally, the first identification result, the second identification result, and the third identification result all indicate the major category of the animal species and the probability of that major category.

[0132] Optionally, the target animal species to be identified can be the one with the highest probability among the major categories of animal species indicated by the first identification result, the second identification result, and the third identification result. For example, if the animal species indicated by the first identification result is a mammal with an 80% probability, the animal species indicated by the second identification result is a reptile with a 70% probability, and the animal species indicated by the third identification result is a mammal with a 50% probability, then the target animal species is determined to be a mammal.

[0133] It should be noted that if any two or three of the probability values ​​of the major categories of animal species indicated by the first, second, and third identification results are the same, then the animal species can be determined based on the contribution values ​​of the first, second, and third feature data to the target animal species. Specifically, the feature data with the largest contribution value to the target animal species among the first, second, and third feature data is identified, and the animal species indicated by the identification result output by the corresponding model of the feature data with the largest contribution value is the target animal species.

[0134] Optionally, the target animal species can also be determined by a weighted fusion method using the first, second, and third identification results. The weighting factor for the first identification result can indicate the relative importance of the first feature number in determining the target animal species; the weighting factor for the second identification result can indicate the relative importance of the second feature number; and the weighting factor for the third identification result can indicate the relative importance of the third feature number. Each weighting factor can be determined empirically. The probability of the major categories of animal species indicated by the first, second, and third identification results is recalculated using the weighted fusion method. Specifically, the probability of the same major category can be calculated by summing the weighting factors, while the probability of different major categories can be calculated independently using the weighting factors. Then, the major category corresponding to the highest probability among the calculated major categories is determined as the target animal species. For example, the first identification result indicates that the animal species is a mammal with a probability of 80%, the second identification result indicates that the animal species is a reptile with a probability of 70%, and the third identification result indicates that the animal species is a mammal with a probability of 50%. The weighting factors for the first and second identification results are 0.4 and 0.2, respectively. Therefore, the probability that the target animal species is a mammal is 80% * 0.4 + 50% * 0.2 = 42%, and the probability that the target animal species is a reptile is 70% * 0.4 = 28%. The probability of the target animal species being a mammal, calculated using weighted fusion, is greater than the probability of it being a reptile. Therefore, the target animal species is determined to be a mammal.

[0135] Scenario 3

[0136] Optionally, the first identification result, the second identification result, and the third identification result all indicate the subclass of the animal species.

[0137] If at least two of the first, second, or third identification results indicate the same subclass of the animal species, then the target animal species to be identified is the same subclass of the animal species.

[0138] If the animal species indicated by the first, second, and third identification results are all different subclasses, the target animal species can be determined based on the contribution values ​​of the first, second, and third feature data to the target animal species. Specifically, the feature data with the largest contribution value to the target animal species among the first, second, and third feature data is identified. The animal species indicated by the identification result output by the model corresponding to the feature data with the largest contribution value is the target animal species of the animal to be identified. For example, if the first feature data has the largest contribution value to the target animal species, then the target animal species of the animal to be identified is the subclass of the animal species indicated by the first identification result.

[0139] Case 4

[0140] Optionally, the first identification result, the second identification result, and the third identification result all indicate the subclass of the animal species and the probability of that subclass.

[0141] Optionally, the target animal species to be identified can be the one with the highest probability among the subclasses indicated by the first, second, and third identification results. For example, if the first identification result indicates that the animal species is a tiger and the probability of it being a tiger is 60%, the second identification result indicates that the animal species is a lion and the probability of it being a lion is 70%, and the third identification result indicates that the animal species is a tiger and the probability of it being a tiger is 50%, then the target animal species is determined to be a lion.

[0142] It should be noted that if any two or three of the probabilities of the subclass of the animal species indicated by the first, second, and third identification results are the same, then the animal species can be determined based on the contribution values ​​of the first, second, and third feature data to the target animal species. Specifically, the feature data with the largest contribution value to the target animal species among the first, second, and third feature data is identified, and the animal species indicated by the identification result output by the corresponding model of the feature data with the largest contribution value is the target animal species.

[0143] Optionally, the target animal species can also be determined by a weighted fusion method using the first, second, and third identification results. The weighting factor for the first identification result can indicate the relative importance of the first feature number in determining the target animal species; the weighting factor for the second identification result can indicate the relative importance of the second feature number; and the weighting factor for the third identification result can indicate the relative importance of the third feature number. Each weighting factor can be determined empirically. The probability of the subclass of the animal species indicated by the first, second, and third identification results is recalculated using the weighted fusion method. Specifically, the probability of the same subclass can be calculated by summing the weighting factors, and the probability of different subclasses can be calculated independently using the weighting factors. Then, the subclass corresponding to the highest probability among the calculated subclasses is determined as the target animal species. For example, the first identification result is a tiger with a probability of 60%, the second identification result is a lion with a probability of 70%, and the third identification result is a tiger with a probability of 50%. The weighting factors for the first and second identification results are 0.4 and 0.2 respectively. Therefore, the probability that the target animal is a tiger is 60% * 0.4 + 50% * 0.2 = 34%, and the probability that it is a lion is 70% * 0.4 = 28%. Using weighted fusion, the recalculated probability of a tiger is greater than the probability of a lion, so the target animal is determined to be a tiger.

[0144] Optionally, S140 includes: the first device acquiring sound data corresponding to the video stream of the animal to be identified, inputting the sound data into the fifth model, and outputting a fifth identification result. The first device determines the target animal species of the animal to be identified based on the first identification result, the second identification result, and the third identification result, including: the first device determining the target animal species of the animal to be identified based on the first identification result, the second identification result, the third identification result, and the fifth identification result.

[0145] Optionally, the audio data corresponding to the video stream of the animal to be identified can be understood as: audio data containing animal sounds detected during the time the video stream of the animal to be identified was acquired. Alternatively, the audio data corresponding to the video stream of the animal to be identified can be understood as: audio data after audio preprocessing of the audio data containing animal sounds detected during the time the video stream of the animal to be identified was acquired.

[0146] Optionally, sound preprocessing may include filtering out non-animal sounds, such as rain, wind, and the rustling of leaves.

[0147] It should be noted that the method of determining the target animal species of the animal to be identified based on the first identification result, the second identification result, the third identification result, and the fifth identification result is similar to the method of determining the target animal species of the animal to be identified based on the first identification result, the second identification result, and the third identification result. To avoid redundancy, the embodiments of this application will not be described in detail.

[0148] The above describes the process of determining the target animal species. By analyzing multi-dimensional feature data from the first image, multi-dimensional first, second, and third recognition results are obtained. These results are then combined to determine the target animal species. A fifth recognition result, incorporating sound data, can further improve the accuracy of species identification. When feature information in a certain dimension of the first image is missing—for example, if the animal is occluded—complete second feature data may not be available. The second recognition result obtained from this second feature data and the second model may be relatively inaccurate. However, by inputting the extracted features from other dimensions into the corresponding model, a relatively accurate animal species identification result can be obtained. Therefore, the combined result of these methods leads to a more accurate target animal species identification, thus improving the overall accuracy of animal species recognition.

[0149] After determining the target animal species in the aforementioned method 100, the target animal species in this application embodiment can also be used to verify whether the animal species indicated by the fourth identification result is accurate.

[0150] Optionally, the first device inputs the acquired sound data into the fourth model and outputs a fourth recognition result, which indicates the species of the animal to be identified. The fourth recognition result can be either a major category of the animal species or a minor category of the animal species.

[0151] Optionally, the first device verifies the accuracy of the animal species indicated by the fourth identification result by determining whether the target animal species matches the animal species indicated by the fourth identification result. If the target animal species matches the animal species indicated by the fourth identification result, it indicates that the animal species indicated by the fourth identification result is correct; if the target animal species does not match the animal species indicated by the fourth identification result, it indicates that the animal species indicated by the fourth identification result is incorrect.

[0152] It should be noted that the matching of the target animal species with the animal species indicated by the fourth identification result means: if the target animal species and the animal species indicated by the fourth identification result are both major categories of animal species or both are minor categories of animal species, and the target animal species is the same as the animal species indicated by the fourth identification result, then the target animal species matches the animal species indicated by the fourth identification result; if the target animal species is a major category of animal species, and the animal species indicated by the fourth identification result is a minor category of animal species, and the animal species indicated by the fourth identification result belongs to the target animal species, then the target animal species matches the animal species indicated by the fourth identification result; if the target animal species is a minor category of animal species, and the animal species indicated by the fourth identification result is a major category of animal species, and the target animal species belongs to the animal species indicated by the fourth identification result, then the target animal species matches the animal species indicated by the fourth identification result.

[0153] It should be noted that the discrepancy between the target animal species and the animal species indicated by the fourth identification result means: if both the target animal species and the animal species indicated by the fourth identification result belong to the same major category or both belong to the same minor category, and the target animal species and the animal species indicated by the fourth identification result are different, then the target animal species and the animal species indicated by the fourth identification result do not match; if the target animal species belongs to the same major category, and the animal species indicated by the fourth identification result belongs to the same minor category, and the animal species indicated by the fourth identification result does not belong to the target animal species, then the target animal species and the animal species indicated by the fourth identification result do not match; if the target animal species belongs to the same minor category, and the animal species indicated by the fourth identification result belongs to the same major category, and the target animal species does not belong to the animal species indicated by the fourth identification result, then the target animal species and the animal species indicated by the fourth identification result do not match.

[0154] Optionally, if the target animal species matches the animal species indicated by the fourth identification result, the first device updates the fourth model based on the fourth identification result and the sound data.

[0155] Optionally, if the target animal species does not match the animal species indicated by the fourth recognition result, the first device outputs a first instruction, which is used to instruct manual calibration. The manual person calibrates the fourth recognition result based on the sound data to obtain the calibrated fourth recognition result. The first device updates the fourth model based on the calibrated fourth recognition result and the sound data.

[0156] The following is a description of updating the first model, second model, and third model in S130 based on the target animal species determined in the aforementioned method 100 and the first animal species obtained from the second image.

[0157] Optionally, S130 includes: determining the second image with the highest similarity to the first image in the animal information database, and inputting the second image into the YOLO algorithm to obtain the first animal species.

[0158] Optionally, the animal information database may be an animal information database that includes different animal species, and the animal information database for each animal may include an animal body surface database, an animal posture database, and an animal predation information database for each animal.

[0159] It should be noted that the animal body surface database for each animal includes at least one of the following: the outline of the animal body surface, the color of the animal body surface, or the texture of the animal body surface. The animal posture database for each animal includes animal posture. The animal predation information database for each animal includes animal predation information, wherein the animal predation information for each animal may be the prey of the animal.

[0160] Optionally, determining the second image with the highest similarity to the first image in the animal information database includes: obtaining the first body surface data with the highest similarity to the first feature data in the animal body surface database, obtaining the first posture data with the highest similarity to the second feature data in the animal posture database, obtaining the first predation information with the highest similarity to the third feature data in the animal predation information database, and obtaining the second image based on the first body surface data, the first posture data, and the first predation information.

[0161] It should be noted that obtaining the similarity between the animal body surface database and the first feature data refers to obtaining the similarity between the animal body surface database and the first feature data for each individual animal. Obtaining the similarity between the animal posture database and the second feature data refers to obtaining the similarity between the animal posture database and the second feature data for each individual animal. Obtaining the similarity between the animal predation information database and the third feature data refers to obtaining the similarity between the animal predation information database and the third feature data for each individual animal.

[0162] Optionally, the similarity calculation method can be any one of Euclidean distance, Manhattan distance, Pearson correlation coefficient or cosine distance. Taking the maximum value of the three similarity sets respectively can yield the corresponding first body surface data, first posture data and first predation information.

[0163] Optionally, obtaining the second image based on the first body surface data, the first posture data, and the first predation information can be achieved by simply weighting and superimposing the obtained first body surface data, first posture data, and first predation information.

[0164] Optionally, the target animal species and the first animal species can both be major categories of animal species, and the output parameters of the first model, the second model, and the third model can also be major categories of animal species or major categories of animal species and the probability of that major category; or, the target animal species and the first animal species can both be minor categories of animal species, and the output parameters of the first model, the second model, and the third model can also be minor categories of animal species or minor categories of animal species and the probability of that minor category.

[0165] Optionally, if the first animal species is the same as the target animal species, the first model can be updated based on the target animal species and the first feature data.

[0166] Optionally, if the first animal species is different from the target animal species, a second instruction is output. The second instruction is used to instruct manual calibration. The manual person calibrates the target animal species according to the first image to obtain the calibrated target animal species. The first model can be updated according to the calibrated target animal species and the first feature data.

[0167] Optionally, if the first animal species is the same as the target animal species, the second model can be updated based on the target animal species and the second feature data.

[0168] Optionally, if the first animal species is different from the target animal species, the second model can be updated based on the calibrated target animal species and the second feature data.

[0169] Optionally, if the first animal species is the same as the target animal species, the third model can be updated based on the target animal species and the third feature data.

[0170] Optionally, if the first animal species is different from the target animal species, the third model can be updated based on the calibrated target animal species and the third feature data.

[0171] After determining the target animal species in the aforementioned method 100, this embodiment of the application is further used to determine the health status of the animals in the target animal species.

[0172] Optionally, the first device acquires audio data corresponding to the video stream of the animal to be identified, and determines the health status of the target animal species based on the audio data.

[0173] Optionally, determining the health status of a target animal species based on sound data includes: establishing a sound database for the target animal species in various established animal sound databases, where each animal sound database includes normal and abnormal sounds of each animal; matching the sound data with the normal and abnormal sounds in the sound database of the target animal species; if the sound data matches the normal sounds in the sound database of the target animal species, the target animal species is determined to be healthy; if the sound data matches the abnormal sounds in the sound database of the target animal species, the target animal species is determined to be abnormal.

[0174] Optionally, determining the health status of the target animal species based on the sound data also includes: determining the health status of the target animal species based on the first health status, second health status, third health status and sound data.

[0175] Optionally, the first device determines the animal information database for the target animal species from the established animal information databases based on the target animal species. Each animal information database includes an animal body surface database, an animal posture database, and an animal predation information database.

[0176] It should be noted that the animal body surface database for each animal includes both normal and abnormal body surface data. The animal posture database for each animal includes both normal and abnormal posture data. The animal predation information database for each animal includes both normal and abnormal predation data.

[0177] It should be noted that the first device determines the first health status of the target animal species based on the first characteristic data, the second health status of the target animal species based on the second characteristic data, and the third health status of the target animal species based on the third characteristic data.

[0178] Optionally, the first device matches the first feature data with the normal and abnormal body surfaces in the animal body surface database of the target animal species. If the first feature data matches the normal body surface in the animal body surface database of the target animal species, the first health condition of the target animal species is determined to be healthy; if the first feature data matches the abnormal body surface in the animal body surface database of the target animal species, the first health condition of the target animal species is determined to be abnormal.

[0179] Optionally, the first device matches the second feature data with normal and abnormal postures in the animal posture database of the target animal species. If the second feature data matches the normal posture in the animal posture database of the target animal species, the second health status of the target animal species is determined to be healthy. If the second feature data matches the abnormal posture in the animal posture database of the target animal species, the second health status of the target animal species is determined to be abnormal.

[0180] Optionally, the first device matches the third feature data with normal predation and abnormal predation in the animal predation information database of the target animal species. If the third feature data matches normal predation in the animal predation information database of the target animal species, the third health status of the target animal species is determined to be healthy; if the third feature data matches abnormal predation in the animal predation information database of the target animal species, the third health status of the target animal species is determined to be abnormal.

[0181] Optionally, determining the health status of the target animal species based on its first, second, and third health statuses and vocal data can be achieved by: determining that the target animal species is abnormal if any one of the first, second, and third health statuses, or the health status determined by the vocal data, indicates an abnormality. For example, if the first health status is healthy, the second health status is abnormal, the third health status is healthy, and the health status determined by the vocal data is healthy, then the target animal species is determined to be abnormal. If the first, second, and third health statuses, as well as the health status determined by the vocal data, are all normal, then the target animal species is determined to be healthy.

[0182] The above is the process of judging the health status of the target animal species. After identifying the target animal species, the health status of the target animal species is also monitored. If there are any abnormal physical conditions, it can prompt the animal protection department to take corresponding rescue measures, which helps zoologists or animal protection departments to study and grasp more comprehensive animal resources.

[0183] After determining the target animal species in the aforementioned method 100, this embodiment of the application is further used to determine the quantity of the target animal species.

[0184] Optionally, the first device determines the number of target animal species by labeling animals of the same species in the first image with the same label, and labeling animals of different species with different labels, to obtain a labeled first image, counting the number of labels for the target animal species in the labeled first image, and determining the number of target animal species.

[0185] Optionally, the first device can determine the number of target animal species by labeling the target animal species in the first image after determining the target animal species, obtaining a labeled first image, counting the number of labels in the labeled first image, and determining the number of target animal species.

[0186] The above describes the statistical process for the number of target animal species. After identifying the target animal species, the statistical process for the number of target animal species was also completed, which helps zoologists or animal protection departments to study and grasp more comprehensive animal resources.

[0187] To better understand the scheme of this application, Figure 2 A flowchart of a method for determining animal species is given as an example, such as... Figure 2 As shown, the first device is an edge computing device.

[0188] S201, camera rotates.

[0189] Specifically, the edge computing device determines whether there are animal sounds in the original sound information collected by the sound sensor. If there are animal sounds, it locates the direction of the sound source and outputs a first control command. The first control command is used to control the camera to rotate to the direction of the sound source.

[0190] For example, in the aforementioned S110, the first device detects animal sounds by collecting sound data from the sound sensor. If there are animal sounds, the first device can also determine the direction of the sound source of the animal sounds based on the extracted sound data and output a control command to the camera. The control command is used to control the camera to rotate to the direction of the sound source to collect the original video stream, which can be S201.

[0191] S202, the camera captures raw video information.

[0192] Specifically, the original video information can be video information with or without animals.

[0193] For example, the raw video stream captured by the camera in S110 mentioned above can be Figure 2 The original video information in the video.

[0194] S203 uses the YOLO algorithm to detect the presence of animals.

[0195] Specifically, using the YOLO algorithm to detect the presence of animals means using the YOLO algorithm to detect whether there are animals in the original video information. If there are animals, execute S204; if there are no animals, execute S205.

[0196] For example, the first device in S110 above can perform animal detection on the original video stream in S203.

[0197] S204, extract video information containing animals, preprocess the video information containing animals, and obtain image A.

[0198] Specifically, extracting video information containing animals refers to extracting video information containing animals from the original video information. Preprocessing the extracted video information containing animals includes: image enhancement, image denoising, and image deduplication. The obtained image A is any image from the preprocessed video information.

[0199] For example, in the aforementioned S110, the first device extracts a video stream containing animals from the original video stream, and preprocessing the extracted video stream containing animals can be S204, and the first image in method 100 can be image A.

[0200] S205, Delete video information that does not contain animals.

[0201] For example, in the aforementioned S110, if the original video stream does not contain animals, then deleting the original video stream can be S205.

[0202] S206, extract the body surface features, posture features, and predation features of image A.

[0203] Specifically, the body surface features of image A include at least one of the following: the outline of the animal body surface of the first animal to be identified, the color of the animal body surface, or the texture of the animal body surface; the posture features of image A include the posture of the first animal to be identified; and the predation features of image A include predation information of the first animal to be identified.

[0204] For example, the aforementioned S120 can be S206, the first feature data in the aforementioned S120 can be the body surface features in S206, the second feature data in the aforementioned S120 can be the posture features in S206, and the third feature data in the aforementioned S120 can be the predation features in S206.

[0205] S207, Input the body surface features into the animal body surface recognition model, and output animal type 1.

[0206] For example, the first model in method 100 could be Figure 2 In the animal body surface recognition model, the first recognition result in method 100 can be... Figure 2 Animal species 1.

[0207] S208, Input the posture features into the animal posture recognition model and output animal type 2.

[0208] For example, the second model in method 100 could be Figure 2 In the animal pose recognition model, the second recognition result in method 100 can be... Figure 2 Animal species 2.

[0209] S209, input predation characteristics into the animal predation information recognition model, and output animal species 3.

[0210] For example, the third model in method 100 could be Figure 2 In the animal predation information recognition model, the third recognition result in method 100 can be... Figure 2 Animal species 3.

[0211] S210, based on animal species 1, animal species 2 and animal species 3, determine the animal species of the first animal to be identified as animal species 5.

[0212] For example, S130 mentioned above can be S210, and the target animal species of the animal to be identified in method 100 can be... Figure 2 The number of animal species is 5.

[0213] S211, Extract the most similar body surface features from the animal information database.

[0214] For example, in method 100, the first body surface data with the highest similarity to the first feature data extracted from the animal information database could be... Figure 2 In S211 of method 100, the first body surface data can be... Figure 2 Similar surface features in the body.

[0215] S212, Extract the similar posture feature with the highest similarity to the posture feature from the animal information database.

[0216] For example, in method 100, the first pose data with the highest similarity to the second feature data extracted from the animal information database could be... Figure 2 In S212, the first attitude data in method 100 can be Figure 2 Similar posture features in the text.

[0217] S213, extract the most similar predator features from the animal information database.

[0218] For example, in method 100, the first predator information that has the highest similarity to the third feature data in the animal information database can be... Figure 2 In S213 of method 100, the first predation information can be... Figure 2 Similar predatory characteristics in the species.

[0219] S214. Image A' is obtained based on similar body surface features, similar posture features, and similar predation features.

[0220] For example, obtaining the second image based on the first body surface data, the first posture data, and the first predation information in method 100 could be... Figure 2 In S214, the second image in method 100 can be Figure 2 Image A' in the image.

[0221] S215, Use the YOLO algorithm to identify the first animal to be identified in image A' as animal species 6.

[0222] For example, in the aforementioned S130, inputting the second image into the YOLO algorithm to obtain the first animal species could be S215, and the first animal species in method 100 could be... Figure 2 The number of animal species in the text is 6.

[0223] S216, Determine whether animal species 5 and animal species 6 are the same.

[0224] Specifically, if animal species 5 is the same as animal species 6, then S217 is executed; if animal species 5 is not the same as animal species 6, then a first manual calibration instruction is output, and S218 is executed. The first manual calibration instruction is used to instruct animal species 5 to be manually calibrated.

[0225] For example, determining whether the first animal species is the same as the target animal species in method 100 could be... Figure 2 S216 in the middle.

[0226] S217, Update the animal species identification model based on animal species 5 and image A.

[0227] Specifically, updating the animal species recognition model based on animal species 5 and image A means: updating the animal body surface recognition model based on the body surface features in image A and animal species 5; updating the animal posture recognition model based on the posture features in image A and animal species 5; and updating the animal predation information recognition model based on the predation features in image A and animal species 5.

[0228] For example, in method 100, if the first animal species is the same as the target animal species, the first model can be updated based on the target animal species and the first feature data; the second model can be updated based on the target animal species and the second feature data; and the third model can be updated based on the target animal species and the third feature data. Figure 2 S217 in the middle.

[0229] S218, perform manual calibration of animal species 5 to animal species 7, and update the animal species recognition model based on animal species 7 and image A.

[0230] Specifically, updating the animal species recognition model based on animal species 7 and image A means: updating the animal body surface recognition model based on the body surface features in image A and animal species 7; updating the animal posture recognition model based on the posture features in image A and animal species 7; and updating the animal predation information recognition model based on the predation features in image A and animal species 7.

[0231] For example, in method 100, if the first animal species is different from the target animal species, a second instruction is output. The second instruction is used to instruct the target animal species to be manually calibrated. The first model can be updated based on the calibrated target animal species and the first feature data, the second model can be updated based on the calibrated target animal species and the second feature data, and the third model can be updated based on the calibrated target animal species and the third feature data. Figure 2 In S218 of method 100, the calibrated target animal species can be... Figure 2 The number of animal species is 7.

[0232] S219, the sound sensor collects raw sound information.

[0233] For example, the sound data collected by the sound sensor in method 100 can be Figure 2 The original sound information in it.

[0234] S220, determine whether there are animal sounds in the original sound information.

[0235] Specifically, if the original sound information contains animal sounds, execute S221; if the original sound information does not contain animal sounds, execute S222.

[0236] For example, in method 100, the first device can perform animal sound detection on the sound data collected by the sound sensor in step S220.

[0237] S221, extract the sound information containing animal sounds, and perform sound preprocessing on the sound information containing animal sounds to obtain preprocessed sound information.

[0238] Specifically, extracting sound information containing animal sounds refers to extracting sound information containing animal sounds from the original sound information, while preprocessing the sound information containing animal sounds refers to filtering out non-animal sounds.

[0239] For example, in method 100, the sound preprocessing of the sound data containing animal sounds detected during the time period of acquiring the video stream of the animal to be identified can be step S221, and the sound data after sound preprocessing of the sound data containing animal sounds detected during the time period of acquiring the video stream of the animal to be identified in method 100 can be... Figure 2 The preprocessed audio information.

[0240] S222, Delete the original sound information that does not contain animal sounds.

[0241] For example, in method 100, if there are no animal sounds in the sound data collected by the sound sensor, deleting the sound data can be step S222.

[0242] S223, input the preprocessed sound information into the sound recognition model, and output the first animal to be identified as animal species 4.

[0243] For example, the fourth model in method 100 could be Figure 2 In the voice recognition model, the fourth recognition result in method 100 can be... Figure 2 In the animal species 4, the first device in method 100 will input the sound data corresponding to the video stream of the animal to be identified into the fourth model, and the output of the fourth recognition result can be S223.

[0244] S224, determine whether animal species 4 and animal species 5 are the same.

[0245] Specifically, if animal species 4 and animal species 5 are the same, then S225 is executed; if animal species 4 and animal species 5 are different, a second manual calibration instruction is output, and S226 is executed. The second manual calibration instruction is used to instruct animal species 4 to undergo manual calibration.

[0246] S225, update the sound recognition model based on animal species 4 and the preprocessed sound information.

[0247] For example, if the target animal species in method 100 matches the animal species indicated by the fourth identification result, then updating the fourth model based on the animal species indicated by the fourth identification result and the sound data can be S225.

[0248] S226, perform manual calibration of animal species 4 to animal species 8, and update the sound recognition model based on animal species 8 and the preprocessed sound information.

[0249] For example, if the target animal species in method 100 does not match the animal species indicated by the fourth identification result, the first device outputs a first instruction. The first instruction is used to instruct the fourth identification result to be manually calibrated. The first device can update the fourth model based on the calibrated fourth identification result and sound data in step S226.

[0250] S227, input the preprocessed sound information and animal species 5 into the sound database.

[0251] S228, determine whether there are any abnormalities in the preprocessed audio information.

[0252] Specifically, if there is an anomaly in the preprocessed audio information, then S229 is executed; if there is no anomaly in the preprocessed audio information, then S239 is executed.

[0253] S229, Output animal species 5 with physical abnormalities.

[0254] S230, Determine the surface health status of animal species 5 based on animal species 5 and its surface characteristics.

[0255] S231, determine whether there are any abnormalities on the animal's body surface.

[0256] Specifically, if the animal of animal species 5 has an abnormal body surface, then execute S232; if the animal of animal species 5 has a normal body surface, then execute S239.

[0257] S232, Output animal species 5 with physical abnormalities.

[0258] S233, Determine the postural health status of animal species 5 based on animal species 5 and postural characteristics.

[0259] S234, determine whether there is any abnormality in the animal's posture.

[0260] Specifically, if the animal posture of animal species 5 is abnormal, then S235 is executed; if the animal posture of animal species 5 is normal, then S239 is executed.

[0261] S235, Output animal species 5 with physical abnormalities.

[0262] S236, Determine the predatory health status of animal species 5 based on animal species 5 and predatory characteristics.

[0263] S237, determine whether there are any abnormalities in animal predation.

[0264] Specifically, if animal species 5 predation is abnormal, then S238 is executed; if animal species 5 predation is normal, then S239 is executed.

[0265] S238, Output animal species 5 with physical abnormalities.

[0266] S239, Output animal species 5, all in good health.

[0267] The above S227 to S239 are the process of judging the health status of animal species 5 after determining that the first animal species to be identified is animal species 5.

[0268] S240, count the number of animal species 5.

[0269] Specifically, animals of the same species in image A are labeled with the same tag, and animals of different species are labeled with different tags to obtain labeled image A. The number of tags for the target animal species in labeled image A is counted to determine the number of animal species 5.

[0270] It should be noted that this application may include, but is not limited to, the following: Figure 2 More steps or fewer steps Figure 2 Different combinations of steps can form different embodiments. For example, S201 to S210 can constitute an embodiment in which the first device acquires first feature data, second feature data, and third feature data from an image of an animal to be identified, inputs these multi-dimensional feature data into different models, obtains multi-dimensional animal species identification results, and uses the multi-dimensional identification results to determine the target animal species to be identified. S201 to S217 can constitute an embodiment in which the first device updates the first model, the second model, and the third model by comparing the identified target animal species with the animal species identified in the animal information database that has the highest similarity to the image of the animal to be identified. S201 to S226 can constitute another embodiment in which the first device updates the fourth model by comparing the identified target animal species with the identified animal species identified by sound data. S201 to S239 can constitute another embodiment in which, after determining the target animal species, the health status of the target animal species is judged and output. S201 to S240 can constitute yet another embodiment in which, after determining the target animal species, the health status of the target animal species and the number of target animal species are statistically output.

[0271] above Figure 2 The first animal species to be identified was determined by extracting features from the acquired image A, obtaining body surface features, posture features, and predation features. These multi-dimensional features were then input into different models to obtain multi-dimensional animal species 1, 2, and 3. Even if a feature in one dimension of the image is missing, potentially leading to inaccurate identification in that dimension, the extracted features input into the corresponding models can still yield relatively accurate animal species identification results. This comprehensive approach results in a more accurate animal species identification, thus improving the overall accuracy of animal species identification. Furthermore, after determining the first animal species, the health status and population of the animal species were also assessed and statistically analyzed, which helps zoologists and animal protection departments to gain a more comprehensive understanding of animal resources.

[0272] Figure 3 A schematic block diagram of a device for determining animal species provided in an embodiment of this application, such as... Figure 3As shown, the apparatus provided in this embodiment includes:

[0273] The acquisition unit 310 is used to acquire a video stream of the animal to be identified, the video stream including a first image.

[0274] The acquisition unit 310 is further configured to acquire first feature data, second feature data and third feature data of the first image. The first feature data includes at least one of the outline of the animal body surface of the animal to be identified, the color of the animal body surface or the texture of the animal body surface. The second feature data includes the animal posture of the animal to be identified. The third feature data includes the predation information of the animal to be identified.

[0275] The processing unit 320 is used to input the first feature data into the first model and output the first recognition result.

[0276] The processing unit 320 is also used to input the second feature data into the second model and output the second recognition result.

[0277] The processing unit 320 is also used to input the third feature data into the third model and output the third recognition result.

[0278] The processing unit 320 is also used to determine the target animal species of the animal to be identified based on the first identification result, the second identification result and the third identification result.

[0279] Figure 3 The device described above can perform the functions of the first device in the above method embodiments, and will not be described in detail here to avoid redundancy.

[0280] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0281] Based on the same inventive concept Figure 4A schematic block diagram of another apparatus for determining animal species provided in an embodiment of this application includes a processor coupled to a memory. When the processor executes a computer program or instructions stored in the memory, it implements the method of the first aspect or any embodiment of the first aspect described above.

[0282] Based on the same inventive concept, this application provides a computer storage medium storing a computer program, which, when executed by a processor, implements the method of the first aspect or any embodiment of the first aspect described above.

[0283] If the integrated units described above are implemented as software functional units and sold or used as independent products, they can be stored in a device. Based on this understanding, all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer chip, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable storage medium can include at least: any entity or device capable of carrying computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0284] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0285] In the embodiments provided in this application, it should be understood that the disclosed apparatus / devices and methods can be implemented in other ways. For example, the apparatus / device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0286] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0287] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0288] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for determining animal species, characterized in that, The method is applied to a first device and includes: Acquire a video stream of the animal to be identified, the video stream including a first image; Extract first feature data, second feature data, and third feature data from the first image. The first feature data includes at least one of the following: the outline of the animal's body surface, the color of the animal's body surface, or the texture of the animal's body surface. The second feature data includes the animal's posture. The third feature data includes the predation information of the animal, which indicates the prey of the animal, including animals, leaves, or grass. The first feature data is input into the first model, and the first recognition result is output. The second feature data is input into the second model, and the second recognition result is output. The third feature data is input into the third model, and the third recognition result is output. The target animal species of the animal to be identified is determined based on the first identification result, the second identification result, and the third identification result.

2. The method as described in claim 1, characterized in that, The method further includes: Obtain the audio data corresponding to the video stream; The sound data is input into the fourth model, and a fourth recognition result is output, which indicates the species of the animal to be identified. If the target animal species matches the species of the animal to be identified indicated by the fourth identification result, the fourth model is updated based on the fourth identification result and the sound data; If the target animal species does not match the species of the animal to be identified indicated by the fourth identification result, a first instruction is output, which is used to instruct the fourth identification result to be manually calibrated. Receive the calibrated fourth identification result; The fourth model is updated based on the calibrated fourth recognition result and the sound data.

3. The method as described in claim 1, characterized in that, The method further includes: Obtain the audio data corresponding to the video stream; The sound data is input into the fifth model, and a fifth recognition result is output, which indicates the species of the animal to be identified. The step of determining the target animal species of the animal to be identified based on the first identification result, the second identification result, and the third identification result includes: The target animal species of the animal to be identified is determined based on the first identification result, the second identification result, the third identification result, and the fifth identification result.

4. The method as described in claim 1, characterized in that, The method further includes: Obtain the audio data corresponding to the video stream; Based on the sound data, the health status of the target animal species is determined.

5. The method as described in claim 4, characterized in that, After determining the target animal species of the animal to be identified based on the first identification result, the second identification result, and the third identification result, the method further includes: The first health status of the target animal species is determined based on the first feature data; The second health status of the target animal species is determined based on the second characteristic data; The third health status of the target animal species is determined based on the third characteristic data; The step of determining the health status of the target animal species based on the sound data includes: The health status of the target animal species is determined based on the first health status, the second health status, the third health status, and the sound data.

6. The method as described in claim 1, characterized in that, The method further includes: Animals of the same species in the first image are labeled with the same tag, and animals of different species are labeled with different tags, thus obtaining the labeled first image; After determining the target animal species of the animal to be identified based on the first identification result, the second identification result, and the third identification result, the method further includes: The number of tags for the target animal species in the first image after labeling is counted to determine the number of target animal species.

7. The method as described in claim 1, characterized in that, The first identification result is used to indicate the species of the animal to be identified, the second identification result is used to indicate the species of the animal to be identified, and the third identification result is used to indicate the species of the animal to be identified; After determining the target animal species of the animal to be identified based on the first identification result, the second identification result, and the third identification result, the method further includes: Obtain the second image with the highest similarity to the first image from the animal information database; The second image is input into the YOLO algorithm, which only requires browsing once to obtain the first animal species; If the first animal species is the same as the target animal species, update the first model based on the target animal species and the first feature data; update the second model based on the target animal species and the second feature data; update the third model based on the target animal species and the third feature data; or... If the first animal species is different from the target animal species, a second instruction is output, which is used to instruct the target animal species to be manually calibrated. Receive the calibrated target animal species; The first model is updated based on the calibrated target animal species and the first feature data; the second model is updated based on the calibrated target animal species and the second feature data; and the third model is updated based on the calibrated target animal species and the third feature data.

8. The method as described in claim 7, characterized in that, The animal information database includes an animal body surface database, an animal posture database, and an animal predation information database. The step of obtaining the second image with the highest similarity to the first image from the animal information database includes: Obtain the first body surface data in the animal body surface database that has the highest similarity to the first feature data; Obtain the first posture data in the animal posture database that has the highest similarity to the second feature data; Obtain the first predator information in the animal predator information database that has the highest similarity to the third feature data; The second image is obtained based on the first body surface data, the first posture data, and the first predation information.

9. An apparatus for determining animal species, comprising a processor coupled to a memory, the processor being configured to execute a computer program or instructions stored in the memory to implement the method as described in any one of claims 1-8.

10. A computer storage medium, characterized in that, The computer storage medium stores a computer program, which, when executed by a processor, implements the method as described in any one of claims 1-8.

Images