Species detection method and device, electronic device and computer readable storage medium

Abstract

The invention relates to a species detection method and device, an electronic device and a computer readable storage medium, and the method comprises the steps: obtaining a to-be-recognized animal species image; according to the trained species type identification model, identifying the animal species image, and determining a target species type to which the animal species image belongs; the species type identification model is obtained by training based on a first data set marked with different animal species types; under the condition that the target species type is configured with state detection, based on a trained species state detection model corresponding to the target species type, identifying the animal species image, and determining a species state corresponding to the animal species image; wherein the species state detection model is obtained by training a second data set which takes the species type as a target species type and is marked with different health state degrees. Based on model training, species type identification and species state detection of animal images can be realized.

Description

Technical Field

[0001] The present application relates to the field of image detection, and in particular to a species detection method, device, electronic device and computer-readable storage medium. Background Art

[0002] Animals are an important part of the earth's ecological environment and the foundation for the development of human society. Nowadays, protecting animals and protecting the ecological environment are issues that human society is increasingly concerned about. At present, wildlife monitoring is often carried out by placing wildlife monitoring cameras in places where wild animals frequently appear. The wildlife monitoring camera senses animals through infrared rays to generate snapshots or videos, and uploads them to the platform through the Internet. After that, the background classification method configured by the platform will classify wild animals of roughly the same size into one category based on equivalent length.

[0003] Therefore, in the above-mentioned related technologies, only the capture of wild animal images and rough classification based on body length can be achieved, and the identification of animal species and the detection of animal status cannot be achieved.

[0004] There is currently no effective solution to the problem that related technologies cannot identify animal species and detect animal status. Summary of the invention

[0005] In this embodiment, a species detection method, device, electronic device and computer-readable storage medium are provided to solve the problem that the related art cannot realize the identification of animal species type and the detection of animal status.

[0006] In a first aspect, a species detection method is provided in this embodiment, comprising: Obtain an image of an animal species to be identified; According to the trained species type recognition model, the animal species image is recognized to determine the target species type to which the animal species image belongs; the species type recognition model is trained based on a first data set labeled with different animal species types; In the case where the target species type is configured with status detection, the animal species image is identified based on a trained species status detection model corresponding to the target species type, and the species status corresponding to the animal species image is determined; wherein the species status detection model is trained based on a second data set whose species type is the target species type and is annotated with different health status levels.

[0007] In some embodiments, obtaining an image of an animal species to be identified includes: Acquire images of animal species captured by wildlife monitoring cameras and / or images of animal species captured by smart monitoring cameras.

[0008] In some embodiments, the training process of the species type recognition model includes: Obtaining an original data set for model training and annotation information of animal species types corresponding to the original data set; According to the labeling information, data enhancement processing is performed on the minority class samples and the attention class samples in the original data set based on an oversampling strategy to obtain the first data set; Based on the first data set, the preset neural network model is iteratively trained until a preset iteration termination condition is reached to obtain the trained species type recognition model.

[0009] In some embodiments, data enhancement processing is performed on minority class samples and attention class samples in the original data set according to the annotation information to obtain the first data set, including: Determine the K nearest neighbors corresponding to the minority class samples and the attention class samples in the original data set respectively; Determine the boundary samples in the original data set according to the K nearest neighbors; Interpolation processing is performed on the boundary samples, and the first data set is obtained based on the interpolation processing result.

[0010] In some of the embodiments, the species type identification model and the species state detection model have the same algorithm structure.

[0011] In some embodiments, the species type recognition model includes a backbone network, a neck network, and a classification structure; the input data of the species type recognition model passes through the backbone network, the neck network, and the classification structure in sequence; wherein: The backbone network includes: a feature extraction module, a network depth increase module and a feature map space size adjustment module; The neck network includes: a feature extraction module, a sampling module and a feature splicing module; The classification structure includes: a classifier and a regressor.

[0012] In some of the embodiments, after determining the species status corresponding to the animal species image, the method further includes: Obtaining a body temperature recognition result corresponding to the animal species image; The health status information corresponding to the animal species image is determined according to the body temperature recognition result and the species status.

[0013] In a second aspect, a species detection device is provided in this embodiment, including: an acquisition module, a species identification module and a state detection module; wherein: The acquisition module is used to acquire an image of an animal species to be identified; The species recognition module is used to recognize the animal species image according to the trained species type recognition model to determine the target species type to which the animal species image belongs; the species type recognition model is trained based on a first data set labeled with different animal species types; The status detection module is used to identify the animal species image and determine the species status corresponding to the animal species image based on a trained species status detection model corresponding to the target species type when the target species type is configured with status detection; wherein the species status detection model is trained based on a second data set whose species type is the target species type and is annotated with different health status levels.

[0014] In a third aspect, an electronic device is provided in this embodiment, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the species detection method described in the first aspect when executing the computer program.

[0015] In a fourth aspect, a storage medium is provided in this embodiment, on which a computer program is stored, and when the program is executed by a processor, the species detection method described in the first aspect is implemented.

[0016] Compared with the related art, the present embodiment provides a species detection method, device, electronic device and computer-readable storage medium. The species detection method obtains an image of an animal species to be identified; the animal species image is identified according to a trained species type recognition model to determine the target species type to which the animal species image belongs; the species type recognition model is trained based on a first data set labeled with different animal species types; when the target species type is configured with state detection, the animal species image is identified based on a trained species state detection model corresponding to the target species type to determine the species state corresponding to the animal species image; wherein the species state detection model is trained based on a second data set whose species type is the target species type and is labeled with different health status levels. It can realize species type recognition and species state detection of the animal image based on model training.

[0017] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 is a hardware structure block diagram of a terminal of the species detection method of this embodiment; Figure 2 is a flow chart of the species detection method of this embodiment; Figure 3 is a schematic diagram of constructing a first data set in some embodiments; Figure 4 is a schematic diagram of constructing a second data set in some of the embodiments; Figure 5 is a schematic diagram of the structure of the backbone network of this embodiment; Figure 6 is a schematic diagram of the structure of a CGS module in some of the embodiments; Figure 7 is a schematic structural diagram of a CCRC module in some of the embodiments; Figure 8 is a schematic diagram of the specific structure of the Resx module in some embodiments; Fig. 9 is a schematic diagram of the structure of the CP module of some embodiments; Fig.10 is a schematic diagram of the structure of the neck network of some embodiments; Fig.11 is a schematic diagram of the algorithm structure of a species state detection model and a species type identification model in some embodiments; Fig.12 is a flow chart of a species detection method of some embodiments; Fig.13a is a schematic diagram of input and output of a training species type recognition model in this embodiment; Fig.13b is a schematic diagram of input and output of a training species state model of this embodiment; Fig.14 4 is a structural block diagram of the species detection device of this embodiment. DETAILED DESCRIPTION

[0019] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0020] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with general skills in the technical field to which this application belongs. The words "one", "a", "the", "these" and the like in this application do not indicate a quantitative limitation, and they may be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" may mean: A exists alone, A and B exist at the same time, and B exists alone. Generally, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0021] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 FIG. 1 is a hardware structure diagram of a terminal of the species detection method of this embodiment. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 and a memory 104 for storing data, wherein the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0022] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the species detection method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0023] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by a communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0024] In this embodiment, a species detection method is provided. Figure 2 is a flow chart of the species detection method of this embodiment, such as Figure 2 As shown, the process includes the following steps: Step S210, obtaining an image of an animal species to be identified. The image of the animal species can be obtained by capturing or recording the wild animals in the nature reserve through a camera deployed in the nature reserve. Afterwards, the camera can upload the image of the animal species to a platform for species identification and status detection through the network.

[0025] Step S220, identifying the animal species image according to the trained species type recognition model to determine the target species type to which the animal species image belongs; the species type recognition model is trained based on a first data set labeled with different animal species types.

[0026] After receiving the data in step S210, the platform will display the information of the animal species image. At this time, the animal species image still lacks the associated species information and health status information waiting to be identified. It can be determined whether the currently reported animal species image needs to be identified according to the pre-set business configuration information; if so, the species type of the above animal species image is identified based on the trained species type identification model, and the target species type corresponding to the identified animal species image is updated in the information display. If species type identification is not required, the subsequent process ends.

[0027] Among them, species type, or species classification, is a core concept in biology, used to describe and distinguish different biological groups. The classification of species types mainly relies on the biological classification system, which classifies organisms hierarchically according to their similarities. The species type recognition model in this step is pre-trained based on a training set labeled with different animal species types. Therefore, the species type recognition model has the ability to recognize different animal species.

[0028] Exemplarily, when constructing the above-mentioned first data set, images or videos can be uploaded through a camera (after the video is uploaded, it can be separated into single frame images by frame through video processing tools), the obtained images are filtered, all species data therein are filtered out, and the corresponding species are labeled respectively, so as to construct the first data set. Figure 3 A schematic diagram of constructing a first data set in some of the embodiments is shown in FIG. Figure 3 As shown, when constructing the first data set, the original reported data is first obtained. The original reported data can be the original data collected by the camera or queried online for constructing the first data set. All species data are filtered out from the original reported data to form a species original data set. The species type of each species sample in the species original data set is labeled, for example, "giant panda", "red panda", "antelope", etc. After that, the first data set labeled with different animal species types is obtained, and the above-mentioned trained species type recognition model is obtained through model training.

[0029] Step S230, when the target species type is configured with status detection, the animal species image is identified based on a trained species status detection model corresponding to the target species type to determine the species status corresponding to the animal species image; wherein the species status detection model is trained based on a second data set whose species type is the target species type and is annotated with different health status levels.

[0030] The species status may specifically be the health status corresponding to a certain species type. In the construction stage of the second data set, the health status may be annotated based on the image information such as the animal gait, body shape, fur color, and whether there is a wound, etc. presented by the training sample. The corresponding status detection requirements may be configured for the species type that needs to be detected. When it is determined based on step S220 that the target species type to which the animal species image belongs is the species type that needs to be detected, the animal species image is then subjected to status detection according to the trained species status detection model corresponding to the target species type. It can be understood that the first data set may be a training set containing a variety of different animal species types. The second data set may be a training set constructed for a certain species type. That is, for different species types, the corresponding second data sets are different, and the corresponding trained species status detection models are also different. For example, when the target species type is a species image of a giant panda, it needs to be input into the trained species status detection model corresponding to the giant panda for status detection. At this time, the second data set corresponding to the species status detection model is constructed for species samples containing giant pandas with different health statuses. Similarly, when the above-mentioned target species type is an image of an antelope, it needs to be input into the trained species state detection model corresponding to the antelope for state detection. At this time, the second data set corresponding to the species state detection model is constructed by species samples containing antelopes with different health status. Optionally, the annotation information in the second data set can be divided into two types: "healthy" and "unhealthy". Healthy and unhealthy species images can be distinguished from various species samples of the target species type, so as to construct a second data set for identifying the health status of species.

[0031] Figure 4 A schematic diagram of constructing a second data set in some of the embodiments is shown in FIG. Figure 4 As shown in the figure, firstly, the original reported data is obtained, and the giant panda species samples (for species samples of the giant panda species type) are screened out to construct a giant panda health status dataset. For species samples of the giant panda species type, "healthy" and "unhealthy" can be marked according to the body shape, fur color and other information of the animal in the picture to construct a second dataset corresponding to the giant panda species type. Similarly, second datasets corresponding to species types such as red pandas and antelopes can also be constructed.

[0032] In some embodiments, the algorithm cores for identifying species types and identifying the degree of species health status may be the same, and only the data sets used for model training are different, so that the trained algorithm recognition effects are different and the model output results are different. Among them, in the training stage, the model for identifying species types inputs a first data set marked with species types, and the model will output classification information of species types. The model for identifying the degree of species health status inputs a second data set marked with the degree of species health status of a certain species, and for the same species type, the corresponding input training sets are all the same species, and the output result is the degree of health status of the species. Before performing species status detection, it is necessary to first identify the species type to which the animal species image belongs. Optionally, the above-mentioned species type recognition and species status detection can be performed by the intelligent monitoring camera after completing the general recognition of the animal as a bird, beast or reptile, etc., and reporting the species monitoring event according to the recognition information, and then the associated algorithm server completes the specific species type and species status recognition and detection.

[0033] In related technologies, it is often only possible to capture images of wild animals and roughly classify them based on body length, but it is not possible to identify the type of animal species or detect the animal's status. In addition, the morphological characteristics of different species vary greatly, and the body characteristics of species with different health levels vary greatly. Therefore, the generalization of monitoring algorithms is particularly important, and traditional image recognition is no longer sufficient to meet current usage scenarios.

[0034] This embodiment first identifies the species type based on the trained model, and then analyzes the species status after identifying the species type. For unhealthy animals, an alarm message can also be generated to remind relevant personnel to treat the animals in time. Therefore, compared with related technologies, this embodiment can realize the identification of specific species types and species health status, and the identification range is wider.

[0035] Through the above steps S210 to S230, the animal species image to be identified is obtained; according to the trained species type identification model, the animal species image is identified to determine the target species type to which the animal species image belongs; the species type identification model is trained based on the first data set labeled with different animal species types; in the case where the target species type is configured with state detection, the animal species image is identified based on the trained species state detection model corresponding to the target species type to determine the species state corresponding to the animal species image; wherein the species state detection model is trained based on the second data set whose species type is the target species type and is labeled with different health status levels. It can realize the species type identification and species state detection of the animal image based on model training.

[0036] In one embodiment, based on the above step S210, obtaining an image of an animal species to be identified may include: Acquire images of animal species captured by wildlife monitoring cameras and / or images of animal species captured by smart monitoring cameras.

[0037] Among them, the above-mentioned animal species images can be collected by either an intelligent monitoring camera or a wildlife monitoring camera. A wildlife monitoring camera, also known as a wildlife protection camera, is a camera device specifically used to monitor and record the behavior of wild animals. It detects the activities of animals through infrared sensors or motion sensors. When the animal enters the monitoring range of the camera, the camera will automatically trigger the photo or video function to capture the image information of the animal in the natural environment. It is suitable for deployment in natural environments such as nature reserves and legal hunting areas. An intelligent monitoring camera, also known as an intelligent camera or an intelligent monitoring camera, is a monitoring device that integrates image acquisition, processing, analysis and network communication functions. An intelligent monitoring camera is a product of the combination of traditional cameras and modern artificial intelligence technologies (such as machine learning, computer vision, etc.). An intelligent monitoring camera can not only capture images and videos, but also analyze and process these data. In this embodiment, it can identify the approximate category of the target object in the image, such as birds, beasts, etc. The intelligent monitoring camera can be deployed on urban roads, commercial places, or other outdoor places that can provide a charging environment. Therefore, according to the physical environment conditions of the actual application scenario, a wildlife monitoring camera and / or an intelligent monitoring camera can be selected to collect animal species images.

[0038] Specifically, when the intelligent monitoring camera identifies an animal, it generates a snapshot image or video, and uploads the identification information to the platform through the network. The wildlife monitoring camera detects the passing of animals through infrared sensing, generates a snapshot image or video, and uploads the image or video to the platform through the network. Thus, the platform can obtain the above-mentioned image of the animal species to be identified. It can be understood that when constructing the above-mentioned first data set and second data set, data can also be collected based on wildlife monitoring cameras or intelligent monitoring cameras. This embodiment can realize real-time image acquisition of wild animals, thereby realizing timely species identification and status monitoring.

[0039] In particular, in some of the embodiments, when an image of an animal species that needs to be paid attention to in the current monitoring scene is identified based on species type recognition and species state detection, the training set can be supplemented based on the identified image. For example, by performing species type recognition and species state detection on a certain animal species image currently collected, it is determined that the animal species image is an image of a red panda in an unhealthy state, and the current monitoring scene needs to detect unhealthy red pandas in a timely manner. In this case, the animal species image can be added to the training set as an image that needs to be paid attention to and has detection significance for the current monitoring scene to enhance the quality and quantity of the images in the training set. By continuously expanding the training set based on the recognition results during the recognition process, and continuing to further upgrade and optimize the above-mentioned species type recognition model and species state detection model based on the training set, the recognition capabilities of the above-mentioned species type recognition model and species state detection model can be further enhanced, thereby improving the accuracy of the final recognition result.

[0040] Additionally, in one embodiment, the training process of the species type recognition model may specifically include: The original data set used for model training and the annotation information of the animal species types corresponding to the original data set are obtained; according to the annotation information, data enhancement processing is performed on the minority class samples and the attention class samples in the original data set based on the oversampling strategy to obtain a first data set; based on the first data set, the preset neural network model is iteratively trained until the preset iteration termination condition is reached, and a trained species type recognition model is obtained.

[0041] When constructing the first data set, data enhancement processing may be performed on several minority class samples and attention class samples.

[0042] The minority class samples can be samples of a species type with a small number in the original reported data, and the attention class samples can be samples of a species type that are predetermined and need to be focused on from the original reported data. For example, in the data uploaded by the current reserve, it is necessary to pay attention to giant pandas, so the samples of the species type giant panda are the attention class samples; at the same time, there are more monkey images and fewer black bear images in the current reported data, so the samples of the species type black bear are the minority class samples. Therefore, data enhancement can be performed on the image samples of giant pandas and black bears.

[0043] Specifically, the images of the filtered original reported data are randomly cropped, for example, they can be cropped to a size of 640×640, and then the filtered species data set is labeled, and the corresponding species information and species health status can be specifically labeled. In addition, the data enhancement strategy can be adjusted according to business needs and data in the data set. The current reserve is concerned about the health of giant pandas, and it is necessary to identify unhealthy giant pandas. Based on the oversampling strategy, a data enhancement strategy is used for unhealthy giant pandas to enhance the model's ability to identify unhealthy giant pandas. Specifically, the Borderline-SMOTE oversampling strategy can be used to enhance the model's recognition ability for the focus class and minority class.

[0044] In addition, after completing the above-mentioned data enhancement processing based on the oversampling strategy, part of the data in the original data set can also be randomly flipped horizontally to enhance the model's ability to identify species in different directions and improve the model's robustness. This embodiment preprocesses the image in the above manner to enrich the data set, improve the model's robustness, prevent the model from overfitting, and simulate real animal monitoring scenarios.

[0045] In one embodiment, according to the annotation information, data enhancement processing is performed on the minority class samples and the attention class samples in the original data set to obtain the first data set, which may specifically include: Determine the K nearest neighbors corresponding to the minority class samples and the attention class samples in the original data set respectively; determine the boundary samples in the original data set based on the K nearest neighbors; perform interpolation processing on the boundary samples, and obtain the first data set based on the interpolation processing result.

[0046] In this embodiment, the minority class samples and the attention class samples can be divided into three categories by the Border-line SMOTE algorithm. Specifically, the K nearest neighbors of each minority class sample or attention class sample are calculated. The K nearest neighbors are the K nearest samples of a sample. If for a minority class sample or attention class sample, all the K nearest neighbor samples belong to the majority class, then the minority class sample or attention class sample is classified as a noise point and is not processed; if for a minority class sample or attention class sample, more than half but less than all of the K nearest neighbor samples belong to the majority class, then the minority class sample or attention class sample is classified as a dangerous point, and the dangerous point is a boundary sample, which requires data enhancement processing. If for a minority class sample or attention class sample, less than half of the K nearest neighbor samples belong to the majority class (or in other words, more than half of the K nearest neighbor samples belong to the minority class), then the minority class sample or attention class sample is classified as a safe point and is not processed.

[0047] Next, the majority class mentioned above is explained by taking species type as an example. One species type in the data set can be used as the majority class, or multiple species types can be used as the majority class. In this embodiment, the majority class refers to a data category with a relatively large number of samples in the data set (such as the species type in the first data set). For example, in the first data set, when one or several species types have similar or similar numbers of samples, and are significantly higher than other species types, they can be regarded as the majority class. For example, there are species type A, species type B, and species type C in the data set. Among them, species type A has 600 samples, species type B has 550 samples, and species type C has only 100 samples, then species type A and species type B can both be regarded as the majority class.

[0048] That is, data enhancement is only performed on the boundary samples identified as dangerous points. Specifically, a new sample is generated by randomly selecting from all boundary samples and then interpolating through the Synthetic Minority Oversampling Technique (SMOTE) algorithm. The specific process of interpolating and generating new samples is as follows: (1) In formula (1), x n is the new sample generated, x i is a sample selected from the boundary samples, x zi represent x i A random K-nearest neighbor sample of β It is a random number between 0 and 1, used to control the sample difference.

[0049] Since all the noise points are majority classes, the generated samples will overlap with the majority class. More than half of the samples are minority class samples, which are safe points. Oversampling the safe points can easily lead to overfitting, and the newly synthesized samples will not provide much useful information. Based on this, this embodiment oversamples the dangerous points (boundary samples) and generates new samples around the boundary samples, which can improve the generalization ability of the model and reduce the risk of overfitting. Therefore, the Borderline-SMOTE oversampling strategy can enhance the model's recognition ability for the focus class and the minority class.

[0050] Additionally, in one embodiment, the algorithm structure of the species type recognition model is consistent with that of the species state detection model. The species type recognition model and the species state detection model can be constructed using the same algorithm structure. However, the specific data sets used for training are different, so the recognition capabilities and recognition results of the two models are ultimately different.

[0051] In one embodiment, the species type recognition model includes a backbone network, a neck network and a classification structure; the input data of the species type recognition model passes through the backbone network, the neck network and the classification structure in sequence; wherein: the backbone network includes: a feature extraction module, a network depth increase module and a feature map space size adjustment module; the neck network includes: a feature extraction module, a sampling module and a feature splicing module; the classification structure includes: a classifier and a regressor.

[0052] The algorithm structure of the species type recognition model is consistent with that of the species state detection model. The following mainly uses the algorithm structure of the species type recognition model as an example to illustrate. Among them, the backbone network can include a feature extraction module (CGS module), a network depth increase module (CCRC module) and a feature map space size adjustment module (CP module).

[0053] Figure 5 is a schematic diagram of the structure of the backbone network of this embodiment, such as Figure 5 As shown in the figure, the input image data with a dimension of 3×640×640 first extracts feature information through the CGS module to obtain a feature map with 32 channels and a size of 320×320. The feature information is further extracted through the CGS module to obtain a feature map with 64 channels and a size of 160×160. Then, the network depth is increased through the CCRC module (output 64×160×160), the feature expression ability is improved, and the module diversity is increased. After that, it passes through the CGS module (output 128×80×80), the CCRC module (output 128×80×80), the CGS module (output 256×40×40), and the CCRC module (output 256×40×40), and the CP module (output 512×20×20) is used to reduce the spatial size of the feature map, reduce the computational complexity, and extract important features.

[0054] in, Figure 6 FIG. 1 is a schematic diagram of a CGS module in some of the embodiments. Figure 6 As shown, the GCS module can include a convolution layer (Conv), a group normalization layer (GN), and a loss function (Silu). The dimension of the input image data is 640×640×3, and the convolution operation is performed through the convolution layer to extract image features. The convolution kernel size can be 6×6, the step size is 2, the padding is 2, the number of channels is consistent with the number of channels of the input data, and the number of convolution kernels is 32. After the convolution operation of the convolution layer, the size of the output feature map can be shown as follows (2): (2) In formula (2), N represents the output size, W represents the input size, F represents the convolution kernel size, P represents the padding value size, S represents the step size, and the feature map size after the convolution operation is 320.

[0055] After the convolution operation is completed, the feature map is normalized through the group normalization layer, as follows: (3) Among them, in formula (3), x is the input data, μ is the mean of the input data, σ is the variance of the input data, This is a constant set to prevent division by 0. γ and β is a learnable parameter that can be adjusted during training. Group normalization divides the features into several groups and normalizes them within these groups.

[0056] After group normalization, nonlinearity will be introduced through the SiLu activation function to alleviate problems such as gradient disappearance. The activation function can guide network training and accelerate network convergence. The definition of the SiLu activation function is as follows: f(x)=x×sigmoid(x) (4) Among them, sigmoid(x) is the standard sigmoid function, and the specific form of the sigmoid function is shown in formula (5): (5) Additionally, Figure 7 Schematic diagram of the structure of a CCRC module in some of the embodiments, such as Figure 7 As shown, the CCRC module includes a CGS module, a Resx module and a concat module. The specific structure of the CGS module can refer to the above embodiments and Figure 6 , which will not be described here. The Resx module is a residual network. Figure 8 Detailed structural diagram of the Resx module in some embodiments is shown in FIG. Figure 8 As shown in the figure, the Resx module mainly includes a convolutional layer and an activation function Relu. The residual network directly superimposes the output F(x) of the L-1th layer onto the output of the Lth convolutional layer to output F(x)+x. The x-identity mapping means that when F(x) approaches 0, F(x)+x=x. At this time, the range of functions that can be fitted by the L-1 convolutional layer can completely cover the range of functions that can be fitted by the L-1 convolutional layer. The residual network can solve the problems of gradient diffusion and feature redundancy, and can make the deep network model achieve better results. The expression of the residual network can be: (6) For the The input of the convolutional layer, For the The input of the convolutional layer, For the The output of the convolutional layer.

[0057] In addition, the function of the Concat module is to concatenate and combine the incoming feature maps to retain more feature information. The expression of the Concat module can be: (7) Fig. 9 Schematic diagram of the structure of the CP module in some embodiments, such as Fig. 9 As shown in the figure, the CP module includes a pooling layer and a 1×1 convolution layer. The pooling layer reduces the spatial size of the feature map, reduces the computational complexity, extracts important features, and prevents overfitting.

[0058] Additionally, Fig.10 is a schematic diagram of the structure of the neck network in some embodiments, such as Fig.10 As shown, for the neck network, it can include the above CGS module, the upsample module (Upsample module) and the above CCRC module. The Upsample module can use bilinear interpolation, as shown in the following formula (8):

[0059] Among them, in formula (8), f ( x , y ) is the point to be interpolated. Q 11 For ( x 1, y 1) A known point at Q 21 For ( x 2, y 1) A known point at Q 12 For ( x 1, y 2) known points at Q 22 For ( x 2, y 2) A known point at.

[0060] The final classification structure receives the feature information input by the neck network and performs target detection and recognition. In the species type recognition model, it is the species type recognition, and in the species state detection model, it is the species health status recognition. The classification structure can include a classifier and a bounding box regressor to predict the target category and position. The classifier predicts the object category in the bounding box and obtains the feature output through convolution operation. The regressor is responsible for predicting the position and size of the bounding box. The classification prediction of the classification structure obtains the final recognition result.

[0061] It should be noted that the specific structures of different network modules and the data dimensions of input and output in the above embodiments are only examples. Those skilled in the art can make adaptive adjustments based on actual conditions and requirements for efficiency and accuracy of model training, and this embodiment does not specifically limit this. According to the needs of actual application scenarios, specific components in the corresponding network modules can be added, modified, deleted, and replaced, and the number of network layers can be adjusted. Optionally, the normalization layer in the CGS module can be replaced by a group normalization layer (Group Normalization, referred to as GN) with a batch normalization layer (BatchNormalization, referred to as BN). The structure of the convolution layer and the pooling layer in the CP module is replaced with a structure of convolution, pooling, and feature fusion.

[0062] Fig.11 Schematic diagram of the algorithm structure of the species state detection model and the species type identification model in some of the embodiments, such as Fig.11 As shown, the structure of the species state detection model is consistent with that of the species type recognition model, and both can include: a backbone network 112, a neck network 114, and a classification structure 116. The specific structures of the backbone network 112, the neck network 114, and the classification structure 116 can refer to the above embodiments, and will not be repeated here.

[0063] In this embodiment, firstly, the specific animal species is identified based on the species type recognition model, and then, according to the specific animal species and business configuration, it is determined whether it is necessary to analyze the health status of the species. If necessary, the species health status is detected based on the species status detection model. The core implementation method of the species status detection model is consistent with the species type recognition model. The difference is that the training set and test set of the species status detection model and the species type recognition model are different, and the output result information is different. The species status detection model includes a variety of algorithm models, such as the giant panda algorithm model, the red panda algorithm model, the Tibetan antelope algorithm model, etc., mainly based on which category or categories of species need to be concerned in the current monitored area (such as a protected area). The output result of the species status detection model is the degree of species health status, for example, it can be a value between 0 and 1, and the larger the value, the healthier the species.

[0064] In this embodiment, considering that animals of different species have large differences in body shape and different characteristic information. In view of such differences, traditional image recognition is no longer sufficient to meet the needs of accurate species detection. Based on deep learning technology, this embodiment extracts image feature information through image preprocessing, convolution, group normalization, feature map splicing, residual network, and upsampling, and performs classification based on the extracted feature information to predict the species type, thereby improving the accuracy of species type recognition.

[0065] In addition, the same species often presents different physical characteristics and has different physical information under different health conditions. Based on this difference, this embodiment processes the data set data and adjusts the training direction for the species type that needs to pay attention to the health status. One species type corresponds to one algorithm model. The specific species category is identified through the species type recognition model. After that, according to the identified species type, the corresponding species status detection model is selected to identify the health status and predict the health status of the animal.

[0066] Additionally, in one embodiment, after determining the species status corresponding to the animal species image, the species detection method may further include: Obtain a body temperature recognition result corresponding to the animal species image; and determine health status information corresponding to the animal species image based on the body temperature recognition result and the species status.

[0067] Specifically, you can configure the species that need to be identified for health status according to business needs. According to the identified target species type, determine whether the target species type needs to be identified for species health status. If the target species type is not identified, the detection process ends. If the target species type is identified but the target species type does not need to be detected for health status, the detection process ends.

[0068] If the target species type needs to be tested for health status, the species status is first obtained based on the species status detection model, and then the species health status is analyzed through the animal's body temperature. The animal's body temperature can be reported by the camera device that supports the body temperature detection function deployed at the front end. Combining the body temperature recognition results and the species status, the health status information corresponding to the animal species image is determined and the data display on the platform is updated. For unhealthy animals, alarm information can be generated and reported to remind relevant personnel to treat the animals in time.

[0069] Specifically, the temperature recognition result can be one of 0, -1, and 1. 0 means that the temperature is not recognized or the front-end device does not report the temperature information; -1 means that the temperature is in an abnormal temperature range; 1 means that the temperature is in a normal temperature range. Multiply the temperature recognition result by the preset coefficient and then add it to the value of the species status to obtain the final species health status information: (9) f is the species health status information; is the species status output by the species status detection model, 0.32 is the coefficient that needs to be multiplied by the body temperature recognition result, which is an empirical value. Technical personnel in this field can also set other values ​​according to the requirements of the actual application scenario; p is the body temperature recognition result, and its value is one of 0, -1, and 1.

[0070] Therefore, the species detection method of this embodiment has a high recognition rate, a wide recognition range and can achieve real-time and accurate monitoring of the health status of species.

[0071] Fig.12 is a flow chart of species detection methods in some of the embodiments, such as Fig.12 As shown, the species detection method includes the following steps: Step S1201, obtaining an image of an animal species captured by a smart monitoring camera.

[0072] Step S1202, obtaining the animal species image collected by the wildlife monitoring camera. Among them, step S1201 and step S1202 can be performed one by one, or both. The execution order of step S1201 and S1202 is not fixed, and the two steps are independent of each other. The intelligent monitoring camera or wildlife monitoring camera uploads the data to the platform through the network for data display.

[0073] Step S1203, determine whether the species identification function is configured; if so, execute step S1204; otherwise, end the process.

[0074] Step S1204: identifying the animal species image based on the trained species type recognition model.

[0075] Step S1205, determine whether the species type is successfully identified; if so, execute step S1206 and step S1209; otherwise, end the process.

[0076] Step S1206, determine whether the identified target species type needs to be tested for health status; if so, execute step S1207; otherwise, end the process.

[0077] Step S1207: Detect the animal species image based on the trained species state detection model.

[0078] Step S1208, determine whether the species status is successfully identified; if so, execute step S1209; otherwise, end the process.

[0079] Step S1209: update platform display information.

[0080] In the above steps S1201 to S1209, based on deep learning technology, algorithmic analysis is performed on species images to identify specific species information. Compared with related technologies, the system is more intelligent and has a higher recognition rate, avoiding cumbersome manual operation processes and reducing labor costs. In addition, this embodiment can identify more species types and can also identify the health status of species. From the perspective of technical implementation, it is more comprehensive and has richer detection functions, which can improve better species protection conditions. It is suitable for protected areas with high requirements for species protection and can provide better monitoring and early warning effects. Species information and health status are reported in a timely and efficient manner to remind relevant personnel to handle them. In addition, the functional technology of this embodiment is highly flexible, and whether species identification is required and which species need to be tested for health status can be configured and implemented in advance.

[0081] Fig.13a Schematic diagram of input and output of a training species type recognition model in this embodiment. Fig.13a As shown, a first data set labeled with different species types is input into a preset algorithm structure for model training, and the identified species types are output. Fig.13b is a schematic diagram of input and output of a training species state model in this embodiment, Fig.13b As shown, the second data set labeled with the health status of different species is input into the preset algorithm structure for model training, and the identified species status is output. For example, based on the second data set labeled with the health status of giant pandas, a species status detection model for realizing status detection of giant pandas can be trained; based on the second data set labeled with the health status of red pandas, a species status detection model for realizing status detection of red pandas can be trained; based on the second data set labeled with the health status of antelopes, a species status detection model for realizing status detection of antelopes can be trained.

[0082] In this embodiment, a species detection device is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments, and the descriptions that have been made will not be repeated. The terms "module", "unit", "subunit", etc. used below can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0083] Fig.14 is a structural block diagram of the species detection device 14 of this embodiment, as shown in Fig.14 As shown, the species detection device 14 includes: an acquisition module 142, a species identification module 144 and a state detection module 146; wherein: An acquisition module 142 is used to acquire an image of an animal species to be identified; The species identification module 144 is used to identify the animal species image according to the trained species type identification model to determine the target species type to which the animal species image belongs; the species type identification model is trained based on the first data set labeled with different animal species types; The state detection module 146 is used to identify the animal species image and determine the species state corresponding to the animal species image based on a trained species state detection model corresponding to the target species type when the target species type is configured with state detection; wherein the species state detection model is trained based on a second data set whose species type is the target species type and is annotated with different health status levels.

[0084] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0085] In this embodiment, an electronic device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0086] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0087] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program: S1, obtaining an image of the animal species to be identified; S2, identifying the animal species image according to the trained species type recognition model to determine the target species type to which the animal species image belongs; the species type recognition model is trained based on a first data set labeled with different animal species types; S3, when the target species type is configured with status detection, the animal species image is identified based on a trained species status detection model corresponding to the target species type to determine the species status corresponding to the animal species image; wherein the species status detection model is trained based on a second data set whose species type is the target species type and is annotated with different health status levels.

[0088] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0089] In addition, in combination with the species detection method provided in the above embodiments, a storage medium may be provided in this embodiment to implement the method. The storage medium stores a computer program; when the computer program is executed by a processor, any species detection method in the above embodiments is implemented.

[0090] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.

[0091] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0092] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

[0093] The term "embodiment" in this application refers to a specific feature, structure or characteristic described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various locations in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is clearly or implicitly understood by those of ordinary skill in the art that the embodiments described in this application can be combined with other embodiments without conflict.

[0094] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. A species detection method, characterized in that: include: Obtain an image of an animal species to be identified; According to the trained species type recognition model, the animal species image is recognized to determine the target species type to which the animal species image belongs; the species type recognition model is trained based on a first data set labeled with different animal species types; In the case where the target species type is configured with status detection, the animal species image is identified based on a trained species status detection model corresponding to the target species type, and the species status corresponding to the animal species image is determined; wherein the species status detection model is trained based on a second data set whose species type is the target species type and is annotated with different health status levels.

2. The species detection method according to claim 1, characterized in that: Obtain images of the animal species to be identified, including: Acquire images of animal species captured by wildlife monitoring cameras and / or images of animal species captured by smart monitoring cameras.

3. The species detection method according to claim 1, characterized in that: The training process of the species type recognition model includes: Obtaining an original data set for model training and annotation information of animal species types corresponding to the original data set; According to the labeling information, data enhancement processing is performed on the minority class samples and the attention class samples in the original data set based on an oversampling strategy to obtain the first data set; Based on the first data set, the preset neural network model is iteratively trained until a preset iteration termination condition is reached to obtain the trained species type recognition model.

4. The species detection method according to claim 3, characterized in that: According to the labeling information, data enhancement processing is performed on the minority class samples and the attention class samples in the original data set to obtain the first data set, including: Determine the K nearest neighbors corresponding to the minority class samples and the attention class samples in the original data set respectively; Determine the boundary samples in the original data set according to the K nearest neighbors; Interpolation processing is performed on the boundary samples, and the first data set is obtained based on the interpolation processing result.

5. The species detection method according to claim 1, characterized in that: The algorithm structure of the species type identification model is consistent with that of the species status detection model.

6. The species detection method according to claim 5, characterized in that: The species type recognition model includes a backbone network, a neck network and a classification structure; the input data of the species type recognition model passes through the backbone network, the neck network and the classification structure in sequence; wherein: The backbone network includes: a feature extraction module, a network depth increase module and a feature map space size adjustment module; The neck network includes: a feature extraction module, a sampling module and a feature splicing module; The classification structure includes: a classifier and a regressor.

7. The species detection method according to any one of claims 1 to 6, characterized in that: After determining the species status corresponding to the animal species image, the method further includes: Obtaining a body temperature recognition result corresponding to the animal species image; The health status information corresponding to the animal species image is determined according to the body temperature recognition result and the species status.

8. A species detection device, characterized in that: include: Acquisition module, species identification module and status detection module; among which: The acquisition module is used to acquire an image of an animal species to be identified; The species recognition module is used to recognize the animal species image according to the trained species type recognition model to determine the target species type to which the animal species image belongs; the species type recognition model is trained based on a first data set labeled with different animal species types; The status detection module is used to identify the animal species image and determine the species status corresponding to the animal species image based on a trained species status detection model corresponding to the target species type when the target species type is configured with status detection; wherein the species status detection model is trained based on a second data set whose species type is the target species type and is annotated with different health status levels.

9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the species detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the species detection method according to any one of claims 1 to 7 are implemented.

Images