Wildlife monitoring image false trigger image screening method and storage medium

Abstract

The embodiment of the application discloses a kind of wild animal monitoring image false trigger image screening method and storage medium, method includes: obtaining the monitoring image that camera is shot, constructs training set;Using the training set, network model is trained, and image screening model is obtained;From all images that camera is shot in the screening, obtain a empty shot image as reference image, the remaining image is as screening image;The reference image and screening image are input into the image screening model, and screening result is obtained.The false trigger image discrimination technology based on triplet loss degree measurement learning is main body in the present application, can discriminate infrared trigger camera false trigger image and then filter it, finally realize the automation of invalid data filtering in infrared trigger camera wild animal monitoring image original data, avoid the manual screening of massive monitoring data, help to promote the intelligentization and informatization of wild animal protection work.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer software, in particular to a wild animal monitoring image false triggering image screening method and a storage medium. BACKGROUND

[0002] Wild animal monitoring work is of great significance to wild animal protection and biodiversity protection. At present, infrared triggered cameras are one of the main tools for obtaining wild animal monitoring images. When wild animals move within the sensing range of the infrared camera, the camera will automatically take pictures of the wild animals. After collecting the data, some cameras can access the Internet through a gateway node through WSN, etc., and transmit the image data to the server end. Or the camera only stores the image in the SD card, waiting for the staff to collect the data. Infrared triggered cameras are usually deployed in a grid, with a scale of dozens to hundreds, and the data is collected every few months.

[0003] Due to the effect of infrared triggered cameras and natural conditions such as the outdoor environment, infrared triggered cameras generally have a high false triggering (empty shooting) rate, that is, many images taken by the camera do not contain wild animals. On the other hand, the amount of data collected by infrared cameras is extremely large, such as the Serengeti wild animal data set which has about 1.2 million images, only about 300,000 of which are valid images containing animals, with a false triggering rate of up to 75%. Artificially screening valid images from the massive images will consume a huge amount of manpower and resources.

[0004] Some traditional image processing algorithms such as the ViBe algorithm can screen false triggering images to some extent, but the effect is limited. The ViBe algorithm detects the foreground by modeling the background and considers the foreground, i.e. the moving target, as wild animals, thereby distinguishing false triggering images. However, in the wild animal monitoring scenario, due to the large time span, day and night alternation, weather changes, and vegetation growth, the image background itself has relatively complex changes, so the ViBe algorithm, which establishes a background model by comparing the pixel values of the same position and adjacent points of multiple images and has a relatively slow background update strategy, is difficult to adapt to the monitoring scenario and has poor effect.

[0005] In addition, some image classification or object detection algorithms based on deep learning and machine learning are also applied to the false trigger image screening problem. However, the current false trigger image screening method based on image classification usually adds an "no animal" category in the species category. However, the diversity of the infrared trigger camera layout site leads to a significant change in the background, which is more complex than the change in the wild animal species category. Therefore, the background images with different complexity are treated equally as a category and a species category, which makes it difficult for the model to generalize to various monitoring scenes. Although the false trigger screening method based on object detection has higher accuracy than the classification-based method, the label making cost is also higher. The reason is that the infrared trigger camera is easily disturbed by light changes, complex background information and other noise. The monitoring images have the characteristics of large intra-class difference and high inter-class similarity. If traditional deep learning methods are used for direct classification or detection of images, the model is difficult to learn effective features from a large number of false trigger images, and the accuracy cannot meet the practical standard. However, the metric learning method has made progress in recent years, which can effectively handle the discrimination problem in the scene with intra-class diversity and inter-class similarity. SUMMARY

[0006] In view of the technical defects in the prior art, the purpose of the embodiments of the present application is to provide a wild animal monitoring image false trigger image screening method and a storage medium, which realizes the automatic screening of false trigger images in wild animal monitoring images by using a computer, and can conveniently filter out invalid images according to the determination results of each image

[0007] To achieve the above purpose, in a first aspect, the embodiments of the present application provide a wild animal monitoring image false trigger image screening method, comprising:

[0008] obtaining a monitoring image shot by a camera, and constructing a training set;

[0009] training a network model using the training set to obtain an image screening model;

[0010] obtaining an empty shot image from all images shot by a camera to be screened as a reference image, and the remaining images as images to be screened;

[0011] inputting the reference image and the image to be screened into the image screening model to obtain a screening result.

[0012] Further, the training set includes a plurality of image triplets, each image triplet includes a benchmark sample, a positive sample and a negative sample, the benchmark sample of each image triplet is the same, and the positive sample and the negative sample of each image triplet are different; wherein,

[0013] randomly selecting one empty shot image from the camera as a benchmark sample;

[0014] Randomly select one false trigger image without animals from the monitoring image data set captured by the camera as a positive sample;

[0015] Randomly select one valid image containing wild animals from the monitoring image data set captured by the camera as a negative sample.

[0016] Further, the network model is trained using the training set to obtain an image screening model, comprising:

[0017] Each image triplet is sequentially input into the network model, and the model total loss is output.

[0018] The iteration is continuously performed, and if the model total loss meets the preset condition, the training of the network model is stopped, and the current network model is taken as the image screening model.

[0019] Further, the input of each image triplet into the network model and the output of the model total loss comprise:

[0020] The species classification loss is calculated according to the negative sample of the image triplet;

[0021] The reference sample, the positive sample and the negative sample of the image triplet are subjected to feature extraction to obtain a reference sample feature vector, a positive sample feature vector and a negative sample feature vector;

[0022] The first actual distance between the positive sample feature vector and the reference sample feature vector and the second actual distance between the negative sample feature vector and the reference sample feature vector are calculated based on the metric learning;

[0023] The triplet loss is calculated according to the first gap between the first actual distance and the first ideal distance and the second gap between the second actual distance and the second ideal distance; wherein the first ideal distance is the ideal distance between the positive sample feature vector and the reference sample feature vector, which is 0, and the second ideal distance is the ideal distance between the feature vector and the reference sample feature vector, which is 1;

[0024] The model total loss is calculated by weighting the triplet loss and the species classification loss.

[0025] Further, the model total loss meets the preset condition, comprising:

[0026] The model total loss tends to a constant value.

[0027] Further, the input of the reference image and the image to be screened into the image screening model to obtain a screening result comprises:

[0028] The reference image and the image to be screened are input into the image screening model to obtain a feature vector of the reference image and a feature vector of the image to be screened.

[0029] If the feature vector distance between the feature vector of the image to be screened and the feature vector of the reference image is greater than a preset threshold, the image to be screened is determined as a valid image, otherwise the image to be screened is determined as a false trigger image.

[0030] Further, the feature vector distance between the feature vector of the image to be screened and the feature vector of the reference image is Mahalanobis distance.

[0031] Further, the threshold value is 0.4.

[0032] Further, the network model includes one of a convolutional neural network model, a Transformer model or an MLP model.

[0033] In a second aspect, the present application also provides a computer readable storage medium, which stores a computer program, the computer program including program instructions, the program instructions causing a processor to execute the method of the first aspect when executed by the processor.

[0034] The present application takes the false trigger image discrimination technology based on triple loss metric learning as the main body, can discriminate the false trigger image of the infrared trigger camera and then filter it, and finally realizes the automation of filtering invalid data in the wild animal monitoring image raw data of the infrared trigger camera, avoids manual screening of massive monitoring data, and helps to promote the intelligentization and informatization of wild animal protection work. BRIEF DESCRIPTION OF DRAWINGS

[0035] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the drawings needed in the specific embodiments or prior art description will be briefly introduced below.

[0036] Figure 1 is a flowchart of a wild animal monitoring image false trigger image screening method provided by an embodiment of the present application. DETAILED DESCRIPTION

[0037] The technical solutions in the embodiments of the present application will be described below in a clear and complete manner with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0038] It should be understood that the terms "comprises" and "comprising" when used in this specification and the following claims, indicate the presence of the described features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0039] It should also be understood that the terms used in the specification of the application herein are used for the purpose of describing particular embodiments only and are not intended to limit the application. As used in the specification and the appended claims of the application, the singular forms "a," "an" and "the" are intended to include plural forms unless the context clearly dictates otherwise.

[0040] It should further be understood that the term "and / or" used in the specification of the application and the appended claims means one or more of the associated listed items as well as all possible combinations of the items.

[0041] As used in the specification and the appended claims of the application, the term "if' can be interpreted as meaning "when" or "once" or "in response to a determination" or "in response to detecting" depending on the context. Similarly, the phrases "if determined" or "if detected [the described condition or event]" can be interpreted to mean "once determined" or "in response to a determination" or "once detected [the described condition or event]" or "in response to detecting [the described condition or event]" depending on the context.

[0042] It should be noted that unless otherwise stated, technical terms or scientific terms used in this application should be understood as their common meanings to those skilled in the art to which the application belongs.

[0043] As Figure 1 shown, the embodiment provides a flowchart of a wild animal monitoring image false triggering image screening method. The method can include the following steps:

[0044] S100: Obtain a monitoring image captured by a camera, and construct a training set.

[0045] Specifically, the training set includes a plurality of image triplets, each image triplet includes one reference sample, one positive sample and one negative sample, the reference sample of each image triplet is the same, and the positive sample and the negative sample of each image triplet are different. The data set is used as the initial source of data in the triplets.

[0046] wherein one aerial image randomly selected from the aerial images captured by the camera is taken as a reference sample; one false trigger image without animals randomly selected from the monitoring image dataset captured by the camera is taken as a positive sample; and one effective image containing wild animals randomly selected from the monitoring image dataset captured by the camera is taken as a negative sample.

[0047] It can be understood that, in the construction of the training set, the image photos captured by the ordinary camera can also be used, or the image photos captured by the infrared trigger camera can also be used. The effective image as the negative sample includes sample type and species category number, and the false trigger image as the positive sample includes sample type and camera number.

[0048] S200: training the network model using the training set to obtain an image screening model.

[0049] In the embodiment, the network model can be one of a convolutional neural network model, a Transformer model or an MLP model. That is, the method is not limited to a specific deep learning network or architecture, and a model with better extraction capability can theoretically obtain better results through the method.

[0050] Specifically, training the network model using the training set can include the following steps:

[0051] S201: sequentially inputting each image triple into the network model to output a total model loss.

[0052] Each image triple is sequentially inputted into the preset network model to train the network model, and the network model can calculate the total model loss after each training. Specifically, the method for calculating the total model loss includes:

[0053] S2011: calculating a species classification loss according to the negative sample of the image triple.

[0054] Specifically, the species classification loss is calculated according to the following formula:

[0055]

[0056] wherein i is the training sample number, N is the training set size, C is the number of species categories, p ij represents the probability that the sample i is predicted to be classified into category j; y ij represents whether the sample i belongs to category j, j=1 when the classification is correct, represents that the sample i belongs to category j, j=0 when the classification is incorrect, and represents that the sample i does not belong to category j. The role of the species classification loss is to guide the network model to focus on animal features.

[0057] S2012: Extract features from the reference sample, positive sample, and negative sample of the image triplet to obtain the reference sample feature vector, positive sample feature vector, and negative sample feature vector.

[0058] S2013: Calculate the first actual distance between the feature vector of a positive sample and the feature vector of a reference sample, and the second actual distance between the feature vector of a negative sample and the feature vector of a reference sample, based on metric learning.

[0059] S2014: The triplet loss is calculated based on the first gap between the first actual distance and the first ideal distance, and the second gap between the second actual distance and the second ideal distance; wherein, the first ideal distance is the ideal distance between the positive sample feature vector and the benchmark sample feature vector, which is 0, and the second ideal distance is the ideal distance between the feature vector and the benchmark sample feature vector, which is 1.

[0060] Specifically, triplet loss The calculation formula is as follows:

[0061]

[0062] in, As a baseline sample, For positive samples, f(x) represents the mapping function for calculating the image feature vector. These are negative samples. `m` is a specific threshold to ensure a minimum margin; it is a constant greater than 0, and in the experiment, it is set to 0.0001. Triplet loss is used. By pushing negative samples further away from the baseline and positive samples, and bringing positive samples closer to the baseline, the network model's ability to correctly classify images is enhanced.

[0063] S2015: The total model loss is obtained by weighting the triplet loss and the species classification loss.

[0064] Specifically, the total model loss The calculation formula is as follows:

[0065]

[0066] Where α is an adjustable weighting factor, which is set to 0.5 in the experiment.

[0067] S202: Continue iterating. If the total loss of the model meets the preset conditions, stop training the network model and use the current network model as the image selection model.

[0068] Specifically, the total model loss meets a preset condition, that is, the total model loss tends to a constant value. After several iterations, the total model loss gradually stabilizes and tends to a constant value, at which point it can be considered that the network model has been trained to convergence.

[0069] S300: Obtain an empty photograph image from all images photographed by the camera to be screened as a reference image, and the remaining images as images to be screened.

[0070] The images photographed by the camera to be screened include both empty photograph images, i.e., false trigger images, and valid images containing animals. A person needs to randomly select an empty photograph image from all images photographed by the camera to be screened as a reference image, and the remaining images as images to be screened.

[0071] S400: Input the reference image and the image to be screened into the image screening model to obtain a screening result.

[0072] Specifically, the method comprises the following steps:

[0073] S401: Input the reference image and the image to be screened into the image screening model to obtain a feature vector of the reference image and a feature vector of the image to be screened.

[0074] S402: If a feature vector distance between the feature vector of the image to be screened and the feature vector of the reference image is greater than a preset threshold, it is determined that the image to be screened is a valid image, otherwise, it is determined that the image to be screened is a false trigger image.

[0075] In the embodiment, the feature vector distance between the feature vector of the image to be screened and the feature vector of the reference image is Mahalanobis distance. According to experimental experience, the threshold is 0.4.

[0076] The method for screening false trigger images of wild animal monitoring images of infrared trigger cameras based on triplet loss metric learning can accurately and effectively filter false trigger images in wild animal monitoring images collected by infrared trigger cameras, can make wild animal researchers and protectors spend more time and effort on research and protection rather than data processing, can save a large amount of manpower and material resources, and can accelerate the intelligentization and informatization of wild animal protection and biodiversity protection. On the other hand, the application is inspired by the characteristics of the target problem, and combines metric learning and deep learning technology organically, thereby providing a new idea for the wide application of artificial intelligence.

[0077] Further, based on the same inventive concept, the embodiments of the application also provide a readable storage medium storing a computer program, the computer program comprising program instructions, and the program instructions are executed by a processor to implement the above-mentioned method for screening false trigger images of wild animal monitoring images.

[0078] The computer readable storage medium can be an internal storage unit of the background server in the foregoing embodiments, such as a hard disk or a memory of the system. The computer readable storage medium can also be an external storage device of the system, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, and the like. Further, the computer readable storage medium can include both the internal storage unit and the external storage device of the system. The computer readable storage medium is used to store the computer program and other programs and data required by the system. The computer readable storage medium can also be used to temporarily store data that has been output or will be output.

[0079] Those skilled in the art can appreciate that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in the foregoing description in general terms. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0080] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0081] When the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art, or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0082] The above merely illustrates the specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any skilled person in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, and these modifications or replacements shall be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A wildlife monitoring image false trigger image screening method, characterized by, The method comprises the following steps: acquiring monitoring images captured by a camera to construct a training set; training a network model using the training set to obtain an image screening model; acquiring one aerial image from all images captured by the camera to be screened as a reference image, and the remaining images as images to be screened; inputting the reference image and the image to be screened into the image screening model to obtain a screening result; the training set comprises a plurality of image triplets, each image triplet comprises one benchmark sample, one positive sample and one negative sample, the benchmark samples of each image triplet are the same, and the positive sample and the negative sample of each image triplet are different; wherein, randomly selecting one aerial image captured by the camera as a benchmark sample; randomly selecting one false trigger image without animals from the monitoring image data set captured by the camera as a positive sample; randomly selecting one effective image containing wild animals from the monitoring image data set captured by the camera as a negative sample; training the network model using the training set to obtain an image screening model, comprising: inputting each image triplet into the network model in turn to output the total loss of the model; continuously iterating, if the total loss of the model meets the preset condition, stopping training the network model, and taking the current network model as the image screening model; the total loss of the model is obtained by inputting each image triplet into the network model in turn, comprising: calculating the species classification loss based on the negative sample of the image triplet; extracting features of the benchmark sample, the positive sample and the negative sample of the image triplet to obtain a benchmark sample feature vector, a positive sample feature vector and a negative sample feature vector; calculating the first actual distance between the positive sample feature vector and the benchmark sample feature vector and the second actual distance between the negative sample feature vector and the benchmark sample feature vector based on metric learning; calculating the triplet loss based on the first gap between the first actual distance and the first ideal distance and the second gap between the second actual distance and the second ideal distance; wherein, the first ideal distance is the ideal distance between the positive sample feature vector and the benchmark sample feature vector, which is 0, and the second ideal distance is the ideal distance between the feature vector and the benchmark sample feature vector, which is 1; weighting the triplet loss and the species classification loss to obtain the total loss of the model.

2. A method of false trigger image screening for wildlife monitoring images as claimed in claim 1 wherein, the total loss of the model meets the preset condition, comprising: the total loss of the model tends to a constant value.

3. A method of false trigger image screening for wildlife monitoring images as claimed in claim 1 wherein, the reference image and the image to be screened are input into the image screening model to obtain the screening result, comprising: inputting the reference image and the image to be screened into the image screening model to obtain the feature vector of the reference image and the feature vector of the image to be screened; if the feature vector distance between the feature vector of the image to be screened and the feature vector of the reference image is greater than a preset threshold, it is determined that the image to be screened is an effective image, otherwise it is determined that the image to be screened is a false trigger image.

4. A method of false trigger image screening for wildlife monitoring images as claimed in claim 3 wherein, the feature vector distance between the feature vector of the image to be screened and the feature vector of the reference image is Mahalanobis distance.

5. A method of false trigger image screening for wildlife monitoring images as claimed in claim 3 wherein, the threshold value is 0.

4.

6. A method of false trigger image screening for wildlife monitoring images as claimed in claim 1 wherein, The network model includes one of the following: convolutional neural network model, Transformer model, or MLP model.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method as described in any one of claims 1-6.

Images