Wild animal monitoring system and method

Through the wildlife monitoring system, combining the multi-target multi-camera target tracking model and event monitoring model, the movement trajectory of wild animals under different seasons and meteorological conditions is analyzed, and pedestrians and animals are judged in real time, solving the difficulty in identifying and The timely response to problems has improved the response speed and effectiveness of protective measures.

CN119027873APending Publication Date: 2024-11-26SHENZHEN QICHANG TECH CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411109144.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-13
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

Existing wildlife monitoring technology is difficult to identify and analyze the movement trajectories of wildlife in combination with seasonal and meteorological conditions, and it is impossible to detect contact between wildlife and pedestrians in a timely manner, resulting in lagging protection measures.

Method used

Design a wildlife monitoring system, including a remote main control platform, a biological monitoring module and a meteorological monitoring module, and use a multi-objective multi-camera target tracking model and event monitoring model, and feature extraction is performed through Transformer algorithm, Cascade-RCNN algorithm and DeepSORT algorithm. Identify and trajectory prediction, analyze wildlife movement trajectories based on meteorological data, and judge pedestrians and animals in real time to generate early warning information.

Benefits of technology

It has realized the recording and analysis of wildlife movement trajectories under different seasons and meteorological conditions, timely discover pedestrians and animals contact, and improved the response speed and effectiveness of protection measures.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027873A_ABST
    Figure CN119027873A_ABST
Patent Text Reader

Abstract

The invention belongs to the technical field of wild animal monitoring, and discloses a wild animal monitoring system and method, and the system comprises a remote main control platform; the biological monitoring module is in communication connection with the remote main control platform and is used for collecting motion images of wild animals and uploading motion image data to the remote main control platform; the meteorological monitoring module is in communication connection with the remote main control platform and is used for acquiring meteorological data in the wild animal protection area and uploading the meteorological data to the remote main control platform; the remote main control platform is used for identifying and analyzing the motion image data based on a pre-constructed multi-target multi-camera target tracking model to obtain a tracking result, and the tracking result is used for representing the motion trail of the wild animal under the current meteorological condition. According to the invention, the motion trails of various organisms in different seasons and different weather conditions can be recorded, so that the activity characteristics of various organisms can be known, and protection measures can be provided for specific organisms in different seasons.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of wild animal monitoring, and in particular relates to a wild animal monitoring system and method. Background Art

[0002] Wildlife is an important part of nature and plays a key role in the ecosystem. Monitoring wildlife helps us understand their population size, distribution and health status, so as to develop effective conservation measures. In addition, monitoring can prevent the spread of diseases and protect the well-being of humans and wildlife.

[0003] Wildlife monitoring uses a variety of methods, including field surveys, infrared camera monitoring, drone monitoring, acoustic monitoring, etc. These methods can help researchers understand the population size, habitat status and behavioral habits of wild animals.

[0004] In different seasons and under different meteorological conditions, the activities of wild animals show different movement trajectories. The above method is difficult to identify and analyze the movement trajectories of wild animals in combination with factors such as seasons and meteorological conditions. It is impossible to obtain the activity characteristics of wild animals in different seasons and meteorological conditions, and it is difficult to provide protection measures for specific wild animals.

[0005] Secondly, the above monitoring methods are unable to detect contact between wild animals and pedestrians in a timely manner, and protection measures cannot be implemented in the first place, which may cause harm to pedestrians or wild animals. Summary of the invention

[0006] The purpose of the present invention is to provide a wildlife monitoring system and method to solve the problem that existing animal monitoring is difficult to identify and analyze the movement trajectory of wild animals in combination with factors such as season and meteorological conditions, and is unable to promptly detect contact between wild animals and pedestrians.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions: In a first aspect, the present invention provides a wildlife monitoring system, the system comprising: Remote control platform; The biological monitoring module is connected to the remote control platform for collecting moving images of wild animals and uploading the moving image data to the remote control platform; The meteorological monitoring module is connected to the remote main control platform for collecting meteorological data in the wildlife protection area and uploading the meteorological data to the remote main control platform; The remote control platform is used to: identify and analyze motion image data based on a pre-built multi-target multi-camera target tracking model to obtain tracking results, which are used to characterize the movement trajectory of wild animals under current meteorological conditions; And determine whether there are pedestrians in the moving image. When there are pedestrians in the moving image, the moving image data is identified based on a pre-built event monitoring model to obtain an event recognition result, which is used to characterize whether the wild animal comes into contact with the pedestrian.

[0008] Preferably, the system further comprises: an administrator terminal, the administrator terminal is communicatively connected with the remote main control platform, and the remote main control platform is further used for: generating warning information when wild animals come into contact with pedestrians; and sending the warning information to the administrator terminal.

[0009] Preferably, the event monitoring model is a MonoGRNet model.

[0010] Preferably, the multi-target multi-camera target tracking model includes: The single-camera tracking model is used to extract features from motion image data based on the Transformer algorithm to obtain multi-dimensional animal features, identify multi-dimensional animal features based on the Cascade-RCNN algorithm to obtain single-camera target recognition results, and predict the trajectory of single-camera target recognition results based on the DeepSORT algorithm to obtain single-camera animal motion trajectories; The cross-camera object Re-ID model is used to re-identify multi-dimensional animal features based on the Re-ID algorithm to obtain cross-camera object recognition results; A camera connection model is used to associate single-camera animal motion trajectories based on movement direction, bidirectional transition time window and appearance similarity to obtain the associated animal motion trajectory, and use the associated animal motion trajectory as the tracking result, wherein the appearance similarity is determined based on the cross-camera target recognition result.

[0011] In a second aspect, the present invention further provides a method for monitoring wild animals, the method comprising: Obtaining the motion images of wild animals collected by the biological monitoring module and the meteorological data collected by the meteorological monitoring module; Recognize and analyze the motion image data based on a pre-built multi-target multi-camera target tracking model to obtain a tracking result, which is used to characterize the movement trajectory of wild animals under current meteorological conditions; Determine whether there are pedestrians in the moving image; When there are pedestrians in the motion image, the motion image data is identified based on a pre-built event monitoring model to obtain an event recognition result, which is used to characterize whether the wild animal comes into contact with the pedestrian.

[0012] Preferably, the multi-target multi-camera target tracking model includes: a single camera tracking model, a cross-camera target Re-ID model and a camera connection model; Based on the pre-built multi-target multi-camera target tracking model, the motion image data is identified and analyzed to obtain tracking results, including: The single-camera tracking model extracts features from motion image data based on the Transformer algorithm to obtain multi-dimensional animal features, recognizes the multi-dimensional animal features based on the Cascade-RCNN algorithm to obtain single-camera target recognition results, and predicts the trajectory of the single-camera target recognition results based on the DeepSORT algorithm to obtain the single-camera animal motion trajectory; The cross-camera target Re-ID model re-identifies multi-dimensional animal features based on the Re-ID algorithm to obtain a cross-camera target recognition result; The camera connection model associates the single-camera animal motion trajectories based on the moving direction, the bidirectional transition time window and the appearance similarity to obtain the associated animal motion trajectory, and uses the associated animal motion trajectory as the tracking result; wherein the appearance similarity is determined based on the cross-camera target recognition result.

[0013] Preferably, the method further comprises: When the event identification result is a contact between wild animals and pedestrians, an early warning message is generated; Send the warning information to the management terminal.

[0014] Preferably, the method further comprises: in the process of constructing the cross-camera target Re-ID model, optimizing the global features of the cross-camera target Re-ID model based on ternary loss and cross entropy loss.

[0015] Preferably, the method further comprises: obtaining the moving direction, comprising: The correct moving direction of each target in the single-camera target recognition result is obtained by manual marking, and the action area of ​​each target is obtained; the action area includes: an entry area and an exit area; wherein the entry area is the first area of ​​the action area passed by the target, and the exit area is the last area of ​​the action area passed by the target; Based on the entry and exit regions of each target, the movement direction of each target is determined.

[0016] Preferably, the method further comprises: determining the appearance similarity, comprising: Based on the cross-camera target recognition results, determine the single-camera animal motion trajectory of each target on each single camera; Perform average pooling processing on the single-camera animal motion trajectories on each single camera to obtain several trajectory features; Calculate the distance between every two trajectory features; Based on the distance, the time mask and the direction mask between each two trajectory features, a similarity matrix is ​​determined, and the similarity matrix is ​​used to characterize the appearance similarity.

[0017] Beneficial effects: The present invention realizes biological monitoring, meteorological monitoring, and event detection. It can record the movement trajectories of various organisms in different seasons and different weather conditions to understand the activity characteristics of various organisms and provide protection measures for specific organisms in different seasons; and it can monitor in real time whether there are pedestrians in contact with animals. When pedestrians actively contact animals or animals actively contact pedestrians, an alarm will be sounded and the address of the current camera will be sent to forest rangers or relevant managers to facilitate managers to implement protection measures as soon as possible. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present invention, but do not constitute a limitation on the embodiments of the present invention. In the accompanying drawings: Figure 1 is a block diagram of a wildlife monitoring system provided by one embodiment of the present invention; Figure 2 It is a flow chart of a method for monitoring wild animals provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be briefly introduced below in combination with the drawings and the description of the embodiments or the prior art. Obviously, the following description of the structure of the drawings is only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention.

[0020] Embodiment 1 Figure 1 is a block diagram of a wildlife monitoring system provided by one embodiment of the present invention, such as Figure 1 As shown, this embodiment provides a wildlife monitoring system, the system comprising: a remote main control platform, a biological monitoring module and a meteorological monitoring module; The biological monitoring module is connected to the remote main control platform for collecting motion images of wild animals and uploading the motion image data to the remote main control platform; The meteorological monitoring module is connected to the remote main control platform for collecting meteorological data in the wildlife protection area and uploading the meteorological data to the remote main control platform; The remote control platform is used to: identify and analyze motion image data based on a pre-built multi-target multi-camera target tracking model to obtain tracking results, which are used to characterize the movement trajectory of wild animals under current meteorological conditions; And determine whether there are pedestrians in the moving image. When there are pedestrians in the moving image, the moving image data is identified based on a pre-built event monitoring model to obtain an event recognition result, which is used to characterize whether the wild animal comes into contact with the pedestrian.

[0021] In this embodiment, the biological monitoring module can be a number of cameras arranged in a wildlife protection area, and the cameras are arranged in different positions to monitor the activities of wild animals; the meteorological data includes: wind direction, temperature, wind force, humidity, air quality and other data; the meteorological data of this embodiment can be obtained from the weather network API interface, or at various meteorological monitoring stations in the wildlife protection area, the meteorological data can be obtained through various meteorological monitoring stations.

[0022] In this embodiment, the present invention realizes biological monitoring, meteorological monitoring, and event detection. It can record the movement trajectories of various organisms in different seasons and different weather conditions to understand the activity characteristics of various organisms and provide protection measures for specific organisms in different seasons; and monitor whether there are pedestrians in contact with organisms, including pedestrians actively contacting organisms and organisms actively contacting pedestrians, and then issue an alarm and send the address of the current camera to the forest ranger or relevant management personnel to facilitate the management personnel to implement protection measures as soon as possible.

[0023] As a further optimization of this embodiment, the system also includes: an administrator terminal, which is communicatively connected to the remote main control platform, and the remote main control platform is also used to: generate warning information when wild animals come into contact with pedestrians; and send the warning information to the administrator terminal.

[0024] As a further optimization of this embodiment, the multi-target multi-camera target tracking model includes: a single camera tracking model, a cross-camera target Re-ID model and a camera connection model; Based on the Transformer algorithm (Transformer is a neural network based on the attention mechanism), the feature extraction of motion image data is performed to obtain multi-dimensional animal features. Based on the Cascade-RCNN algorithm, the multi-dimensional animal features are identified to obtain the single-camera target recognition results. Based on the DeepSORT algorithm, the trajectory of the single-camera target recognition results is predicted to obtain the single-camera animal motion trajectory. The cross-camera object Re-ID model is used to re-identify multi-dimensional animal features based on the Re-ID algorithm to obtain cross-camera object recognition results; A camera connection model is used to associate single-camera animal motion trajectories based on movement direction, bidirectional transition time window and appearance similarity to obtain the associated animal motion trajectory, and use the associated animal motion trajectory as the tracking result, wherein the appearance similarity is determined based on the cross-camera target recognition result.

[0025] In this embodiment, the Re-ID algorithm is a basic task in multi-target multi-camera target tracking. This embodiment utilizes the effectiveness of the Transformer algorithm in Re-ID. Through the Transformer algorithm, this embodiment can extract more discriminative multi-dimensional animal features for single-camera tracking and cross-camera Re-ID. In order to consider the large visual appearance changes, lighting conditions and target directions between different cameras, in the process of building the cross-camera target Re-ID model, ternary loss and cross entropy loss are used simultaneously to optimize the global features of the cross-camera target Re-ID model. The optimized global features can help reduce potential associated candidates, thereby improving the accuracy of Re-ID.

[0026] Specifically, the triplet loss optimizes the first triplet set in the feature space by making the distance of the positive sample pair smaller than the distance of the negative sample pair a , the second triple set p , the third triplet set n The distance of the optimized feature space for: ; in, N is the sum of the training samples, i For the i training samples, are the features of anchor points, positive samples, and negative samples extracted from the cross-camera target Re-ID model, respectively. m is the distance of the negative sample pair, d () represents the distance between two features. The distance formula can be cosine distance or Euclidean distance.

[0027] Cross Entropy Loss Function For model optimization, the formula is as follows: ; in, y is the true value, is the predicted value, For the flag bit.

[0028] In this embodiment, target detection is a key factor affecting tracking performance. In order to produce high-quality detection results, this embodiment uses the Cascade-RCNN algorithm (two-stage target detection network algorithm) to identify multi-dimensional animal features and obtain single-camera target recognition results. First, the single-camera tracking model needs to be trained before identifying multi-dimensional animal features; the single-camera tracking model of this embodiment can be pre-trained on the COCO dataset, and then fine-tuned on the training and verification data collected on the actual project to be deployed; wherein the COCO dataset is a large-scale dataset that can be used for image detection (image detection), semantic segmentation (semantic segmentation) and image captioning (image captioning); it has more than 330K images (of which 220K are annotated images), including 1.5 million targets, 80 target categories (object categories: pedestrians, cars, elephants, etc.), 91 material categories (stuff categories: grass, wall, sky, etc.), each image contains five sentences of image description, and there are 250,000 pedestrians with key point annotations.

[0029] In this embodiment, when identifying the motion trajectory of the target, the framework of DeepSORT (multi-target tracking algorithm based on deep learning) is adopted. DeepSORT uses a Kalman filter to predict the motion trajectory of the target and associates it with appearance and motion similarity.

[0030] In order to prevent false positive trajectory prediction results, this embodiment uses the following processing method to optimize the trajectory prediction results of the DeepSORT framework, such as filtering out all trajectories with shorter lengths and not associating them in the camera connection model.

[0031] In addition to filtering short trajectories, this embodiment uses a confidence score cascade association method to improve the accuracy of association. In this embodiment, the trajectory prediction results obtained with high confidence are first associated, and then those unassociated trajectory prediction results are associated with the trajectory prediction results with low confidence in the second round. Finally, those unassociated trajectory prediction results will be initialized as new trajectory segments.

[0032] The above processing method can improve the performance and reliability of the DeepSORT algorithm for a single camera.

[0033] After obtaining single-camera tracking results for each camera, the camera linking model can associate the tracks based on several cues, including movement direction, transition time window, and appearance similarity.

[0034] Movement direction: It is very important for the camera link model and can be used to filter out impossible association candidates. In order to obtain the movement direction of each target, this embodiment uses the entry / exit action area to classify the movement direction of each target. The action area is manually marked to obtain the correct movement direction of each target. The first area and the last area passed by the target are regarded as the entry and exit areas of the target, respectively. According to the information of the entry / exit area, this embodiment can classify the movement direction of the target.

[0035] Bidirectional transition time window: The transition time window of each camera pair can be determined based on the distance between the two cameras and several other factors (e.g., the orientation of the two cameras, the distance between the captured areas). In order to ensure that the time window associated with the camera link model is perfect, this embodiment is carefully adjusted according to the above factors; it is worth noting that the optimal time window for a camera pair may be different depending on the moving direction of the target. This means that the optimal time window for cam.A (camera A) → cam.B (camera B) and cam.B (camera B) → cam.A (camera A) may be different. This may be due to the difference in the rotation angle of the cameras in the two moving directions. Therefore, this embodiment adopts a bidirectional transition time window, that is, different transition time windows are considered so that the camera link model can select the best time window for cross-camera association.

[0036] Appearance similarity: The appearance similarity is determined based on the cross-camera target recognition results; specifically, based on the cross-camera target recognition results, the single-camera animal motion trajectory of each target on each single camera is determined; the single-camera animal motion trajectory on each single camera is averaged and pooled to obtain a number of trajectory features; the distance between every two trajectory features is calculated; based on the distance between every two trajectory features, the time mask and the direction mask, a similarity matrix is ​​determined, and the similarity matrix is ​​used to characterize the appearance similarity.

[0037] This embodiment considers the time mask and the direction mask to associate the camera pairs. By introducing the time and direction masks, the final similarity of each track is The expression is: ; where d(·) is the distance between two track-based features, which is obtained by averaging all features of each track from the single-camera tracking result. For the i The characteristics of the samples, For the j The characteristics of the samples, is the temporal mask, which represents the temporal window filtering mechanism in the camera link model, through t 1and t 2 To determine the lower and upper limits of the time window, Indicates the time when the animal appears; The definition is as follows: ; is a direction mask that can be inferred based on the target’s moving direction. It is defined as follows: ; After obtaining the similarity matrix, the camera link model performs the Hungarian algorithm for each pair of cameras, associates the trajectories across cameras, and filters out associations with low similarity. Finally, after association, a unique global ID is assigned to each trajectory to characterize the movement trajectory of each animal.

[0038] As a further optimization of this embodiment, the event monitoring model is a MonoGRNet model, which is a non-modal 3D target positioning unified network based on monocular images. The core idea of ​​the network is to decompose the three-dimensional positioning problem into several progressive subtasks that can only be solved with monocular RGB data. The network starts from perceiving the semantics of the two-dimensional image plane and then performs geometric reasoning in three-dimensional space.

[0039] In this embodiment, event monitoring is achieved through 3D target detection. The purpose of using 3D target detection is to obtain the spatial position of the organism in order to more accurately determine the relative position relationship between pedestrians and organisms. In order to save performance, event detection will only be triggered when a pedestrian is detected.

[0040] The MonoGRNet algorithm proposes to accurately estimate the 3D center depth of instances without computing pixel-level depth maps. The algorithm adopts an instance-level depth estimation (IDE) module that explores the large receptive field of the deep feature map to obtain the coarse instance depth, and then combines higher-resolution early features to refine the IDE.

[0041] The MonoGRNet algorithm proposes a network that can estimate the 3D target bounding box using a monocular RGB image. The algorithm uses the RGB image within the 3D target box to estimate the 3D depth and regress the estimated 3D bounding box to the center point.

[0042] Given a monocular RGB image, the goal is to localize objects of a specific class in 3D space. The target object is represented by a class label and an ABBox-3D (a three-dimensional bounding box) that bounds the entire object without considering occlusion or truncation.

[0043] 2D boundary detection: The two-dimensional detection module is the basic module for stable feature learning, and also provides the region of interest for the subsequent geometric reasoning module. It detects the center b and the size (w, h) of the projected 2D box B2d surrounding the ABBox-3D. 3D boundary detection: Locate the 3D center C by predicting its depth Zc and 2D projection C. Finally, based on local features, regress the local corner point O relative to the 3D center.

[0044] Instance depth detection: The IDE subnet estimates the depth of the ABBox-3D center Zc. Given the grid G ​​divided from the backbone in the feature map, each grid cell G predicts the 3D center depth of the nearest instance within the distance threshold σ. Considering the depth information, the cell is assigned a closer instance.

[0045] 3D bounding box location detection: This subnet estimates the location of the 3D center C = (Xc, Yc, Zc) of the object of interest in each grid G.

[0046] Embodiment 2 Figure 2 is a flow chart of a method for monitoring wild animals provided by one embodiment of the present invention, such as Figure 2 As shown, this embodiment provides a method for monitoring wild animals, the method comprising: Step S10: acquiring the motion images of wild animals collected by the biological monitoring module and the meteorological data collected by the meteorological monitoring module.

[0047] In this embodiment, the biological monitoring module can be a number of cameras arranged in a wildlife protection area, and the cameras are arranged in different positions to monitor the activities of wild animals; the meteorological data includes: wind direction, temperature, wind force, humidity, air quality and other data; the meteorological data of this embodiment can be obtained from the weather network API interface, or at various meteorological monitoring stations in the wildlife protection area, the meteorological data can be obtained through various meteorological monitoring stations.

[0048] Step S20: Based on a pre-built multi-target multi-camera target tracking model, the motion image data is identified and analyzed to obtain a tracking result, which is used to characterize the movement trajectory of the wild animal under the current meteorological conditions.

[0049] In this embodiment, the multi-target multi-camera target tracking model includes: a single camera tracking model, a cross-camera target Re-ID model and a camera connection model; Specifically, based on the pre-built multi-target multi-camera target tracking model, the motion image data is identified and analyzed to obtain tracking results, including: Step a01: The single-camera tracking model extracts features from the motion image data based on the Transformer algorithm to obtain multi-dimensional animal features, recognizes the multi-dimensional animal features based on the Cascade-RCNN algorithm to obtain single-camera target recognition results, and predicts the trajectory of the single-camera target recognition results based on the DeepSORT algorithm to obtain the single-camera animal motion trajectory; Step a02: the cross-camera target Re-ID model re-identifies multi-dimensional animal features based on the Re-ID algorithm to obtain a cross-camera target recognition result; Step a03: The camera connection model associates the single-camera animal motion trajectories based on the moving direction, the two-way transition time window and the appearance similarity to obtain the associated animal motion trajectory, and uses the associated animal motion trajectory as the tracking result; wherein the appearance similarity is determined based on the cross-camera target recognition result.

[0050] In this embodiment, the Re-ID algorithm is a basic task in multi-target multi-camera target tracking. This embodiment utilizes the effectiveness of the Transformer algorithm in Re-ID. Through the Transformer algorithm, this embodiment can extract more discriminative multi-dimensional animal features for single-camera tracking and cross-camera Re-ID. In order to consider the large visual appearance changes, lighting conditions and target directions between different cameras, in the process of building the cross-camera target Re-ID model, ternary loss and cross entropy loss are used simultaneously to optimize the global features of the cross-camera target Re-ID model. The optimized global features can help reduce potential associated candidates, thereby improving the accuracy of Re-ID.

[0051] Specifically, the triplet loss is achieved by making the distance between positive sample pairs smaller than that between negative sample pairs. m To optimize the first triple set in the feature space a , the second triple set p , the third triplet set n The distance of the optimized feature space for: ; in, N is the sum of the training samples, i For the i training samples, are the features of anchor points, positive samples, and negative samples extracted from the cross-camera target Re-ID model, respectively. d () represents the distance between two features. The distance formula can be cosine distance or Euclidean distance.

[0052] Cross Entropy Loss Function For model optimization, the formula is as follows: ; in, y is the true value, is the predicted value, For the flag bit.

[0053] In this embodiment, target detection is a key factor affecting tracking performance. In order to produce high-quality detection results, this embodiment uses the Cascade-RCNN algorithm (two-stage target detection network algorithm) to identify multi-dimensional animal features and obtain single-camera target recognition results. First, the single-camera tracking model needs to be trained before identifying multi-dimensional animal features; the single-camera tracking model of this embodiment can be pre-trained on the COCO dataset, and then fine-tuned on the training and verification data collected on the actual project to be deployed; wherein the COCO dataset is a large-scale dataset that can be used for image detection (image detection), semantic segmentation (semantic segmentation) and image captioning (image captioning); it has more than 330K images (of which 220K are annotated images), including 1.5 million targets, 80 target categories (object categories: pedestrians, cars, elephants, etc.), 91 material categories (stuff categories: grass, wall, sky, etc.), each image contains five sentences of image description, and there are 250,000 pedestrians with key point annotations.

[0054] As a further optimization of this embodiment, the method further includes: obtaining the moving direction, including: Step b01: manually mark the correct moving direction of each target in the single-camera target recognition result to obtain the action area of ​​each target; the action area includes an entry area and an exit area; the entry area is the first area of ​​the action area passed by the target, and the exit area is the last area of ​​the action area passed by the target; Step b02: Determine the moving direction of each target based on the entry area and exit area of ​​each target.

[0055] In this embodiment, the moving direction of the target is very important for the camera link model and can be used to filter out impossible association candidates. In order to obtain the moving direction of each target, this embodiment uses the entry / exit action area to classify the moving direction of each target. The action area is manually marked to obtain the correct moving direction of each target. The first area and the last area that the target passes through are respectively regarded as the entry and exit areas of the target. According to the information of the entry / exit area, this embodiment can classify the moving direction of the target.

[0056] In this embodiment, the transition time window of each camera pair can be determined based on the distance between the two cameras and several other factors (such as the orientation of the two cameras, the distance between the captured areas). In order to ensure that the time window associated with the camera link model is perfect, this embodiment is carefully adjusted according to the above factors; it is worth noting that the optimal time window for a camera pair may be different depending on the moving direction of the target. This means that the optimal time window for cam.A (camera A) → cam.B (camera A) and cam.B (camera B) → cam.A (camera A) may be different. This may be due to the difference in the rotation angle of the cameras in the two moving directions. Therefore, this embodiment adopts a bidirectional transition time window, that is, different transition time windows are considered so that the camera link model can select the best time window for cross-camera association.

[0057] As an improved optimization of this embodiment, the method further includes: determining the appearance similarity, including: Step c01: based on the cross-camera target recognition results, determine the single-camera animal motion trajectory of each target on each single camera; Step c02: average pooling processing is performed on the single-camera animal motion trajectories on each single camera to obtain a number of trajectory features; Step c03: Calculate the distance between every two trajectory features; Step c04: Based on the distance, time mask and direction mask between every two trajectory features, a similarity matrix is ​​determined, where the similarity matrix is ​​used to characterize the appearance similarity.

[0058] In this embodiment, the time mask and the direction mask are considered to associate the camera pairs. By introducing the time and direction masks, the final similarity of each track is The expression is: ; where d(·) is the distance between two track-based features, which is obtained by averaging all features of each track from the single-camera tracking result. For the i The characteristics of the samples, For the j The characteristics of the samples, is the temporal mask, which represents the temporal window filtering mechanism in the camera link model, through t 1 and t 2 To determine the lower and upper limits of the time window, Indicates the time when the animal appears; The definition is as follows: ; is a direction mask that can be inferred based on the target’s moving direction. It is defined as follows: ; After obtaining the similarity matrix, the camera link model performs the Hungarian algorithm for each pair of cameras, associates the trajectories across cameras, and filters out associations with low similarity. Finally, after association, a unique global ID is assigned to each trajectory to characterize the movement trajectory of each animal.

[0059] Step S30: determining whether there is a pedestrian in the moving image; Step S40: When there are pedestrians in the motion image, the motion image data is identified based on the pre-built event monitoring model to obtain an event recognition result, which is used to characterize whether the wild animal is in contact with the pedestrian. In this embodiment, the event monitoring model is the MonoGRNet model, which is a non-modal 3D target positioning unified network based on monocular images. The core idea of ​​the network is to decompose the three-dimensional positioning problem into several progressive subtasks, which can only be solved with monocular RGB data. The network starts with perceiving the semantics of the two-dimensional image plane and then performs geometric reasoning in three-dimensional space.

[0060] In this embodiment, event monitoring is achieved through 3D target detection. The purpose of using 3D target detection is to obtain the spatial position of the organism in order to more accurately determine the relative position relationship between pedestrians and organisms. In order to save performance, event detection will only be triggered when a pedestrian is detected.

[0061] The MonoGRNet algorithm proposes to accurately estimate the 3D center depth of instances without computing pixel-level depth maps. The algorithm adopts an instance-level depth estimation (IDE) module that explores the large receptive field of the deep feature map to obtain the coarse instance depth, and then combines higher-resolution early features to refine the IDE.

[0062] The MonoGRNet algorithm proposes a network that can estimate the 3D target bounding box using a monocular RGB image. The algorithm uses the RGB image within the 3D target box to estimate the 3D depth and regress the estimated 3D bounding box to the center point.

[0063] Given a monocular RGB image, the goal is to localize objects of a specific class in 3D space. The target object is represented by a class label and an ABBox-3D (a three-dimensional bounding box) that bounds the entire object without considering occlusion or truncation.

[0064] 2D boundary detection: The 2D detection module is a basic module for stable feature learning and also provides the region of interest for the subsequent geometric reasoning module. Detect the center b and the 2D box B2d of size (w, h) surrounding the projection of ABBox-3D; 3D boundary detection: locate the 3D center C by predicting its depth Zc and 2D projection C. Finally, based on local features, regress the local corner point O relative to the 3D center; Instance depth detection: The IDE subnet estimates the depth of the ABBox-3D center Zc. Given the grid G ​​divided from the backbone in the feature map, each grid cell G predicts the 3D center depth of the nearest instance within the distance threshold σ. Considering the depth information, the cell is assigned a closer instance. 3D bounding box location detection: This subnet estimates the location of the 3D center C = (Xc, Yc, Zc) of the object of interest in each grid G.

[0065] As a further optimization of this implementation, the method further includes: Step d01: When the event recognition result is that a wild animal comes into contact with a pedestrian, a warning message is generated; Step d02: Send the warning information to the management terminal. In this embodiment, the present invention realizes biological monitoring, meteorological monitoring, and event detection. It can record the movement trajectories of various organisms in different seasons and different weather conditions to understand the activity characteristics of various organisms and provide protection measures for specific organisms in different seasons; and monitor whether pedestrians come into contact with organisms. When pedestrians actively come into contact with animals or animals actively come into contact with pedestrians, an alarm is sounded, and the address of the current camera is sent to the forest ranger or relevant management personnel to facilitate the management personnel to implement protection measures as soon as possible.

[0066] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0067] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0068] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A wildlife monitoring system, characterized in that: The system comprises: Remote control platform; The biological monitoring module is connected to the remote control platform for collecting moving images of wild animals and uploading the moving image data to the remote control platform; The meteorological monitoring module is connected to the remote main control platform for collecting meteorological data in the wildlife protection area and uploading the meteorological data to the remote main control platform; The remote control platform is used to: identify and analyze motion image data based on a pre-built multi-target multi-camera target tracking model to obtain tracking results, which are used to characterize the movement trajectory of wild animals under current meteorological conditions; And determine whether there are pedestrians in the moving image. When there are pedestrians in the moving image, the moving image data is identified based on a pre-built event monitoring model to obtain an event recognition result, which is used to characterize whether the wild animal comes into contact with the pedestrian.

2. The wildlife monitoring system according to claim 1, characterized in that: The system also includes: a management terminal, which is connected to the remote main control platform for communication. The remote main control platform is also used to: generate warning information when wild animals come into contact with pedestrians; and send the warning information to the management terminal.

3. The wildlife monitoring system according to claim 1, characterized in that: The event monitoring model is the MonoGRNet model.

4. The wildlife monitoring system according to claim 1, characterized in that: The multi-target multi-camera target tracking model includes: The single-camera tracking model is used to extract features from motion image data based on the Transformer algorithm to obtain multi-dimensional animal features, identify multi-dimensional animal features based on the Cascade-RCNN algorithm to obtain single-camera target recognition results, and predict the trajectory of single-camera target recognition results based on the DeepSORT algorithm to obtain single-camera animal motion trajectories; The cross-camera object Re-ID model is used to re-identify multi-dimensional animal features based on the Re-ID algorithm to obtain cross-camera object recognition results; A camera connection model is used to associate single-camera animal motion trajectories based on movement direction, bidirectional transition time window and appearance similarity to obtain the associated animal motion trajectory, and use the associated animal motion trajectory as the tracking result, wherein the appearance similarity is determined based on the cross-camera target recognition result.

5. A method for monitoring wild animals, characterized in that: The method comprises: Obtaining the motion images of wild animals collected by the biological monitoring module and the meteorological data collected by the meteorological monitoring module; Recognize and analyze the motion image data based on a pre-built multi-target multi-camera target tracking model to obtain a tracking result, which is used to characterize the movement trajectory of wild animals under current meteorological conditions; Determine whether there are pedestrians in the moving image; When there are pedestrians in the motion image, the motion image data is identified based on a pre-built event monitoring model to obtain an event recognition result, which is used to characterize whether the wild animal comes into contact with the pedestrian.

6. The method for monitoring wild animals according to claim 5, characterized in that: The multi-target multi-camera target tracking model includes: a single camera tracking model, a cross-camera target Re-ID model and a camera connection model; Based on the pre-built multi-target multi-camera target tracking model, the motion image data is identified and analyzed to obtain tracking results, including: The single-camera tracking model extracts features from motion image data based on the Transformer algorithm to obtain multi-dimensional animal features, recognizes the multi-dimensional animal features based on the Cascade-RCNN algorithm to obtain single-camera target recognition results, and predicts the trajectory of the single-camera target recognition results based on the DeepSORT algorithm to obtain the single-camera animal motion trajectory; The cross-camera target Re-ID model re-identifies multi-dimensional animal features based on the Re-ID algorithm to obtain a cross-camera target recognition result; The camera connection model associates the single-camera animal motion trajectories based on the moving direction, the bidirectional transition time window and the appearance similarity to obtain the associated animal motion trajectory, and uses the associated animal motion trajectory as the tracking result; wherein the appearance similarity is determined based on the cross-camera target recognition result.

7. The method for monitoring wild animals according to claim 5, characterized in that: The method further comprises: When the event identification result is a contact between wild animals and pedestrians, an early warning message is generated; Send the warning information to the management terminal.

8. The method for monitoring wild animals according to claim 6, characterized in that: The method also includes: in the process of building the cross-camera target Re-ID model, optimizing the global features of the cross-camera target Re-ID model based on the ternary loss and the cross entropy loss.

9. The method for monitoring wild animals according to claim 6, characterized in that: The method further includes: obtaining a moving direction, including: The correct moving direction of each target in the single-camera target recognition result is obtained by manual marking, and the action area of ​​each target is obtained; the action area includes: an entry area and an exit area; wherein the entry area is the first area of ​​the action area passed by the target, and the exit area is the last area of ​​the action area passed by the target; Based on the entry and exit regions of each target, the movement direction of each target is determined.

10. The method for monitoring wild animals according to claim 6, characterized in that: The method further includes: determining appearance similarity, including: Based on the cross-camera target recognition results, determine the single-camera animal motion trajectory of each target on each single camera; Perform average pooling processing on the single-camera animal motion trajectories on each single camera to obtain several trajectory features; Calculate the distance between every two trajectory features; Based on the distance, the time mask and the direction mask between each two trajectory features, a similarity matrix is ​​determined, and the similarity matrix is ​​used to characterize the appearance similarity.

Citation Information

Cited By

  • Method and system for monitoring and protecting rare species based on internet of things

    CN122718665A