Multi-camera cooperative visual tracking system and method for invading animals of transformer substation

Abstract

The invention discloses a multi-camera cooperative visual tracking system and method for invading animals of a transformer substation, and the system achieves the precise monitoring and real-time tracking of the invading animals, such as birds, mice, cats and dogs, in a transformer substation region through the cooperation of a plurality of high-resolution cameras. The method comprises the following steps: firstly, acquiring a region video stream by a plurality of cameras, setting a main tracking camera (MTU), and extracting a dynamic target region through a frame difference method and a threshold segmentation technology; and then background noise is removed through area filtering, and a target contour is obtained. Target tracking adopts SIFT feature extraction and a brute force matching algorithm (BFMatcher) to realize cross-frame matching, and a Kalman filter is used to predict and update a target position. In order to ensure cooperative work of multiple cameras, a cross-view tracking technology is introduced into the system, and when a target crosses the view field of the cameras, adjacent cameras share target information, and a detection mode is activated in advance. The method overcomes the problem of a single visual angle blind area, effectively improves the monitoring efficiency, and is suitable for continuous monitoring and accurate tracking of animals around the transformer substation in a complex environment.

Description

Technical Field

[0001] The present invention belongs to the fields of digital image processing, significant target detection and computer vision technology, and in particular relates to a multi-camera coordinated substation intrusive animal visual tracking system and method. Background Art

[0002] Substations are critical power infrastructure, and animal intrusion around them has long posed a threat to equipment safety and stable operation. In particular, the activities of animals such as birds, mice, cats and dogs may not only cause power equipment failures, but may also lead to safety hazards and equipment damage. Therefore, monitoring and precise tracking of animals around substations have become a necessary means to improve the safety of power equipment and prevent accidents. With the continuous advancement of intelligent technology, especially the gradual maturity of visual monitoring technology and multi-camera collaborative tracking technology, the use of multi-camera systems for automatic detection and tracking of animal intrusions has become an effective solution.

[0003] At present, target detection and tracking technology, especially in the field of deep learning, has made significant progress. The application of convolutional neural networks (CNN) in animal detection has gradually matured. Although existing deep learning target detection models (such as YOLO, SSD, etc.) have good recognition effects, their real-time performance in high-resolution video scenes is still insufficient. In addition, these methods usually focus on target recognition and tracking under a single camera, and cannot effectively cope with multi-animal, multi-perspective tracking tasks in complex environments. In addition, for substation monitoring scenes with high real-time requirements, existing algorithms are often complex in calculation and slow in response. Other traditional methods, such as background subtraction and optical flow, also have their own limitations. Background subtraction performs well in static backgrounds, but is easily disturbed by dynamic backgrounds; optical flow can effectively capture motion, but its high computational complexity limits its real-time performance and application effect. In addition, the movement trajectories of ground animals such as cats, dogs, and mice are more complex and changeable, and may be disturbed by other factors in the environment. Therefore, in order to comprehensively improve the monitoring capabilities of animals around substations, it is necessary to take into account the movement characteristics of different animals and develop a multi-camera collaborative tracking method with stronger compatibility and wider adaptability, which can achieve accurate detection and tracking of various animals (birds, cats, dogs, mice, etc.).

[0004] In view of the shortcomings of these methods, a multi-camera collaborative substation intrusion animal visual tracking system and method is proposed, which can comprehensively utilize the collaborative advantages of multiple cameras, and through image preprocessing, motion target detection, feature point matching, Kalman filtering and other technologies, it can track different types of animal activities in the substation area in real time and accurately, avoiding the loss of targets due to viewing angle limitation or occlusion. This method can not only solve the blind spot problem under a single viewing angle, but also effectively improve the detection accuracy and tracking stability. It can realize the continuous monitoring of various animals such as birds, cats, dogs, mice, etc. in the substation area, and provide strong technical support for ensuring the safe operation of power equipment. Summary of the invention

[0005] In order to solve the above problems, the present invention provides a multi-camera coordinated substation intrusive animal visual tracking system and method, which can detect and track intrusive animal targets in real time, quickly and stably in the monitoring screen of power equipment.

[0006] In order to achieve the above-mentioned purpose, the present invention specifically adopts the following technical solutions:

[0007] A multi-camera coordinated substation intrusion animal visual tracking system, including the following modules:

[0008] The image acquisition module is composed of a high-resolution camera, a CMOS image sensor, a power supply circuit and an STM32 single-chip microcontroller. There are multiple high-resolution cameras and all of them are equipped with a pan-tilt. The high-resolution camera and the network communication module transmit data via wifi. The network communication module includes a PoE switch, a Wi-Fi 6 router, a transmission protocol converter and a buffer memory. The network communication module and the main control center realize data transmission through a 5G wireless connection. The main control center module is equipped with a high-performance CPU and GPU. The CPU and GPU realize data interaction with the network communication module through Ethernet. The edge processing module includes a feature extraction and matching module, a target prediction module, a cross-view collaborative tracking module, a trajectory visualization module, a feedback and control circuit, and a video stream processing and transmission interface.

[0009] A multi-camera coordinated visual tracking method for substation intruders includes the following steps:

[0010] S1 adopts a multi-camera collaborative tracking mechanism, using multiple high-resolution surveillance cameras (ball cameras) to independently collect real-time video streams of the substation area in advance. When a camera detects an intruder target, the camera is set as the master tracking unit (MTU) to send the collected video stream to the main control machine as the input of the system to facilitate subsequent processing of target detection and tracking issues;

[0011] S2 information preprocessing, which performs video frame preprocessing on the input video stream, including grayscale, scaling and filtering, to reduce the complexity of subsequent calculations;

[0012] S3 position estimation: the frame difference method is used to perform frame difference processing on adjacent frame images to obtain a differential image. The differential image is converted into a clear foreground (changed area)-background (unchanged area) separation image through binarization processing to obtain the area of ​​interest where the target may move;

[0013] S4 fine target recognition: In order to refine the precise location of the target and conduct subsequent tracking, SIFT feature point detection is performed on the region of interest obtained after frame difference processing to extract key points, and then the position of the same bird in different time frames is determined by feature point matching, thereby tracking the movement trajectory of the invading animal;

[0014] S5 builds a state space model of Kalman filtering through the matched feature point positions to estimate and predict the next position of the target;

[0015] S6 target motion situation estimation: the system uses Kalman filtering to predict the target's motion direction and activates adjacent cameras to enter detection mode in advance to ensure that the target is not lost when it is about to leave the current MTU field of view, thus maintaining tracking continuity.

[0016] S7 camera status adjustment, when the target enters other adjacent cameras, the adjacent cameras become MTU, and the feature point matching and Kalman filtering steps in S4 to S6 are repeated;

[0017] S8 uses straight lines to connect multiple target locations in the video to form a complete motion path and visualize the trajectory of the target.

[0018] Further, step S1 includes:

[0019] S11 divides the monitoring field of view of each camera into a rectangular array of three rows and three columns, and marks it into nine areas in sequence;

[0020] S12 tracks the target by adjusting the rotation and translation of the camera. First, determine the position of the target in the field of view. If the offset r of the target is greater than the radius R of the camera, the camera will rotate according to the relative position of the target and gradually bring the target back to the center of the field of view. For example, if the target is in the upper right corner, the camera will rotate to the lower left corner and accelerate until the target returns to the center of the field of view. The offset calculation formula r can be expressed as:

[0021] r=(xC x ,yC y ) (1)

[0022] Where (x, y) is the coordinate of the target, (C x, C y ) are the coordinates of the center point of the camera's field of view;

[0023] Each camera in S13 will be in different behavior states according to the location and status of the target.

[0024] In order to coordinate the work of multiple cameras, the following states are defined:

[0025] Waiting state: When the camera loses communication with the host or controller, it enters the waiting state and the camera does not perform any operation until the communication is restored;

[0026] Detection state: When the target is out of the field of view of the current camera or is lost, the camera enters the detection state. The camera uses the shared information from other units to detect the target and adjust the camera so that it can re-lock the target;

[0027] Tracking state: After the camera successfully identifies and locks the target, it enters the tracking state. The camera continuously adjusts and continues to track the target. If the target completely leaves the field of view, it switches to the detection state.

[0028] Each camera of S14 will share the current position, speed, direction and other information of the target in real time.

[0029] Further, step S2 includes:

[0030] S21 establishes two-dimensional space coordinates;

[0031] S22 performs scaling processing on the video to reduce the video frame size and reduce the computational complexity. The calculation formula is as follows:

[0032]

[0033] Among them, new_width, new_height are the width and height of the reduced image, respectively, scale_percent;

[0034] S23 performs grayscale conversion to convert the BGR color image into a grayscale image, removing color information and only including brightness information to reduce the processing dimension.

[0035] Further, step S3 includes:

[0036] The frame difference method described in S31 is a method for obtaining a differential image by performing frame difference calculation on two adjacent frame images. The process of obtaining a complete target contour set by differentiating the two adjacent frame images can be expressed as:

[0037] D(x,y)=|I courent (x,y)-I prev (x, y) (4)

[0038] Among them, I courent (x, y) represents the grayscale image of the current frame pixel, I prev (x, y) represents the grayscale image of the previous frame of pixels, and D(x, y) represents the image after difference;

[0039] S32 uses binarization processing to process the differenced image to further remove the slight difference noise in the background (such as slight pixel changes caused by illumination changes). The binarization processing method is: all points with pixel values ​​less than the threshold T are set to zero, and points greater than the threshold are set to 1 (white), so as to retain the significant change area and remove the inconspicuous noise area. The process can be expressed by the formula:

[0040]

[0041] Among them, T binary (x, y) is the image after thresholding, and T is the set threshold;

[0042] S33 uses an area filtering method to process the differential image after binarization to further improve the accuracy of identifying invasive animal targets. The area filtering method calculates the area A of each connected area and sets a minimum area threshold A. min , if A<A min Then filter this part of the noise and retain only the larger area that meets the target characteristics to improve the accuracy of detection. For the filtered connected area, extract its bounding box and calculate the coordinates of the center point. The calculation formula of the bounding box is:

[0043] BoundingBox=(x,y,w,h). (6)

[0044] Center point coordinate calculation formula:

[0045]

[0046] Where (x, y) is the coordinate of the top left corner of the bounding box, and w and h are the width and height respectively.

[0047] Further, step S4 includes:

[0048] S41 constructs a scale space, applies Gaussian filtering to blur the input image I (x, y) at different scales, and then calculates the blur degree of the image at a certain scale. The Gaussian function of different feature points in the image can be detected and used to smooth the image. The formula is as follows:

[0049]

[0050] By changing the value of σ, the image is Gaussian blurred to different degrees to generate blurred images of multiple scales. The construction expression of the scale space is as follows:

[0051] D(x,y,σ)=I(x,y)*G(x,y,σ) (9)

[0052] Where I(x, y) represents the pixel value of the input image, G(x, y, σ) represents the Gaussian filter kernel;

[0053] After finding the potential key points, S42 uses Taylor Expansion to accurately calculate the exact position of each extreme point, and removes the feature points at the edge according to the directional gradient information of the feature points. The feature points at the edge are easily affected by noise;

[0054] S43 In the area around the feature point, SIFT divides the area into 4x4 grids, calculates the gradient direction in each grid, and then counts the gradient information in these directions to form a 128-dimensional descriptor vector;

[0055] S44For each feature point, SIFT calculates a descriptor that represents the features of the local area around the feature point. The BFMatcher (Brute Force Matcher) brute force matching algorithm is used to compare each feature point descriptor of one frame image with all feature point descriptors in another frame image one by one. The matching effect is judged by calculating the Euclidean distance between the feature point descriptors. The smaller the Euclidean distance, the higher the matching effect. The formula for calculating the Euclidean distance is as follows:

[0056]

[0057] Where f1(i) and f2(i) are the values ​​of the i-th dimension of descriptor 1 and descriptor 2, and n is the dimension of the descriptor.

[0058] Further, step S5 includes:

[0059] S51 establishes a state space model to define the motion characteristics of the invading animal target. After determining the target position, the Kalman filter is introduced to predict and update the target position to make the tracking smoother and more stable. The state space model uses a four-dimensional state vector to represent the position information (x, y) and speed information (v x , v y ), define the invading animal target state vector:

[0060]

[0061] Define the measurement vector:

[0062]

[0063] Among them, x and y represent the position of the invading animal in the current frame, and v x and v y represents the velocity component;

[0064] S52 establishes a state transition matrix (transitionMatrix) to define how the state of the moving object evolves from one frame to the next frame, so as to describe the position (x, y) of the moving object as the speed (v x , v y ) changes, the state transfer matrix can be expressed as:

[0065]

[0066] S53 establishes a measurement matrix (measurementMatrix) to map the observed position (x, y) to the measurement space. The measurement matrix H is expressed as follows:

[0067]

[0068] S54 defines the process noise covariance matrix (processNoiseCov) to reflect the uncertainty in the state transfer process. The covariance matrix Q can be expressed as:

[0069]

[0070] S55 uses the Kalman filter to predict the next frame position of the target through the state transfer model to obtain the predicted value. The Kalman filter state expression is:

[0071]

[0072] in is the predicted state at the next moment, F is the state transfer matrix, X k-1 is the previous state vector;

[0073] S56 obtains a measured value through actual measurement, compares the measured value with the predicted value, and corrects the predicted value according to the measured value to complete the measurement update. The measurement update includes two steps:

[0074] S561 calculates the Kalman gain to balance the weight between prediction and measurement. The formula is:

[0075] H k =P k|k-1 H T (HP k|k-1 H T +R) -1 (17)

[0076] Among them, P k|k-1 is the predicted covariance, H is the measurement matrix, and R is the measurement noise covariance matrix;

[0077] S562 status update, use the measured value to correct the predicted value, the formula is:

[0078]

[0079] Among them, Z k is the actual measured value, X k is the updated state vector. This step uses the measured deviation (residual) to correct the predicted value, making the estimated position closer to the actual observed value;

[0080] S57 updates the covariance matrix to reduce uncertainty and prepare for the next prediction. The updated covariance matrix will affect the credibility and measurement weight of the next prediction. The Kalman filter continuously predicts and updates the position information to achieve smooth tracking of invading animals in complex motion scenes. The updated covariance matrix can be expressed as:

[0081] P k =(IK k H)P k|k-1 (19)

[0082] Where I represents the identity matrix, K k represents the Kalman gain matrix, and H represents the observation matrix.

[0083] Further, step S6 includes:

[0084] Based on the prediction results of Kalman filtering, the S61 system will activate adjacent cameras in advance and adjust their status to "detection mode". When the target is about to enter the field of view of the camera, the camera is already in detection mode and is ready to continue tracking.

[0085] S62 When the target enters the field of view of a new camera, the new camera does not need to perform feature point extraction and frame difference detection from scratch; the new camera only needs to use the received shared target position and feature descriptors to match with the previous camera data to ensure seamless transfer of the target position;

[0086] Further, step S8 includes:

[0087] S81 connects the position points of the target in the time series in each frame by linear interpolation to form a motion trajectory;

[0088] When S82 detects a new target position, it adds the position to the motion trajectory and updates the display, using Kalman filtering to smooth the motion trajectory.

[0089] Compared with the prior art, the beneficial effects are:

[0090] (1) By using the Kalman filter to predict and smooth the target position, the amount of calculation and processing time are reduced, thereby significantly improving the real-time performance while ensuring the tracking accuracy, meeting the actual needs of high-dynamic scenes around power equipment;

[0091] (2) By combining the frame difference method with the scale-invariant feature transform (SIFT), the accuracy of target feature extraction is effectively improved, and it can stably identify and track fast-moving targets in complex environments. At the same time, with the help of this combined method, the system can effectively suppress the interference of weather, lighting changes and background motion, significantly improving the accuracy and robustness of detection;

[0092] (3) The introduction of Kalman filter and SIFT feature matching effectively predicts and associates the target smoothly, improves the persistence and consistency of the target, and effectively solves the problem of target loss or incorrect tracking due to the high similarity between targets;

[0093] (4) Through the collaborative work of multiple cameras, the advantages of multi-camera perspectives are fully utilized. Multiple cameras can cover different monitoring areas, avoiding the loss of targets caused by viewing angle limitations or occlusions, achieving all-round, no-blind-angle animal monitoring, and significantly improving the coverage and real-time performance of monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0094] Figure 1 This is a diagram of the architecture of a multi-camera collaborative substation intrusive animal visual tracking system proposed by the present invention;

[0095] Figure 2 A modular flow chart of a multi-camera collaborative visual tracking method for substation intruder animals proposed by the present invention;

[0096] Figure 3 A flowchart of a multi-camera coordinated visual tracking method for substation intruder animals proposed by the present invention;

[0097] Figure 4 This is a diagram of the field of view area division of the camera monitoring range proposed in the present invention. DETAILED DESCRIPTION

[0098] The specific implementation methods of the multi-camera coordinated substation intrusive animal visual tracking system and method of the present invention are described in detail below with reference to the accompanying drawings.

[0099] like Figure 1 As shown, a multi-camera coordinated substation intrusion animal visual tracking system of the present invention includes the following modules:

[0100] The image acquisition module is composed of a high-resolution camera, a CMOS image sensor, a power supply circuit and an STM32 single-chip microcontroller. There are multiple high-resolution cameras and all of them are equipped with a pan-tilt. The high-resolution camera and the network communication module transmit data via wifi. The network communication module includes a PoE switch, a Wi-Fi 6 router, a transmission protocol converter and a buffer memory. The network communication module and the main control center realize data transmission through a 5G wireless connection. The main control center module is equipped with a high-performance CPU and GPU. The CPU and GPU realize data interaction with the network communication module through Ethernet. The edge processing module includes a feature extraction and matching module, a target prediction module, a cross-view collaborative tracking module, a trajectory visualization module, a feedback and control circuit, and a video stream processing and transmission interface.

[0101] The feature extraction and matching module uses SIFT to extract key feature points, and identifies and tracks the target through feature matching technology, which is used to accurately identify the target and match it with the target in subsequent frames to ensure continuous tracking of invading animals. The target prediction module is used to predict the next position and motion trajectory of the target, so as to activate adjacent cameras in advance and ensure that the target is not lost during the field of view transfer. The cross-view collaborative tracking module is responsible for working together between multiple camera perspectives to ensure that the tracking information of the target in multiple perspectives can be seamlessly connected. When the target moves from one camera field of view to another, the module maintains the tracking of the target by collaboratively switching the "main tracking unit (MTU)". The trajectory visualization module is used to visualize the target's motion trajectory in the video, helping operators to intuitively view the behavioral trajectory of invading animals and improve the real-time monitoring effect. The feedback and control circuit is used to receive instructions from the main control center or edge computing device, and adjust the camera's tracking strategy and image processing parameters according to the target position and motion state. The video stream processing and transmission interface is used to transmit the processed video stream, target position information and tracking trajectory information to the main control center to ensure real-time sharing of image data and tracking information within the system.

[0102] Combination Figure 2 , Figure 3 , a multi-camera collaborative substation intrusion animal visual tracking method, comprising the following steps:

[0103] S1 adopts a multi-camera collaborative tracking mechanism, using multiple high-resolution surveillance cameras (ball cameras) to independently collect real-time video streams of the substation area in advance. When a camera detects an intruder target, the camera is set as the master tracking unit (MTU) to send the collected video stream to the main control machine as the input of the system to facilitate subsequent processing of target detection and tracking issues;

[0104] S2 performs video frame preprocessing on the input video stream, including grayscale, scaling and filtering, to reduce the complexity of subsequent calculations;

[0105] S3 uses the frame difference method to perform frame difference processing on adjacent frame images to obtain a differential image, and converts the differential image into a clear foreground (changed area)-background (unchanged area) separation image through binarization processing to obtain the area of ​​interest where the target may move;

[0106] In order to refine the precise location of the target and conduct subsequent tracking, S4 performs SIFT feature point detection on the region of interest obtained after frame difference processing to extract key points, and then determines the position of the same bird in different time frames through feature point matching, thereby tracking the movement trajectory of the invading animal;

[0107] S5 builds a state space model of Kalman filtering through the matched feature point positions to estimate and predict the next position of the target;

[0108] The S6 system uses Kalman filtering to predict the target's direction of movement and activates adjacent cameras to enter detection mode in advance to ensure that the target is not lost when it is about to leave the current MTU field of view, maintaining tracking continuity;

[0109] S7 When the target enters other adjacent cameras, the adjacent cameras become MTUs, and the feature point matching and Kalman filtering steps from 4 to 6 are repeated;

[0110] S8 uses straight lines to connect multiple target locations in the video to form a complete motion path and visualize the trajectory of the target.

[0111] Preferably, step S1 includes:

[0112] S11 divides the field of view and combines Figure 4 , the field of view is divided into a rectangular array of three rows and three columns, and marked into nine areas in sequence;

[0113] S12 tracks the target by adjusting the rotation and translation of the camera. First, determine the position of the target in the field of view. If the offset r of the target is greater than the radius R of the camera, the camera will rotate according to the relative position of the target and gradually bring the target back to the center of the field of view. For example, if the target is in the upper right corner, the camera will rotate to the lower left corner and accelerate until the target returns to the center of the field of view. The offset calculation formula r can be expressed as:

[0114] r=(xC x ,yC y ) (1)

[0115] Where (x, y) is the coordinate of the target, (C x , C y ) are the coordinates of the center point of the camera's field of view;

[0116] Each camera in S13 will be in different behavior states according to the location and state of the target. In order to coordinate the cooperation of multiple cameras, the following states are defined:

[0117] Waiting state: When the camera loses communication with the host or controller, it enters the waiting state and the camera does not perform any operation until the communication is restored;

[0118] Detection state: When the target is out of the field of view of the current camera or is lost, the camera enters the detection state. The camera uses the shared information from other units to detect the target and adjust the camera so that it can re-lock the target;

[0119] Tracking state: After the camera successfully identifies and locks the target, it enters the tracking state. The camera continuously adjusts and continues to track the target. If the target completely leaves the field of view, it switches to the detection state.

[0120] Each camera of S14 will share the current position, speed, direction and other information of the target in real time.

[0121] Preferably, step S2 includes:

[0122] S21 establishes two-dimensional space coordinates;

[0123] S22 performs scaling processing on the video to reduce the video frame size and reduce the computational complexity. The calculation formula is as follows:

[0124]

[0125] Among them, new_width, new_height are the width and height of the reduced image respectively; width, height are the width and height of the original image respectively; scale_percent indicates the scaling ratio;

[0126] S23 performs grayscale conversion to convert the BGR color image into a grayscale image, removing color information and only including brightness information to reduce the processing dimension.

[0127] Preferably, step S3 includes:

[0128] The frame difference method described in S31 is a method for obtaining a differential image by performing frame difference calculation on two adjacent frame images. The process of obtaining a complete target contour set by differentiating the two adjacent frame images can be expressed as:

[0129] D(x,y)=|I courent (x,y)-I prev (x, y)|, (4)

[0130] Among them, Icourent (x, y) represents the grayscale image of the current frame pixel, I prev (x, y) represents the grayscale image of the previous frame of pixels, and D(x, y) represents the image after difference;

[0131] S32 uses binarization processing to process the differenced image to further remove the slight difference noise in the background (such as slight pixel changes caused by illumination changes). The binarization processing method is: all points with pixel values ​​less than the threshold T are set to zero, and points greater than the threshold are set to 1 (white), so as to retain the significant change area and remove the inconspicuous noise area. The process can be expressed by the formula:

[0132]

[0133] Among them, T binary (x, y) is the image after thresholding, and T is the set threshold;

[0134] S33 uses an area filtering method to process the differential image after binarization to further improve the accuracy of identifying invasive animal targets. The area filtering method calculates the area A of each connected area and sets a minimum area threshold A. min , if A<A min Then filter this part of the noise and retain only the larger area that meets the target characteristics to improve the accuracy of detection. For the filtered connected area, extract its bounding box and calculate the coordinates of the center point. The calculation formula of the bounding box is:

[0135] BoundingBox=(x,y,w,h). (6)

[0136] Center point coordinate calculation formula:

[0137]

[0138] Where (x, y) is the coordinate of the top left corner of the bounding box, and w and h are the width and height respectively.

[0139] Preferably, step S3 includes:

[0140] S31 constructs a scale space, applies Gaussian filtering to blur the input image I (x, y) at different scales, and then calculates the blur degree of the image at a certain scale. The Gaussian function of different feature points in the image can be detected and used to smooth the image. The formula is as follows:

[0141]

[0142] By changing the value of σ, the image is Gaussian blurred to different degrees to generate blurred images of multiple scales. The construction expression of the scale space is as follows:

[0143] D(x,y,σ)=I(xy)*G(x,y,σ) (9)

[0144] Where I(x, y) represents the pixel value of the input image, G(x, y, σ) represents the Gaussian filter kernel;

[0145] After S32 finds the potential key points, the Taylor Expansion is used to accurately calculate the exact position of each extreme point, and the feature points at the edge are eliminated according to the directional gradient information of the feature points, and the feature points at the edge are easily affected by noise;

[0146] S33 In the area around the feature point, SIFT divides the area into 4x4 grids, calculates the gradient direction in each grid, and then counts the gradient information in these directions to form a 128-dimensional descriptor vector;

[0147] S34 For each feature point, SIFT calculates a descriptor that represents the features of the local area around the feature point. The brute force matching algorithm {BFMatcher) is used to compare each feature point descriptor of one frame image with all feature point descriptors in another frame image one by one. The matching effect is judged by calculating the Euclidean distance between the feature point descriptors. The smaller the Euclidean distance, the higher the matching effect. The formula for calculating the Euclidean distance is as follows:

[0148]

[0149] Where f1(i) and f2(i) are the values ​​of the i-th dimension of descriptor 1 and descriptor 2, and n is the dimension of the descriptor.

[0150] Preferably, step S5 includes:

[0151] S51 establishes a state space model to define the motion characteristics of the invading animal target. After determining the target position, the Kalman filter is introduced to predict and update the target position to make the tracking smoother and more stable. The state space model uses a four-dimensional state vector to represent the position information (x, y) and speed information (v x , v y ), define the invading animal target state vector:

[0152]

[0153] Define the measurement vector:

[0154]

[0155] Among them, x and y represent the position of the invading animal in the current frame, and v xand v y represents the velocity component;

[0156] S52 establishes a state transition matrix (transitionMatrix) to define how the state of the moving object evolves from one frame to the next frame, so as to describe the position (x, y) of the moving object as the speed (v x , v y ) changes, the state transfer matrix can be expressed as:

[0157]

[0158] S53 establishes a measurement matrix (measurementMatrix) to map the observed position (x, y) to the measurement space. The measurement matrix H is expressed as follows:

[0159]

[0160] S54 defines the process noise covariance matrix (processNoiseCov) to reflect the uncertainty in the state transfer process. The covariance matrix Q can be expressed as:

[0161]

[0162] S55 uses the Kalman filter to predict the next frame position of the target through the state transfer model to obtain the predicted value. The Kalman filter state expression is:

[0163]

[0164] in is the predicted state at the next moment, F is the state transfer matrix, X k-1 is the previous state vector;

[0165] S56 obtains a measured value through actual measurement, compares the measured value with the predicted value, and corrects the predicted value according to the measured value to complete the measurement update. The measurement update includes two steps:

[0166] S561 calculates the Kalman gain to balance the weight between prediction and measurement. The formula is:

[0167] H k =P k|k-1 H T (HP k|k-1 H T +R) -1 (17)

[0168] Among them, P k|k-1 is the predicted covariance, H is the measurement matrix, and R is the measurement noise covariance matrix;

[0169] S562 status update, use the measured value to correct the predicted value, the formula is:

[0170]

[0171] Among them, Z k is the actual measured value, X k is the updated state vector. This step uses the measured deviation (residual) to correct the predicted value, making the estimated position closer to the actual observed value;

[0172] S57 updates the covariance matrix to reduce uncertainty and prepare for the next prediction. The updated covariance matrix will affect the credibility and measurement weight of the next prediction. The Kalman filter continuously predicts and updates the position information to achieve smooth tracking of invading animals in complex motion scenes. The updated covariance matrix can be expressed as:

[0173] P k =(IK k H)P k|k-1 (19)

[0174] Where I represents the identity matrix, K k represents the Kalman gain matrix, and H represents the observation matrix.

[0175] Preferably, step S6 includes:

[0176] Based on the prediction results of Kalman filtering, the S61 system will activate adjacent cameras in advance and adjust their status to "detection mode". When the target is about to enter the field of view of the camera, the camera is already in detection mode and is ready to continue tracking.

[0177] S62 When the target enters the field of view of a new camera, the new camera does not need to perform feature point extraction and frame difference detection from scratch; the new camera only needs to use the received shared target position and feature descriptors to match with the previous camera data to ensure seamless transfer of the target position;

[0178] Preferably, step S8 includes:

[0179] S81 connects the position points of the target in the time series in each frame by linear interpolation to form a motion trajectory;

[0180] When S82 detects a new target position, it adds the position to the motion trajectory and updates the display, using Kalman filtering to smooth the motion trajectory.

[0181] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention and are not intended to limit the present invention. A person skilled in the art should understand that the specific implementation of the present invention can be modified or replaced by equivalents, but these modifications or changes are within the scope of protection of the pending claims.

Claims

1. A multi-camera coordinated substation intrusion animal visual tracking system, characterized in that: It includes image acquisition module, network communication module, main control center module and edge processing module; The image acquisition module is composed of a high-resolution camera, a CMOS image sensor, a power supply circuit and an STM32 single-chip microcontroller. There are multiple high-resolution cameras and all of them are equipped with a pan-tilt platform. The high-resolution camera and the network communication module transmit data via wifi. The network communication module includes a PoE switch, a Wi-Fi 6 router, a transmission protocol converter and a buffer memory. The network communication module and the main control center realize data transmission through a 5G wireless connection. The main control center module is equipped with a high-performance CPU and GPU. The CPU and GPU realize data interaction with the network communication module through Ethernet. The edge processing module includes a feature extraction and matching module, a target prediction module, a cross-view collaborative tracking module, a trajectory visualization module, a feedback and control circuit, and a video stream processing and transmission interface. The feature extraction and matching module uses SIFT to extract key feature points and uses feature matching technology to identify and track targets, which is used to accurately identify targets and match them with targets in subsequent frames to ensure continuous tracking of invading animals; The target prediction module is used to predict the next position and movement trajectory of the target, so as to activate adjacent cameras in advance and ensure that the target is not lost during the field of view transfer process; The cross-view collaborative tracking module is responsible for working in collaboration between camera views to ensure that the tracking information of the target in multiple views can be seamlessly connected; When the target moves from one camera's field of view to another camera's field of view, the cross-view cooperative tracking module maintains tracking of the target by cooperatively switching the "main tracking unit MTU"; The trajectory visualization module is used to visualize the target's motion trajectory in the video, helping operators to intuitively view the behavior trajectory of invading animals and improve real-time monitoring effects; The feedback and control circuit is used to receive instructions from the autonomous control center or edge computing device, and adjust the tracking strategy and image processing parameters of the camera according to the target position and motion state; The video stream processing and transmission interface is used to transmit the processed video stream, target location information and tracking trajectory information to the main control center to ensure real-time sharing of image data and tracking information within the system.

2. The visual tracking method of a multi-camera coordinated substation intrusion animal visual tracking system according to claim 1 is characterized in that: The following steps are involved: S1 adopts a multi-camera collaborative tracking mechanism, and uses high-resolution surveillance cameras to independently collect real-time video streams in the substation area in advance. When a camera detects an intruder target, it is set as the main tracking unit MTU to send the collected video stream to the main control machine as the input of the system to facilitate subsequent processing of target detection and tracking issues; S2 information preprocessing, which performs video frame preprocessing on the input video stream, including grayscale, scaling and filtering, to reduce the complexity of subsequent calculations; S3 position estimation, using the frame difference method to perform frame difference processing on adjacent frame images to obtain a differential image, and then converting the differential image into a clear foreground-background separation image through binarization processing to obtain the region of interest where the target may move; S4 fine target recognition: In order to refine the precise location of the target and conduct subsequent tracking, SIFT feature point detection is performed on the region of interest obtained after frame difference processing to extract key points, and then the position of the same bird in different time frames is determined by feature point matching, thereby tracking the movement trajectory of the invading animal; S5 builds a state space model of Kalman filtering through the matched feature point positions to estimate and predict the next position of the target; S6 target motion situation estimation: the system uses Kalman filtering to predict the target's motion direction and activates adjacent cameras to enter detection mode in advance to ensure that the target is not lost when it is about to leave the current MTU field of view, thus maintaining tracking continuity. S7 camera status adjustment, when the target enters other adjacent cameras, the adjacent cameras become MTU, and the feature point matching and Kalman filtering steps in S4 to S6 are repeated; S8 uses straight lines to connect multiple target locations in the video to form a complete motion path and visualize the trajectory of the target.

3. The multi-camera coordinated substation intrusion animal visual tracking method according to claim 2 is characterized in that: Step S1 includes: S11 divides the field of view into a rectangular array of three rows and three columns, and marks it into nine areas in sequence; S12 tracks the target by adjusting the rotation and translation of the camera. First, the position of the target in the field of view is determined. If the offset r of the target is greater than the radius R of the camera, the camera will rotate according to the relative position of the target and gradually bring the target back to the center of the field of view. The offset calculation formula r can be expressed as: r=(x-C x ,y-C y ) (1) Where (x, y) is the coordinate of the target, (C x , C y ) are the coordinates of the center point of the camera's field of view; Each camera in S13 will be in different behavior states according to the location and state of the target. In order to coordinate the collaborative work of multiple cameras, the following states are defined: Waiting state: When the camera loses communication with the host or controller, it enters the waiting state and the camera does not perform any operation until the communication is restored; Detection state: When the target is out of the field of view of the current camera or is lost, the camera enters the detection state. The camera uses the shared information from other units to detect the target and adjust the camera so that it can re-lock the target; Tracking state: After the camera successfully identifies and locks the target, it enters the tracking state. The camera continuously adjusts and continues to track the target. If the target completely leaves the field of view, it switches to the detection state. Each camera of S14 will share the current position, speed and direction information of the target in real time.

4. The multi-camera coordinated substation intrusion animal visual tracking method according to claim 2 is characterized in that: Step S2 includes: S21 establishes two-dimensional space coordinates; S22 performs scaling processing on the video to reduce the video frame size and reduce the computational complexity. The calculation formula is as follows: Among them, new_width, new_height are the width and height of the reduced image respectively; width, height are the width and height of the original image respectively; scale_percent indicates the scaling ratio; S23 performs grayscale conversion to convert the BGR color image into a grayscale image, removing color information and only including brightness information to reduce the processing dimension.

5. The multi-camera coordinated substation intrusion animal visual tracking method according to claim 2 is characterized in that: Step S3 includes: The frame difference method described in S31 is a method for obtaining a differential image by performing frame difference calculation on two adjacent frame images. The process of obtaining a complete target contour set by differentiating the two adjacent frame images is expressed by the formula: D(x,y)=|I courent (x,y)-I prev (x,y)|, (4) Among them, I courent (x, y) represents the grayscale image of the current frame pixel, I prev (x, y) represents the grayscale image of the previous frame of pixels, and D(x, y) represents the image after difference; S32 uses binarization to process the differenced image to further remove the slight difference noise in the background. The binarization method is: all points with pixel values ​​less than the threshold T are set to zero, and points greater than the threshold are set to 1, so as to retain the significant change area and remove the inconspicuous noise area. The process is expressed by the formula: Among them, T binary (x, y) is the image after thresholding, and T is the set threshold; S33 uses an area filtering method to process the differential image after the binary processing to further improve the accuracy of invasive animal target identification. The area filtering method calculates the area A of each connected area and sets a minimum area threshold A min , if A<A min Then filter this part of the noise and retain only the larger area that meets the target characteristics to improve the accuracy of detection. For the filtered connected area, extract its bounding box and calculate the coordinates of the center point. The calculation formula of the bounding box is: Bounding Box=(x,y,w,h). (6) Center point coordinate calculation formula: Where (x, y) is the coordinate of the top left corner of the bounding box, and w and h are the width and height respectively.

6. The multi-camera coordinated substation intrusion animal visual tracking method according to claim 2 is characterized in that: Step S4 includes: S41 constructs a scale space, applies Gaussian filtering to blur the input image I (x, y) at different scales, and then calculates the blur degree of the image at a certain scale. The Gaussian function of different feature points in the image can be detected and used to smooth the image. The formula is as follows: By changing the value of σ, the image is Gaussian blurred to different degrees to generate blurred images of multiple scales; the construction expression of the scale space is as follows: D(x,y,σ)=I(x,y)*G(x,y,σ) (9) Where I(x, y) represents the pixel value of the input image, G(x, y, σ) represents the Gaussian filter kernel; After S42 finds the potential key points, it uses Taylor Expansion to accurately calculate the exact position of each extreme point, and removes the feature points at the edge according to the directional gradient information of the feature points. The feature points at the edge are easily affected by noise. S43 In the area around the feature point, SIFT divides the area into 4x4 grids, calculates the gradient direction in each grid, and then counts the gradient information in these directions to form a 128-dimensional descriptor vector; S44For each feature point, SIFT calculates a descriptor that represents the features of the local area around the feature point. The BFMatcher brute force matching algorithm is used to compare each feature point descriptor of one frame image with all feature point descriptors in another frame image one by one. The matching effect is judged by calculating the Euclidean distance between the feature point descriptors. The smaller the Euclidean distance, the higher the matching effect. The formula for calculating the Euclidean distance is as follows: Where f1(i) and f2(i) are the values ​​of the i-th dimension of descriptor 1 and descriptor 2, and n is the dimension of the descriptor.

7. The multi-camera coordinated substation intrusion animal visual tracking method according to claim 2 is characterized in that: Step S5 includes: S51 establishes a state space model to define the motion characteristics of the invading animal target. After determining the target position, the Kalman filter is introduced to predict and update the target position to make the tracking smoother and more stable. The state space model uses a four-dimensional state vector to represent the position information (x, y) and speed information (v x , v y ), define the invading animal target state vector: Define the measurement vector: Among them, x and y represent the position of the invading animal in the current frame, and v x and v y represents the velocity component; S52 establishes a state transition matrix (transitionMatrix) to define how the state of the moving object evolves from one frame to the next frame, so as to describe the position (x, y) of the moving object as the speed (v x , v y ) changes, the state transfer matrix can be expressed as: S53 establishes a measurement matrix (measurementMatrix) to map the observed position (x, y) to the measurement space. The measurement matrix H is expressed as follows: S54 defines the process noise covariance matrix (processNoiseCov) to reflect the uncertainty in the state transfer process. The covariance matrix Q is expressed as follows: S55 uses the Kalman filter to predict the next frame position of the target through the state transfer model to obtain the predicted value. The Kalman filter state expression is: in is the predicted state at the next moment, F is the state transfer matrix, X k-1 is the previous state vector; S56 obtains a measured value through actual measurement, compares the measured value with the predicted value, and corrects the predicted value according to the measured value to complete the measurement update. The measurement update includes two steps: S561 calculates the Kalman gain to balance the weight between prediction and measurement. The formula is: H k =P k|k-1 H T (HP k|k-1 H T +R) -1 (17) Among them, P k|k-1 is the predicted covariance, H is the measurement matrix, and R is the measurement noise covariance matrix; S562 status update, use the measured value to correct the predicted value, the formula is: Among them, Z k is the actual measured value, X k is the updated state vector; this step uses the measured deviation to correct the predicted value, making the estimated position closer to the actual observed value; S57 updates the covariance matrix to reduce uncertainty and prepare for the next prediction. The updated covariance matrix will affect the credibility and measurement weight of the next prediction. The Kalman filter continuously predicts and updates the position information to achieve smooth tracking of invading animals in complex motion scenes. The updated covariance matrix can be expressed as: P k =(I-K k H)P k|k-1 (19) Where I represents the identity matrix, K k represents the Kalman gain matrix, and H represents the observation matrix.

8. The multi-camera coordinated substation intrusion animal visual tracking method according to claim 2 is characterized in that: Step S6 includes: Based on the prediction results of Kalman filtering, the S61 system will activate adjacent cameras in advance and adjust their status to "detection mode". When the target is about to enter the field of view of the camera, the camera is already in detection mode and is ready to continue tracking. S62 When the target enters the field of view of a new camera, the new camera does not need to perform feature point extraction and frame difference detection from scratch; the new camera only needs to use the received shared target position and feature descriptors to match with the previous camera data to ensure seamless transfer of the target position.

9. The multi-camera coordinated substation intrusion animal visual tracking method according to claim 2 is characterized in that: Step S8 includes: S81 connects the position points of the target in the time series in each frame by linear interpolation to form a motion trajectory; When S82 detects a new target position, it adds the position to the motion trajectory and updates the display, using Kalman filtering to smooth the motion trajectory.

Images