Computer vision-based migratory bird flight health monitoring method and device
By extracting static and dynamic attitude parameters of migratory birds from flight videos using computer vision-based methods, a multi-dimensional feature assessment of migratory bird health status is constructed. This solves the problems of single feature and large equipment interference in existing technologies, and realizes high-precision, low-interference large-scale migratory bird flight health monitoring.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- QINGDAO QINGSHU TECH CO LTD
- Filing Date
- 2026-03-11
- Publication Date
- 2026-06-19
AI Technical Summary
Existing technologies for assessing the health status of migratory birds during flight suffer from limitations such as single feature dimensions, incomplete reference benchmarks, and poor species adaptability, making it difficult to achieve accurate early warning. Furthermore, traditional monitoring methods are inefficient, subject to significant equipment interference, and are costly, making large-scale population monitoring impossible.
A computer vision-based approach is adopted to extract static and dynamic attitude parameters of migratory birds from flight videos. Cross-frame temporal analysis is performed using the ST-OverloCK backbone network and ConvLSTM to construct a multi-dimensional feature assessment of the flight health status of migratory birds. The flight health index is calculated by comparing the health sample parameter database of various flight modes and dynamically adjusting the weights.
It has achieved high-precision and large-scale monitoring of migratory bird health during flight, avoiding interference with the ecological environment, improving monitoring efficiency and coverage, and enabling early identification of health abnormalities.
Smart Images
Figure CN122244751A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the fields of computer vision and wildlife health monitoring, and in particular to a method and device for monitoring the health of migratory birds during flight based on computer vision. Background Technology
[0002] With the development of computer vision and deep learning technologies, video-based target pose analysis methods are gradually being applied to the field of wildlife monitoring. However, existing related technologies mostly focus on basic tasks such as migratory bird species identification and behavior classification, and there is a lack of specialized research on flight health status assessment. Moreover, existing health assessment methods generally suffer from multiple technical shortcomings, such as single feature dimensions, incomplete reference benchmarks, and poor species adaptability, making it difficult to meet the actual needs of accurate early warning.
[0003] Therefore, overcoming the aforementioned technical problems and defects has become a key issue that needs to be addressed. Summary of the Invention
[0004] To address the aforementioned technical problems, this application provides a computer vision-based method and apparatus for monitoring the health of migratory birds during flight, which can improve the monitoring efficiency and accuracy of migratory bird health.
[0005] According to one aspect of this application, a computer vision-based method for monitoring the health of migratory birds during flight is provided, the method comprising:
[0006] Multiple consecutive flight images are obtained from the flight video of the target migratory bird to obtain the flight target sequence image of the target migratory bird;
[0007] For each frame of the flight image sequence, the coordinates of key points of the target migratory bird in the flight image are detected using a detection model, and a set of key point coordinates for the corresponding frame is generated.
[0008] Based on the set of key point coordinates of consecutive frames within the time window W, the static attitude and flight attitude changes of the target migratory bird are determined, and the static attitude parameter vector and the temporal flight feature vector are obtained.
[0009] The static attitude parameter vector and the time-series flight feature vector are compared with the corresponding parameters in the preset parameter library to obtain the comparison results;
[0010] Based on the comparison results, the difference between the actual flight posture and the normal flight posture of the target migratory bird is determined, and the flight health index of the target migratory bird is obtained.
[0011] In the above scheme, the key points include the left wingtip, left wing root, right wingtip, right wing root, head, neck, trunk center, tail center, left tail feather tip, and right tail feather tip.
[0012] In the above scheme, the detection model includes the ST-OverloCK backbone network, which introduces a cross-frame temporal attention mechanism. The ST-OverloCK backbone network is constructed by connecting a lightweight temporal information encoding module in parallel with the Focus-Net module of the bionic vision backbone network OverloCK. The temporal information encoding module is used to transfer and update contextual information between feature maps of consecutive frames using a convolutional long short-term memory network (ConvLSTM).
[0013] In the above scheme, determining the static attitude and flight attitude changes of the target migratory bird based on the set of key point coordinates within a continuous frame of time window W, and obtaining the static attitude parameter vector and the temporal flight feature vector, includes:
[0014] For each frame of image within a time window W, a skeleton model of the target migratory bird in the corresponding frame is generated based on the corresponding set of key point coordinates.
[0015] Based on the skeleton model, the static attitude parameters of the target migratory bird in the corresponding frame are determined, and the corresponding static attitude parameter vector is obtained; the static attitude parameter vector includes wingspan length, left wingspan angle, right wingspan angle, trunk pitch angle, roll angle and tail opening angle;
[0016] A temporal analysis is performed on the static attitude parameter vectors of consecutive frames to determine the flight attitude changes of the target migratory bird, and the temporal flight feature vector of the target migratory bird is obtained. The temporal flight feature vector includes flapping frequency, left and right wing flapping phase difference, attitude parameter change rate, attitude change smoothness, and trajectory curvature.
[0017] In the above scheme, the method further includes determining the static attitude parameters in the static attitude parameter vector using the following method:
[0018] The wingspan length is determined based on the positional difference between the left and right wingtips;
[0019] The left wing span angle is determined based on the angle between the left wing vector and the torso vector, and the right wing span angle is determined based on the angle between the right wing vector and the torso vector.
[0020] The torso pitch angle is determined based on the angle between the torso vector and the horizontal direction;
[0021] The roll angle is determined based on the ratio of the vertical position difference between the left and right wingtips to the wingspan.
[0022] The tail opening angle is determined based on the angle formed by the left tail feather tip, the center of the tail, and the right tail feather tip.
[0023] In the above scheme, the method further includes determining the flight feature parameters in the time-series flight feature vector using the following method:
[0024] The flapping frequency is determined based on the fast Fourier transform results of the wingspan angle sequence;
[0025] The phase difference of the left and right wing flapping is determined based on the difference in the peak position of the cross-correlation function between the left and right wing span angle sequences.
[0026] The rate of change of the attitude parameters is determined based on the mean of the magnitude of the first time derivative of the static attitude parameter vector.
[0027] The smoothness of the attitude change is determined based on the integral norm of the second-order time derivative of the static attitude parameter vector.
[0028] The trajectory curvature is determined based on the coordinate changes of the sequence of key points at the center of the torso within the time window W.
[0029] In the above scheme, determining the difference between the actual flight posture and the normal flight posture of the target migratory bird based on the comparison results, and obtaining the flight health index of the target migratory bird, includes:
[0030] Based on the deviation between the static attitude parameter vector and the reference static attitude parameter vector, the attitude parameter deviation is determined. ;
[0031] Determine the degree of symmetry deviation based on span deviation and phase deviation. The aspect ratio deviation represents the deviation between the left wing aspect ratio and the right wing aspect ratio, and the phase deviation represents the deviation between the flapping phase difference of the left and right wings and the reference phase difference.
[0032] Based on the stability of the flapping wing frequency and the deviation of the flapping wing frequency from the reference frequency, the rhythmic deviation is determined. ;
[0033] Based on the smoothness of the attitude change and the deviation of the trajectory curvature from the normal baseline, the stability deviation is determined. ;
[0034] Based on the attitude parameter deviation The aforementioned symmetry deviation The rhythmic deviation The stability deviation The flight health index of the target migratory bird is determined using a preset weight vector.
[0035] In the above scheme, the deviation based on the attitude parameters The aforementioned symmetry deviation The rhythmic deviation The stability deviation Using a preset weight vector, the flight health index of the target migratory bird is determined, including:
[0036] Based on the static attitude parameter vector, determine the static attitude parameters that deviate from the reference range, and obtain the deviation static attitude parameters;
[0037] Based on the attitude parameter deviation The aforementioned symmetry deviation The rhythmic deviation and the stability deviation The correlation between each deviation index and the deviation from the static attitude parameter is analyzed, and the preset weight vector is dynamically updated to obtain the target weight vector.
[0038] Based on the target weight vector, , , and Linear weighting is applied to obtain the flight health index of the target migratory birds; where,
[0039] Dynamically update the preset weight vector, including increasing the weight of the deviation index associated with the deviation from the static attitude parameters in the preset weight vector.
[0040] In the above scheme, the preset parameter library includes health sample parameters for various typical flight modes; the typical flight modes include cruise, hovering, takeoff, and landing.
[0041] According to another aspect of this application, a computer vision-based migratory bird flight health monitoring device is provided, the device including a data acquisition unit, a key point detection unit, a feature extraction unit, and a processing unit;
[0042] The acquisition unit is used to acquire multiple consecutive frames of flight images from the flight video of the target migratory bird to obtain a flight target sequence image of the target migratory bird;
[0043] The key point detection unit is used to detect the coordinates of key points of the target migratory bird in each frame of the flight image sequence using a detection model, and generate a set of key point coordinates for the corresponding frame.
[0044] The feature extraction unit is used to determine the static attitude and flight attitude changes of the target migratory bird based on the set of key point coordinates of consecutive frames within the time window W, and to obtain a static attitude parameter vector and a temporal flight feature vector.
[0045] The processing unit is configured to compare the static attitude parameter vector and the temporal flight feature vector with the corresponding parameters in a preset parameter library to obtain a comparison result; and to determine the difference between the actual flight attitude and the normal flight attitude of the target migratory bird based on the comparison result, thereby obtaining the flight health index of the target migratory bird.
[0046] The computer vision-based method and apparatus for monitoring the health of migratory birds in flight provided in this application extracts static and dynamic posture parameters of migratory birds from video sequences, enabling feature analysis results to take into account the dynamic and static multi-dimensional features of migratory bird postures, thereby improving the accuracy of health monitoring results. Furthermore, since computer vision technology is introduced for analysis and monitoring, there is no need to capture migratory birds or attach equipment. This not only avoids intervention in the migratory bird's living environment and reduces interference with the migratory bird's ecological environment, but also obtains the real parameters of migratory birds in their natural state, thereby improving the accuracy of analysis and monitoring results. At the same time, it can also achieve continuous monitoring of large-scale populations, significantly improving monitoring efficiency and coverage.
[0047] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description
[0048] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application, and do not constitute an undue limitation of this application.
[0049] Figure 1 A schematic flowchart of a computer vision-based method for monitoring the flight health of migratory birds, provided for an embodiment of this application;
[0050] Figure 2 This is a flowchart illustrating step S103 of the computer vision-based migratory bird flight health monitoring method in this application embodiment.
[0051] Figure 3 This is a flowchart illustrating step S105 of the computer vision-based migratory bird flight health monitoring method in an embodiment of this application.
[0052] Figure 4 This is a schematic diagram of a computer vision-based migratory bird flight health monitoring device provided in an embodiment of this application. Detailed Implementation
[0053] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0054] As an important component of the global ecosystem, the health status of migratory birds directly reflects the quality of the regional ecological environment. Their trans-regional migration makes them key vectors for infectious diseases such as avian influenza. Timely identification of abnormal flight health in migratory birds is of great significance for infectious disease control and biodiversity conservation. Traditional methods for monitoring migratory bird health mainly fall into two categories: manual monitoring and technical monitoring. Manual monitoring relies primarily on visual judgment, which suffers from low efficiency and strong subjectivity, making it difficult to achieve continuous monitoring of large-scale populations. Technical monitoring methods often employ GPS devices and sensors, which are prone to interfering with the birds' condition, and the equipment is expensive and has limited battery life, making them unsuitable for monitoring large-scale wild migratory bird populations.
[0055] To address these issues, computer vision technology can be introduced to analyze the health status of migratory birds by extracting posture parameters from videos. However, current technologies, when applied to migratory bird posture analysis, often rely solely on a single posture parameter or dynamic feature to identify anomalies. They fail to construct a fusion of static and dynamic features, or a multi-dimensional collaborative feature set. This approach cannot capture subtle temporal deviations such as wing movement symmetry and posture change smoothness in the early stages of health anomalies, and also ignores the inherent relationships between features. This results in low anomaly identification accuracy and high false positive and false negative rates, indicating a somewhat one-sided approach to feature extraction. Furthermore, existing benchmarks are mostly based on samples from migratory birds in a single flight mode, failing to fully consider the differences in the bird's posture baseline under different modes, leading to limitations in benchmark design. Limitations: Furthermore, existing methods largely rely on general biological attitude detection frameworks, lacking customized designs tailored to the physiological structure and flight mechanics of migratory birds. Key point definitions are often limited to basic areas such as the head, trunk, and wingtips, failing to cover crucial feature points for flight attitude control, such as tail feather tips and wing roots. Moreover, attitude parameter calculations do not align with the unique flight mechanisms of migratory birds, such as wingspan changes and tail flapping adjustments, making it difficult to accurately capture subtle attitude distortions caused by health abnormalities and hindering accurate early identification of health anomalies. Therefore, there is an urgent need for a migratory bird flight attitude assessment method that can consider both static and dynamic characteristics, adapt to various flight modes, and accurately quantify health abnormalities. This would enable early warning of migratory bird flight health and provide technical support for ecological protection and infectious disease control.
[0056] Based on this, in various embodiments of this application, by extracting static and dynamic posture parameters of migratory birds from video sequences, the feature analysis results can take into account the dynamic and static multi-dimensional features of migratory bird postures, thereby improving the accuracy of health monitoring results. Furthermore, since computer vision technology is introduced for analysis and monitoring, there is no need to capture migratory birds or attach devices. This not only avoids interfering with the migratory bird's living environment and reduces interference with the migratory bird's ecological environment, but also obtains the real parameters of migratory birds in their natural state, thereby improving the accuracy of analysis and monitoring results. At the same time, it also enables continuous monitoring of large-scale populations, significantly improving monitoring efficiency and coverage.
[0057] This application provides a computer vision-based method for monitoring the health of migratory birds during flight, such as... Figure 1 As shown, the method may include S101 to S105; S101 to S105 will be described in detail below with reference to specific embodiments.
[0058] S101: Obtain multiple consecutive frames of flight images from the flight video of the target migratory bird to obtain a flight target sequence image of the target migratory bird.
[0059] Specifically, a flight video sequence V of migratory birds can be acquired first. Then, the video sequence V can be processed using an image stabilization algorithm based on optical flow field estimation and a target tracking algorithm based on YOLOv12-DeepSORT to obtain a flight target sequence image of T consecutive frames containing only a single target migratory bird. That is, a sequence of images of flying targets, in which, , This represents the flight image of frame t. .
[0060] In practical applications, the target migratory bird can also be referred to as the migratory bird to be evaluated.
[0061] S102: For each frame of the flight image in the flight target sequence image, the coordinates of key points of the target migratory bird in the flight image are detected by the detection model, and a set of key point coordinates of the corresponding frame is generated.
[0062] In one embodiment, the key points include the left wingtip, left wing root, right wingtip, right wing root, head, neck, trunk center, tail center, left tail feather tip, and right tail feather tip.
[0063] Here, by using key feature points such as tail feather tips and wing roots, which are crucial for flight attitude control, for detection and analysis, the comprehensiveness and reliability of the analysis parameters are improved, thereby enhancing the accuracy and reliability of attitude detection results. This enables precise capture of subtle attitude distortion patterns caused by health abnormalities, achieving accurate early identification of health abnormalities.
[0064] In practical applications, the sequence of images of the flying target can be prepared first. Input a pre-trained detection model, which detects each frame of a sequence of images of flying targets. Keypoint detection and analysis are performed, and the coordinates of N key points on the body of the target migratory bird are output, resulting in a set of two-dimensional coordinates, i.e., the keypoint coordinate set. , .
[0065] In practical applications, the detection model can also be called a migratory bird posture detection model, or a migratory bird posture detection network.
[0066] In one embodiment, the detection model includes an ST-OverloCK backbone network that incorporates a cross-frame temporal attention mechanism; the ST-OverloCK backbone network is constructed by connecting a lightweight temporal information encoding module in parallel to the Focus-Net module of the biomimetic vision backbone network OverloCK; the temporal information encoding module is used to transfer and update contextual information between feature maps of consecutive frames using ConvLSTM.
[0067] In practical applications, the timing information encoding module can be configured with 256 input channels, 128 hidden layer channels, and 2 layers.
[0068] Here, by using a parallel temporal information encoding module, the shortcomings of the original OverloCK backbone network, which only focuses on single-frame static feature extraction and lacks temporal correlation capture, can be overcome. This allows for the full exploitation of cross-frame contextual information in migratory bird flight sequence images, making attitude key point detection more closely match the dynamic characteristics of continuous flight of migratory birds and improving the accuracy of analysis results.
[0069] S103: Based on the set of key point coordinates of consecutive frames within the time window W, determine the static attitude and flight attitude changes of the target migratory bird, and obtain the static attitude parameter vector and the temporal flight feature vector.
[0070] In one embodiment, such as Figure 2 As shown, the static attitude and flight attitude changes of the target migratory bird are determined based on the set of key point coordinates of consecutive frames within the time window W, resulting in a static attitude parameter vector and a temporal flight feature vector, i.e., S103, which may include S201 to S203. The following describes S201 to S203 in detail with reference to specific embodiments.
[0071] S201: For each frame of image within the time window W, generate the skeleton model of the target migratory bird in the corresponding frame based on the corresponding set of key point coordinates.
[0072] In practical applications, the key point coordinates of each frame of the image can be used as a reference. Construct a skeleton model of the target migratory bird in the corresponding frame; the skeleton model can also be called a migratory bird skeleton model, and this application embodiment does not limit it, as long as it can achieve its function.
[0073] S202: Based on the skeleton model, determine the static attitude parameters of the target migratory bird in the corresponding frame to obtain the corresponding static attitude parameter vector; the static attitude parameter vector includes wingspan length, left wingspan angle, right wingspan angle, trunk pitch angle, roll angle and tail opening angle.
[0074] In practical applications, based on the coordinates of each key point in the skeleton model, the static pose parameters of the frame can be calculated, resulting in the corresponding static pose parameter vector. Static attitude parameter vector It can be represented as Among them, static attitude parameters may include wingspan. Left wing angle Right wing angle Torso pitch angle Roll angle and tail opening angle .
[0075] In one embodiment, the method may further include determining the static attitude parameters in the static attitude parameter vector using the following method:
[0076] The wingspan length is determined based on the positional difference between the left and right wingtips;
[0077] The left wing span angle is determined based on the angle between the left wing vector and the torso vector, and the right wing span angle is determined based on the angle between the right wing vector and the torso vector.
[0078] The torso pitch angle is determined based on the angle between the torso vector and the horizontal direction;
[0079] The roll angle is determined based on the ratio of the vertical position difference between the left and right wingtips to the wingspan.
[0080] The tail opening angle is determined based on the angle formed by the left tail feather tip, the center of the tail, and the right tail feather tip.
[0081] In practical applications, wingspan length The calculation formula can be expressed as: ;in, Indicates the coordinates of the left wingtip. This indicates the coordinates of the right wingtip.
[0082] In practical applications, the left wing span angle It could be the angle between the left wing vector and the torso vector, or the right wing span angle. It could be the angle between the right wing vector and the torso vector.
[0083] In practical applications, the torso pitch angle It can be the angle between the torso vector and the horizontal direction.
[0084] In practical applications, roll angle It can be determined by the ratio of the vertical position difference between the left and right wingtips to the wingspan.
[0085] In practical applications, the tail opening angle It can be determined by the angle formed by the three points: the tip of the left tail feather, the center of the tail, and the tip of the right tail feather.
[0086] In practical applications, S201 and S202 can be understood as the process of extracting static attitude parameters.
[0087] S203: Perform time-series analysis on the static attitude parameter vectors of consecutive frames to determine the flight attitude changes of the target migratory bird and obtain the time-series flight feature vector of the target migratory bird; the time-series flight feature vector includes flapping frequency, left and right wing flapping phase difference, attitude parameter change rate, attitude change smoothness, and trajectory curvature.
[0088] In practical applications, a non-overlapping sliding strategy can be used to transfer the acquired T-frame flight target sequence images. The time window is divided according to its length W; for example, the time window W can be set to 60 frames, resulting in a sequence of images of the flying target. The frame rate is configured to 300 frames per second (T=300). The 300-frame sequence is divided into 60-frame / window segments using a non-overlapping sliding strategy. Within the time window... Within, it can process continuous static attitude parameter sequences. Perform time series analysis to extract time-series flight feature vectors Time-series flight feature vector Flight characteristic parameters include flapping frequency Phase difference of left and right wing flapping Rate of change of attitude parameters Smoothness of posture changes and trajectory curvature .
[0089] In one embodiment, the method further includes determining flight feature parameters in the time-series flight feature vector using the following method:
[0090] The flapping frequency is determined based on the fast Fourier transform results of the wingspan angle sequence;
[0091] The phase difference of the left and right wing flapping is determined based on the difference in the peak position of the cross-correlation function between the left and right wing span angle sequences.
[0092] The rate of change of the attitude parameters is determined based on the mean of the magnitude of the first time derivative of the static attitude parameter vector.
[0093] The smoothness of the attitude change is determined based on the integral norm of the second-order time derivative of the static attitude parameter vector.
[0094] The trajectory curvature is determined based on the coordinate changes of the sequence of key points at the center of the torso within the time window W.
[0095] In practical applications, the flapping frequency It can be achieved by analyzing the left wing span sequence The dominant frequency is obtained by applying the Fast Fourier Transform (FFT) and is expressed as follows: .
[0096] In practical applications, the phase difference between the flapping of the left and right wings The left wing span sequence can be calculated and right wing span sequence The peak position of the cross-correlation function is determined.
[0097] In practical applications, the rate of change of attitude parameters It can be a static attitude parameter vector The mean of the magnitudes of the first-order time derivatives.
[0098] In practical applications, the smoothness of attitude changes It can be derived from the static attitude parameter vector. The integral norm of the second-order time derivative is expressed as follows: .
[0099] In practical applications, trajectory curvature Based on time window The coordinates of the sequence of key points at the center of the inner torso were calculated.
[0100] Here, by configuring 10 key attitude points and a parameter calculation method adapted to flight mechanics when detecting the static attitude of migratory birds, and combining it with the ST-OverloCK backbone network, the accuracy of attitude detection and parameter extraction is improved, adapting to the unique flight mechanism of migratory birds. At the same time, by taking into account both static attitude parameters and temporal dynamic features, multi-dimensional features are constructed, which can accurately capture dynamic features such as spatial attitude, flapping rhythm, and motion symmetry, so as to identify subtle attitude changes corresponding to early minor health abnormalities and reduce the false negative rate.
[0101] S104: Compare the static attitude parameter vector and the time-series flight feature vector with the corresponding parameters in the preset parameter library to obtain the comparison result.
[0102] In practical applications, healthy samples can be collected first to prepare data, and the detection model can be trained and a preset parameter library can be constructed. The preset parameter library can also be called the normal flight reference parameter library or the reference parameter library.
[0103] Specifically, high-definition cameras can be used to collect flight videos of various common migratory birds in migratory bird nature reserves, such as egrets, cranes and geese. The video resolution is set to 1920×1080 and the frame rate is set to 30fps. 1,000 healthy samples are collected, and each sample lasts for 8 to 15 seconds.
[0104] In practical applications, after the collection of healthy samples is completed, a reference parameter library for normal flight can be constructed based on the migratory bird flight samples. That is, the preset parameter library.
[0105] In practical applications, when constructing a preset parameter library, the flight mode types of migratory birds in healthy samples can be expanded to cover diverse healthy sample postures, thereby improving the accuracy of migratory bird flight posture analysis results.
[0106] Based on this, in one embodiment, the preset parameter library includes health sample parameters for various typical flight modes; the typical flight modes include cruise, hovering, takeoff, and landing.
[0107] In practical applications, the number of samples for each flight mode can be set to no less than 240, and the sample size for each migratory bird should be evenly distributed.
[0108] Here, by configuring the reference parameter library to cover multiple flight modes and diverse health samples, and combining it with Gaussian mixture model statistical modeling, misjudgments caused by a single-mode benchmark can be avoided, thereby improving the accuracy of attitude detection and analysis results.
[0109] In practical applications, after obtaining healthy samples, the acquired video can be manually labeled frame by frame to identify key points. Then, the YOLOv12-DeepSORT and ST-OverloCK pose estimation networks can be trained using the labeled dataset to obtain the trained model. The training parameters can be set as follows: batch size of 16, initial learning rate of 1e-4, optimizer of AdamW, weight decay coefficient of 1e-5, training epoch of 100, and early stopping strategy with patience=10. During training, methods such as random flipping, brightness adjustment, and slight rotation can be used to improve the model's generalization ability.
[0110] In practical applications, the extracted healthy samples can be processed by a trained detection model to extract effective data windows to construct a normal flight reference parameter library. For example, 20,000 valid data windows can be extracted to build a normal flight reference parameter library. This refers to a pre-defined parameter library; specifically, it allows for the extraction of static pose parameter vectors from each healthy sample. and time-series flight feature vector Statistical modeling was performed on these data to construct a normal flight reference parameter library containing parameters of a multivariate Gaussian mixture model. .
[0111] Specifically, parameter library The following statistical parameters can be stored as a normal baseline:
[0112] The overall mean vector of static attitude parameters and the overall covariance matrix ;
[0113] flapping frequency overall mean and standard deviation ;
[0114] Beat phase difference overall mean and standard deviation ;
[0115] attitude change rate overall mean and standard deviation ;
[0116] Smoothness of attitude change overall mean and standard deviation ;
[0117] trajectory curvature overall mean and standard deviation .
[0118] In practical applications, after building a preset parameter library, when executing S104, the baseline parameters in the preset parameter library can be compared with the actually obtained parameters.
[0119] S105: Based on the comparison results, determine the difference between the actual flight posture and the normal flight posture of the target migratory bird, and obtain the flight health index of the target migratory bird.
[0120] In practical applications, for target migratory birds, their feature vectors within a time window W can be obtained, i.e., static attitude parameter vectors. and time-series flight feature vector and with the preset parameter library Compare and calculate the abnormal deviation index.
[0121] In practical applications, the multi-parameter deviation comparison results between the baseline parameters and the actual acquired parameters can be used to obtain the multi-dimensional abnormal deviation index analysis results, thereby obtaining the flight health index of the target migratory bird.
[0122] Based on this, in one embodiment, such as Figure 3 As shown, based on the comparison results, the difference between the actual flight posture and the normal flight posture of the target migratory bird is determined, and the flight health index of the target migratory bird, namely S105, is obtained, which may include S301 to S305; S301 to S305 will be described in detail below with reference to specific embodiments.
[0123] S301: Determine the attitude parameter deviation based on the deviation between the static attitude parameter vector and the reference static attitude parameter vector. .
[0124] In practical applications, Mahalanobis distance can be used to calculate the deviation of attitude parameters. The calculation formula can be expressed as: = .
[0125] S302: Determine the degree of symmetry deviation based on span deviation and phase deviation. The aspect ratio deviation represents the deviation between the left wing aspect ratio and the right wing aspect ratio, and the phase deviation represents the deviation between the flapping phase difference of the left and right wings and the reference phase difference.
[0126] In practical applications, the span of the left wing can be adjusted. With right wing angle Differences and beat phase differences Deviation from normal range Weighted calculation, based on the weighted results, calculates the degree of symmetry deviation. Specifically, the calculation formula can be expressed as: in , The preset weighting coefficients, This represents the average left and right wingspan angles within the time window W.
[0127] S303: Determine the rhythmic deviation based on the stability of the flapping wing frequency and the deviation between the flapping wing frequency and the reference frequency. .
[0128] In practical applications, it can be based on the flapping frequency. Compared with the normal population mean in the preset parameter library The difference determines the flapping frequency The stability, and based on the flapping frequency Stability and its relationship with normal frequency range Deviation calculation of rhythmic deviation The specific calculation formula can be expressed as: .
[0129] S304: Determine the stability deviation based on the smoothness of the attitude change and the deviation of the trajectory curvature from the normal baseline. .
[0130] In practical applications, it can be based on the smoothness of attitude changes. and trajectory curvature The deviation from the normal baseline is used to calculate the stability deviation. The specific calculation formula can be expressed as: ,in These are the preset weighting coefficients.
[0131] S305: Based on the deviation of the attitude parameters The aforementioned symmetry deviation The rhythmic deviation The stability deviation The flight health index of the target migratory bird is determined using a preset weight vector.
[0132] In practical applications, a preset weight vector can be configured in advance; for the preset weight vector, it can be dynamically adjusted based on the static attitude parameters extracted by S103. This yields the actual weight vector, i.e., the target weight vector.
[0133] Therefore, in one embodiment, the deviation based on the attitude parameters... The aforementioned symmetry deviation The rhythmic deviation The stability deviation Using a preset weight vector, the flight health index of the target migratory bird, i.e., S305, can be determined, which may include:
[0134] Based on the static attitude parameter vector, determine the static attitude parameters that deviate from the reference range, and obtain the deviation static attitude parameters;
[0135] Based on the attitude parameter deviation The aforementioned symmetry deviation The rhythmic deviation and the stability deviation The correlation between each deviation index and the deviation from the static attitude parameter is analyzed, and the preset weight vector is dynamically updated to obtain the target weight vector.
[0136] Based on the target weight vector, , , and Linear weighting is applied to obtain the flight health index of the target migratory birds; where,
[0137] Dynamically update the preset weight vector, including increasing the weight of the deviation index associated with the deviation from the static attitude parameters in the preset weight vector.
[0138] In practical applications, the acquired static attitude parameters can be compared with the normal static attitude parameters for the corresponding flight mode in the preset parameter library to determine whether each static attitude parameter deviates from the normal reference range. For static attitude parameters that deviate from the normal reference range, i.e., deviate from the static attitude parameter, the weight of the corresponding deviation index in the preset weight vector can be increased. Conversely, for deviation indices not associated with the deviated static attitude parameter, their weight can be decreased, thereby updating the preset weight vector and obtaining the target weight vector. ,in, , , , They are respectively , , and The target weight.
[0139] In practical applications, after determining the target weight vector, the four deviation indicators can be linearly weighted according to the target weight vector to obtain the flight health index. The specific calculation formula can be expressed as: ,in, , Deviation index The deviation value after normalization.
[0140] Here, the health index is calculated by weighting and integrating four deviation indicators, which enables the quantitative assessment and classification of abnormal risks. At the same time, by dynamically adjusting the weights, the accuracy of the weighted results can be improved, making the assessment results more targeted and practical while adapting to different flight modes.
[0141] In practical applications, the flight status of migratory birds can be pre-configured into four risk levels: "healthy", "low risk", "medium risk" and "high risk" based on the preset threshold range of the flight health index H. For example, for the threshold range of [0,40), [40,60), [60,80), and [80,100] where the flight health index H is located, the flight status of migratory birds can be configured into four risk levels: "healthy", "low risk", "medium risk" and "high risk" respectively. After determining the flight health index H of the target migratory bird according to S105, the corresponding risk level can be determined according to the range where the flight health index H is located, thereby judging whether the flight health of the target migratory bird is abnormal.
[0142] In summary, the computer vision-based migratory bird flight health monitoring method provided in this application extracts static and dynamic posture parameters of migratory birds from video sequences, enabling feature analysis results to take into account the dynamic and static multi-dimensional features of migratory bird postures, thereby improving the accuracy of health monitoring results. Furthermore, since computer vision technology is used for analysis and monitoring, there is no need to capture migratory birds or attach equipment. This not only avoids intervention in the migratory bird's living environment and reduces interference with the migratory bird's ecological environment, but also obtains the real parameters of migratory birds in their natural state, thereby improving the accuracy of analysis and monitoring results. At the same time, it can also achieve continuous monitoring of large-scale populations, significantly improving monitoring efficiency and coverage.
[0143] To realize the computer vision-based migratory bird flight health monitoring method of this application, embodiments of this application also provide a computer vision-based migratory bird flight health monitoring device, such as... Figure 4 As shown, the device may include an acquisition unit 401, a key point detection unit 402, a feature extraction unit 403, and a processing unit 404.
[0144] The acquisition unit 401 can be used to acquire multiple consecutive frames of flight images from the flight video of the target migratory bird to obtain a flight target sequence image of the target migratory bird.
[0145] The key point detection unit 402 can be used to detect the coordinates of key points of target migratory birds in each frame of the flight image sequence using a detection model, and generate a set of key point coordinates for the corresponding frame.
[0146] The feature extraction unit 403 is used to determine the static attitude and flight attitude changes of the target migratory bird based on the set of key point coordinates of consecutive frames within the time window W, and to obtain the static attitude parameter vector and the temporal flight feature vector.
[0147] The processing unit 404 can be used to compare the static attitude parameter vector and the temporal flight feature vector with the corresponding parameters in the preset parameter library to obtain the comparison result; and, based on the comparison result, to determine the difference between the actual flight attitude and the normal flight attitude of the target migratory bird, and obtain the flight health index of the target migratory bird.
[0148] In one embodiment, the key points include the left wingtip, left wing root, right wingtip, right wing root, head, neck, trunk center, tail center, left tail feather tip, and right tail feather tip.
[0149] In one embodiment, the detection model includes an ST-OverloCK backbone network that incorporates a cross-frame temporal attention mechanism; the ST-OverloCK backbone network is constructed by connecting a lightweight temporal information encoding module in parallel to the Focus-Net module of the biomimetic vision backbone network OverloCK; the temporal information encoding module is used to transfer and update contextual information between feature maps of consecutive frames using ConvLSTM.
[0150] In one embodiment, the feature extraction unit 403 may specifically be used for:
[0151] For each frame of image within a time window W, a skeleton model of the target migratory bird in the corresponding frame is generated based on the corresponding set of key point coordinates.
[0152] Based on the skeleton model, the static attitude parameters of the target migratory bird in the corresponding frame are determined, and the corresponding static attitude parameter vector is obtained; the static attitude parameter vector includes wingspan length, left wingspan angle, right wingspan angle, trunk pitch angle, roll angle and tail opening angle;
[0153] A temporal analysis is performed on the static attitude parameter vectors of consecutive frames to determine the flight attitude changes of the target migratory bird, and the temporal flight feature vector of the target migratory bird is obtained. The temporal flight feature vector includes flapping frequency, left and right wing flapping phase difference, attitude parameter change rate, attitude change smoothness, and trajectory curvature.
[0154] In one embodiment, the feature extraction unit 403 can also be used to determine the static attitude parameters in the static attitude parameter vector using the following method:
[0155] The wingspan length is determined based on the positional difference between the left and right wingtips;
[0156] The left wing span angle is determined based on the angle between the left wing vector and the torso vector, and the right wing span angle is determined based on the angle between the right wing vector and the torso vector.
[0157] The torso pitch angle is determined based on the angle between the torso vector and the horizontal direction;
[0158] The roll angle is determined based on the ratio of the vertical position difference between the left and right wingtips to the wingspan.
[0159] The tail opening angle is determined based on the angle formed by the left tail feather tip, the center of the tail, and the right tail feather tip.
[0160] In one embodiment, the feature extraction unit 403 can also be used to determine the flight feature parameters in the time-series flight feature vector using the following method:
[0161] The flapping frequency is determined based on the fast Fourier transform results of the wingspan angle sequence;
[0162] The phase difference of the left and right wing flapping is determined based on the difference in the peak position of the cross-correlation function between the left and right wing span angle sequences.
[0163] The rate of change of the attitude parameters is determined based on the mean of the magnitude of the first time derivative of the static attitude parameter vector.
[0164] The smoothness of the attitude change is determined based on the integral norm of the second-order time derivative of the static attitude parameter vector.
[0165] The trajectory curvature is determined based on the coordinate changes of the sequence of key points at the center of the torso within the time window W.
[0166] In one embodiment, the processing unit 404 may specifically be used for:
[0167] Based on the deviation between the static attitude parameter vector and the reference static attitude parameter vector, the attitude parameter deviation is determined. ;
[0168] Determine the degree of symmetry deviation based on span deviation and phase deviation. The aspect ratio deviation represents the deviation between the left wing aspect ratio and the right wing aspect ratio, and the phase deviation represents the deviation between the flapping phase difference of the left and right wings and the reference phase difference.
[0169] Based on the stability of the flapping wing frequency and the deviation of the flapping wing frequency from the reference frequency, the rhythmic deviation is determined. ;
[0170] Based on the smoothness of the attitude change and the deviation of the trajectory curvature from the normal baseline, the stability deviation is determined. ;
[0171] Based on the attitude parameter deviation The aforementioned symmetry deviation The rhythmic deviation The stability deviation The flight health index of the target migratory bird is determined using a preset weight vector.
[0172] In one embodiment, the processing unit 404 may specifically be used for:
[0173] Based on the static attitude parameter vector, determine the static attitude parameters that deviate from the reference range, and obtain the deviation static attitude parameters;
[0174] Based on the attitude parameter deviation The aforementioned symmetry deviation The rhythmic deviation and the stability deviation The correlation between each deviation index and the deviation from the static attitude parameter is analyzed, and the preset weight vector is dynamically updated to obtain the target weight vector.
[0175] Based on the target weight vector, , , and Linear weighting is applied to obtain the flight health index of the target migratory birds; where,
[0176] Dynamically update the preset weight vector, including increasing the weight of the deviation index associated with the deviation from the static attitude parameters in the preset weight vector.
[0177] In one embodiment, the preset parameter library includes health sample parameters for various typical flight modes; the typical flight modes include cruise, hovering, takeoff, and landing.
[0178] It should be noted that the above-described migratory bird flight health monitoring device based on computer vision is only illustrated by the division of the above-described program modules when performing migratory bird flight health monitoring based on computer vision. In practical applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the processing described above. In addition, the migratory bird flight health monitoring device based on computer vision provided in the above-described embodiments and the migratory bird flight health monitoring method embodiments based on computer vision belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be repeated here.
[0179] It should be noted that terms such as "first" and "second" are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.
[0180] Furthermore, the technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.
[0181] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application.
Claims
1. A method for monitoring the health of migratory birds during flight based on computer vision, characterized in that, The method includes: Multiple consecutive flight images are obtained from the flight video of the target migratory bird to obtain the flight target sequence image of the target migratory bird; For each frame of the flight image sequence, the coordinates of key points of the target migratory bird in the flight image are detected using a detection model, and a set of key point coordinates for the corresponding frame is generated. Based on the set of key point coordinates of consecutive frames within the time window W, the static attitude and flight attitude changes of the target migratory bird are determined, and the static attitude parameter vector and the temporal flight feature vector are obtained. The static attitude parameter vector and the time-series flight feature vector are compared with the corresponding parameters in the preset parameter library to obtain the comparison results; Based on the comparison results, the difference between the actual flight posture and the normal flight posture of the target migratory bird is determined, and the flight health index of the target migratory bird is obtained.
2. The method according to claim 1, characterized in that, The key points include the left wingtip, left wing root, right wingtip, right wing root, head, neck, center of the trunk, center of the tail, left tail feather tip, and right tail feather tip.
3. The method according to claim 1, characterized in that, The detection model includes the ST-OverloCK backbone network, which incorporates a cross-frame temporal attention mechanism. The ST-OverloCK backbone network is constructed by connecting a lightweight temporal information encoding module in parallel with the Focus-Net module of the biomimetic vision backbone network OverloCK. The temporal information encoding module is used to transfer and update contextual information between feature maps of consecutive frames using ConvLSTM.
4. The method according to claim 1, characterized in that, The set of keypoint coordinates based on consecutive frames within a time window W determines the static attitude and flight attitude changes of the target migratory bird, resulting in a static attitude parameter vector and a temporal flight feature vector, including: For each frame of image within a time window W, a skeleton model of the target migratory bird in the corresponding frame is generated based on the corresponding set of key point coordinates. Based on the skeleton model, the static attitude parameters of the target migratory bird in the corresponding frame are determined, and the corresponding static attitude parameter vector is obtained; the static attitude parameter vector includes wingspan length, left wingspan angle, right wingspan angle, trunk pitch angle, roll angle and tail opening angle; A temporal analysis is performed on the static attitude parameter vectors of consecutive frames to determine the flight attitude changes of the target migratory bird, and the temporal flight feature vector of the target migratory bird is obtained. The temporal flight feature vector includes flapping frequency, left and right wing flapping phase difference, attitude parameter change rate, attitude change smoothness, and trajectory curvature.
5. The method according to claim 4, characterized in that, The method further includes determining the static attitude parameters in the static attitude parameter vector using the following method: The wingspan length is determined based on the positional difference between the left and right wingtips; The left wing span angle is determined based on the angle between the left wing vector and the torso vector, and the right wing span angle is determined based on the angle between the right wing vector and the torso vector. The torso pitch angle is determined based on the angle between the torso vector and the horizontal direction; The roll angle is determined based on the ratio of the vertical position difference between the left and right wingtips to the wingspan. The tail opening angle is determined based on the angle formed by the left tail feather tip, the center of the tail, and the right tail feather tip.
6. The method according to claim 4, characterized in that, The method further includes determining the flight feature parameters in the temporal flight feature vector using the following method: The flapping frequency is determined based on the fast Fourier transform results of the wingspan angle sequence; The phase difference of the left and right wing flapping is determined based on the difference in the peak position of the cross-correlation function between the left and right wing span angle sequences. The rate of change of the attitude parameters is determined based on the mean of the magnitude of the first time derivative of the static attitude parameter vector. The smoothness of the attitude change is determined based on the integral norm of the second-order time derivative of the static attitude parameter vector. The trajectory curvature is determined based on the coordinate changes of the sequence of key points at the center of the torso within the time window W.
7. The method according to claim 1, characterized in that, Based on the comparison results, the difference between the actual flight posture and the normal flight posture of the target migratory bird is determined, and the flight health index of the target migratory bird is obtained, including: Based on the deviation between the static attitude parameter vector and the reference static attitude parameter vector, the attitude parameter deviation is determined. ; Determine the degree of symmetry deviation based on span deviation and phase deviation. The aspect ratio deviation represents the deviation between the left wing aspect ratio and the right wing aspect ratio, and the phase deviation represents the deviation between the flapping phase difference of the left and right wings and the reference phase difference. Based on the stability of the flapping wing frequency and the deviation of the flapping wing frequency from the reference frequency, the rhythmic deviation is determined. ; Based on the smoothness of the attitude change and the deviation of the trajectory curvature from the normal baseline, the stability deviation is determined. ; Based on the attitude parameter deviation The aforementioned symmetry deviation The rhythmic deviation The stability deviation The flight health index of the target migratory bird is determined using a preset weight vector.
8. The method according to claim 7, characterized in that, The deviation based on the attitude parameters The aforementioned symmetry deviation The rhythmic deviation The stability deviation Using a preset weight vector, the flight health index of the target migratory bird is determined, including: Based on the static attitude parameter vector, determine the static attitude parameters that deviate from the reference range, and obtain the deviation static attitude parameters; Based on the attitude parameter deviation The aforementioned symmetry deviation The rhythmic deviation and the stability deviation The correlation between each deviation index and the deviation from the static attitude parameter is analyzed, and the preset weight vector is dynamically updated to obtain the target weight vector. Based on the target weight vector, , , and Linear weighting is applied to obtain the flight health index of the target migratory birds; where, Dynamically update the preset weight vector, including increasing the weight of the deviation index associated with the deviation from the static attitude parameters in the preset weight vector.
9. The method according to claim 1, characterized in that, The preset parameter library includes health sample parameters for various typical flight modes; the typical flight modes include cruise, hovering, takeoff, and landing.
10. A computer vision-based migratory bird flight health monitoring device, characterized in that, The device includes an acquisition unit, a key point detection unit, a feature extraction unit, and a processing unit; The acquisition unit is used to acquire multiple consecutive frames of flight images from the flight video of the target migratory bird to obtain a flight target sequence image of the target migratory bird; The key point detection unit is used to detect the coordinates of key points of the target migratory bird in each frame of the flight image sequence using a detection model, and generate a set of key point coordinates for the corresponding frame. The feature extraction unit is used to determine the static attitude and flight attitude changes of the target migratory bird based on the set of key point coordinates of consecutive frames within the time window W, and to obtain a static attitude parameter vector and a temporal flight feature vector. The processing unit is configured to compare the static attitude parameter vector and the temporal flight feature vector with the corresponding parameters in a preset parameter library to obtain a comparison result; and to determine the difference between the actual flight attitude and the normal flight attitude of the target migratory bird based on the comparison result, thereby obtaining the flight health index of the target migratory bird.