An asian elephant motion amount visual measurement method and system based on human-computer collaborative calibration and key point fusion

Abstract

The application discloses an Asian elephant motion amount visual measurement method and system based on man-machine cooperative calibration and key point fusion, and the method comprises the following steps: acquiring artificial line segments and automatic line segments in an Asian elephant motion scene image and forming a line segment set, estimating vanishing points of the line segments in the line segment set, and determining a camera projection matrix based on the estimated vanishing points; inputting multiple video clips into a pre-trained Asian elephant detection model to obtain a detection frame and key point coordinates; classifying the detection frame based on a pre-set confidence threshold, matching existing tracking trajectories with the classified detection frame, and if the matching is successful, fusing corresponding key point coordinates to obtain fused key point pixel coordinates and generating a target pixel trajectory; converting the fused key point pixel coordinates into corresponding physical coordinates based on the camera projection matrix, and calculating the motion amount of a target Asian elephant based on the target pixel trajectory. The application can accurately acquire Asian elephant motion behavior data without contact.

Description

Technical Field

[0001] This invention relates to image processing, computer vision, and pattern recognition, and belongs to the field of Asian elephant motion measurement technology. In particular, it relates to a visual measurement method and system for Asian elephant motion based on human-computer collaborative calibration and key point fusion. Background Technology

[0002] In the management of zoos and wildlife sanctuaries, the health monitoring of Asian elephants is directly related to their welfare level and conservation effectiveness, and the amount of exercise is an important indicator reflecting the physical vitality and health status of Asian elephants. Manual observation is labor-intensive and cannot be conducted continuously, therefore, it is necessary to introduce unmanned, precise, and reliable visual monitoring methods.

[0003] Compared to methods such as wearing sensors, computer vision technology has advantages such as being contactless, low-cost, and easy to understand, and has become the main method for monitoring animal behavior. The core of Asian elephant movement statistics is the recognition of movement behavior and accurate measurement of trajectory, but existing methods have the following problems: First, Asian elephants are highly social animals, and individuals frequently engage in social behaviors such as physical contact and close walking, causing severe occlusion, leading to target loss or identity changes; second, cameras are usually deployed at high positions, resulting in severe perspective distortion in the images, and directly relying on pixel displacement statistics will produce huge errors; third, the subtle physiological movements of Asian elephants when standing still, such as ear flicking and breathing, will produce invalid displacement noise, and without a precise filtering mechanism, it will be impossible to obtain true movement data. Summary of the Invention

[0004] To address the aforementioned issues, the present invention aims to provide a visual measurement method and system for Asian elephant movement based on human-machine collaborative calibration and key point fusion. This method and system can acquire Asian elephant movement behavior data without contact and with high accuracy, and can be applied to Asian elephant video monitoring systems in zoos and wildlife reserves.

[0005] The present invention adopts the following technical solution.

[0006] The first aspect of this invention provides a visual measurement method for Asian elephant movement based on human-machine collaborative calibration and key point fusion, comprising the following:

[0007] Artificial and automatic line segments are acquired from images of Asian elephant movement scenes and a line segment set is constructed. The vanishing point of the line segments in the line segment set is estimated, and the camera projection matrix is ​​determined based on the estimated vanishing point.

[0008] The long video captured by the camera is cut into multiple video segments. The multiple video segments are input into a pre-trained Asian elephant detection model for frame-by-frame processing to obtain the confidence score and key point coordinates of the detection box containing the target Asian elephant.

[0009] The detection boxes are classified based on a preset confidence threshold. The existing tracking trajectory is matched with the classified detection boxes. If the match is successful, the corresponding key point coordinates are fused to obtain the fused key point pixel coordinates and generate the target pixel trajectory.

[0010] The camera projection matrix is ​​used to convert the pixel coordinates of the fused key points into corresponding physical coordinates, and the actual movement distance of the target Asian elephant is calculated based on the target pixel trajectory to obtain the movement amount of the target Asian elephant.

[0011] Optionally, acquiring artificial and automatic line segments from images of Asian elephant movement scenes and constructing a line segment set includes:

[0012] Extract valid parallel line segments from images of Asian elephant movement scenes to obtain automatic line segments;

[0013] The quality score is obtained by evaluating the quality of effective parallel line segments based on multiple evaluation indicators, including: the total number of effective parallel line segments, the number of line segments in two orthogonal directions, the line segment distribution entropy, and the angle standard deviation. Among them, the effective parallel line segments are clustered by angle to obtain the number of line segments in two orthogonal directions, and the angle standard deviation is calculated for the line segments in the same direction cluster.

[0014] If the quality score is less than the preset quality threshold, or the number of valid line segments in any orthogonal direction is less than the preset number threshold, at least two sets of parallel lines that are orthogonal in physical space shall be manually added, and at least two parallel line segments shall be drawn in each orthogonal direction.

[0015] Calculate the length weight, position distribution weight, and parallel consistency weight of the artificial line segments to obtain the comprehensive weight of each artificial line segment; cluster the automatic line segments by angle and calculate the comprehensive weight of the automatic line segments.

[0016] After normalizing the combined weights of the manually generated line segments and the automatically generated line segments, the manually generated line segments and the automatically generated line segments are merged to construct a line segment set.

[0017] Optionally, vanishing point estimation for line segments in the set of line segments includes:

[0018] Randomly select two line segments from the set of line segments and calculate their intersection point as candidate vanishing points;

[0019] For each line segment in the set of line segments, calculate the direction vector of the line connecting each endpoint to the candidate vanishing point. If the angle between the direction vector corresponding to each endpoint of the line segment and the direction of the line segment is less than the preset interior point threshold, then the line segment is determined to be an interior point line segment.

[0020] Calculate the sum of the comprehensive weights of the interior line segments of the candidate vanishing points, and update the optimal vanishing point based on the sum of the comprehensive weights until the maximum number of iterations is reached or the optimal vanishing point is not updated after a preset number of iterations.

[0021] The cost function is the sum of the weighted squared angular deviations of each interior point line segment to the optimal vanishing point. After approximating the angular deviations as geometric distance deviations, the cost function is transformed into the sum of the weighted squared distances. The weighted least squares method is then used to solve for the final estimated vanishing point.

[0022] Optionally, determining the camera projection matrix based on the estimated vanishing point includes:

[0023] By estimating the vanishing points of line segments in the set of line segments in the orthogonal directions, the first vanishing point and the second vanishing point are obtained.

[0024] Calculate the third vanishing point based on the geometric constraints of the first and second vanishing points;

[0025] Based on the first vanishing point, the second vanishing point, and the third vanishing point, the following system of equations is established:

[0026]

[0027] in, These represent the x and y coordinates of the final estimated vanishing point, respectively. , These represent the first vanishing point, respectively. x and y coordinates , These represent the second vanishing points. x and y coordinates , Represents the third vanishing point x and y coordinates , These represent the focal lengths along the x-axis and y-axis in the pixel coordinate system of a two-dimensional image, respectively.

[0028] Solving the system of equations yields the focal lengths along the x and y axes in the pixel coordinate system of the two-dimensional image. Based on the focal lengths and the final estimated vanishing point, an intrinsic parameter matrix is ​​constructed, and the rotation matrix is ​​calculated based on the intrinsic parameter matrix.

[0029] A real-world coordinate system is constructed by selecting a preset location, and the translation vector is calculated based on the camera position; the camera projection matrix is ​​constructed based on the intrinsic parameter matrix, rotation matrix, and translation vector.

[0030] Optionally, the method further includes:

[0031] Video data is collected from deployed cameras, and frames are extracted from the video data at equal intervals to form a training set;

[0032] In the training set, rectangular bounding boxes were used to label the Asian elephant as a whole, and these rectangular bounding boxes were used as detection boxes. Keypoints were also labeled for each sample to form a keypoint detection dataset.

[0033] The Asian elephant detection model based on YOLOv11-pose was trained using a keypoint detection dataset.

[0034] Optionally, the detection boxes are classified based on a preset confidence threshold, and the existing tracking trajectories are matched with the classified detection boxes, including:

[0035] Set a preset confidence threshold, classify detection boxes with confidence scores greater than or equal to the preset confidence threshold as high confidence detection boxes, and classify detection boxes with confidence scores less than the preset confidence threshold as low confidence detection boxes;

[0036] The high-confidence detection box is matched with the existing tracking trajectory for the first time; the low-confidence detection box is matched with the tracking trajectory that failed to match in the first match for the second time.

[0037] Trajectories that successfully match in two matching attempts are marked as successfully tracked. High-confidence detection boxes that still do not match after the second matching attempt are initialized as new trajectories. Trajectories that do not match detection boxes for multiple consecutive frames are marked as lost and deleted.

[0038] Specifically, the position of the existing tracking trajectory in the current frame is predicted based on Kalman filtering to obtain the predicted position. The predicted position is then matched with the Euclidean distance between the predicted position and the center point of the detection box. If the match is successful, the coordinates of the key point corresponding to the current detection box are associated with the corresponding trajectory and the Kalman filter corresponding to the trajectory is updated.

[0039] Optional key points include: left shoulder, right shoulder, left front foot, right front foot, left back foot, and right back foot.

[0040] Optionally, differentiated weights can be set for key points, and the pixel coordinates of the fused key points can be calculated by weighted average.

[0041] Optionally, the method further includes:

[0042] Store the pixel coordinates of the fusion key points in chronological order to generate pixel trajectories;

[0043] Calculate the cumulative pixel displacement value of the pixel trajectory within the preset window length. If the cumulative pixel displacement value is greater than the preset displacement threshold, the frame in the window will be determined as a motion frame. Motion frames with an interval less than the preset frame threshold will be merged to obtain a motion segment.

[0044] Coordinate transformation is performed based on the pixel coordinates of the fusion key points corresponding to all frames within the extracted motion segment.

[0045] A second aspect of the present invention provides a visual measurement system for Asian elephant motion based on human-machine collaborative calibration and key point fusion, realizing the aforementioned visual measurement method for Asian elephant motion based on human-machine collaborative calibration and key point fusion, the system comprising:

[0046] The multi-target calibration module is used to acquire a set of line segments composed of artificial and automatic line segments in the Asian elephant movement scene image, estimate the vanishing point of the line segments in the set, and determine the camera projection matrix based on the estimated vanishing point.

[0047] The detection module is used to cut the long video captured by the camera into multiple video segments, input the multiple video segments into a pre-trained Asian elephant detection model for frame-by-frame processing, and obtain the confidence score and key point coordinates of the detection box containing the target Asian elephant.

[0048] The trajectory generation module is used to classify the detection boxes based on a preset confidence threshold, match the existing tracking trajectory with the classified detection boxes, and if the match is successful, fuse the corresponding key point coordinates to obtain the fused key point pixel coordinates and generate the target pixel trajectory.

[0049] The calculation module is used to convert the pixel coordinates of the fused key points into corresponding physical coordinates based on the camera projection matrix, and to calculate the actual movement distance of the target Asian elephant based on the target pixel trajectory, so as to obtain the movement amount of the target Asian elephant.

[0050] Compared with the prior art, the beneficial effects of the present invention include at least the following:

[0051] 1. A semi-automatic camera calibration method based on human-machine collaboration is proposed. By combining automatic detection quality assessment, weighted line segment fusion and weighted RANSAC algorithm, the method solves the problem of insufficient accuracy of traditional calibration methods in complex scenes and achieves high-precision mapping from pixel coordinates to physical coordinates.

[0052] 2. The YOLOv11-pose multi-task model is used to simultaneously detect the Asian elephant as a whole and six key points. The trajectory stability is improved by using a key point fusion strategy. Kalman filtering and ByteTrack algorithm are combined to achieve anti-occlusion tracking, effectively addressing the problem of frequent occlusion in group environments.

[0053] 3. Set up a minimum displacement threshold filtering mechanism to eliminate physiological micro-motion noise when Asian elephants stand still and obtain effective movement data that truly reflects their physical vitality.

[0054] 4. The calibration process supports human interaction assistance, balancing automation and flexibility; the detection and tracking model has high computational efficiency and low hardware requirements, making it suitable for long-term continuous monitoring applications. Attached Figure Description

[0055] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0056] Figure 1 A flowchart illustrating the steps of a visual measurement method for Asian elephant motion based on human-machine collaborative calibration and key point fusion, provided for an embodiment of the present invention.

[0057] Figure 2 This is a schematic diagram of automatic calibration of an outdoor camera provided in an embodiment of the present invention;

[0058] Figure 3 This is a schematic diagram of automatic calibration of an indoor camera provided in an embodiment of the present invention;

[0059] Figure 4 A schematic diagram of a key point and trajectory point provided in an embodiment of the present invention;

[0060] Figure 5 This is a schematic diagram of an application platform architecture for a visual measurement method of Asian elephant motion based on human-machine collaborative calibration and key point fusion, provided in an embodiment of the present invention. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of this invention. The embodiments described in this application are merely some embodiments of this invention, and not all embodiments. Based on the spirit of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of this invention.

[0062] The method of this invention is divided into two stages: offline preparation and online processing. The offline preparation stage includes steps S1-S3, which are completed all at once in the initial stage of system deployment, establishing calibration parameters, training datasets, and detection models. The online processing stage includes steps S4-S9, which process the monitoring video frame by frame in a loop to achieve real-time motion statistics.

[0063] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0064] like Figure 1 As shown, this invention provides a visual measurement method for Asian elephant movement based on human-machine collaborative calibration and key point fusion. The method specifically includes the following steps:

[0065] S1: Multi-camera calibration: Acquire a set of line segments composed of artificial and automatic line segments in the Asian elephant motion scene image, estimate the vanishing point of the line segments in the set, and determine the camera projection matrix based on the estimated vanishing point.

[0066] For each camera, a frame image with clear scene features is extracted from its captured video stream, and a one-time camera calibration is performed based on this frame. The one-time camera calibration uses only a single calibration board image to solve for the camera's intrinsic parameters, distortion coefficients, and extrinsic parameters in one step. Only a suitable image from the camera's viewpoint needs to be selected, and the camera parameters can be obtained through this one-time calibration. The intrinsic parameters include focal length and principal point; the distortion coefficients are radial or tangential distortions; and the extrinsic parameters include the camera's pose relative to the calibration board image. The one-time camera calibration method is existing technology and will not be elaborated upon here.

[0067] It should be noted that the selected frames for camera calibration include, but are not limited to, lines that are clearly visible to the naked eye in the scene, such as the edge of the ground, railings, and corner lines. The frames should be free of motion blur, overexposure, or underexposure, with sharp edges and clear contrast.

[0068] S1 specifically includes:

[0069] S1-1: Deploy the camera array.

[0070] A multi-channel network camera array will be deployed above the elephant pavilion and event venue. The video capture equipment will have a resolution of no less than 1920×1080 pixels and a frame rate of no less than 15 frames per second. Each camera's field of view must cover the venue floor and have an overlapping field of view.

[0071] S1-2: Extract valid parallel line segments from the Asian elephant movement scene image to obtain automatic line segments.

[0072] The LSD (Line Segment Detector) algorithm is used to extract parallel line features in the scene, and pixels with a length less than a preset length threshold are excluded. Interfering line segments are eliminated, and stable straight line segments that meet the length requirements are retained.

[0073] Specifically, extracting parallel line features in a scene includes: calculating the gradient magnitude and gradient direction of each pixel in the scene; merging pixels with similar gradient directions and gradient magnitudes greater than a preset threshold into multiple connected regions; calculating the minimum bounding rectangle for each connected region; using the principal axis of the rectangle as the line segment direction; and using the line connecting the two endpoints of the rectangle as the detected line segment. After filtering the line segments, clustering is performed according to the line segment angle, including:

[0074] Gradient calculation: The horizontal gradient of each pixel is calculated using a 2×2 gradient operator. and vertical gradient Calculate the gradient magnitude and gradient direction ;

[0075] Region growing: Regions with similar gradient directions and magnitudes greater than a preset threshold are selected. Connect the pixels of each pixel to form a connected region, and remove tiny regions with fewer than a set threshold of pixels;

[0076] Line segment fitting: Calculate the minimum bounding rectangle for each connected region, take the direction of the rectangle's principal axis as the direction of the line segment, and use the line connecting the two endpoints of the rectangle as the detected straight line segment;

[0077] Line segment filtering: Calculate the pixel length of line segments and discard those with a length less than [a certain value]. and line segments whose average gradient magnitude is below a threshold;

[0078] Parallel line clustering: Clustering is performed according to the angle of line segments to identify two main directional clusters, corresponding to orthogonal directions in physical space.

[0079] In one embodiment of the present invention, the length threshold The value is 50. For example... Figure 2 The image shows an outdoor scene in the elephant enclosure. The black lines in the image are the results of automatic detection. The outdoor camera automatically detects a large number of lines that are evenly distributed, eliminating the need for manual calibration.

[0080] S1-3: Based on multiple evaluation indicators, perform quality assessment on the effective parallel line segments obtained in S1-2. If the quality assessment fails, execute S1-4 and S1-5 to supplement the manual line drawing process; otherwise, execute S1-6 to cluster the effective parallel line segments by angle.

[0081] Evaluation indicators include:

[0082] (a) Statistically detected lengths greater than The total number of line segments is denoted as . ;

[0083] (b) Cluster line segments by angle and calculate the number of line segments in two orthogonal directions. and If the number of line segments in any direction is less than 3, then the coverage in that direction is considered insufficient.

[0084] It is understandable that for any two orthogonal directions in the xyz coordinate system, determining only two of them is sufficient to determine the remaining directions.

[0085] (c) Divide the image into a 3×3 grid, count the number of line segments in each grid, and calculate the distribution entropy value. This indicates the uniformity of line segment distribution in the image. The smaller the value, the more concentrated the distribution of line segments;

[0086] (d) For line segments within the same direction cluster, calculate their angular standard deviation. , The smaller the value, the better the parallelism.

[0087] Based on the above indicators, the automatic detection quality score is calculated. :

[0088]

[0089] In the formula, , , These are the weighting coefficients. This is a threshold for the number of directional line segments. The angle standard deviation normalization factor, Indicates the uniformity of line segment distribution in an image. This represents the standard deviation of the angle of line segments within the same cluster. , This indicates the number of line segments in two orthogonal directions.

[0090] like Below the preset threshold If the number of valid line segments in any orthogonal direction obtained from S1-2 is less than 20, the system determines that the scene lacks sufficient structured features, automatically triggers the manual line drawing assistance process, displays prompt information in the user interface, and guides the user to draw manually.

[0091] In one embodiment of the present invention, It is 0.6. , , , , ,like Figure 3 As shown, this is an indoor scene in the elephant enclosure. The number of automatically detected line segments is insufficient and their distribution is uneven, requiring manual calibration. In the figure, the solid black lines represent the results of automatic detection, and the dashed black lines represent the line segments drawn by manual calibration.

[0092] S1-4: If the quality assessment result is unsatisfactory, at least two sets of orthogonal parallel lines in physical space shall be manually added, with at least two parallel line segments drawn in each orthogonal direction.

[0093] Specifically, the system interface can guide the user to manually draw at least two sets of orthogonal parallel lines in physical space on the image. These lines can be drawn along the ground edge or along temporarily placed markers in the scene. At least two parallel line segments should be drawn in each orthogonal direction, and the line segments should span more than one-third of the image's height or width, and be distributed in different areas of the image as much as possible. Whenever the user completes the drawing of a new line segment, the system calculates the angular deviation between this line segment and existing line segments in the same group. If the deviation exceeds a threshold (preferably 3 degrees), the system will issue a prompt, suggesting that the user adjust or redraw the line.

[0094] After the user completes the drawing, the system will assign each manually drawn line segment to the user. Represented as the following data structure:

[0095]

[0096] In the formula, The pixel coordinates of the starting point of the line segment. The pixel coordinates of the endpoint of the line segment. For the angle of the line segment, by Calculations show that Let be the length of the line segment, from Calculations show that The value of the direction group identifier to which the line segment belongs is 1 or 2, which respectively represent two orthogonal directions.

[0097] S1-5: The system evaluates the quality of each manually drawn line segment according to the quality evaluation method in S1-3, calculates the length weight, position distribution weight, and parallel consistency weight of the manually drawn line segment, and obtains the comprehensive weight of each manually drawn line segment.

[0098] If the quality assessment fails, the user is prompted to redraw the image, and its reference weight in subsequent vanishing point estimation is calculated. The weight calculation considers the following factors:

[0099] (a) Length weight The longer the line segment, the more reliable the directional information it provides. The calculation formula is:

[0100]

[0101] In the formula, The length of the artificial line segment. This is the diagonal length of the current image, used for normalization.

[0102] (b) Location distribution weights The distribution of line segments in the image affects the stability of the vanishing point estimation; line segments located at the image edges exert a stronger constraint on the vanishing point. The calculation formula is:

[0103]

[0104] In the formula, Here are the coordinates of the center point of the line segment. These represent the image width and height, respectively. The closer this value is to 1, the closer the line segment is to the image edge.

[0105] (c) Parallel consistency weight Within the same direction group, the higher the consistency of parallelism among line segments, the higher the overall reliability of that group of line segments. For direction groups... Calculate the standard deviation of the angles of all line segments within the group. Then the parallel consistency weight of each line segment is:

[0106]

[0107] In the formula, For direction group The average angle, exp represents the exponential operation with the natural constant e as the base.

[0108] (d) Multiply the above weights together to obtain the overall weight of each artificial line segment:

[0109]

[0110] The system normalizes the weights of all artificial line segments so that their sum equals the total weight of the artificial line segments. .

[0111] The normalized weights of the artificial line segments are denoted as: .

[0112] S1-6: Calculate the weights of the valid line segments obtained in S1-2 to obtain the comprehensive weights of the automatic line segments. S1-6 includes:

[0113] (a) The automatic line segments are clustered by angle to obtain two main direction clusters, which correspond to two orthogonal directions respectively.

[0114] (b) For those belonging to the direction cluster Automatic line segments Its overall weight for:

[0115]

[0116] In the formula, For directional clusters The average angle, Let be the standard deviation of the angles of this cluster. The weights of all auto-segments are normalized so that their sum equals the total weight of the auto-segments.

[0117]

[0118] In the formula, The total weight of the automatic line segment. The total weight of the manually operated line segment is 1, and the sum of the total weight of the manually operated line segment and the total weight of the automatically operated line segment is 1.

[0119] The normalized automatic segment weights are denoted as .

[0120] S1-7: After normalizing the combined weights of the manual and automatic line segments, the manual and automatic line segments are merged to construct a weighted line segment set for vanishing point estimation:

[0121]

[0122] In the formula, The number of artificial line segments. The number of automatic line segments. This is the data structure representation of the i-th artificial line segment. The normalized comprehensive weight of the i-th artificial line segment. The data structure representation of the j-th automatic line segment. This represents the comprehensive weight of the j-th automatic line segment after normalization.

[0123] S1-8: Using the weighted RANSAC algorithm to process the set of line segments Perform vanishing point estimation.

[0124] It's important to note that in the real world, parallel lines converge at a single point in an image under perspective projection; this point is called the vanishing point. The vanishing point is used to calculate camera parameters. By establishing a projection matrix, pixel coordinates are converted into physical coordinates, enabling accurate distance calculations in physical space.

[0125] The specific steps for S1-8 are as follows:

[0126] (a) Set the maximum number of iterations The interior point threshold is 1000. The angle is 5°, the current optimal model is empty, and the optimal interior point weighted sum is... .

[0127] (b) From the set of line segments Two line segments are randomly selected, and their intersection is calculated as a candidate vanishing point. When sampling, weighting factors are considered; the higher the weight, the greater the probability of the line segment being selected.

[0128] (c) For the set of line segments For each line segment in the algorithm, calculate the direction vector of the line connecting each endpoint to the candidate vanishing point. If the angle between the direction vectors corresponding to the two endpoints of the line segment and the line segment direction is less than the interior point threshold, then... If the vanishing point is not found in the vanishing point, then the line segment is determined to be an interior point line segment. When determining an interior point, the direction group to which the line segment belongs must be considered; line segments from different direction groups should not be supported by the same vanishing point.

[0129] (d) Calculate the weighted sum of interior points of the current candidate vanishing point:

[0130]

[0131] in, This is the weighted sum of interior points of the current candidate vanishing points. This represents the overall weight of the i-th interior point. This represents the set of interior point segments in the set S that are approximately aligned with the direction of the candidate vanishing point and whose included angle is less than the interior point threshold.

[0132] (e) If Then update the optimal vanishing point. , .

[0133] (f) Repeat step be until the maximum number of iterations is reached or the optimal model is not updated for multiple consecutive iterations.

[0134] (g) Using all interior point segments, the optimal vanishing point is refined using the weighted least squares method to obtain the final vanishing point estimate. .

[0135] Specifically, the steps for refining the weighted least squares method are as follows:

[0136] Constructing the objective function: Let the optimal vanishing point obtained by weighted RANSAC (Random Sample Consensus) estimation be... The set of all interior point segments is The weights of each line segment, i.e., the combined weights of the interior points, are: The cost function is defined as the weighted sum of squared angular deviations of each interior line segment from the vanishing point:

[0137]

[0138] in, For the first The angle between the direction of a line segment and the direction of the line connecting the vanishing point to the center point of the line segment is used as the angular deviation of the interior point line segment from the vanishing point, and represents the number of interior point line segments.

[0139] Linearization: The angular deviation is approximated as the geometric distance deviation. For line segments... Its direction vector is =( The center point of the line segment is Vanishing point to line segment The distance along the line is:

[0140]

[0141] in, Represents the optimized vanishing point pixel coordinates, ( , ), ( , ) represent line segments respectively The start and end pixel coordinates, express to line segment The distance along the line.

[0142] Weighted least squares solution: Transform the cost function into a weighted sum of squared distances:

[0143]

[0144] right and Taking the partial derivatives and setting them to zero, we obtain the normal system of equations:

[0145] =0, =0

[0146] Organize into matrix form ,in:

[0147]

[0148] in, For line segments The unit normal vector, Let be any point on the line segment.

[0149] Finding the final vanishing point: Solving the system of linear equations yields:

[0150]

[0151] This is the final vanishing point estimate after fine-tuning and optimization. .

[0152] In this way, after a rough estimate of the optimal vanishing point, the weighted least squares method is used to further refine the optimal vanishing point. This allows us to first find the approximate location of the vanishing point, narrowing the search range, and then find the optimal vanishing point within that approximate location. By transforming the angle deviation problem into a distance problem from a point to a line, and finally obtaining a more accurate vanishing point by solving a system of linear equations, we can accelerate the speed of vanishing point location, improve the accuracy, stability, and anti-interference ability of the vanishing point, and thus improve the quality of the vanishing point.

[0153] S1-9: Perform the above weighted RANSAC estimation on the two orthogonal directions respectively to obtain the first vanishing point. Second vanishing point .

[0154] To ensure the orthogonality of the two directions, the system calculates the first vanishing point. With the vanishing point If the included angle deviates from 90 degrees and exceeds the preset angle threshold, the system will issue a prompt, suggesting that the user check whether the manually drawn line meets the orthogonality requirement, and supplement or adjust the drawn line if necessary.

[0155] It should be noted that the preferred value for the preset angle threshold is 5 degrees, and those skilled in the art can set the specific value of the preset angle threshold according to the actual application.

[0156] S1-10: Calculate the third vanishing point based on the geometric constraints of orthogonal vanishing points. :

[0157]

[0158] In the formula, Represents the cross product of vectors. Represents the first vanishing point, Represents the second vanishing point.

[0159] In this way, three vanishing points can be guaranteed without relying on the detection of a third set of parallel lines, providing stable and correct geometric constraints for subsequent camera intrinsic and extrinsic parameters, ensuring the accuracy of pixel-to-physical coordinate conversion, and thus improving the accuracy of motion measurement.

[0160] S1-11: Joint Solution of Intrinsic and Extrinsic Parameters: Solving the system of equations yields the focal lengths along the x and y axes in the 2D image pixel coordinate system. Based on the focal lengths and the final estimated vanishing point, an intrinsic parameter matrix is ​​constructed. The rotation matrix is ​​then calculated using this intrinsic parameter matrix. A real-world coordinate system is constructed by selecting a preset location, and the translation vector is calculated based on the camera position. Finally, the camera projection matrix is ​​constructed using the intrinsic parameter matrix, rotation matrix, and translation vector. S1-11 specifically includes:

[0161] (a) Let the coordinates of the principal point be ( , In the two-dimensional image pixel coordinate system, the focal lengths along the x-axis and y-axis are respectively... , It consists of three orthogonal vanishing points. , , Given the satisfied geometric constraints, establish a system of equations:

[0162]

[0163]

[0164]

[0165] in, , These represent the first vanishing point, respectively. x and y coordinates , These represent the second vanishing points. x and y coordinates , Represents the third vanishing point The x and y coordinates.

[0166] Solving the system of equations simultaneously, we get... , , , Construct the camera intrinsic parameter matrix :

[0167]

[0168] (b) Based on the obtained camera intrinsic parameter matrix K, calculate each column of the rotation matrix R:

[0169] , , ,

[0170] in, This represents the inverse of the camera intrinsic parameter matrix. This indicates the direction of the x-axis in the 3D world coordinate system within the camera coordinate system. This indicates the direction of the y-axis in the 3D world coordinate system within the camera coordinate system. This indicates the direction of the z-axis in the 3D world coordinate system within the camera coordinate system. Let R represent the magnitude of the vector, R = [r1r2 r3].

[0171] The rotation matrix is ​​used to describe the orientation relationship between the camera coordinate system and the world coordinate system, representing the camera's rotational attitude in three-dimensional space, characterizing the camera's viewing direction, and providing a geometric basis for subsequent pixel coordinate to physical coordinate conversion.

[0172] (c) Select a suitable location to construct a real-world coordinate system, and determine the camera's position within this coordinate system. The translation vector in the extrinsic parameter matrix can be calculated. .

[0173] (d) Based on the obtained intrinsic parameter matrix K, rotation matrix R, and translation vector t, construct the camera projection matrix P:

[0174] ,

[0175] Understandably, the camera projection matrix is ​​used to transform the three-dimensional world coordinate system into the two-dimensional image pixel coordinate system.

[0176] S2: Establish training datasets for object detection and keypoint detection. S2 specifically includes:

[0177] S2-1: Collect video data from deployed cameras. The video collection time should be no less than 30 minutes to ensure coverage of different postures, lighting conditions, and individual samples of Asian elephants.

[0178] S2-2: Extract frames from the acquired video at equal intervals to form a training set. The extraction interval is adaptively determined based on the video frame rate: when the frame rate is 25 frames / second, extract one frame every 100 frames; when the frame rate is 30 frames / second, extract one frame every 120 frames, ensuring that adjacent extracted frames have sufficient scene differences. The number of images in the training set shall not be less than 5000.

[0179] S2-3: Use the Labelme tool to label the Asian elephants in the training set images with rectangular bounding boxes, and label the shoulder and limb contact points of the Asian elephants with key points. Each individual is labeled with 6 key points (both shoulders, front two feet, and hind two feet). During the labeling, the visibility status is distinguished to form a key point detection dataset.

[0180] S3: Train a model for detecting the Asian elephant as a whole and its key points, and use it as a detection model for the Asian elephant.

[0181] A multi-task model for overall detection and keypoint detection of Asian elephants based on YOLOv11-pose was trained. The model output includes the overall bounding box of the Asian elephant and the coordinates of six keypoints, including the shoulder and the grounding point, providing accurate pixel positions for subsequent trajectory point optimization.

[0182] The above steps S1-S3 are the offline preparation stage. After completion, the online processing stage begins, which requires repeating steps S4-S9 on the long video.

[0183] S4: Long video segmentation. This function segments the input long surveillance video into several short video clips of fixed length. The clip length is set to 300 seconds, and the number of clips is calculated based on the total video length to form a set of short video clips.

[0184] S5: Asian elephant target detection and key point detection.

[0185] like Figure 4 As shown, the short video clip is input into the YOLOv11-pose multi-task model trained in step S3, and frame-by-frame processing is performed to obtain the overall rectangular bounding box of the Asian elephant and the pixel coordinates of 6 key points. The rectangular bounding box is used as the detection box, and the detection result is represented as follows:

[0186] ,

[0187] in, , These are the x and y coordinates of the center point of the detection box, respectively. , These are the width and height of the detection frame, respectively. The category confidence score represents the model's confidence level that the detection box contains an Asian elephant target. It ranges from 0 to 1 and is used to distinguish the target from the background. It also serves as the basis for dividing detection boxes into high-confidence and low-confidence categories in the ByteTrack algorithm. , represents the coordinates of 6 key points. Indicates the coordinates of the left shoulder. Indicates the coordinates of the right shoulder. Indicates the coordinates of the left front foot. Indicates the coordinates of the right front foot. Indicates the coordinates of the left rear foot. This indicates the coordinates of the right rear foot.

[0188] S6: Asian elephant target tracking and keypoint trajectory generation. S6 specifically includes:

[0189] Detection box association tracking. The ByteTrack algorithm is used to track the detection boxes output in step S5. Perform correlation tracking.

[0190] Specifically: A preset confidence threshold is set. Detection boxes with a confidence score greater than or equal to the preset confidence threshold are classified as high-confidence detection boxes, and detection boxes with a confidence score less than the preset confidence threshold are classified as low-confidence detection boxes. High-confidence detection boxes are matched with all existing tracking trajectories for the first time. Low-confidence detection boxes are matched with tracking trajectories that failed to match in the first match for the second time. The status of the trajectory is updated according to the matching results. Trajectories that successfully match in both matches are considered to be successfully tracked. Detection boxes that still do not match after the second match are initialized to new trajectories if they do not match any trajectory. If a trajectory fails to match a detection box for multiple consecutive frames, it is marked as lost and deleted.

[0191] The matching methods include:

[0192] Based on Kalman filtering, the position of an existing trajectory in the current frame is predicted, and the predicted position is calculated relative to the center point of the detection box. Matching is performed using Euclidean distance;

[0193] The state vector of a Kalman filter is defined as follows: ,in The location of the center point of the detection frame. The velocity component represents the speed and direction of the target detection box center point's movement between consecutive frames.

[0194] State transition matrix Using a uniform velocity model, The time interval between adjacent frames:

[0195]

[0196] Observation matrix Extract location information:

[0197]

[0198] State estimation covariance matrix Process noise covariance matrix and observation noise covariance matrix All are initialized as diagonal matrices, with diagonal elements set based on prior experience, for example... Initial value set to , Set as , Set as The specific values ​​can be fine-tuned based on the actual tracking results.

[0199] Perform two steps of prediction and update per frame:

[0200] First, through the state transition matrix Predict the current state and the prediction variance:

[0201] ,

[0202] ,

[0203] in, This represents the predicted state at time k. This represents the optimal estimated state at time k-1. This represents the prediction variance at time k.

[0204] Next, calculate the Kalman gain: Kalman gain reflects the degree to which the observations correct the state estimate; after observation... The optimal estimate is obtained after correction. Update covariance , used for prediction of the next frame and Euclidean distance matching with the detection box, where I represents the identity matrix.

[0205] If the predicted location successfully matches the center point of the detection box, the coordinates of the 6 key points of the current detection box are... Associate it with the trajectory and update the Kalman filter corresponding to the trajectory;

[0206] If the detection box does not match any trajectory, it is initialized with a new trajectory; if a trajectory fails to match for multiple consecutive frames, it is deleted. For successfully tracked trajectories, the coordinates of 6 key points are stored in chronological order to generate a key point trajectory sequence.

[0207]

[0208] in, Represents the sequence of key point trajectories. This represents the coordinates of the k-th left shoulder. This represents the coordinates of the k-th right shoulder. This represents the coordinates of the k-th left front foot. This represents the coordinates of the k-th right front foot. This represents the coordinates of the k-th left rear foot. This represents the coordinate of the kth right rear foot, and n represents the number of keypoint frames contained in a single trajectory, i.e., the duration of the trajectory.

[0209] Existing technologies only detect the target position in the current frame, and target loss occurs when occlusion happens, causing tracking interruption. This invention introduces a Kalman filter prediction step before ByteTrack matching, using historical motion trajectories to estimate the position of the next frame. Even if the Asian elephant is temporarily missing its detection box due to occlusion by its companions, the trajectory continuity can still be maintained based on the predicted position. Accurate tracking can be quickly resumed after the occlusion is removed. Through the ByteTrack high and low confidence secondary matching mechanism, the weak detection signals at the occlusion edges are fully utilized, thereby solving the problems of target loss and identity jump caused by occlusion, and achieving stable individual tracking in a group environment.

[0210] Existing single-stage matching algorithms directly discard low-confidence detection boxes, easily leading to missed detections at occluded edges. This invention employs ByteTrack's high- and low-confidence secondary matching strategy: high-confidence detection boxes are preferentially matched with existing trajectories, while low-confidence detection boxes are then re-associated with unmatched trajectories. This mechanism fully utilizes the boundary detection response output by YOLOv11-pose, incorporating weak detection signals at occluded edges into the tracking, reducing trajectory breaks caused by occlusion.

[0211] Existing technologies often use single points, such as the centroid or the center of the detection box, as trajectory points. When occlusion occurs, the missing key points cause trajectory jumps. This invention integrates six key points from the shoulders and limbs to generate trajectory reference points. Even if some key points are occluded, the remaining visible key points can still provide stable position estimates, reducing the impact of missing single points on the overall trajectory.

[0212] Kalman filtering provides temporal smoothing constraints, while ByteTrack provides inter-frame data correlation. The combination of the two forms a closed-loop tracking system of "prediction-detection-matching-update". Compared with conventional tracking using only Kalman filtering or detection tracking using only ByteTrack, this solution can improve the trajectory integrity rate in occlusion scenarios of Asian elephant herds, reduce identity jumps, and provide a reliable temporal data foundation for subsequent motion statistics.

[0213] S7: Merge six key points to generate merged key point pixel coordinates.

[0214] Based on the trajectory sequence of the six key points output in step S5, the pixel coordinates of the fused key points are calculated. Let the coordinates of the six key points in the current frame be... , , , , , The coordinates of the fused key points are calculated as follows:

[0215]

[0216]

[0217] in, , , The weighting coefficients for the shoulder, front foot, and back foot are respectively selected. , , This method emphasizes the stability of the shoulder base point while fusing information from the limbs to generate pixel coordinates of the fused key points. .

[0218] S8: Generate target pixel trajectory and determine motion state

[0219] S8-1: Store the pixel coordinates of the fusion key points output from step S7 in chronological order to generate pixel trajectories:

[0220]

[0221] in, Elephant logo The pixel trajectory, where n represents the number of keypoint frames contained in a single trajectory, and the duration of the trajectory is n / nfs, where nfs represents the video frame rate.

[0222] S8-2: Calculate instantaneous pixel displacement :

[0223]

[0224] S8-3: Set window length Frames, and cumulative pixel displacement threshold =80 pixels, calculate the cumulative pixel displacement within the window length. :

[0225]

[0226] like > If a frame is identified as a moving frame within the window, it is marked as 1; otherwise, it is identified as a still frame and marked as 0.

[0227] S8-4: Extract continuous motion frame sequences and merge them with intervals less than [missing information]. From the frame segments, we obtain the motion segments. , where m represents the number of motion frames in the motion segment.

[0228] In this embodiment of the invention, a sliding window cumulative displacement threshold determination mechanism is set. The cumulative pixel displacement of a 5-frame window is compared with a cumulative pixel displacement threshold. Only when the displacement of multiple consecutive frames exceeds the cumulative pixel displacement threshold is it determined to be a valid motion state, and continuous motion segments are extracted. The merging interval is less than... By extracting still frames and micro-motion noise from the fragments, invalid displacements are precisely filtered out, and effective motion data that truly reflects the body's vitality is obtained, thus solving the problem of invalid displacement noise caused by physiological micro-movements such as ear shaking and breathing when Asian elephants stand in place.

[0229] S9: Extraction is satisfied if the condition is met. > The video clip is used to calculate the distance traveled during this movement. S9 specifically includes:

[0230] S9-1: For the frames within the motion video segment extracted in step S8, use the camera projection matrix P calculated in step S1 to fuse the keypoint pixel coordinates ( , Convert to physical coordinates , Map pixel coordinates to physical coordinates:

[0231]

[0232]

[0233] In the formula, This represents the k-th motion segment. This represents the element in row 1 and column 4 of P. This represents the element in row 3 and column 4 of P. This represents the element in row 3, column 2 of P. This represents the element in the 2nd row and 2nd column of P. This represents the element in row 2, column 4 of P. This represents the element in row 1 and column 2 of P. This represents the element in row 3, column 1 of P. This represents the element in the first row and first column of P. This represents the element in the 2nd row and 1st column of P.

[0234] S9-2: Calculate the actual distance the elephant traveled :

[0235]

[0236] in, This represents the actual distance the elephant moves in the world coordinate system relative to the (i-1)th frame. , ) represents the true coordinates of the elephant keypoint in the i-th frame in the world coordinate system. , () represents the true coordinates of the same key point in the world coordinate system in the (i-1)th frame.

[0237] This allows for the calculation of motion distances within valid motion segments of the video. :

[0238]

[0239] in, This represents the cumulative actual distance traveled within the k-th motion segment. The same elephant may contain multiple motion segments; the distance for each segment is calculated independently and then summed. This indicates the start frame of the k-th motion segment. This represents the end frame within the k-th motion segment.

[0240] S9-3: Determine if there are any unprocessed short video clips. If they exist, return to step S5 to continue processing; otherwise, proceed to the next step to determine if there are any unprocessed long surveillance videos.

[0241] S9-4: Determine if there is a long video input. If so, return to step S4; otherwise, end the current exercise volume statistics.

[0242] The overall architecture of the actual operation of all the above steps is as follows: Figure 5 As shown, the camera stores video data in an NVR (Network Video Recorder), the local server then reads and stores the data from the NVR, processes the data, and finally displays the results on a visualization screen.

[0243] In this embodiment of the invention, a combination of LSD line segment detection and manual line drawing is used to accurately estimate three orthogonal vanishing points using the weighted RANSAC algorithm. The camera intrinsic parameter matrix K, rotation matrix R, and translation vector t are then solved simultaneously to construct the projection matrix P, establishing a precise mapping relationship from pixel coordinates to physical coordinates. This solves the severe perspective distortion problem caused by high-position camera deployment. By converting the pixel coordinates of fused keypoints into physical coordinates in the world coordinate system, the actual physical displacement between adjacent frames is calculated in the world coordinate system. This solves the problem of huge errors caused by directly relying on pixel displacement statistics, achieving accurate measurement of motion distance.

[0244] This invention also provides a visual measurement system for Asian elephant motion based on human-machine collaborative calibration and key point fusion. Running the aforementioned visual measurement method for Asian elephant motion based on human-machine collaborative calibration and key point fusion, the system includes:

[0245] The multi-target calibration module is used to acquire artificial and automatic line segments in the Asian elephant movement scene image and form a line segment set, estimate the vanishing point of the line segments in the line segment set, and determine the camera projection matrix based on the estimated vanishing point.

[0246] The detection module is used to cut the long video captured by the camera into multiple video segments, input the multiple video segments into a pre-trained Asian elephant detection model for frame-by-frame processing, and obtain the confidence score and key point coordinates of the detection box containing the target Asian elephant.

[0247] The trajectory generation module is used to classify the detection boxes based on a preset confidence threshold, match the existing tracking trajectory with the classified detection boxes, and if the match is successful, fuse the corresponding key point coordinates to obtain the fused key point pixel coordinates and generate the target pixel trajectory.

[0248] The calculation module is used to convert the pixel coordinates of the fused key points into corresponding physical coordinates based on the camera projection matrix, and to calculate the actual movement distance of the target Asian elephant based on the target pixel trajectory, so as to obtain the movement amount of the target Asian elephant.

[0249] Regarding the system in the above embodiments, the specific manner in which each unit performs operations has been described in detail in the embodiments related to the method, and will not be elaborated here.

[0250] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0251] All parts not covered in this invention are the same as or can be implemented using existing technologies.

[0252] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0253] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the claims of the present invention.

Claims

1. A visual measurement method for the movement of Asian elephants based on human-machine collaborative calibration and key point fusion, characterized in that, Includes the following: Artificial and automatic line segments are acquired from images of Asian elephant movement scenes and a line segment set is constructed. The vanishing point of the line segments in the line segment set is estimated, and the camera projection matrix is ​​determined based on the estimated vanishing point. The long video captured by the camera is cut into multiple video segments. The multiple video segments are input into a pre-trained Asian elephant detection model for frame-by-frame processing to obtain the confidence score and key point coordinates of the detection box containing the target Asian elephant. The detection boxes are classified based on a preset confidence threshold. The existing tracking trajectory is matched with the classified detection boxes. If the match is successful, the corresponding key point coordinates are fused to obtain the fused key point pixel coordinates and generate the target pixel trajectory. The camera projection matrix is ​​used to convert the pixel coordinates of the fused key points into corresponding physical coordinates, and the actual movement distance of the target Asian elephant is calculated based on the target pixel trajectory to obtain the movement amount of the target Asian elephant; The process of acquiring artificial and automatic line segments from images of Asian elephant movement and constructing a line segment set includes: Extract valid parallel line segments from images of Asian elephant movement scenes to obtain automatic line segments; The quality score is obtained by evaluating the quality of effective parallel line segments based on multiple evaluation indicators, including: the total number of effective parallel line segments, the number of line segments in two orthogonal directions, the line segment distribution entropy, and the angle standard deviation. Among them, the effective parallel line segments are clustered by angle to obtain the number of line segments in two orthogonal directions, and the angle standard deviation is calculated for the line segments in the same direction cluster. If the quality score is less than the preset quality threshold, or the number of valid parallel line segments in any orthogonal direction is less than the preset number threshold, at least two sets of orthogonal parallel lines in physical space shall be manually added, and at least two parallel line segments shall be drawn in each orthogonal direction. Calculate the length weight, position distribution weight, and parallel consistency weight of the artificial line segments to obtain the comprehensive weight of each artificial line segment; cluster the automatic line segments by angle and calculate the comprehensive weight of the automatic line segments. After normalizing the combined weights of the manual and automatic line segments, the manual and automatic line segments are merged to construct a line segment set. The method further includes: Store the pixel coordinates of the fusion key points in chronological order to generate pixel trajectories; Calculate the cumulative pixel displacement value of the pixel trajectory within the preset window length. If the cumulative pixel displacement value is greater than the preset displacement threshold, the frame in the window will be determined as a motion frame. Motion frames with an interval less than the preset frame threshold will be merged to obtain a motion segment. Coordinate transformation is performed based on the pixel coordinates of the fusion key points corresponding to all frames within the extracted motion segment.

2. The visual measurement method for Asian elephant motion based on human-machine collaborative calibration and key point fusion according to claim 1, characterized in that, Estimating the vanishing point of line segments in a set of line segments includes: Randomly select two line segments from the set of line segments and calculate their intersection point as candidate vanishing points; For each line segment in the set of line segments, calculate the direction vector of the line connecting each endpoint to the candidate vanishing point. If the angle between the direction vector corresponding to each endpoint of the line segment and the direction of the line segment is less than the preset interior point threshold, then the line segment is determined to be an interior point line segment. Calculate the sum of the comprehensive weights of the interior line segments of the candidate vanishing points, and update the optimal vanishing point based on the sum of the comprehensive weights until the maximum number of iterations is reached or the optimal vanishing point is not updated after a preset number of iterations. The cost function is the sum of the weighted squared angular deviations of each interior point line segment to the optimal vanishing point. After approximating the angular deviations as geometric distance deviations, the cost function is transformed into the sum of the weighted squared distances. The weighted least squares method is then used to solve for the final estimated vanishing point.

3. The visual measurement method for Asian elephant motion based on human-machine collaborative calibration and key point fusion according to claim 2, characterized in that, Determining the camera projection matrix based on the estimated vanishing point includes: By estimating the vanishing points of line segments in the set of line segments in the orthogonal directions, the first vanishing point and the second vanishing point are obtained. Calculate the third vanishing point based on the geometric constraints of the first and second vanishing points; Based on the first vanishing point, the second vanishing point, and the third vanishing point, the following system of equations is established: ， in, These represent the x and y coordinates of the final estimated vanishing point, respectively. , These represent the first vanishing point, respectively. x and y coordinates , These represent the second vanishing points. x and y coordinates , Represents the third vanishing point x and y coordinates , These represent the focal lengths along the x-axis and y-axis in the pixel coordinate system of a two-dimensional image, respectively. Solving the system of equations yields the focal lengths along the x and y axes in the pixel coordinate system of the two-dimensional image. Based on the focal lengths and the final estimated vanishing point, an intrinsic parameter matrix is ​​constructed, and the rotation matrix is ​​calculated based on the intrinsic parameter matrix. A real-world coordinate system is constructed by selecting a preset location, and the translation vector is calculated based on the camera position; the camera projection matrix is ​​constructed based on the intrinsic parameter matrix, rotation matrix, and translation vector.

4. The visual measurement method for Asian elephant motion based on human-machine collaborative calibration and key point fusion according to claim 1, characterized in that, The method further includes: Video data is collected from deployed cameras, and frames are extracted from the video data at equal intervals to form a training set; In the training set, rectangular bounding boxes were used to label the Asian elephant as a whole, and these rectangular bounding boxes were used as detection boxes. Keypoints were also labeled for each sample to form a keypoint detection dataset. The Asian elephant detection model based on YOLOv11-pose was trained using a keypoint detection dataset.

5. The visual measurement method for Asian elephant motion based on human-machine collaborative calibration and key point fusion according to claim 1, characterized in that, Based on a pre-set confidence threshold, the detection boxes are classified, and existing tracking trajectories are matched with the classified detection boxes, including: Set a preset confidence threshold, classify detection boxes with confidence scores greater than or equal to the preset confidence threshold as high confidence detection boxes, and classify detection boxes with confidence scores less than the preset confidence threshold as low confidence detection boxes; The high-confidence detection box is matched with the existing tracking trajectory for the first time; the low-confidence detection box is matched with the tracking trajectory that failed to match in the first match for the second time. Trajectories that successfully match in two matching attempts are marked as successfully tracked. High-confidence detection boxes that still do not match after the second matching attempt are initialized as new trajectories. Trajectories that do not match detection boxes for multiple consecutive frames are marked as lost and deleted. Specifically, the position of the existing tracking trajectory in the current frame is predicted based on Kalman filtering to obtain the predicted position. The predicted position is then matched with the Euclidean distance between the predicted position and the center point of the detection box. If the match is successful, the coordinates of the key point corresponding to the current detection box are associated with the corresponding trajectory and the Kalman filter corresponding to the trajectory is updated.

6. The visual measurement method for Asian elephant motion based on human-machine collaborative calibration and key point fusion according to claim 1, characterized in that, Key points include: left shoulder, right shoulder, left front foot, right front foot, left back foot, and right back foot.

7. The visual measurement method for Asian elephant motion based on human-machine collaborative calibration and key point fusion according to claim 1, characterized in that, Differentiated weights are set for key points, and the pixel coordinates of the fused key points are calculated by weighted average.

8. A visual measurement system for Asian elephant motion based on human-machine collaborative calibration and key point fusion, realizing the visual measurement method for Asian elephant motion based on human-machine collaborative calibration and key point fusion as described in any one of claims 1 to 7, characterized in that, The system includes: The multi-target calibration module is used to acquire a set of line segments composed of artificial and automatic line segments in the Asian elephant movement scene image, estimate the vanishing point of the line segments in the set, and determine the camera projection matrix based on the estimated vanishing point. The detection module is used to cut the long video captured by the camera into multiple video segments, input the multiple video segments into a pre-trained Asian elephant detection model for frame-by-frame processing, and obtain the confidence score and key point coordinates of the detection box containing the target Asian elephant. The trajectory generation module is used to classify the detection boxes based on a preset confidence threshold, match the existing tracking trajectory with the classified detection boxes, and if the match is successful, fuse the corresponding key point coordinates to obtain the fused key point pixel coordinates and generate the target pixel trajectory. The calculation module is used to convert the pixel coordinates of the fused key points into corresponding physical coordinates based on the camera projection matrix, and to calculate the actual movement distance of the target Asian elephant based on the target pixel trajectory, so as to obtain the movement amount of the target Asian elephant.

Images