Cattle face recognition method and system based on deep metric learning in complex breeding environment

Abstract

The application relates to the technical field of image recognition, and discloses a cattle face recognition method and system based on deep metric learning under a complex breeding environment. The method comprises the following steps: acquiring a continuous cattle face video, establishing a cross-frame corresponding relationship of a local visual area, comparing a single-source motion model with a double-source motion model under the constraint of single cattle local deformation, and determining a candidate source component; extracting a local identity feature, taking the candidate source component as a removed boundary, confirming different cattle identities according to the template distance change before and after removal; tracking the candidate source component along the cross-frame corresponding relationship, forming a source anchoring interval and a cross-object mixed interval; excluding the mixed interval and video frames that fail to be tracked to the source anchoring interval, forming a trajectory representative feature and performing deep metric matching, and outputting a cattle identity.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and in particular to a method and system for cattle face recognition based on deep metric learning in complex farming environments. Background Technology

[0002] With the application of machine vision technology in large-scale farming, using cow face images for individual cattle identification is gradually becoming a non-contact identification method. Existing cow face recognition methods typically use cameras placed in feeding areas, watering areas, or cattle passage areas to acquire cow face images or continuous video. The method then determines the cow face image region to be processed through cow face target localization and facial structure recognition, extracts visual features from the corresponding image region, and determines the cattle's identity based on the feature matching relationship between the cow face to be identified and registered cow faces. For continuous video, the position of the cow face image in adjacent video frames can also be used. The changes and visual correlations maintain the continuous trajectory of the same cow face, and the cow face features of different video frames within the trajectory are combined for identification. For example, when adjacent cows approach the feeding trough at the same time, one cow is in a state of feeding with its head down, and another cow stretches its head forward from a nearby position. The heads of the two cows will locally overlap in the direction of the camera's imaging. At this time, the cow face candidate region formed in a single video frame may still have a continuous cow face shape, and the facial structure detected inside the region may also maintain a reasonable relative position in the image coordinates. Therefore, the cow face candidate region can continue to enter the subsequent visual feature extraction and identity matching process.

[0003] During the aforementioned process of continuous change in local overlap, the actual cattle objects covered by the candidate cow face region may change with the relative movement of the two cattle. For example, in the initial video frame, this region mainly corresponds to the cattle that are grazing. As the adjacent cattle continue to extend their heads, some facial visual content in the candidate region is gradually provided by the adjacent cattle, and in subsequent video frames, it may again mainly correspond to the adjacent cattle. Since the position, scale, and overall appearance of the candidate cow face region in the image still have continuous change characteristics during this process, the cow face association result based on video continuity may still maintain the same trajectory. Therefore, when performing depth visual feature extraction, the local visual content of the same candidate cow face from different cattle may jointly participate in the body. The features are formed and further aggregated into multi-frame features along the continuous trajectory, so that the identity features corresponding to the trajectory are jointly influenced by the visual information of different cattle. For cattle with similar facial appearances, such features may continue to change in the metric space, and thus may not necessarily show obvious abnormal features. Therefore, when performing individual recognition based on cattle face images and videos in complex breeding environments, it is necessary to further solve how to determine the effective cattle face visual information that can jointly participate in the formation of depth metric features based on the object consistency between visual content within the candidate cattle face region and the relationship of maintaining this object consistency in continuous video frames, so as to avoid the problem of cross-cattle visual content jointly participating in identity feature matching along the continuous cattle face trajectory. Summary of the Invention

[0004] This application proposes a method and system for cattle face recognition based on deep metric learning in complex farming environments to address the problems mentioned in the background art.

[0005] To achieve the above objectives, this application adopts the following technical solution: a cattle face recognition method based on deep metric learning in complex farming environments, comprising:

[0006] Acquire continuous cow face videos, establish cross-frame correspondence between local visual regions within the cow face candidate region, determine the allowable range of local deformation based on local location and head movement amplitude, compare single-source motion models and dual-source motion models within the allowable range of local deformation, and determine two candidate source components.

[0007] A deep metric learning network is used to extract local identity features and aggregate them to form cow face features. The candidate source components are used as the feature removal boundary. The changes in the metric distance between the cow face features and the two cow identity templates before and after removal are compared. Based on the changes in the single-source calibration video of the confirmed identity, it is determined that the two candidate source components support different cow identities.

[0008] Two candidate source components are tracked along the cross-frame correspondence. The identity matching result does not participate in the tracking path formation. After tracking to a stable video interval of a single source, the source anchoring interval is determined. When the identified identity of the source anchoring interval is consistent with the cow identity supported by the corresponding candidate source component, the cross-object mixing interval is determined according to the common existence range of the candidate source components. The cow face trajectory is then determined to be segmented based on the source continuity relationship before and after the cross-object mixing interval.

[0009] Exclude video frames that cross-object mixed regions and those that cannot be traced to the source anchor region along the cross-frame correspondence. Weight the cow face features according to the coverage of local visual regions that can be traced to the same source anchor region along the cross-frame correspondence to form trajectory representative features. Perform depth measurement matching with the cow identity template to output the cow identity.

[0010] A deep metric learning-based cattle face recognition system for complex farming environments includes:

[0011] The candidate source component determination module acquires continuous cow face videos, establishes cross-frame correspondences of local visual regions within the cow face candidate region, determines the allowable range of local deformation based on local position and head movement amplitude, compares single-source motion models and dual-source motion models within the allowable range of local deformation, and determines two candidate source components.

[0012] Different cattle identity verification modules use deep metric learning networks to extract local identity features and aggregate them to form cattle face features. The candidate source components are used as feature removal boundaries. The changes in the metric distance between the cattle face features and the two cattle identity templates before and after removal are compared. Based on the changes in the single-source calibration video of the confirmed identity, it is determined that the two candidate source components support different cattle identities.

[0013] The source tracking and cross-object mixed interval determination module tracks two candidate source components along the cross-frame correspondence. The identity matching result does not participate in the tracking path formation. After tracking to a stable video interval with a single source, the source anchor interval is determined. When the identified identity of the source anchor interval is consistent with the cow identity supported by the corresponding candidate source component, the cross-object mixed interval is determined according to the common existence range of the candidate source components, and the cow face trajectory is determined to be segmented based on the source continuity relationship before and after the cross-object mixed interval.

[0014] The trajectory representation feature formation and identity matching module excludes video frames that cross-object mixed regions and those that cannot be traced to the source anchor region along the cross-frame correspondence. It weights the cow face features based on the coverage of local visual regions that can be traced to the same source anchor region along the cross-frame correspondence, forms trajectory representation features, and performs depth measurement matching with the cow identity template to output the cow identity.

[0015] The beneficial effects of this invention are as follows:

[0016] 1. This invention establishes cross-frame correspondences between local visual regions within a cow face candidate region, and compares single-source motion models and dual-source motion models within the allowable range of normal local deformation of a single cow to determine candidate source components, thereby identifying different physical sources of local visual content while maintaining the overall continuity of the cow face candidate region.

[0017] 2. This invention uses candidate source components as the removal boundary for local identity features. By measuring the change in distance between the cow face features and different cow identity templates before and after removal, and combining it with single-source calibration data for correction, it confirms the independent support of different candidate source components for different cow identities. Subsequently, it only tracks candidate source components along the existing cross-frame correspondence and forms source anchoring intervals and cross-object mixed intervals to achieve cross-verification of cow identity information and physical source relationship.

[0018] 3. In trajectory-level recognition, this invention excludes video frames that are mixed across objects and fail to be tracked to the source anchoring interval. It also fuses the cow face features from the same source according to the source coverage and cross-frame continuity to form trajectory representative features for depth measurement matching. This restricts the local visual content of different cattle from entering the same trajectory identity feature, reducing cross-cattle identity mixing and erroneous association caused by local overlap in complex breeding environments. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort:

[0020] Figure 1 This is a timing diagram of the candidate source components for the bull face and the cross-object mixing interval of the present invention;

[0021] Figure 2 This is a schematic diagram comparing conventional continuous trajectory aggregation with the source-constrained trajectory aggregation of this invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] like Figure 1 and Figure 2As shown, this invention discloses a method for cattle face recognition based on deep metric learning in complex farming environments, including the following specific steps:

[0024] This implementation uses continuous cow face video monitored by a fixed camera at a feeding trough as the main scene. The first cow enters the feeding position first, forming a stable cow face trajectory. The second cow then enters the same camera field of view from the side, resulting in close overlap in front of the feeding trough. The same cow face candidate area simultaneously contains the local visual content of two cows within a continuous time. After the overlap ends, the first cow exits, while the second cow continues to stay. Conventional detection and tracking may maintain the same trajectory number, allowing cow face features from different sources before, during, and after the overlap to be fused into the same trajectory. This implementation sequentially determines the candidate source components, verifies the identities of different cows, tracks the candidate source components along the cross-frame correspondence, and determines the source anchoring interval. Based on this, the input range of the trajectory representative features is limited. The same processing logic is used for close-range overlapping conditions at fixed camera positions such as drinking points and narrow passages.

[0025] Two types of frozen calibration data are pre-established at the same camera position. Single-cow non-overlapping calibration video is used to form the allowable range of local deformation, the complexity constraint of motion model, the joint distribution of false multi-source events, and the normal range of physical continuity. Single-source calibration video with confirmed identity and cow identity template are used to form the identity recognition support conditions, the range of normal measurement distance difference variation, the source anchoring identity confirmation conditions, the cow face feature fusion conditions, the lower limit of effective support duration, and the lower limit of final identity confirmation. The two types of calibration data and cow identity template are kept frozen within one processing cycle.

[0026] In this implementation method, when empirical quantile statistics are involved, the quantile level is only used to determine the position of the statistical boundary read from the calibration empirical distribution. The actual statistical boundary is taken as the observation at the corresponding quantile position. The 0.99, 0.05, and other quantile levels are not directly used as threshold values. The quantile values ​​adopt a unified empirical quantile calculation rule: first, the observations are arranged in ascending order of value; when time-weighted statistics are used, the actual sampling time weights corresponding to each observation are normalized and then accumulated sequentially. The observation value that first reaches the corresponding quantile level by the accumulated weight is taken as the empirical quantile value; when time-weighted statistics are not used, each independent sample is given the same weight and is taken according to the same rule. The empirical quantile calculation does not perform interpolation or extrapolation outside the sample range. The calibration samples are composed of independent samples based on non-overlapping continuous video intervals or non-repeating local correspondences.

[0027] For empirical quantile levels above 0.90 or below 0.10, the number of independent samples in the target tail is first calculated using the preferred value. When the target tail contains at least 3 independent samples, the preferred value is adopted. When the data volume is sufficient, the preferred value is to ensure that the target tail contains no less than 5 independent samples. If the preferred value cannot satisfy 3 independent samples in the tail, the distribution is adjusted towards the middle within the corresponding publicly available preferred range, and the quantile level that is closest to the preferred value and ensures that the target tail contains 3 independent samples is selected. If the requirement is still not met within the corresponding preferred range, the statistical boundary is not formed, and the processing that depends on the statistical boundary enters the corresponding pending state. Each statistical boundary is formed directly based on the empirical distribution, without pre-setting a normal distribution or other parameter distribution. The preferred value of each quantile level is based on satisfying the sample sufficiency rule of this section.

[0028] This implementation uses a single suspected cross-cattle visual mixing event as the processing time boundary. The processing object is a local visual region as the smallest visual unit. The candidate source component consists of continuously connected local visual regions with existing cross-frame correspondences. The cow face candidate region only limits the visual search range of the current video frame and is not used as a physical source or cattle identity boundary. The candidate multi-source interval is a continuous video interval where the dual-source motion model is continuously superior to the single-source motion model, both temporary candidate sources satisfy the independent motion model solution conditions, and can be maintained along the cross-frame correspondence. The source anchoring interval is located outside the candidate multi-source interval and satisfies the conditions of single-source motion, independent source solution, and identity support. The cross-object mixing interval... To determine the maximum continuous range that includes candidate multi-source intervals and where two candidate source components satisfy the source existence condition within the same cow face candidate region, the timestamps of adjacent original videos should strictly increase. An empirical distribution of sampling intervals is formed using the time intervals between adjacent effective frames of a single cow non-overlapping calibration video. A high-side empirical quantile level in the range of 0.98 to 0.995 is used to form the upper limit of the sampling continuity interval. In this implementation, 0.99 is preferred. The actual upper limit of the interval is the time interval corresponding to the 99th percentile. When the current adjacent timestamps do not increase or the time interval exceeds the upper limit of the sampling continuity interval, the cross-frame correspondence and continuous video interval are broken at that point. The times on both sides of the breakpoint must not be combined to calculate the event duration.

[0029] In this implementation, for all time-weighted empirical quantiles, time-weighted angular distances, time-weighted identity support statistics, or other time-weighted statistics performed on video frames, the actual sampling time unit for each video frame is determined according to a unified rule based on the original video timestamp. Within a continuous statistical interval that satisfies the sampling continuity condition, the actual sampling time unit for non-endpoint video frames is half the sum of the previous and next adjacent time intervals. The first and last frames of the interval are each half the time interval between the first and last frames and the only adjacent valid video frame. Isolated continuous intervals containing only one valid video frame do not form cumulative time weights, and each actual sampling time unit must not cross a sampling continuity breakpoint. For statistical quantities such as empirical quantiles, angular distances, and identity support quantities that are not time-based, the corresponding actual sampling time units are normalized within the current statistical interval and used as time weights. For time-cumulative quantities such as valid support duration, the second dimension of the actual sampling time unit is retained.

[0030] When the frame rate is fixed and there are no dropped frames, the time-weighted empirical quantile is equivalent to the ordinary empirical quantile. Within one processing cycle, the candidate source components and cross-frame correspondences are passed to identity verification and source tracking. The source tracking results are passed to trajectory feature formation. When using online processing, for suspected cross-cattle visual mixing events, relevant video frames are cached first. Before the source anchoring relationship and trajectory relationship are formed, only the pending confirmation status is output. Subsequent video frames are only used to complete the candidate source component tracking, source anchoring interval confirmation, and trajectory relationship confirmation of the currently cached event. They are not used to change the already output confirmed cattle identity. If any stage lacks necessary input or corresponding calibration threshold, the transmission to the confirmed identity branch is stopped at that stage: insufficient source conditions are recorded as source undetermined, insufficient identity conditions are recorded as identity pending confirmation, insufficient trajectory relationship is recorded as source relationship pending confirmation, and insufficient fusion or final matching conditions are output as pending identification status.

[0031] Continuous video of the feeding trough area is acquired using a fixed camera. Cow face detection is performed on each video frame to obtain candidate cow face regions. These candidate regions are normalized to a uniform size and formed into multiple spatially continuous local visual regions according to spatial partitioning rules determined during system calibration and remaining unchanged within the current processing cycle. Spatial partitioning can employ regular grid partitioning or fixed partitions corresponding to local cow face locations. Each local visual region corresponds to at least one effective pooling unit on the depth feature map. The calibration video and the video to be identified use the same normalized size and spatial partitioning rules. For each local visual region, the normalized position of the entire image, the normalized position within the cow face candidate region, and the normalized local visual description are recorded. The local visual description is obtained by performing region pooling and unit-length normalization on the corresponding local feature map of the pre-trained visual feature extraction network. The normalized position of the entire image is used for cross-frame motion prediction, and the normalized position within the cow face candidate region is used for calibrating the allowable range of local deformation.

[0032] Between adjacent video frames, forward and backward consistent matching is performed based on the similarity of local visual descriptions and the proximity of image planes. When no historical displacement has been formed in the local visual region, the current image position is used as the predicted position for the next frame. When a historical displacement has been formed, the original time interval corresponding to the displacement is removed by the most recent effective cross-frame position to obtain the image plane normalized velocity. Then, this velocity is multiplied by the original time interval of the current adjacent video frame and superimposed on the current image position to obtain the predicted position for the next frame. If the most recent historical time interval or the current adjacent time interval does not meet the sampling continuity condition, cross-frame extrapolation is not performed using the historical displacement, and continuous matching is terminated at the corresponding time breakpoint. The search radius calibration distribution is formed by the distance between the local predicted position obtained by the single-Newton non-overlapping calibration video according to the same time scale prediction rule and the actual corresponding position. The search radius is determined by the high-side empirical quantile level in the range of 0.98 to 0.995. In this embodiment, 0.99 is preferred when the sample is sufficient. The actual search radius is the distance value of the corresponding empirical quantile position.

[0033] Within the search radius, the candidate local visual region with the smallest local visual descriptor angular distance is selected as the forward matching result. When there are the same minimum angular distances, the candidate with the smaller image plane distance is selected. Then, the current frame is retrieved in reverse from the matching result of the next frame using the same rules. Only the matching relationship of the original local visual region returned by the reverse retrieval is retained to form a cross-frame correspondence. For the forward and backward projection position deviations and the local visual descriptor angular distances of adjacent frames, the number of samples with values ​​no greater than the current value in the single non-overlapping calibration samples is counted. Then, the number of samples is divided by the total number of corresponding calibration samples to obtain two abnormal ranks between 0 and 1. The larger value of the two abnormal ranks is subtracted from 1 to obtain the reliability of the cross-frame correspondence. When the local visual content enters another candidate region of the cow face in the adjacent frame, the existing cross-frame correspondence continues to extend with the local visual content.

[0034] Based on the position of the local visual region within the candidate region of the cow face and the amplitude of head movement, the allowable range of local deformation is determined by the single cow non-overlapping calibration video. For adjacent frames in the single cow non-overlapping calibration video, a shared head motion model is used to represent the 2D projection motion of a single cow head. Preferably, a 2D affine model is used. The horizontal and vertical coordinates of the image plane of the local visual region are multiplied by the corresponding coefficients of the 2D affine transformation matrix and summed. Then, the 2D translation is superimposed to obtain the predicted position of the local visual region in the next frame. The 2D affine model contains 6 continuous parameters to be solved. To independently solve a motion model, at least 3 spatially non-collinear local visual regions with effective cross-frame correspondences are retained. When the shared head motion model uses other 2D parameterized motion models, the independent solution conditions and model complexity are determined according to the actual number of parameters to be solved.

[0035] In the single-cow non-overlapping calibration video, the normalized distance between the actual position of the local visual region and the predicted position of the shared head motion model is calculated frame by frame as the local motion residual. At the same time, the average normalized displacement of all effective local visual regions after being processed by the shared head motion model is calculated as the head motion amplitude. The calibration frames are arranged in ascending order of head motion amplitude, and adjacent motion state intervals are divided according to the principle of equal sample size. The number of effective samples in each motion state interval should meet the sample requirements of the corresponding empirical quantile statistics. If it does not meet the requirements, it is merged with the adjacent motion state interval. Within each motion state interval, the upper empirical quantile value of the local motion residual is calculated according to the position of the local visual region within the cow face candidate region. The upper empirical quantile level is preferably 0.90 to 0.98, and in this embodiment, 0.95 is preferred. The allowable amount of actual local deformation is taken as the residual value corresponding to the 95th percentile.

[0036] The current video frame to be identified is first fitted with a single-source motion model using all local visual regions that satisfy the cross-frame correspondence. The average normalized displacement obtained after applying this model to all effective local visual regions is used as the current head motion amplitude. This single-source motion model is only used to select the motion state corresponding to the local deformation allowable range and is not used as the result of determining whether there is a second source. The current head motion amplitude is used as the nominal motion state index of the local deformation allowable range. At the same time, based on the actual inter-frame normalized displacement of each effective cross-frame corresponding local visual region, the calibration motion state interval covered by the current local motion amplitude is determined. When the coverage range only involves one calibration motion state interval or two adjacent calibration motion state intervals where the current nominal motion state is located, the local deformation allowable value is read according to the aforementioned motion state index and interpolation rules. When the coverage range spans more than two calibration motion state intervals, the local deformation allowable value corresponding to each calibration motion state interval covered is read for each local position, and the maximum value is taken as the current local deformation allowable value. When the coverage range exceeds the motion amplitude coverage range of a single non-overlapping calibration video, the corresponding video frame is recorded as having an undetermined source. When the current effective local visual region is insufficient to independently solve the single-source motion model, the corresponding video frame is marked as having an undetermined source. When reading the local deformation allowance for the current local position, the bilinear interpolation of adjacent calibration positions is preferred for regular grid calibration maps, while the inverse distance weighted interpolation of adjacent calibration positions is used for irregular fixed partitions. When located at the calibration boundary, only the actually obtainable adjacent calibration positions are used. When the current head movement amplitude is between two calibration motion state intervals, linear interpolation is performed according to the distance between the current head movement amplitude and the center of the two intervals. When the range of the single-Newton non-overlapping calibration video exceeds the coverage area, the corresponding video frame is marked as having an undetermined source. When the interpolated local deformation allowance is 0 and there is a non-zero local motion residual in the calibration sample of the same motion state, the minimum non-zero local motion residual in that motion state is taken as the lower limit of the local deformation allowance. When the calibration sample cannot form a positive local deformation allowance, the corresponding video frame is marked as having an undetermined source.

[0037] Under the same allowable range of local deformation, single-source motion models and dual-source motion models are fitted respectively. For each local visual region, the normalized distance between the actual motion position and the predicted position of the current motion model is first calculated, and then the local deformation allowable amount corresponding to the local visual region is subtracted. When the difference is less than 0, it is counted as 0, and when the difference is not less than 0, the difference is retained. Then, the difference is divided by the sum of the local deformation allowable amount and the numerical stability amount to obtain the normalized effective motion residual. The numerical stability amount is preferably 0.00000001 to 0.000001, and in this embodiment, it is preferably 0.0000001. The numerical stability amount is only used to ensure that the denominator remains positive when the local deformation allowable amount is close to 0. It does not participate in the source access, identity determination or any statistical threshold formation. When using single-precision or double-precision floating-point calculation, a positive number that can be stably represented is selected within the range. Then, the square of the normalized effective motion residual is calculated, and the square is divided by the sum of the square and 1 to obtain the bounded residual value between 0 and 1.

[0038] The single-source motion model and the dual-source motion model use the same model evaluation rules. Weights are determined for local visual regions based on the reliability of cross-frame correspondence. The bounded residual values ​​of each local visual region are multiplied by their corresponding weights, summed, and then divided by the total weights to obtain the normalized motion interpretation cost. When the total weights are 0, the corresponding video frame is not evaluated and is marked as having an undetermined source. The model complexity is calculated by multiplying the total number of continuous parameters to be determined in the motion model by the natural logarithm of the number of local visual regions involved in the fitting, and then dividing by the number of local visual regions involved in the fitting. The model complexity is multiplied by the complexity weight and then added to the normalized motion interpretation cost to obtain the fitting evaluation value.

[0039] The complexity weight is determined by the single-source non-overlapping calibration video. The upper limit of the allowed proportion of false multi-source events is preferably 1% to 5%, and in this embodiment, it is preferably 2%. For each independent calibration event, the normalized motion interpretation cost of the single-source motion model is subtracted from the normalized motion interpretation cost of the dual-source motion model to obtain the interpretation cost difference; the complexity difference is obtained by subtracting the complexity of the single-source motion model from the complexity of the dual-source motion model; when the complexity difference is greater than 0, the interpretation cost difference is divided by the complexity difference to obtain the critical complexity weight that makes the fitting evaluation values ​​of the two motion models equal. The 0 and the non-negative critical complexity weight are combined to form a candidate value set and arranged from small to large. The proportion of false multi-source events corresponding to each event is counted one by one. The smallest candidate value that makes the proportion no higher than 2% is taken as the complexity weight. When the calibration events are insufficient to form candidate values ​​that meet the proportion requirement, the complexity weight is not determined.

[0040] When solving the dual-source motion model, the current effective local visual region is divided into two temporary candidate sources. The initial division uses the actual inter-frame displacement vectors of the local visual region for two-class clustering. The initial cluster centers are the displacement vectors corresponding to the two local visual regions with the largest Euclidean distance between the displacement vectors. After the initial division is completed, a shared head motion model is fitted to the two temporary candidate sources respectively. Then, according to the normalized effective motion residuals of each local visual region under the two motion models, the local visual region is redistributed to the motion model with the smaller residual, and the two motion models are refitted.

[0041] If a local visual region has the same normalized effective motion residual under two motion models and the local visual region already has a source attribution from the previous round, the source attribution from the previous round is retained. If the attribution cannot be distinguished in the initial iteration, the corresponding video frame is recorded as having an undetermined source. After each round of redistribution, the source attribution of all local visual regions is compared. When the source attribution is completely consistent with the previous round, the alternating solution ends. If a source attribution state that has appeared before but is different from the previous round appears, it is considered that periodic attribution oscillation has occurred and the corresponding video frame is recorded as having an undetermined source. When the two temporary candidate sources satisfy the solution conditions of independent motion models, the dual-source solution result is retained. When using a 2D affine model, if any temporary candidate source cannot retain at least three spatially non-collinear local visual regions with effective cross-frame correspondence, the current dual-source solution result is discarded and the corresponding video frame is recorded as having an undetermined source. The numbering of the two temporary candidate sources is kept consistent according to the continuation of the common cross-frame correspondence in adjacent video frames.

[0042] Based on the difference in fitting evaluation between the dual-source motion model and the single-source motion model, the duration of the difference, and the proportion of sources with lower local visual regions in the dual-source motion model, the results are compared with the joint distribution of the same parameters in the false multi-source events of the single-Newton non-overlapping calibration video. The relative fitting improvement is obtained by subtracting the fitting evaluation value of the dual-source motion model from the fitting evaluation value of the single-source motion model frame by frame, and then dividing by the sum of the fitting evaluation value of the single-source motion model and the numerical stability.

[0043] The candidate multi-source interval starts from the first video frame that satisfies the condition that the fitting evaluation value of the dual-source motion model is lower than that of the single-source motion model and that both temporary candidate sources can maintain the cross-frame correspondence. When this condition is not met for the first time, the previous video frame is taken as the end frame. The time-weighted low-side empirical quantile value of the relative fitting improvement, the actual duration of the interval, and the time-weighted low-side empirical quantile value of the proportion of the local visual area occupied by the smaller temporary candidate source are calculated for the candidate multi-source interval to form three joint parameters of the current event. The low-side empirical quantile level is preferably 0.20 to 0.30, and in this embodiment, it is preferably 0.25. The actual statistic is the observation value corresponding to the 25th percentile.

[0044] The single-cow non-overlapping calibration video generates spurious multi-source events and three joint parameters in the same way. For each joint parameter, the number of calibration events whose parameter value is not greater than the current parameter value is counted, and then divided by the total number of calibration events to obtain the empirical rank between 0 and 1. The three empirical ranks together constitute the normalized event state. For each calibration event, the Euclidean distance to its nth nearest neighbor calibration event is calculated as the local support radius, where n is the floor value of the square root of the number of spurious multi-source calibration events and not less than 3. The nth nearest neighbor is formed when the number of calibration events is at least n plus 1.

[0045] The high-side empirical quantile value is taken as the joint distribution allowable boundary for all local support radii. The high-side empirical quantile level is preferably 0.98 to 0.995, and in this embodiment, it is preferably 0.99. The actual joint distribution allowable boundary is taken as the local support radius corresponding to the 99th percentile. The same empirical rank calculation method is used for the current event, and the Euclidean distance from the current normalized event state to the nth nearest neighbor calibration event is calculated. When this distance exceeds the joint distribution allowable boundary and both temporary candidate sources meet the independent motion model solution conditions, the two temporary candidate sources are determined as two candidate source components.

[0046] The spurious multi-source events used to form the joint distribution allowable boundary are preferably taken from single-Newton non-overlapping calibration segments that do not overlap with the complexity weight calibration samples. If the number of spurious multi-source events is insufficient to meet the nth nearest neighbor solution conditions or the tail independent sample requirements of the high-side empirical quantile, non-overlapping single-Newton non-overlapping calibration segments are added, and spurious multi-source events are generated according to the determined complexity weights. If the supplementation still cannot meet the corresponding sample sufficiency requirements, the joint distribution allowable boundary is not formed, and the corresponding candidate multi-source interval is marked as source undetermined.

[0047] In the main scene of the feeding trough, during the stage when the first cow is feeding alone, the frame-by-frame fit evaluation value of the single-source motion model is no higher than that of the dual-source motion model, and no candidate multi-source events exceeding the joint distribution allowable boundary are formed. After the second cow enters the same cow face candidate region, the newly entered local visual region and the local visual region that the first cow continues to exist form motion sources that can be solved independently. When the dual-source fit improvement, the event duration and the smaller source visual support exceed the joint boundary formed by the single cow non-overlapping calibration video, two candidate source components are determined.

[0048] If the head movement amplitude exceeds the calibration range, any temporary candidate source does not meet the independent motion model solution conditions, there are more than two equally optimal source divisions in the dual-source solution and the source number cannot be uniquely maintained according to the cross-frame correspondence, or there are still persistent local visual regions that cannot be attributed to any independently solvable source after the dual-source solution, the corresponding video frame is recorded as source undetermined. If a continuous and effective cross-frame correspondence has been established before the overlap but approximately co-motion occurs during the overlap stage, the uninterrupted cross-frame correspondence and candidate source component number are retained; after the resolvable motion is subsequently recovered, the tracking continues along the cross-frame correspondence.

[0049] After identifying two candidate source components, the local visual regions contained within them are used as identity features to remove the boundaries. Adjacent single-source stable video intervals are retained. Within a single-source stable video interval, each frame satisfies the condition that the fitting evaluation value of the single-source motion model is not higher than that of the dual-source motion model, and the event state formed according to the candidate multi-source event rules does not exceed the joint distribution allowable boundary. The attribution of the candidate source components and their local visual regions is transferred to identity verification, and the cross-frame correspondence is transferred to source tracking.

[0050] A pre-trained, parameter-frozen deep metric learning network is used to extract local identity features. The network input is a local visual region image of a cow's face, and the output is a unit-length normalized local identity feature. Multiple local identity features are aggregated and normalized in the same way as those in the identification stage to form a cow face feature. The cow face feature and the cow identity template are located in the same deep metric feature space. The local visual description used to establish cross-frame correspondence only serves the purpose of local content matching and is not used as a local identity feature or a cow identity template. The local visual description and local identity features can be formed by different outputs of the same visual backbone or by different frozen feature extraction networks, but the calibration video and the video to be identified use the same extraction method.

[0051] The deep metric learning network is trained offline using single-source cow face images with confirmed cow identities. During the training phase, random deletion or occlusion is applied to local regions of the cow faces, ensuring that some cow face features formed by different combinations of visible regions using the same aggregation rule are still subject to the metric constraints of the corresponding cow identity. Candidate source component aggregation features that meet the identity recognition support conditions are compared with the cow identity template using the same metric distance. After network training is completed, parameters are frozen, and only feature extraction is performed in the current processing cycle. For multiple confirmed single-source cow face features of the same cow identity, the sum of angular distances between each cow face feature and the remaining cow face features of the same identity is calculated, and the actual cow face feature with the smallest sum of angular distances is selected as the cow identity template. When the template library needs to be updated, it is updated offline using only independently confirmed data obtained after the processing cycle ends. The backbone structure of the deep metric learning network is implemented using existing metric learning networks in this field.

[0052] The aggregation weight is determined based on the visibility of the local visual region and the reliability of the cross-frame correspondence. The visibility of the local visual region only evaluates whether the current image content can form an identity feature, and is not used to determine which cow the content belongs to. Source: Pixels that are actually mapped from the original image during the normalization process of the cow face candidate region, do not fall into the filling region, and can form effective local features are determined as effective visible pixels. The visibility between 0 and 1 is obtained by dividing the number of effective visible pixels by the total number of pixels in the local visual region.

[0053] For each local visual region, the visibility is multiplied by the reliability of the cross-frame correspondence to obtain the original aggregation value. Then, the original aggregation value of the local visual region is divided by the sum of the original aggregation values ​​of all valid local visual regions in the current video frame to obtain the normalized aggregation weight. When the number of valid visible pixels is 0 or no valid unit feature is obtained from the local identity feature extraction, the corresponding aggregation weight is 0. The valid cross-frame correspondence of the local visual region can still be used for source tracking. When the sum of the original aggregation values ​​of all local visual regions in the current video frame is 0, the cow face feature of the current video frame is not formed, and the current video frame does not enter the identity verification and subsequent trajectory feature fusion. When the sum of the original aggregation values ​​is greater than 0, each local identity feature is multiplied by its corresponding aggregation weight and then summed. When the norm of the weighted sum vector is greater than 0, unit length normalization is performed to form the cow face feature of the current video frame. When the norm of the weighted sum vector is 0, the cow face feature is not formed, and the current video frame does not enter the identity verification and subsequent trajectory feature fusion.

[0054] The number of neighboring cattle identities retained at each retrieval entry is preferably 3 to 5, and in this embodiment, 3 is preferred. The actual number retained does not exceed the number of available cattle identities in the current cattle identity template library. If there are fewer than 2 available cattle identities in the template library, candidate identity pair comparison is not performed, and the current interval is recorded as identity pending confirmation. For each candidate source component, the local visual regions belonging to the candidate source component in the current video frame are aggregated and summed to obtain the identity support quantity. Then, the identity support quantity in the candidate multi-source interval is calculated according to the original video timestamp to obtain the time-weighted low-side empirical quantile value, and the continuous identity support quantity is obtained. The low-side empirical quantile level is preferably 0.20 to 0.30, and in this embodiment, 0.25 is preferred.

[0055] When calibrating the lower bound of identity recognition support using a single-source calibration video with confirmed identity, this lower bound is not used as a pre-gating control. The aggregation weights in the calibration frame are first normalized according to the original aggregation amounts of all effective local visual regions in the frame, and then the aggregation weights of the selected local visual region combinations are summed. The selected combinations are not re-normalized. For each independent calibration segment, the cow identity template is retrieved using the actual local visual region combinations. The minimum total aggregation weight is recorded in the region combinations where the correct cow identity can enter the nearest template set, which is used as the minimum support amount for the calibration segment. If the correct cow identity still cannot enter the nearest template set when all effective local visual regions participate in the aggregation of an independent calibration segment, then the calibration segment indicates that the current model and template library are insufficient to form an identity support threshold, and the lower bound of identity recognition support is not used for deterministic determination.

[0056] For independent calibration fragments that can be used to form the minimum support, a high-side empirical quantile in the range of 0.90 to 0.98 is adopted. In this embodiment, 0.95 is preferred when the sample is sufficient. The aggregate weight value of the corresponding empirical quantile position is determined as the lower limit of identity recognition support. When the continuous identity support is not lower than the lower limit of identity recognition support, the candidate source component meets the identity recognition support condition. When the lower limit of identity recognition support cannot be formed, the current candidate multi-source interval is recorded as identity pending confirmation.

[0057] The cow face feature and the cow identity template are compared using a unit feature angle distance. First, a vector inner product is performed on the cow face feature and the cow identity template, and the inner product result is limited to between -1 and 1. Then, the inverse cosine value of the inner product result is calculated, and finally divided by pi to obtain a metric distance between 0 and 1. For each candidate source component that meets the identity recognition support condition, the local identity features belonging to the candidate source component are taken frame by frame in the candidate multi-source interval. Each local identity feature is multiplied by its aggregation weight, which was determined when the complete cow face feature was aggregated, and then summed. The non-zero summation vector is normalized by unit length to form the source aggregation feature of the current video frame. The aggregation weights are not reallocated within the candidate source components.

[0058] Calculate the total time-weighted angular distance between each valid actual frame source aggregate feature and other valid frame source aggregate features of the same candidate source component, and select the actual frame source aggregate feature with the smallest total distance as the source representative feature of the candidate source component. For adjacent single-source stable video intervals of the candidate multi-source interval, form stable cow face representative features from the actual cow face features according to the same principle. The candidate cow identity is formed by retrieving the cow identity template only by the source representative features of the two candidate source components and the stable cow face representative features of the adjacent single-source stable video intervals. The mixed cow face feature formed by all local identity features in the cow face candidate area does not participate in the determination of the candidate cow identity. When there are fewer than two candidate cow identities, the current candidate multi-source interval is recorded as identity pending confirmation.

[0059] Using candidate source components as feature removal boundaries, compare the changes in the metric distance difference between the cow face feature and the two cow identity templates before and after removal. For any two candidate cow identities, first calculate the metric distance between the current cow face feature and the two candidate cow identity templates respectively. Then, subtract the metric distance between the current cow face feature and the first candidate cow identity template from the metric distance between the second and first candidate cow identity templates to obtain the metric distance difference before removal. Subsequently, for one candidate source component, remove the local identity features belonging to that candidate source component from the local identity feature aggregation. The remaining local identity features continue to use the original aggregation weights without reallocation. Sum the remaining local identity features according to the original aggregation weights and perform unit length normalization to form conditional cow face features. If there are no remaining valid local identity features after removal, or if the weighted sum of the remaining local identity features cannot form a non-zero unit feature, do not calculate the change in the metric distance difference corresponding to that candidate source component, and record the current candidate multi-source interval as identity pending confirmation. Perform the same processing on the two candidate source components respectively.

[0060] For each conditional cow face feature, calculate the difference in metric distance between it and the same two candidate cow identity templates. Then, subtract the difference in metric distance after removal from the difference in metric distance before removal to obtain the change in the original metric distance difference of the corresponding candidate source component. When the change in the original metric distance difference is positive, the corresponding candidate source component supports the first candidate cow identity relative to the second candidate cow identity; when the change in the original metric distance difference is negative, the corresponding candidate source component supports the second candidate cow identity relative to the first candidate cow identity.

[0061] Based on the confirmed single-source calibration video, the original metric distance difference change is corrected. For the currently removed candidate source component, the joint region of its contained local visual regions in normalized bull's face coordinates is taken as the current spatial footprint, and the sum of the aggregated weights of the local visual regions within the candidate source component is calculated. For the candidate local region combination in the confirmed single-source calibration video, the intersection area of ​​the joint region of the combination and the current spatial footprint is divided by the union area to obtain the spatial overlap degree between 0 and 1. Then, the absolute difference of the total aggregated weights of the two region combinations is calculated, and this absolute difference is also... From 0 to 1, subtract the spatial overlap from 1 and then add the absolute difference of the total aggregate weight to obtain the dimensionless matching cost. Organize the single-source calibration videos with confirmed identities into non-overlapping continuous calibration segments. For each continuous calibration segment, only the region combination with the smallest matching cost is retained as a statistical sample. Then, according to the matching cost from small to large, the calibration samples whose true identity belongs to one of the current two candidate cattle identities are selected until the number of samples meets the tail sample requirements of the subsequent 0.05 and 0.95 empirical quantile statistics. If the number of samples is still insufficient, the current candidate multi-source interval is recorded as identity pending confirmation.

[0062] The same local identity feature removal as the current candidate source component is performed on the selected single-source calibration region combination to obtain the measurement distance difference change sample under single-source conditions. The central coverage ratio of the normal measurement distance difference change range is preferably 80% to 95%, and in this embodiment, it is preferably 90%. When the central coverage ratio is 90%, the uncovered 10% is evenly distributed to the low-value side and high-value side of the distribution. The lower boundary is the measurement distance difference change value corresponding to the 5th percentile, and the upper boundary is the measurement distance difference change value corresponding to the 95th percentile. When other central coverage ratios are used, the tail of the low-value side and the tail of the high-value side are distributed in the same way. When the current original measurement distance difference change is higher than the upper boundary, the positive correction amount is obtained by subtracting the upper boundary from the current original measurement distance difference change. When it is lower than the lower boundary, the negative correction amount is obtained by subtracting the current original measurement distance difference change from the lower boundary and assigning a negative sign. When it is within the normal measurement distance difference change range, the corrected measurement distance difference change is 0. The calibration sample used to form the normal measurement distance difference change range does not overlap with the calibration verification sample used to form the identity split allowable value.

[0063] For each ordered candidate identity pair, candidate identity 1 is used as the supporting object of candidate source component 1, and candidate identity 2 is used as the supporting object of candidate source component 2. When candidate source component 1 forms a positive correction value and candidate source component 2 forms a negative correction value, the absolute values ​​of the positive correction value and the negative correction value are compared, and the smaller value is taken as the identity split value of the video frame. When the two directions are not simultaneously established, the identity split value is 0. For video frames that meet the identity recognition support conditions in the candidate multi-source interval, the identity split value is counted according to the original video timestamp, and the interval-level identity split strength is formed by using the low-side empirical quantile level in the range of 0.20 to 0.30. In this embodiment, 0.25 is preferred, and the actual interval-level identity split strength is the identity split value corresponding to the 25th percentile.

[0064] The permissible value for identity splitting is formed separately for each candidate cattle identity pair. The actual duration of the current candidate multi-source interval is used as the truncation length. Non-overlapping continuous calibration segments are truncated from the confirmed single-source calibration verification video of the identity that has never participated in the normal measurement distance difference change range. The tail segment with a length less than the current duration is not included in the calibration. Within each continuous calibration segment, the two non-overlapping local regions with the lowest matching cost of the spatial footprint and total aggregation weight of candidate source component 1 and candidate source component 2 are selected respectively. The same local identity feature removal, normal change correction and interval-level identity splitting calculation are performed to obtain the false identity splitting intensity distribution of the corresponding candidate cattle identity pair under the single-source condition.

[0065] The distribution is used to form the identity splitting allowance value with a high-side empirical quantile level in the range of 0.98 to 0.995. In this implementation, 0.99 is preferred. The actual identity splitting allowance value is the false identity splitting intensity corresponding to the 99th percentile. When only one ordered identity pair in the candidate cattle identities has an interval-level identity splitting intensity that exceeds the corresponding identity splitting allowance value, two candidate source components are determined to support different cattle identities. When two or more ordered identity pairs exceed the corresponding identity splitting allowance value at the same time, the current interval is recorded as identity pending confirmation.

[0066] In the main scene of the feeding trough, candidate source component 1 continues to support the identity of the first cow along the local visual content that existed before the overlap, and candidate source component 2 continues to support the identity of the second cow along the local visual content that newly entered during the overlap stage. Within the same video frame, the two candidate source components form contributions in opposite directions that exceed the range of corresponding normal metric distance difference changes. When the contribution continuously exceeds the identity splitting allowable value in the candidate multi-source interval, it is confirmed that the two candidate source components support different cow identities.

[0067] When both candidate source components meet the identity recognition support conditions, but there is no unique ordered identity pair exceeding the identity splitting allowable value, cross-object identity confirmation is not formed; when the independent retrieval of both candidate source components and adjacent single-source stable video intervals points to the same cattle identity, the candidate multi-source interval is treated as a same identity motion interval, and the cattle face features in this interval do not enter the formation of the current trajectory representative features. If any of the following conditions is insufficient: identity recognition support conditions, normal measurement distance difference variation range, number of candidate cattle identities, or identity splitting allowable value, the current interval is recorded as identity pending confirmation, and it will only enter source tracking after confirming that the two candidate source components support different cattle identities.

[0068] Two candidate source components are tracked along the cross-frame correspondence. The identity matching result does not participate in the tracking path formation. The local visual region in the two candidate source components is used as the tracking starting point. The tracking only extends forward and backward along the established cross-frame correspondence. For the correspondence of each adjacent video frame, a shared head motion model with the same form as the candidate source component determination stage is fitted according to the continuously maintained local visual region in the current candidate source component. The normalized motion prediction deviation between the actual position and the predicted position of the local visual region is obtained. The angular distance of the normalized local visual description corresponding to the adjacent frame is calculated to obtain the change of local visual description, and the reliability of the cross-frame correspondence is read.

[0069] The reliability of motion prediction bias, local visual description variation, and cross-frame correspondence is mapped to the abnormal rank of the single-Newton non-overlapping calibration video. For motion prediction bias and local visual description variation, the number of calibration samples with values ​​no greater than the current value is counted and then divided by the total number of corresponding calibration samples to obtain the abnormal rank between 0 and 1. For the reliability of cross-frame correspondence, the number of calibration samples with a reliability no less than the current reliability is counted and then divided by the total number of calibration samples, so that the lower the reliability, the larger the corresponding abnormal rank.

[0070] After forming abnormal ranks, the three input quantities are converted into dimensionless values ​​of 0 to 1. The maximum value among the three abnormal ranks is then taken as the physical continuity cost of the current cross-frame correspondence. Cross-dimensional calculations are not performed directly on the original position distance, feature angle distance, and reliability. Instead, the physical continuity cost is calculated in the same way using single-Newton non-overlapping calibration samples that are independent of the current processing cycle. The physical continuity allowable value is formed using a high-side empirical quantile level in the range of 0.98 to 0.995. In this implementation, 0.99 is preferred. The actual physical continuity allowable value is the physical continuity cost value corresponding to the 99th percentile. When the physical continuity cost exceeds the physical continuity allowable value, the source propagation of the corresponding local visual region is terminated. If other local visual regions of the same candidate source component still meet the source existence condition, the entire candidate source component is not terminated due to the termination of a single correspondence.

[0071] Source tracing employs both source existence conditions and independent motion model solution conditions. The source existence condition is satisfied when a candidate source component retains at least one local visual region that extends continuously from the candidate multi-source interval and whose physical continuity cost does not exceed the allowable value of physical continuity. When using a 2D affine model, the independent motion model solution condition is satisfied when a candidate source component retains at least three spatially non-collinear effective local visual regions in the current video frame. The edges of cross-object mixing intervals are determined according to the source existence condition, and the source anchoring candidate interval must also satisfy the independent motion model solution condition.

[0072] After a candidate source component leaves the candidate multi-source interval along the cross-frame correspondence, it continuously satisfies the independent motion model solution conditions within the continuous video interval. This continuous video interval contains at least two adjacent valid video frames that satisfy the sampling continuity condition, and each frame satisfies that the single-source motion model fitting evaluation value is not higher than that of the dual-source motion model, and no candidate multi-source event exceeding the joint distribution allowable boundary is formed. When there is no other candidate source component in the same cow face candidate region that continuously satisfies the source existence condition along the valid cross-frame correspondence, the continuous video interval is determined as the source anchoring candidate interval. The source anchoring candidate interval only indicates that the candidate source component has been tracked to the single-source stable video interval along the existing cross-frame correspondence, and does not directly assign the identity of a cow.

[0073] The weights of local visual regions within the source anchoring candidate interval that can participate in the formation of identity features are aggregated and summed. The time-weighted low-side empirical quantile value of the aggregated weights is calculated according to the original video timestamp. The low-side empirical quantile level is preferably 0.20 to 0.30, and in this embodiment, it is preferably 0.25. The actual continuous identity support statistic is taken as the aggregated weights corresponding to the 25th percentile. When the statistic is not lower than the identity recognition support lower limit, the source anchoring candidate interval is determined as the source anchoring interval; when it is lower than the identity recognition support lower limit, only the source tracking result and the source anchoring candidate interval are retained, and source anchoring identity confirmation is not performed.

[0074] After the source anchoring interval is determined, candidate cow face features are selected one by one from the valid cow face features in the source anchoring interval. The total time-weighted angular distance between each candidate cow face feature and other valid cow face features in the same interval is calculated, and the actual cow face feature with the smallest total distance is selected as the source anchoring representative feature. As an alternative implementation, the valid cow face features in the source anchoring interval can be weighted and summed according to the aggregation weight and normalized by unit length to form the source anchoring representative feature. The source anchoring representative feature is then matched with the cow identity template using depth measurement to determine the first nearest neighbor cow identity template with the smallest measurement distance and the second nearest neighbor cow identity template with the second smallest measurement distance. Then, the measurement distance between the source anchoring representative feature and the second nearest neighbor cow identity template is subtracted from the measurement distance between the source anchoring representative feature and the first nearest neighbor cow identity template to obtain the identity confirmation interval of the source anchoring interval.

[0075] For the first nearest neighbor cattle identity and the second nearest neighbor competing identity in the source anchoring stage, the actual duration of the current source anchoring interval is used as the truncation length. Non-overlapping continuous calibration intervals are extracted from the confirmed identity single source calibration video whose real identity belongs to the first nearest neighbor cattle identity, and calibration source anchoring representative features are formed in the same way.

[0076] Only the real cattle identity template is retained as the first nearest neighbor in the complete template library retrieval, and the current second nearest neighbor competing identity template is the calibration interval of the closest competing template. The metric distance between the calibration source anchor representative feature and the real cattle identity template and the competing identity template are calculated respectively. Then, the metric distance with the real cattle identity template is subtracted from the metric distance with the competing identity template to obtain the positive calibration identity confirmation interval. The lower limit of identity confirmation is formed by using the low-side empirical quantile level in the range of 0.03 to 0.10 for the positive calibration identity confirmation interval. In this embodiment, 0.05 is preferred when the sample is sufficient. The actual identity confirmation lower limit is the identity confirmation interval value of the corresponding empirical quantile position. The identity confirmation lower limit should be greater than 0. If the sample is insufficient, the calibration interval cannot keep the real identity as the first nearest neighbor, or the formed identity confirmation lower limit is not greater than 0, the identity pair is not used for deterministic identity confirmation and is recorded as the source identity to be confirmed. When the identity confirmation interval of the source anchor interval is not lower than the effective identity confirmation lower limit, the identification identity of the source anchor interval is determined.

[0077] The consistency between the identified identity of the source anchoring interval and the cow identity supported by the corresponding candidate source component during the identity verification process is verified. The two candidate source components are tracked to the source anchoring intervals of different cow identities, and when the identified identity of the source anchoring interval is consistent with the cow identities supported by the two candidate source components, it is determined that the two candidate source components have independent physical source and identity correspondence. When the identified identities are inconsistent, it is recorded as a source identity conflict. In the main scene of the feeding trough, candidate source component 1 is tracked to the single-source stable video interval of the first cow along the cross-frame correspondence before the overlap, and candidate source component 2 is tracked to the single-source stable video interval formed by the second cow continuing to stay along the cross-frame correspondence after the overlap. When the identified identities of the two source anchoring intervals are consistent with the cow identities corresponding to the identity verification stage, the correspondence between the physical source and the cow identity of the candidate source component is completed.

[0078] After the identified identity in the source anchoring interval matches the cow identity supported by the corresponding candidate source component, the cross-object mixing interval is determined based on the common existence range of the candidate source components within the same cow face candidate region. First, a continuous video range in the candidate multi-source interval is determined where both candidate source components continuously satisfy the independent motion model solution conditions. Then, the range is extended forward and backward along the cross-frame correspondence of the two candidate source components. In the extended edge video frames, only when the candidate source component is still located within the same evaluated cow face candidate region and retains at least one local visual region directly connected to the continuous video range with a physical continuity cost not exceeding the physical continuity allowable value, is it determined that the candidate source component still satisfies the source existence condition within the cow face candidate region. When the candidate source component has entered another independent cow face candidate region, its cross-frame correspondence can still be used for source anchoring, but it is no longer used to extend the current cross-object mixing interval.

[0079] The maximum continuous range containing multiple candidate source intervals and where two candidate source components both satisfy the source existence condition within the same cow face candidate region is defined as the cross-object mixing interval. Its starting frame is the frame where the two candidate source components first simultaneously satisfy the condition when extending forward, and its ending frame is the frame where the two candidate source components last simultaneously satisfy the condition when extending backward.

[0080] The determination of whether the cow face trajectory is segmented is based on the source continuity relationship before and after the cross-object mixing interval. If the candidate source component that exists continuously before the cross-object mixing interval does not have any source existence evidence after the interval that can continue to propagate to the same source anchor interval along the existing cross-frame correspondence, and the remaining candidate source components continue to be tracked to the corresponding source anchor interval after the interval, cow face trajectories are formed before and after the cross-object mixing interval respectively. The video frames within the cross-object mixing interval do not participate in the formation of the trajectory representative features of any cow face trajectory. If the candidate source component that exists continuously before the cross-object mixing interval is tracked to the source anchor interval with the same cow identity before and after the interval, the cow face trajectory remains continuous, and the video frames within the cross-object mixing interval do not participate in the formation of the trajectory representative features of the cow face trajectory.

[0081] In the main scenario of the feeding trough, the first cow exits after the cross-object mixing interval, and candidate source component 1 no longer continues; candidate source component 2 continues to exist after this interval and is tracked to the source anchoring interval of the second cow. Therefore, the trajectory of the first cow and the trajectory of the second cow are formed before and after the cross-object mixing interval, respectively, and the cross-object mixing interval serves as the isolation interval between the two trajectories.

[0082] If two candidate source components continue to coexist after the cross-object mixing interval, or if the source anchoring relationship before and after the cross-object mixing interval is insufficient to determine the cow face trajectory segmentation relationship, it is recorded as a source relationship pending confirmation. If the video starts from a state of severe overlap and lacks source anchoring information before the cross-object mixing interval, and the two candidate source components are tracked to different source anchoring intervals in subsequent videos, it is only determined that there are two different cow sources and a cross-object mixing interval in the current video interval, and the cow face trajectory attribution before the start of the video is not presumed. When two cow heads show approximately consistent 2D projection motion in the cross-object mixing interval, the candidate source component number is maintained if the existing cross-frame correspondence is not interrupted; when the cross-frame correspondence is interrupted, the corresponding source tracking is terminated at the interruption position.

[0083] Once the source anchoring interval, cross-object mixed interval, and cow face trajectory segmentation or continuity relationship are determined, trajectory representative features are formed. Video frames that cross-object mixed intervals, fail to be tracked to the source anchoring interval, and video frames corresponding to source identity conflicts, unconfirmed source identities, and unconfirmed source relationships do not participate in the formation of the current trajectory representative features.

[0084] For video frames that are not excluded, the fusion weights for the cow face feature are determined based on the coverage of local visual regions that can be traced to the same source anchor interval and the continuity of cross-frame correspondences. First, the aggregate weights of the local visual regions that can be traced to the same source anchor interval along the cross-frame correspondences are summed to obtain the source coverage value. Then, only from these local visual regions that can be traced to the same source anchor interval, valid forward and backward cross-frame correspondences that are adjacent to the current video frame and whose physical continuity cost does not exceed the physical continuity allowable value are read. This avoids local correspondences from other sources from participating in the continuity calculation of the current source. When the physical continuity allowable value is greater than 0, each physical continuity cost is divided by the physical continuity allowable value to form a normalization between 0 and 1. Continuity ratio; when the physical continuity allowable value is equal to 0, only the correspondence with a physical continuity cost of 0 is retained, and its normalized continuity ratio is recorded as 0. For the first and last frames of the current bull face trajectory, only the actual existing single-sided effective cross-frame correspondence is used. When the current video frame does not have an effective cross-frame correspondence consistent with the current source, the video frame is not eligible for trajectory feature fusion. When there is an effective cross-frame correspondence, the normalized continuity ratio is adopted at the middle and high side empirical quantile level in the range of 0.70 to 0.80. In this implementation, 0.75 is preferred. Then, the normalized continuity ratio of the corresponding empirical quantile position is subtracted from 1 to obtain the cross-frame continuity value. The source coverage value is multiplied by the cross-frame continuity value to obtain the fusion weight between 0 and 1.

[0085] The fusion conditions for cow face features are determined by single-source calibration videos with confirmed identities. The fusion weights for the calibration videos are calculated in the same way as for the current video, and only video frames that meet the source anchoring identity confirmation conditions are retained. A lower limit for fusion is formed for these fusion weights using a low-side empirical quantile level in the range of 0.03 to 0.10. In this implementation, 0.05 is preferred when there are sufficient samples. The actual lower limit for fusion is the fusion weight value at the corresponding empirical quantile position, rather than directly using 0.05 as the fusion weight threshold. When the current video frame is not excluded and the fusion weight is greater than 0 and not lower than the lower limit for fusion, it is included in the cow face feature set corresponding to the cow face trajectory. When a lower limit for fusion cannot be formed, the corresponding cow face trajectory is recorded as to be identified.

[0086] The effective support duration is determined based on the fusion weight and video sampling time. First, the admitted video frames are divided into continuous admission segments that do not cross the prohibition time according to the cross-object mixing interval, video frames that cannot be tracked to the source anchor interval, sampling continuity breakpoints, and other prohibited states. For a continuous admission segment containing at least 2 video frames, the adjacent original timestamps ordered by time are set as the timestamp of the current frame and the timestamps of the frames before and after it. The actual sampling time unit of the non-endpoint video frames in the segment is taken as 1 / 2 of the sum of the previous adjacent time interval and the next adjacent time interval. The first and last frames of the continuous admission segment are respectively taken as 1 / 2 of the time interval with the only adjacent admission frame. A continuous admission segment containing only 1 video frame does not form an effective support duration that can be accumulated. The actual sampling time units do not overlap with each other and do not cross the sampling continuity breakpoint or the excluded video period.

[0087] The fusion weight of each valid video frame is multiplied by its actual sampling time unit, and then the results of the current cow face trajectory calculation are summed to obtain the effective support duration in seconds. When calibrating the lower limit of the effective support duration using the single-source calibration video with confirmed identity, the lower limit of the effective support duration is not used as a pre-gating. The original video timestamp is used as the candidate start and end boundary to form continuous calibration sub-intervals with different actual durations. Each sub-interval only executes the determined fusion conditions, source anchor representative feature formation rules, scale parameter rules, and final identity confirmation rules. In this calibration process, if there are no fused cow face features outside the source anchor interval, the source anchor representative features are directly used as the trajectory representative features, and the lower limit of the effective support duration that has not yet been formed is not checked.

[0088] For each independent calibration segment, the minimum effective support duration that can still correctly output the confirmed identity is recorded. Then, the minimum effective support duration of each segment is used to form the lower limit of effective support duration by using the high-side empirical quantile level in the range of 0.90 to 0.98. In this implementation, 0.95 is preferred. The actual effective support duration lower limit is the duration value corresponding to the 95th percentile. When the current effective support duration is lower than the effective support duration lower limit, the status of pending identification is output, and the trajectory representative feature solution is not continued.

[0089] When the current effective support duration is not less than the lower limit of the effective support duration, the source anchor representative features already formed in the source anchoring stage are directly called as the anchor features for trajectory-level solution, and new source anchor representative features are not formed again in the trajectory fusion stage.

[0090] The scale parameter is determined by the angular distance between the bullface feature within the source anchoring interval and the source anchoring representative feature. When there are at least 4 valid bullface features with different original timestamps within the source anchoring interval, the unit feature angular distance between each valid bullface feature and the source anchoring representative feature is calculated. The angular distance scale is read at the middle and high side empirical quantile level in the range of 0.70 to 0.80. In this embodiment, 0.75 is preferred. The actual scale parameter is the angular distance value at the corresponding empirical quantile position. When the frame rate is fixed and the 4 samples are equally weighted, the 75th percentile corresponds to the 3rd ordered angular distance.

[0091] When using time-weighted statistics, if the target quantile position is determined by the maximum angular distance of a single endpoint and its time weight, the scale parameter is not directly determined by the sample of that single endpoint. Instead, the scale parameter is formed from the confirmed single-source calibration video of the same camera position and the same cattle identity according to the same angular distance statistical rules. If there are fewer than 4 valid cattle face features in the source anchoring interval, the scale parameter is also formed by freezing the calibration video. When the obtained scale parameter is 0, the minimum value of the non-zero unit feature angular distance in the frozen calibration video is taken as the lower limit of the scale parameter. If a positive scale parameter still cannot be formed, the corresponding cattle face trajectory is recorded as to be identified.

[0092] Within the unit depth metric feature space, according to the fusion weights, a bounded angular distance calculation logic limited by the scale parameter is used to weight and solve the source anchor representative features and the same source bullface features outside the source anchor interval to obtain the trajectory representative features. For any candidate trajectory feature, the angular distance between the source anchor representative feature and the candidate trajectory feature is first calculated. The square of the angular distance is then divided by the sum of the square of the angular distance and the square of the scale parameter to obtain the source anchor angular distance cost. Then, the angular distance between each same source bullface feature outside the source anchor interval and the candidate trajectory feature is calculated respectively, and the corresponding angular distance cost is obtained in the same way.

[0093] The anchor weight is obtained by summing the fusion weights of the video frames that meet the fusion conditions within the source anchoring interval. When the anchor weight is equal to 0, the corresponding cow face trajectory is recorded as to be identified, and no trajectory representative feature optimization is performed. When the anchor weight is greater than 0, the anchor weight is multiplied by the source anchoring angular distance cost. The angular distance costs of each cow face feature outside the source anchoring interval are multiplied by the corresponding fusion weight and summed. The two results are then added together to obtain the total weighted cost of the candidate trajectory features. Under the unit length constraint, the candidate trajectory feature with the minimum total weighted cost is determined and used as the trajectory representative feature.

[0094] The trajectory representation features can be solved using existing numerical optimization methods capable of handling unit norm constraints. The optimization variable is the unit depth metric feature, and the optimization objective is the total weighted cost of the candidate trajectory features. Therefore, the numerical optimization method does not introduce new identification or source determination quantities. The same optimization is performed using the source-anchored representation feature and the actual cow face feature with higher fusion weights as initial values. Only the results that satisfy the unit norm constraint and are determined to be converged by the optimizer are considered valid converged solutions. Unconverged initial values ​​are not included in subsequent comparisons. When no valid converged solution exists, the corresponding cow face trajectory is recorded as unrecognized. When two or more valid converged solutions exist and their first nearest neighbor cow identities are inconsistent, the corresponding cow face trajectory is recorded as unrecognized. When the first nearest neighbor cow identities of valid converged solutions are consistent, the comparison is performed. Comparing the total weighted costs of each effective converged solution, the effective converged solution with the smallest total weighted cost is selected as the trajectory representative feature. As an alternative implementation that does not rely on a continuous numerical optimizer, a finite candidate set can be formed by the source anchor representative feature and all admitted bull face features in the current bull face trajectory. The total weighted cost of each candidate feature is calculated one by one according to the bounded angular distance calculation logic limited by the scale parameter. The unit feature with the smallest total weighted cost is selected as the trajectory representative feature. When there are multiple identical minimum total weighted costs, the source anchor representative feature is selected first. When there are no bull face features that meet the fusion conditions outside the source anchor interval, and the effective support time of the source anchor interval itself is not lower than the lower limit of the effective support time, the source anchor representative feature is directly used as the trajectory representative feature.

[0095] The trajectory representation features are matched with the cattle identity templates using depth measurement. If there are fewer than two available cattle identities in the cattle identity template library, a second nearest neighbor competing identity is not formed and the state to be identified is output. When the template library meets the comparison conditions, the first nearest neighbor cattle identity template with the smallest measurement distance from the trajectory representation features and the second nearest neighbor cattle identity template with the second smallest measurement distance are determined. Then, the measurement distance between the trajectory representation features and the second nearest neighbor cattle identity template is subtracted from the measurement distance between the trajectory representation features and the first nearest neighbor cattle identity template to obtain the identity confirmation interval. Finally, the identity confirmation priority is to call the identity confirmation lower limit that has been formed and is greater than 0 for the same first nearest neighbor cattle identity and second nearest neighbor competing identity in the source anchoring stage. If the final identity pair is different from that in the source anchoring stage, the identity confirmation lower limit of the identity pair is formed according to the same calibration interval as in the source anchoring stage, the real identity first nearest neighbor verification and quantile statistics rules. The lower side empirical quantile level is preferably 0.03 to 0.10. In this implementation, 0.05 is preferred when the sample is sufficient.

[0096] If there is a lack of corresponding identity pair calibration data that meets the sample requirements, or if the lower limit of identity confirmation is not greater than 0, the status to be identified is output. If the identity of the first nearest neighbor cattle is consistent with the identity identified in the source anchoring interval, and the identity confirmation interval is not lower than the corresponding lower limit of identity confirmation, the identity of the first nearest neighbor cattle is output; otherwise, the status to be identified is output. The results of unidentified, identity unconfirmed, source undetermined, source identity conflict, and source relationship unconfirmed generated in the current processing cycle are not used to update the cattle identity template or calibration data. The cattle identity template and calibration data are only updated offline with independent confirmation data after the processing cycle ends.

[0097] In the main scene of the feeding trough, the first trajectory representative feature is formed by the access cow face features that can be tracked to the first cow source anchoring interval before the cross-object mixing interval, and the second trajectory representative feature is formed by the access cow face features that can be tracked to the second cow source anchoring interval after the cross-object mixing interval. The two trajectories are matched with the cow identity template by depth measurement, and the corresponding cow identity is output according to the identity confirmation interval.

[0098] Using multi-frame fusion of conventional detection trajectories in a takeover scenario from the same source as a comparison, the conventional method uses the detection trajectory number as the boundary of trajectory composition. It does not perform candidate source component determination, local identity feature removal verification, candidate source component tracking to the source anchoring interval, or exclusion of cross-object mixed intervals. The number of video frames entering the trajectory feature formation within the cross-object mixed interval is divided by the number of complete video frames in that interval to obtain the mixed interval feature admission rate. According to the conventional trajectory composition rule, this rate is 100%, while according to the trajectory composition rule of this implementation, this rate is 0%. The proportion of the duration of the cross-object mixed interval included in the trajectory observation time is used as the mixed time inclusion rate. According to the conventional rule, this rate is 100%, while according to this implementation, it is 0%. Both values ​​are directly calculated by the corresponding trajectory composition rule and are only used to quantify the difference in trajectory input boundaries. They are not used as experimental recognition rate, recognition accuracy, or performance improvement.

[0099] The differences in this comparison are formed by the trajectory input boundary after four steps of linkage: the candidate source component gives the boundary for the removal of local identity features, the identity verification confirms the cross-cow identity relationship, the source tracking forms the source anchoring interval and the cross-object mixed interval, and the trajectory fusion only uses the cow face features that can be tracked to the source anchoring interval and meet the fusion conditions.

[0100] In the case of short-term intrusion of the source, there are source components that persist before and after the cross-object mixing interval among the candidate source components, and there are source components that enter the current cow face candidate region only within the cross-object mixing interval and terminate after that interval. When the persistent candidate source components are tracked to the source anchoring interval of the same cow identity before and after the cross-object mixing interval, the cow face trajectory remains continuous. The video frames within the cross-object mixing interval do not participate in the formation of trajectory representative features. The cow face features that can be tracked to the same source anchoring interval before and after the cross-object mixing interval jointly form the trajectory representative features.

[0101] In cases where the video starts from a state of severe overlap, if there is a lack of source anchoring information before the cross-object mixing interval, the historical source relationship that does not exist is not constructed. If the subsequent two candidate source components are tracked to different source anchoring intervals along the existing cross-frame correspondence, only the two different cow sources and the cross-object mixing interval in the current video interval are confirmed. When the two cow heads show approximately consistent 2D projection motion during the overlap stage, the cross-frame correspondence that has been formed and has not been interrupted continues to retain the candidate source component number. When the cross-frame correspondence is interrupted, the tracking of the corresponding source is terminated at the interruption position.

[0102] When frame drops or frame rate changes occur, the event duration, interval quantile statistics, and effective support duration all use the original video timestamp and are constrained by the upper limit of the sampling continuity interval. Time breaks exceeding this upper limit are not accumulated as continuous event times. When the installation pose of the camera equipment, video resolution, optical zoom, or other acquisition configurations that change the normalized imaging geometry of the cow face change, the calibration data corresponding to the current camera position stops being used for deterministic determination. Single cow non-overlapping calibration videos and single-source calibration videos with confirmed identities are re-acquired, and the allowable range of local deformation, the allowable boundary of joint distribution, the allowable value of physical continuity, and the identity-related threshold are re-formed. Local visual region segmentation, local visual description extraction, depth metric learning network backbone, and unit norm constraint optimization methods can adopt commonly used implementations in this field. However, the calibration data and the video to be identified must adopt consistent normalized imaging rules, local visual region definitions, and depth metric feature spaces.

[0103] According to the processing flow of this embodiment, the input range of the trajectory representation feature is determined by the result of the candidate source component being tracked to the source anchoring interval. The identity verification of the candidate source component confirms the cross-cow identity relationship. The source anchoring interval and the cross-object mixing interval determine the temporal boundary of the cow face feature from the same source. The fusion weight and effective support duration further determine the video frames that can participate in the trajectory representation and their contributions. The mixed cow face features in the cross-object mixing interval do not enter the trajectory representation from the same source, so that the final depth metric matches the set of cow face features verified by the source.

[0104] This invention also discloses a cattle face recognition system based on deep metric learning for complex farming environments, comprising:

[0105] The candidate source component determination module acquires continuous cow face videos, establishes cross-frame correspondences of local visual regions within the cow face candidate region, determines the allowable range of local deformation based on local position and head movement amplitude, compares single-source motion models and dual-source motion models within the allowable range of local deformation, and determines two candidate source components.

[0106] Different cattle identity verification modules use deep metric learning networks to extract local identity features and aggregate them to form cattle face features. The candidate source components are used as feature removal boundaries. The changes in the metric distance between the cattle face features and the two cattle identity templates before and after removal are compared. Based on the changes in the single-source calibration video of the confirmed identity, it is determined that the two candidate source components support different cattle identities.

[0107] The source tracking and cross-object mixed interval determination module tracks two candidate source components along the cross-frame correspondence. The identity matching result does not participate in the tracking path formation. After tracking to a stable video interval with a single source, the source anchor interval is determined. When the identified identity of the source anchor interval is consistent with the cow identity supported by the corresponding candidate source component, the cross-object mixed interval is determined according to the common existence range of the candidate source components, and the cow face trajectory is determined to be segmented based on the source continuity relationship before and after the cross-object mixed interval.

[0108] The trajectory representation feature formation and identity matching module excludes video frames that cross-object mixed regions and those that cannot be traced to the source anchor region along the cross-frame correspondence. It weights the cow face features based on the coverage of local visual regions that can be traced to the same source anchor region along the cross-frame correspondence, forms trajectory representation features, and performs depth measurement matching with the cow identity template to output the cow identity.

[0109] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for cattle face recognition based on deep metric learning in complex farming environments, characterized in that, include: Acquire continuous cow face videos, establish cross-frame correspondence between local visual regions within the cow face candidate region, determine the allowable range of local deformation based on local location and head movement amplitude, compare single-source motion models and dual-source motion models within the allowable range of local deformation, and determine two candidate source components. A deep metric learning network is used to extract local identity features and aggregate them to form cow face features. The candidate source components are used as the feature removal boundary. The changes in the metric distance between the cow face features and the two cow identity templates before and after removal are compared. Based on the changes in the single-source calibration video of the confirmed identity, it is determined that the two candidate source components support different cow identities. Two candidate source components are tracked along the cross-frame correspondence. The identity matching result does not participate in the tracking path formation. After tracking to a stable video interval of a single source, the source anchoring interval is determined. When the identified identity of the source anchoring interval is consistent with the identity of the cow supported by the corresponding candidate source component, the cross-object mixing interval is determined according to the common existence range of the candidate source components, and the cow face trajectory is determined to be segmented based on the source continuity relationship before and after the cross-object mixing interval. Exclude video frames that cross-object mixed regions and those that cannot be traced to the source anchor region along the cross-frame correspondence. Weight the cow face features according to the coverage of local visual regions that can be traced to the same source anchor region along the cross-frame correspondence to form trajectory representative features. Perform depth measurement matching with the cow identity template to output the cow identity.

2. The method for cattle face recognition based on deep metric learning in complex farming environments according to claim 1, characterized in that, Two candidate source components were identified, specifically including: Based on the position of the local visual region within the candidate region of the cow face and the amplitude of head movement, the allowable range of local deformation is determined by a single cow non-overlapping calibration video. Under the same allowable range of local deformation, single-source motion models and dual-source motion models are fitted respectively. The fitting results are evaluated only by the motion residuals that exceed the allowable range of local deformation. Complexity constraints related to the number of model parameters and the number of local visual regions involved in the fitting are imposed on the dual-source motion model.

3. The method for cattle face recognition based on deep metric learning in complex farming environments according to claim 2, characterized in that, The difference in fitting evaluation between the dual-source motion model and the single-source motion model, the duration, and the proportion of sources with a lower proportion of local visual regions in the dual-source motion model are converted into empirical ranks relative to false multi-source events in single-Newton non-overlapping calibration videos to form normalized event states. The nearest neighbor order n is determined based on the number of false multi-source events. The Euclidean distance from the normalized event state to the nth nearest neighbor calibration event is calculated, and the joint distribution allowable boundary is determined by the distribution of the nth nearest neighbor distance of the false multi-source events. When the Euclidean distance exceeds the joint distribution allowable boundary and both sources in the dual-source motion model satisfy the solution conditions of the independent motion model, two candidate source components are determined. When the head movement amplitude exceeds the calibration range, the candidate source component of the corresponding video frame is uncertain; when any source does not meet the solution conditions of the independent motion model, the candidate source component of the corresponding video frame is uncertain.

4. The method for cattle face recognition based on deep metric learning in complex farming environments according to claim 1, characterized in that, Extracting local identity features, specifically including: The aggregation weights are determined based on the visibility of local visual regions and the reliability of cross-frame correspondences, and the local identity features of the two candidate source components are aggregated respectively. The cattle identity templates are retrieved using the aggregated features of two candidate source components and the cow face features of adjacent single-source stable video intervals. The identity of the candidate cattle is determined solely by the retrieval results.

5. The method for cattle face recognition based on deep metric learning in complex farming environments according to claim 4, characterized in that, The two candidate source components were identified to support different cattle identities, specifically including: Remove the local identity features corresponding to one candidate source component respectively, keep the original aggregation weights of the remaining local identity features unchanged, calculate the difference in metric distance between the cow face features before and after removal and the cow identity templates corresponding to the two candidate cow identities, and obtain the change in metric distance difference; Based on the degree of local location overlap and the difference in aggregation weight, local visual regions are selected from the single-source calibration videos whose identities have been confirmed. The normal metric distance difference variation range of the corresponding candidate source components is determined, and a correction amount is formed based on the metric distance difference variation that exceeds the normal metric distance difference variation range. For the two correspondences between two candidate source components and two candidate cattle identities, when the two candidate source components form correction values ​​in opposite directions in the same video frame, the smaller of the absolute values ​​of the two correction values ​​is taken as the identity splitting value, and the identity splitting value in the candidate multi-source interval is statistically weighted over time to form the interval-level identity splitting intensity. The corresponding identity splitting allowable value is determined by independent single-source calibration verification video. When the interval-level identity splitting strength of only one correspondence exceeds the corresponding identity splitting allowable value, it is determined that two candidate source components support different cattle identities.

6. The method for cattle face recognition based on deep metric learning in complex farming environments according to claim 1, characterized in that, Tracking candidate source components specifically includes: Starting from the local visual region in the candidate source component, the tracking proceeds forward and backward only along the cross-frame correspondence; The reliability of motion prediction bias, local visual description changes, and cross-frame correspondence is mapped to the abnormal rank of the single-Newton non-overlapping calibration video, and the maximum value of the abnormal rank is taken as the physical continuity cost. When the physical continuity cost exceeds the allowable value determined by the single-cow non-overlapping calibration video, the corresponding tracking is terminated, and the identity matching result is not included in the connection of the interrupted cross-frame correspondence.

7. The method for cattle face recognition based on deep metric learning in complex farming environments according to claim 6, characterized in that, When a candidate source component is tracked along the cross-frame correspondence to a stable video interval that continuously satisfies the independent motion model solution conditions, the single-source motion model fitting evaluation value is not higher than the dual-source motion model, no candidate multi-source events exceeding the joint distribution allowable boundary are formed, and there is no other candidate source component continuously existing along the effective cross-frame correspondence within the same cow face candidate region, after the local identity features of the stable video interval satisfy the identity recognition support conditions determined by the single-source calibration video with confirmed identity, the stable video interval is determined as the source anchoring interval, and the cow identity identified by the source anchoring interval is verified to be consistent with the cow identity supported by the corresponding candidate source component; After verification, the maximum continuous range in which the two candidate source components exist continuously along the cross-frame correspondence within the same cow face candidate region is determined as the cross-object mixing interval. When one of the two candidate source components persists before the cross-object mixing interval and terminates after the interval, and the other candidate source component continues to track to the source anchoring interval after the interval, the bull face trajectory is segmented. When candidate source components that persist before the cross-object mixing interval are tracked to the source anchoring interval with the same cow identity on both sides of the interval, the cow face trajectory remains continuous.

8. The method for cattle face recognition based on deep metric learning in complex farming environments according to claim 1, characterized in that, Exclude video frames that cross-object mixing regions and those that cannot be traced to the source anchoring region along the cross-frame correspondence. Determine the fusion weight of the cow face feature based on the coverage of local visual regions that can be traced to the same source anchoring region along the cross-frame correspondence and the continuity of the cross-frame correspondence. The effective support duration is determined based on the fusion weight and video sampling time. When the effective support duration is lower than the lower limit of the duration determined by the single-source calibration video with confirmed identity, the status to be identified is output.

9. The method for cattle face recognition based on deep metric learning in complex farming environments according to claim 8, characterized in that, The source anchor representative features are formed by using the bull face features in the source anchor interval, and the scale parameter is determined by the angular distance between the bull face features and the source anchor representative features in the source anchor interval. Within the unit depth metric feature space, according to the fusion weight, a bounded angular distance function limited by the scale parameter is used to weight and solve the source anchor representative features and the same source bull face features outside the source anchor interval to obtain the trajectory representative features. The trajectory representation features are matched with the cattle identity templates using depth measurement. The identity confirmation interval is obtained by subtracting the measurement distance between the trajectory representation features and the second nearest neighbor cattle identity template from the measurement distance between the trajectory representation features and the first nearest neighbor cattle identity template. When the identity of the first nearest neighbor cattle is consistent with the identity identified in the source anchoring interval, and the identity confirmation interval is not lower than the identity confirmation lower limit determined by the single-source calibration video of the confirmed identity, the identity of the first nearest neighbor cattle is output. Output the status to be identified when the identity of the first nearest neighbor cattle does not match the identity identified in the source anchoring interval; output the status to be identified when the identity confirmation interval is lower than the identity confirmation lower limit.

10. A cattle face recognition system based on deep metric learning in complex farming environments, employing the cattle face recognition method based on deep metric learning in complex farming environments as described in any one of claims 1-9, characterized in that... include: The candidate source component determination module acquires continuous cow face videos, establishes cross-frame correspondences of local visual regions within the cow face candidate region, determines the allowable range of local deformation based on local position and head movement amplitude, compares single-source motion models and dual-source motion models within the allowable range of local deformation, and determines two candidate source components. Different cattle identity verification modules use deep metric learning networks to extract local identity features and aggregate them to form cattle face features. The candidate source components are used as feature removal boundaries. The changes in the metric distance between the cattle face features and the two cattle identity templates before and after removal are compared. Based on the changes in the single-source calibration video of the confirmed identity, it is determined that the two candidate source components support different cattle identities. The source tracking and cross-object mixed interval determination module tracks two candidate source components along the cross-frame correspondence. The identity matching result does not participate in the tracking path formation. After tracking to a single source stable video interval, the source anchoring interval is determined. When the identified identity of the source anchoring interval is consistent with the identity of the cow supported by the corresponding candidate source component, the cross-object mixing interval is determined according to the common existence range of the candidate source components, and the cow face trajectory is determined to be segmented based on the source continuity relationship before and after the cross-object mixing interval. The trajectory representation feature formation and identity matching module excludes video frames that cross-object mixed regions and those that cannot be traced to the source anchor region along the cross-frame correspondence. It weights the cow face features based on the coverage of local visual regions that can be traced to the same source anchor region along the cross-frame correspondence, forms trajectory representation features, and performs depth measurement matching with the cow identity template to output the cow identity.

Images