A method and system for continuous monitoring of termite species in a dam

Abstract

A kind of dam termite sub-grade continuous monitoring method and system, the method obtains the video image of dam termite activity area;Sub-grade termite detection is carried out to current frame, and detection boundary box, detection confidence, termite sub-grade class and target apparent feature are obtained;Scale self-adaptive local group relationship graph is constructed according to historical trajectory prediction state and current frame detection result;Based on space, motion, appearance, grade and graph relationship consistency, candidate association cost is calculated, and dynamic fusion weight is generated according to target density, degree of occlusion, detection confidence and trajectory life cycle state, to realize the adaptive matching of detection target and historical trajectory;Then, through candidate trajectory confirmation, temporary lost trajectory recovery, invalid trajectory inhibition and grade smoothing, stable counting is completed.The method can reduce missed detection, misassociation, identity switching and repeated counting in termite dense occlusion scene, improve the accuracy, continuity and stability of dam termite sub-grade continuous monitoring.

Description

Technical Field

[0001] This invention relates to the field of termite monitoring technology for dikes, specifically to a method and system for continuous monitoring of termites in dikes by grade. Background Technology

[0002] Reservoir dams are crucial infrastructure for flood control, water resource regulation, agricultural irrigation, and regional water supply. Earth-rock dams, due to their material properties, humid and hot environments, vegetation cover, and concealed spaces, easily become suitable habitats for termites to nest, forage, and expand their tunnels. The nest cavities and tunnels formed by termites within the dam structure can disrupt the continuity of the dam structure, creating potential seepage channels. During flood season or under high water levels, this can induce safety hazards such as leakage, piping, collapse, and even local instability. Therefore, timely, continuous, and accurate monitoring of termite activity in dams is of great significance for dam safety management and termite control.

[0003] Current termite monitoring methods for dams largely rely on regular manual inspections, trapping and examination, experience-based judgment, and localized physical detection. While these traditional methods can detect termite activity to some extent, they generally suffer from drawbacks such as insufficient monitoring continuity, high reliance on manual labor, coarse-grained monitoring results, and difficulty in real-time quantification. They typically only determine the presence or absence of termites and cannot further obtain information on the population changes, activity processes, and spatial distribution characteristics of different castes of termites, such as worker termites, soldier termites, nymphs, or larvae. Consequently, they cannot provide sufficient data support for assessing the degree of termite damage to dams and for precise control.

[0004] With the development of computer vision technology, it has become possible to automatically identify termite targets using images or videos. However, the actual monitoring scenario for termites on dams has significant unique characteristics: termites are small in size, have flexible and varied body shapes, and exhibit subtle differences in appearance between different castes. They are often mixed with complex backgrounds such as soil particles, sawdust, attracting materials, water stains, shadows, and dam crevices. Single-frame image detection methods are easily affected by factors such as changes in lighting, background texture interference, and mutual occlusion of targets, resulting in missed detections, false detections, and misclassification of castes, making it difficult to meet the requirements of continuous monitoring for detection accuracy and stability.

[0005] Multi-target tracking technology can achieve cross-frame correlation of targets to a certain extent, but its application in dense termite activity scenarios on dams still faces severe challenges. Termites are highly similar in appearance, move randomly, and interact frequently in local areas. When multiple termites gather near bait materials or termite tunnels, conventional tracking methods that rely solely on spatial distance, cross-union ratio (CUI), or fixed-weight matching are prone to identity switching, false associations, trajectory breaks, and duplicate track establishment due to insufficient target feature differentiation. If the number of detection boxes per frame is counted directly, the counting results will fluctuate significantly due to factors such as missed detections, occlusion recurrence, and duplicate detections, making it difficult to form stable and reliable continuous monitoring results by termite level.

[0006] In summary, there is an urgent need for a graded continuous monitoring method and system for densely populated small-target termite-prone dikes. This system should be able to organically combine graded termite detection, stable trajectory association, and trajectory-level counting correction under complex backgrounds and occlusion conditions, thereby reducing the adverse effects of missed detections, false associations, identity switching, and duplicate counting on the monitoring results. Summary of the Invention

[0007] To address the problems in existing termite monitoring of dikes, such as heavy reliance on manual methods, insufficient continuity, weak ability to identify termite castes, high missed detection rate of dense small targets, frequent identity switching in obstructed environments, and unstable dynamic counting, this invention provides a method and system for continuous monitoring of termites by caste in dikes. This method constructs an integrated technical framework for detection, tracking, and counting. At the detection end, it acquires the caste category and phenotypic characteristics of termite targets. At the tracking end, it constructs a scale-adaptive local colony relationship graph and generates dynamic fusion weights based on scene and trajectory states. At the counting end, it achieves stable counting through trajectory lifecycle management and historical caste probability smoothing, thereby realizing automatic identification, continuous tracking, and dynamic statistics of termite individuals of different castes in dikes.

[0008] The technical solution adopted in this invention is as follows: A method for continuous monitoring of termites in dikes by grade classification includes the following steps: S1: Obtain video images of the termite activity area on the dam to obtain the current frame image; S2: Input the current frame image into the termite detection model to obtain the current frame detection result, which includes the detection bounding box, detection confidence, termite caste category and target appearance features; S3: Predict the position and scale of the historical trajectory in the current frame based on the state of the historical trajectory to obtain the predicted state of the historical trajectory. S4: Represent the current frame's detected target and the historical trajectory prediction status as graph nodes. Based on the scale of the graph nodes, the current frame's target scale statistics, and the local target density, determine the scale-adaptive neighborhood radius, and construct a local group relationship graph within the local area defined by the scale-adaptive neighborhood radius. S5: Based on the local group relationship graph, calculate the spatial matching cost, motion matching cost, appearance matching cost, grade matching cost and graph relationship matching cost between the detected target in the current frame and the historical trajectory, respectively; S6: Calculate the dynamic fusion weights of each matching modality based on the current frame's detected target density, detected target occlusion degree, detection confidence, and trajectory lifecycle state; S7: Generate an adaptive multimodal association cost matrix based on spatial matching cost, motion matching cost, appearance matching cost, grade matching cost, graph relationship matching cost, and the corresponding dynamic fusion weights; S8: Based on the adaptive multimodal association cost matrix, the target detected in the current frame is associated with historical trajectories, and the trajectory status is updated according to the association results; S9: Perform full lifecycle management of trajectories, including candidate confirmation, active maintenance, temporary loss recovery and termination deletion, and perform grade-based counting correction based on the unique trajectory number of the confirmed trajectory and its historical grade status; S10: Output the results of continuous monitoring of termite classification in the dikes.

[0009] In step S2, the processing procedure of the termite detection model by grade includes: S2.1: Extract multi-scale visual features of the current frame image using the backbone feature extraction unit: ; in, This indicates the backbone feature extraction operation. This represents the first layer of visual features. Represents the second layer of visual features. Indicates the first Layered visual features; This represents the preprocessed image of the current frame obtained in step S1.

[0010] S2.2: Enhance the direction, scale and local texture of multi-scale visual features using a morphology-aware feature enhancement unit to obtain morphology-enhanced features, wherein the morphology-aware feature enhancement unit includes a direction-aware branch, a scale-aware branch and a local texture-aware branch. The processing procedure of the morphological feature enhancement unit is represented as follows: ; in: , and These represent orientation-aware features, scale-aware features, and local texture-aware features, respectively.

[0011] Indicates the first The basic visual features obtained from the frame image after backbone feature extraction and feature alignment processing; This represents the orientation-aware feature extraction operation, used to enhance information on the long axis direction of the termite's body, the bending direction of its posture, and the direction of local edges; This represents a scale-aware feature extraction operation, used to enhance information on individual termites at different scales, head size, and overall outline. This represents a local texture-aware feature extraction operation, used to enhance local texture information on the head, thorax, abdomen, and body surface of termites.

[0012] The features extracted from the three branches are fused to obtain the morphological enhancement features: ; in: Indicates the first The morphological enhancement features are obtained by fusing orientation-aware, scale-aware, and local texture-aware features of a frame image.

[0013] in, , and These are the fusion weights for direction, scale, and local texture features, respectively, and they satisfy: ; S2.3: Context-guided compact reorganization units are used to aggregate, compress, and reorganize local background features, adjacent target distribution features, and multi-scale context features. The reorganized context features are then fused with morphological enhancement features to obtain fused features; the details are as follows: ; in, Indicates the first The context reassembly features are obtained after the frame undergoes context feature aggregation, channel compression, and compact reassembly. Indicates local background features, Indicates the distribution characteristics of adjacent targets. Represents multi-scale contextual features, This represents context feature aggregation, channel compression, and compact reassembly operations.

[0014] The feature fusion unit integrates morphological enhancement features, contextual reconstruction features, and shallow, mid-level, and deep semantic features: ; in, This indicates feature concatenation, convolutional weighting, or attention fusion operations. This indicates the final fusion feature.

[0015] S2.4: Input the fused features into the detection output unit, and output the detection bounding box, detection confidence score, termite caste category, and target appearance features: The detection output unit performs bounding box regression, grade classification, confidence assessment, and appearance feature extraction based on fused features, and outputs the detection result for the current frame: ; in, Indicates the first The complete set of termite target detection results for the frame; Indicates the detection output unit; Indicates the first The final fusion features of the frames; This indicates the number of termite targets detected in the current frame; Indicates the first The detection bounding box of each target; Indicates the confidence level of the detection; This represents the probability distribution of grade categories; Describes the target appearance feature, the target appearance feature Used in subsequent steps to detect the appearance matching between the target and historical trajectories.

[0016] No. The caste category of a termite target is represented as follows: ; in, Indicates the first The trajectory in the first The current termite class is determined after the frame is smoothed by historical class probabilities; Indicates the first The termite target belongs to the first The probability of each grade category; This indicates that the grade category with the highest historical grade probability is selected as the current smoothing grade of the trajectory.

[0017] S3 includes: for the already established... For each termite trajectory, based on its historical location, target scale, movement speed, appearance characteristics, historical class status, and life cycle status, the position and scale of the trajectory in the current frame are predicted to obtain the historical trajectory prediction status.

[0018] In the In the frame, the first The state of a trajectory is represented as follows: ; in: Indicates the first The historical trajectory in the first The complete trajectory state at the end of the frame; A unique trajectory number; The trajectory motion state; For trajectory apparent memory features; This represents the probability distribution of historical grades. This refers to the lifecycle state; and These represent the number of consecutive successfully matched frames and the number of consecutive unmatched frames, respectively.

[0019] The trajectory motion state is represented as follows: ; in, Indicates the first The trajectory in the first The motion state vector of the frame, The coordinates of the center of the trajectory bounding box. For the bounding box width and height, For the rate of change of position, The speed of scale change.

[0020] A uniform motion model is used to predict the trajectory state: ; in, The predicted state for the current frame is represented by the state transition matrix. Represented as: ; In the formula, The time interval between adjacent video frames. It is a fourth-order identity matrix; This represents a fourth-order zero matrix.

[0021] Obtain the trajectory prediction bounding box for the current frame from the prediction state: ; In the formula, Indicates the first The historical trajectory in the first Predicted bounding boxes in the frame; and These represent the x-coordinate and y-coordinate of the predicted bounding box center point, respectively. and These represent the width and height of the predicted bounding box, respectively.

[0022] Step S3 is based only on the first The historical trajectory in the first The motion state of the frame, and its position in the first frame. The position and scale in the frame are predicted to obtain the trajectory prediction state. Motion state correction, apparent memory feature update and historical grade probability distribution update after the trajectory successfully matches the target detected in the current frame are performed by step S8; candidate confirmation, active maintenance, temporary loss recovery and termination deletion of the trajectory are performed by step S9.

[0023] Finally, the first The predicted state of the trajectory in the current frame is represented as follows: ; In the formula, Indicates the first The trajectory in the first The complete trajectory state of the frame; Indicates a unique trajectory number; Indicates the trajectory motion state; This represents the apparent memory features of the trajectory; Represents the probability distribution of historical grades; Indicates the trajectory lifecycle state; and These represent the number of consecutive successfully matched frames and the number of consecutive unmatched frames, respectively.

[0024] S4 includes: S4.1: For each graph node Based on the width of its corresponding detection bounding box or trajectory prediction box and height Calculate the scale of the nodes in the graph. ; S4.2: Determine the target scale statistics for the current frame based on the scale of all detected targets in the current frame. The The mean, median, truncated mean, or quantile of the target scale in the current frame; S4.3: For each graph node Calculate its local target density: ; in: To preset the initial statistical radius, With the node in the graph as the center and a radius of , The number of graph nodes within the specified range; S4.4: Determine the scale-adaptive neighborhood radius of the graph node: ; in: , , The neighborhood adjustment coefficient, This is the lower bound of the neighborhood radius. The upper limit of the neighborhood radius, Clip represents the natural logarithm function. Indicates will Limited to to A limiting function within the range.

[0025] S4.5: Based on the scale-adaptive neighborhood radius of each graph node, establish graph edges between graph nodes to form a local group relationship graph. The local group relationship graph is constructed as follows: when the center distance between two graph nodes is less than or greater than the larger of their scale-adaptive neighborhood radii, a graph edge is established, i.e.: ; in: , For any two distinct graph nodes in a local group relationship graph; For the first The first frame The set of edges in a local group relationship graph; , No. Frame-in-Graph Nodes , The center coordinates of the corresponding bounding box; , For graph nodes , In the Scale-adaptive neighborhood radius in the frame.

[0026] It means "if and only if"; This represents the L2 norm, used to calculate the Euclidean distance between the centers of the bounding boxes corresponding to two graph nodes; Indicates a node in the graph Graph Nodes The larger value is selected from the corresponding scale-adaptive neighborhood radius; This indicates the type identifier of the local group relationship diagram; and ;when When, it represents the local group relationship graph formed by the detected targets in the current frame, when At this time, it represents a local group relationship diagram of trajectory prediction composed of historical trajectory prediction states.

[0027] The edge features of the graph include one or more of the following: spatial distance features, scale similarity features, intersection-union ratio features, motion direction consistency features, grade consistency features, apparent similarity features, and relative positional relationship features.

[0028] S5 includes: 5.1: Calculate the detection target With historical trajectory Spatial matching cost between The spatial matching cost Used to characterize the normalized distance between the center point of the detected target and the center point of the trajectory prediction; ; in, Indicates the first The center coordinates of the bounding box of each detected target; Indicates the first The predicted center coordinates of the historical trajectory in the current frame; and These represent the scale of the detected target and the trajectory prediction box, respectively; This indicates a preset positive number to prevent the denominator from being zero; This means that the calculation result will be limited to the range of 0 to 1.

[0029] 5.2: Calculate the detection target With historical trajectory Motion matching cost between The motion matching cost Used to characterize the motion deviation between the detected target position and the trajectory prediction position; Motion matching cost It is used to characterize the motion deviation between the detected target position and the trajectory prediction position.

[0030] ; in, The Mahalanobis distance between the observed position and scale of the detected target and the predicted trajectory state; Mahalanobis distance normalization threshold or motion gating threshold.

[0031] 5.3: Calculate the detection target With historical trajectory Apparent matching cost between The apparent matching cost Used to characterize the degree of dissimilarity between the apparent features of the detected target and the apparent memory features of the trajectory; ; in, Indicates the middle The apparent feature vector of each detected target; Indicates the first The apparent memory feature vector preserved by each historical trajectory; This indicates a preset positive number to prevent the denominator from being zero.

[0032] 5.4: Calculate the detection target With historical trajectory The cost of matching grades between them The grade matching cost Used to characterize the degree of inconsistency between the target quality category and the historical quality status of the trajectory; ; in, This represents the probability vector of the grade category of the currently detected target. The historical grade probability distribution up to the end of the historical trajectory. This represents the degree of similarity between the current detected grade distribution and the historical grade distribution of the trajectory.

[0033] 5.5: Calculate the detection target With historical trajectory Graph relationship matching cost The graph relation matching cost It is used to characterize the structural differences between the local group relationships where the target is located and the local group relationships where the historical trajectory is located.

[0034] ; in, For the current frame number A vector describing the relationship between each detected target in the local group relationship graph; For the first The relationship description vector of each historical trajectory in the local group relationship graph of trajectory prediction.

[0035] In step S6, the target is detected. With historical trajectory The constructed candidate association pairs, computational space reliability Motion reliability Apparent reliability Grade reliability Reliability of graph relationships ; Among them, spatial reliability It is positively correlated with detection confidence and negatively correlated with the degree of target occlusion. Motion reliability Related to the number of consecutive successful trajectory matches or the status of temporary loss recovery; Apparent reliability Related to the quality of the target's appearance features and the density of the detected targets; Grade Reliability It is related to the confidence level of the detection grade and the stability of the historical grade distribution of the trajectory; Reliability of Graph Relationships Related to local target density and the number of neighboring nodes; The dynamic fusion weights for each matched modality are determined using the following formula: ; in, This represents the dynamic fusion weight of the candidate association pair (i,j) in the k-th matching modality; k can be s, m, a, c, or g, and K is the set of matching modalities. It is a preset positive constant to prevent the denominator from being zero.

[0036] In step S7, candidate association pairs are eliminated according to preset gating conditions. The gating conditions include: the spatial distance between candidate association pairs exceeds the spatial gating threshold, the motion deviation exceeds the motion gating threshold, the comprehensive association cost exceeds the association cost threshold, or the degree of grade inconsistency exceeds the grade gating threshold; the comprehensive association costs of the remaining candidate association pairs constitute an adaptive multimodal association cost matrix.

[0037] Let the current frame have a total of There are [number] detection targets, and the historical trajectory set contains [number]... Trajectory. After obtaining the matching cost and corresponding dynamic fusion weights for each matching modality, for the current frame's trajectory. The detection target and the first Candidate association pairs composed of historical trajectories are used to calculate the comprehensive association cost according to the following formula, based on spatial matching cost, motion matching cost, apparent matching cost, grade matching cost, and graph relationship matching cost, and their corresponding dynamic fusion weights. : ; in, , 、 、 、 These are spatial matching cost, motion matching cost, appearance matching cost, grade matching cost, and graph relation matching cost, respectively. 、 、 、 and These are the dynamic fusion weights corresponding to each matching modality, and they satisfy the following: ; The lower the overall correlation cost, the greater the likelihood that the current detection target and the historical trajectory belong to the same termite individual.

[0038] To eliminate obviously unreasonable candidate association pairs, a gating indicator is set. When a candidate association pair simultaneously satisfies the following conditions: spatial distance does not exceed the spatial gating threshold, motion deviation does not exceed the motion gating threshold, grade inconsistency does not exceed the grade gating threshold, and comprehensive association cost does not exceed the association cost threshold, let... Otherwise, Its valid conditions are expressed as follows: ; in, This represents the spatial distance between the detected target and the predicted trajectory location. Indicates motion deviation; , , and These represent the spatial gating threshold, motion gating threshold, grade gating threshold, and comprehensive correlation cost threshold, respectively.

[0039] Based on the gating results, determine the element values ​​of the candidate association pairs in the final association cost matrix: ; in, Indicates the first The first frame The detection target and the first The final associated cost of a historical trajectory after gating; This is a preset maximum value that is greater than the cost of all valid associations. When At that time, the matrix elements retain the original comprehensive correlation cost. ;when When, the matrix elements are set to This indicates that no association is allowed between the detected target and its historical trajectory. In actual calculations, this matrix element can also be directly marked as invalid.

[0040] The adaptive multimodal correlation cost matrix is ​​composed of the gated correlation costs between all detected targets and historical trajectories. ; in, Indicates the first The adaptive multimodal association cost matrix of a frame; Indicates the number of targets detected in the current frame; This indicates the number of historical trajectories associated with the current frame data.

[0041] In step S8, the data association adopts a hierarchical matching strategy: First-level strict matching: Prioritize the association between high-confidence detection targets and active trajectories; The second layer of recovery matching: recovers and associates the detected targets with medium and low confidence, the temporarily lost trajectories, and the local neighborhood candidate targets; The third layer of unmatched target processing: candidate trajectories are established for unmatched detected targets, and historical trajectories that are still unmatched are transferred to a temporary loss state or a termination state based on the number of consecutive unmatched frames and trajectory stability.

[0042] Among them, the first layer of strict matching, the second layer of restored matching, and the third layer of unmatched target processing are all completed based on the adaptive multimodal association cost matrix and the dynamic matching threshold of the corresponding layer.

[0043] The adaptive multimodal association cost matrix generated in step S7 This completes the data association between the detected target in the current frame and the historical trajectory. To balance the accuracy of the data association with the ability to recover the trajectory after short-term occlusion, the data association process is divided into three layers: a first layer of strict matching, a second layer of recovery matching, and a third layer of unmatched target processing. Figure 5 As shown.

[0044] First-level strict matching: Based on the detection confidence threshold The current frame's detected targets are divided into a set of high-confidence detected targets: ; in, Indicates the first The set of high-confidence detected targets in the frame; Indicates the first The first frame One detection target; Indicates the first The detection confidence of each target. This indicates the high-confidence detection threshold.

[0045] Trajectories whose lifecycle state is active are selected from the historical trajectory set to form the active trajectory set: ; in, Indicates the first The set of historical trajectories that are active at the end of the frame and participate in the first-level strict matching of the current frame; Indicates the first The historical trajectory in the first The complete trajectory state of the frame; Indicates the first The life cycle status of a historical trajectory; This indicates an active state, meaning the trajectory has been effectively confirmed and is in a normal, continuous tracking state.

[0046] A one-to-one matching is performed between the high-confidence target set and the active trajectory set based on the association cost matrix. This matching can be expressed as: ; in, This represents the matrix of optimal matching results obtained from the first level of strict matching; This represents the matching decision matrix to be solved; Indicates the first The detection target and the first The final association cost of historical trajectories after dynamic weight fusion and gating processing; Denotes a binary matching decision variable, when the first... The detection target and the first The value is 1 when a historical trajectory is successfully matched, and 0 otherwise.

[0047] One-to-one matching should satisfy: ; in, Indicates the first The detection target and the first The historical trajectory was successfully matched. This indicates that the two did not match.

[0048] Candidate association pairs are allowed to participate in the first-level matching when their association cost satisfies the following formula: ; in, This is the threshold for strict matching at the first layer. For association costs greater than... Alternatively, candidate association pairs marked as invalid in step S7 will not be matched.

[0049] The first layer of matching is mainly used to detect termite targets with high confidence and stable historical status. It prioritizes the accuracy of data association by using strict matching thresholds to reduce the probability of false associations and identity switching.

[0050] Second-level recovery matching: After the first layer of matching is completed, the unmatched high-confidence detected targets, medium- and low-confidence detected targets, and candidate detected targets located within the local search area of ​​the temporarily lost trajectory in the first layer are combined to form the recovered detected target set: ; in, Indicates the first The set of recovery detection targets in the frame that participate in the second-layer recovery matching; This represents the set of unmatched detected targets in the first layer; Indicates the first The first frame One detection target; and These represent the minimum and maximum detection confidence thresholds, respectively. Indicates the first The detection confidence level of each target.

[0051] The unmatched active trajectories in the first layer and the trajectories in a temporarily lost state are combined to form a set of recovered trajectories: ; in, This represents the set of recovery trajectories that participate in the second-level recovery matching of the current frame; This represents the set of unmatched active trajectories in the first layer; Indicates the first The historical trajectory in the first The complete trajectory state at the end of the frame; Indicates the first The life cycle status of a historical trajectory; This indicates a temporary loss of track data due to short-term occlusion or missed detection, but it is still retained for subsequent recovery of the association.

[0052] For the The recovered trajectory searches for candidate targets only near its predicted location. Let the trajectory prediction center be... The detection target center is Then the candidate associations satisfy: ; in, The scale-adaptive neighborhood radius is determined in step S4.

[0053] Among candidate association pairs that satisfy the local neighborhood condition and the corresponding gating condition, a one-to-one matching is performed again based on the comprehensive association cost. The second layer uses a relatively relaxed recovery matching threshold: ; That is, the first layer uses a stricter threshold. To ensure matching accuracy, a more lenient threshold is used in the second layer. Improve the ability to recover trajectories after short-term occlusion, missed detection, and dense interactions. Successfully recovered temporarily lost trajectories retain their original unique trajectory numbers and are not re-established.

[0054] Third-layer unmatched target handling: After the first and second layer matching, for the detection targets that are still not matched, they are initialized as candidate trajectories and passed to step S9 for candidate confirmation.

[0055] For historical trajectories that are still not matched, based on their original lifecycle status, the number of consecutive unmatched frames, and the trajectory confidence, they are passed to step S9 for candidate deletion, temporary retention, or termination.

[0056] Therefore, step S8 is mainly responsible for determining the matching relationship between the detection target and the historical trajectory, and step S9 performs a complete life cycle state transition, specifically candidate confirmation, active maintenance, temporary loss recovery and termination deletion.

[0057] Track update after successful matching For historical trajectories that have been successfully matched in the first or second layer, the trajectory prediction state is corrected based on the position and scale observation results of the corresponding detected targets: ; in, Indicates the first The trajectory in the first The frame's motion state after correction; Indicates the first The predicted state of the trajectory in the current frame. Indicates the first The location and scale observations of the detection target Represents the observation matrix. This represents the state correction coefficient or Kalman gain.

[0058] The apparent memory features of the trajectory and the probability distribution of historical grades are updated as follows: ; in, Indicates the first The trajectory in the first Apparent memory features after frame update; Indicates the first The trajectory in the first Historical grade probability distribution after frame update; This indicates the original apparent memory characteristics of the trajectory. This represents the apparent features of the currently matched and detected target. Indicates the coefficient for retaining apparent historical information; This represents the probability distribution of the original historical grade of the trajectory. This represents the probability distribution of the grade of the current detection target. This represents the retention coefficient for historical information on grade.

[0059] The trajectory confidence is updated based on the current detection confidence and matching quality: ; in: Indicates the first The trajectory in the first Frame matching updated trajectory confidence; This represents the original confidence level of the trajectory. This indicates the detection confidence level of the current target. This represents the normalized overall cost of association, and its value ranges from 0 to 1. This represents the trajectory confidence smoothing coefficient.

[0060] After a successful match, the lifecycle state of the trajectory remains unchanged or is restored to an active state: ; in: Indicates the first The trajectory in the first The lifecycle state after frame update; This indicates an active state, meaning the trajectory has been effectively confirmed and is in a normal, continuous tracking state.

[0061] S9 includes: 9.1 Candidate Confirmation: Unmatched detection targets are established as candidate trajectories. When a candidate trajectory meets the conditions of position continuity, detection confidence level and grade consistency within a preset number of consecutive frames, it is confirmed as an active trajectory and assigned a unique trajectory number. 9.2 Active Maintenance: Update the position, scale, motion state, apparent memory features, historical grade status, and trajectory confidence of continuously matched trajectories; 9.3 Temporary Loss Recovery: For unmatched trajectories caused by occlusion or missed detection, their prediction state is retained within the maximum number of temporary loss frames, allowing them to be re-associated with the target detected in subsequent frames. After successful re-association, the original unique trajectory number is retained. 9.4 Termination of Deletion: When the number of consecutive unmatched frames of a temporarily lost trajectory exceeds the maximum temporary loss threshold, or the recovery confidence is lower than the termination threshold, it is put into the termination state; 9.5, Grade-based counting correction: Individual counting is performed based on the unique trajectory number of the confirmed trajectory. The original trajectory number is restored for termite targets that reappear after short-term occlusion or missed detection. Historical grade probability smoothing is performed on trajectories with short-term fluctuations in grade category, and invalid trajectories are suppressed.

[0062] A continuous monitoring system for termites in dikes, categorized by species, comprising: The video acquisition module is used to acquire video images of the termite activity area on the dam. The termite classification detection module is used to detect termite targets in the current frame image and outputs the detection bounding box, detection confidence, termite classification, and target appearance features. The trajectory prediction module is used to predict the position and scale of the historical trajectory in the current frame based on the state of the historical trajectory, and obtain the predicted state of the historical trajectory. The local group relationship graph construction module is used to uniformly represent the current frame detected target and the historical trajectory prediction state as graph nodes, and construct a scale-adaptive local group relationship graph; The adaptive multimodal matching module is used to calculate the dynamic fusion weights of each matching modality based on the current frame's detected target density, detected target occlusion degree, detection confidence, and trajectory lifecycle state, generate an adaptive multimodal association cost matrix, and complete the data association between the detected target and historical trajectories. The trajectory lifecycle management module is used to perform candidate confirmation, active maintenance, temporary loss recovery, and termination deletion of trajectories; The grade-based counting module is used to perform grade-based counting correction based on the unique trajectory number of the confirmed trajectory and the historical grade status. The monitoring output module is used to output the continuous monitoring results of termite classification in the dike.

[0063] This invention provides a method and system for continuous monitoring of termites in dikes by grade, with the following technical advantages: 1) This invention uses a termite detection model with different castes to identify the castes of termites in complex backgrounds and dense small target scenarios of dams. It can distinguish between different castes such as worker ants, soldier ants, nymphs, and larvae. Compared with the coarse-grained monitoring methods of the prior art that can only determine the presence of termites, this invention provides more refined data support for the assessment of termite damage levels in dams.

[0064] 2) In the data association stage, this invention constructs a scale-adaptive local population relationship graph, dynamically adjusts the neighborhood range according to individual scale, target scale statistics and local density, effectively utilizes the local population spatial structure and relative positional relationship of termites for matching, and significantly reduces the probability of identity switching and misassociation in dense occlusion and similar appearance scenarios.

[0065] 3) Based on the detection target density, occlusion degree, detection confidence and trajectory life cycle state, this invention adaptively generates dynamic fusion weights for each matching modality for each candidate association pair, avoiding the problem of insufficient adaptability of existing methods using fixed weights in different scenarios, and greatly improving the robustness of data association under complex conditions such as occlusion and dense interaction.

[0066] 4) This invention, through a full lifecycle management mechanism of candidate confirmation, active maintenance, temporary loss recovery and termination deletion, can effectively suppress problems such as direct track establishment due to single-frame false detection, track breakage caused by short-term missed detection, repeated track establishment after occlusion, and accumulation of invalid tracks, thereby improving the continuity and stability of termite individual tracking.

[0067] 5) This invention performs graded counting correction based on the unique trajectory number of the confirmed trajectory, restores the original number for targets that are temporarily obscured or missed and reproduces, and performs historical probability smoothing on short-term fluctuations in grade. This avoids the counting fluctuations caused by missed detection, repeated detection and misclassification of single frames in existing methods, and realizes stable and reliable graded continuous monitoring, providing a reliable basis for the analysis of termite activity trends and the judgment of prevention and control timing in dams. Attached Figure Description

[0068] The present invention will be further described below with reference to the accompanying drawings and embodiments; Figure 1 This is an overall flowchart of the method for continuous monitoring of termites in dikes by grade according to the present invention.

[0069] Figure 2 This is a schematic diagram of the structure of the termite detection model of the present invention, which is classified into different grades.

[0070] Figure 3 This is a schematic diagram illustrating the construction of the scale-adaptive local group relationship graph of the present invention.

[0071] Figure 4 This is a flowchart of the candidate pair level dynamic fusion weight and adaptive multimodal matching process of the present invention.

[0072] Figure 5 This is a flowchart of the hierarchical data association and trajectory update process of the present invention.

[0073] Figure 6 This is a flowchart of the trajectory lifecycle management process of the present invention.

[0074] Figure 7 This is a schematic diagram of the ant count and ant population output based on the ant population classification of the present invention. Detailed Implementation

[0075] A method and system for continuous monitoring of termites by caste in dams is proposed. This system is designed for video monitoring scenarios such as reservoir dams, earth-rock dams, dam shoulders, dam slopes, near seepage barriers, termite traps, and areas suspected of termite activity. It can automatically identify, stably track, and dynamically count different castes of termites, including worker termites, soldier termites, nymphs, and larvae, even in complex dam backgrounds, densely packed termite populations, and mutual obstruction. This provides a continuous and quantitative data foundation for assessing the degree of termite damage to dams and for precise prevention and control.

[0076] Example 1: Figure 1 A flowchart illustrating the overall process of the present invention for continuous monitoring of termites in dikes by grade classification is provided. Figure 1 As shown, this method mainly includes the following steps: video stream acquisition and image preprocessing, termite detection by species, historical trajectory state prediction, scale-adaptive local population relationship graph construction, candidate association feature and matching cost calculation, candidate pair dynamic fusion weight calculation, adaptive multimodal association cost matrix generation, hierarchical data association and trajectory update, trajectory lifecycle management and species-level counting correction, and monitoring result output. Each step is described in detail below.

[0077] S1. Video stream acquisition and image preprocessing: Video acquisition devices are deployed at termite activity monitoring points along the dam. These monitoring points include one or more of the following: termite traps, suspected termite tunnel areas, surface monitoring points on the dam slope, concealed areas on the dam shoulder, near drainage ditches, near cracks in the dam body, or key areas for termite control. The video acquisition devices can be ordinary visible light cameras, macro cameras, infrared supplemental lighting cameras, or cameras with low-light imaging capabilities. The cameras should be pointed towards termite trap materials, cracks on the dam surface, suspected termite tunnel areas, or the interior of the monitoring boxes. To reduce interference from irrelevant backgrounds, regions of interest can be pre-defined during deployment, or automatically updated through background modeling or target activity hotspots.

[0078] Video streams of the termite activity area on the dam were acquired using a video capture device, and the data was sampled at a preset frame rate. Frame extraction is performed to obtain a continuous video frame sequence: ; ; in, This represents a sequence of consecutive raw image frames extracted from the video stream according to a preset sampling frame rate. This represents the first frame of the original image in a continuous image frame sequence; This represents the second original image in a continuous image frame sequence; Indicates the first Frame of original image, Indicates the corresponding sampling time; Indicates the video frame number.

[0079] To reduce interference from the dam surface, the trap frame border, soil particles, and other irrelevant areas, the current frame is cropped according to a preset region of interest (ROI). Let the ROI mask be... The cropped image is: ; in, Indicates the first After the original image of the frame is cropped using a region of interest mask, at the pixel position Pixel value at; Indicates the first The original image frame at pixel location Pixel value at the location; monitoring area =1, outside the monitoring area =0.

[0080] Subsequently, the region of interest image is subjected to denoising and brightness equalization. In one implementation, Gaussian filtering and gamma correction are used, and the processing result is expressed as follows: ; in: Indicates the first After Gaussian filtering and gamma brightness correction, the region of interest image of the frame is located at the pixel position. Pixel value at; The standard deviation is expressed as The Gaussian filter kernel, * denotes convolution operation. Indicates the maximum pixel value. This indicates the brightness adjustment factor. It can be set when the image is too dark. When the image is too bright, you can adjust the settings. .

[0081] The processed image is color-normalized to reduce color differences caused by varying lighting and fill light conditions. For the first... The normalized result of the color channels is as follows: ; in, Indicates the first The first frame of the image After color normalization, each color channel is located at a pixel position. Pixel value at; and These represent the mean and standard deviation of the color channel, respectively. To prevent the use of a preset positive number with a denominator of zero, the color-normalized image is scaled proportionally and filled to the input size required by the detection model. , This represents the target width of the input image to the detection model. This represents the target height of the input image to the detection model.

[0082] For monitoring scenarios where the camera device is stationary, static backgrounds can be suppressed by the pixel difference between the current frame image and the background image. The pixel difference is calculated according to the following formula: ; in, Indicates the first Frame image at pixel position The pixel difference value relative to the background image; Indicates the relationship with the first The background image corresponding to the frame is at the pixel position The color channel vector at that location; Represents the background image. When... Greater than the preset threshold At that time, the corresponding area is identified as a potential termite activity prospect area.

[0083] Depending on the ambient lighting, background complexity, and equipment performance, the outline of the termite's body and the edge of its head can also be appropriately enhanced.

[0084] The above processing procedure can be uniformly represented as: ; in, Indicates the first The image obtained after preprocessing the original frame image; This indicates the first [item] extracted from the video stream. Original image frame A binary mask representing the region of interest is used to preserve the termite monitoring area and suppress irrelevant background outside the monitoring area, where the value is 1 within the monitoring area and 0 outside the monitoring area; This represents the standard deviation of the Gaussian filter kernel; Indicates the gamma brightness adjustment factor; This represents the background difference threshold.

[0085] This represents a preprocessing operation consisting of one or more of the following: region of interest cropping, image denoising, brightness equalization, color normalization, scale normalization, background noise suppression, and edge enhancement. The resulting preprocessed image... Input the termite detection model by grade in step S2.

[0086] S2. Grade-based termite detection: The preprocessed image of the current frame obtained in step S1 Input the termite detection model by caste to obtain the termite detection bounding box, detection confidence, caste category, and target appearance features in the current frame. The termite caste categories include at least two of the following: worker termites, soldier termites, nymphs, and larvae. They can also be expanded to include winged adults or other categories according to actual monitoring needs.

[0087] Figure 2 A schematic diagram of the structure of a termite detection model categorized by species is given. For example... Figure 2 As shown, the model includes a backbone feature extraction unit, a morphology-aware feature enhancement unit, a context-guided compact reorganization unit, a feature fusion unit, and a detection output unit. The termite detection model of this invention can be built on a lightweight single-stage detection framework or other detection networks with real-time detection capabilities. Its core lies in the morphology-aware enhancement mechanism, the context-guided compact reorganization mechanism, and the integrated detection-tracking feature output for dam termite scenarios, rather than being limited to a specific network structure name.

[0088] The backbone feature extraction unit performs multi-scale feature extraction on the current frame image: ; in, This indicates the backbone feature extraction operation. This represents the first layer of visual features. Represents the second layer of visual features. Indicates the first Layered visual features; This represents the preprocessed image of the current frame obtained in step S1.

[0089] Shallow features mainly preserve the edges, textures, and limb details of termites, while deep features mainly represent termite castes and target semantic information.

[0090] The morphological feature enhancement unit includes a direction-aware branch, a scale-aware branch, and a local texture-aware branch. These three branches enhance the termite's long axis and posture curvature direction, overall outline and head size, as well as the texture of the head, thorax, abdomen, and body surface, respectively. The processing steps are as follows: ; in: , and These represent orientation-aware features, scale-aware features, and local texture-aware features, respectively.

[0091] Indicates the first The basic visual features obtained from the frame image after backbone feature extraction and feature alignment processing; This represents the orientation-aware feature extraction operation, used to enhance information on the long axis direction of the termite's body, the bending direction of its posture, and the direction of local edges; This represents a scale-aware feature extraction operation, used to enhance information on individual termites at different scales, head size, and overall outline. This represents a local texture-aware feature extraction operation, used to enhance local texture information on the head, thorax, abdomen, and body surface of termites.

[0092] The features extracted from the three branches are fused to obtain the morphological enhancement features: ; in: Indicates the first The morphological enhancement features are obtained by fusing orientation-aware, scale-aware, and local texture-aware features of a frame image.

[0093] in, , and These are the fusion weights for direction, scale, and local texture features, respectively, and they satisfy: ; The fusion weights can be preset or adaptively determined through an attention mechanism.

[0094] Context-guided compact reorganization units aggregate, compress, and reorganize local background features around the target, distribution features of adjacent targets, and multi-scale contextual features: ; in, Indicates the first The context reassembly features are obtained after the frame undergoes context feature aggregation, channel compression, and compact reassembly. Indicates local background features, Indicates the distribution characteristics of adjacent targets. Represents multi-scale contextual features, This unit represents context feature aggregation, channel compression, and compact reorganization operations. It is used to reduce interference from complex backgrounds such as soil particles, sawdust, and shadows, and to enhance the ability to distinguish dense termites.

[0095] The feature fusion unit integrates morphological enhancement features, contextual reconstruction features, and shallow, mid-level, and deep semantic features: ; in, This indicates feature concatenation, convolutional weighting, or attention fusion operations. This indicates the final fusion feature.

[0096] The detection output unit performs bounding box regression, grade classification, confidence assessment, and appearance feature extraction based on fused features, and outputs the detection result for the current frame: ; in, Indicates the first The complete set of termite target detection results for the frame; Indicates the detection output unit; Indicates the first The final fusion features of the frames; This indicates the number of termite targets detected in the current frame; Indicates the first The detection bounding box of each target; Indicates the confidence level of the detection; This represents the probability distribution of grade categories; Describes the target appearance feature, the target appearance feature Used in subsequent steps to detect the appearance matching between the target and historical trajectories.

[0097] No. The caste category of a termite target is represented as follows: ; in, Indicates the first The trajectory in the first The current termite class is determined after the frame is smoothed by historical class probabilities; Indicates the first The termite target belongs to the first The probability of each grade category; This indicates that the grade category with the highest historical grade probability is selected as the current smoothing grade of the trajectory.

[0098] S3. Historical Trajectory State Prediction: For the already established For each termite trajectory, based on its historical location, target scale, movement speed, appearance characteristics, historical class status, and life cycle status, the position and scale of the trajectory in the current frame are predicted to obtain the historical trajectory prediction status.

[0099] In the In the frame, the first The state of a trajectory is represented as follows: ; in: Indicates the first The historical trajectory in the first The complete trajectory state at the end of the frame; A unique trajectory number; The trajectory motion state; For trajectory apparent memory features; This represents the probability distribution of historical grades. This refers to the lifecycle state; and These represent the number of consecutive successfully matched frames and the number of consecutive unmatched frames, respectively.

[0100] The trajectory motion state can be represented as: ; in, Indicates the first The trajectory in the first The motion state vector of the frame, The coordinates of the center of the trajectory bounding box. For the bounding box width and height, For the rate of change of position, The speed of scale change.

[0101] In one implementation, a uniform motion model is used to predict the trajectory state: ; in, The predicted state for the current frame is represented by the state transition matrix. Represented as: ; In the formula, The time interval between adjacent video frames. It is a fourth-order identity matrix; This represents a fourth-order zero matrix.

[0102] Obtain the trajectory prediction bounding box for the current frame from the prediction state: ; In the formula, Indicates the first The historical trajectory in the first Predicted bounding boxes in the frame; and These represent the x-coordinate and y-coordinate of the predicted bounding box center point, respectively. and These represent the width and height of the predicted bounding box, respectively.

[0103] Step S3 is based only on the first The historical trajectory in the first The motion state of the frame, and its position in the first frame. The position and scale in the frame are predicted to obtain the trajectory prediction state. Motion state correction, apparent memory feature update and historical grade probability distribution update after the trajectory successfully matches the target detected in the current frame are performed by step S8; candidate confirmation, active maintenance, temporary loss recovery and termination deletion of the trajectory are performed by step S9.

[0104] Finally, the first The predicted state of the trajectory in the current frame is represented as follows: ; In the formula, Indicates the first The trajectory in the first The complete trajectory state of the frame; Indicates a unique trajectory number; Indicates the trajectory motion state; This represents the apparent memory features of the trajectory; Represents the probability distribution of historical grades; Indicates the trajectory lifecycle state; and These represent the number of consecutive successfully matched frames and the number of consecutive unmatched frames, respectively.

[0105] The predicted state is used in step S4 to construct a scale-adaptive local population relationship graph and to limit the candidate matching range between the current detected target and historical trajectories. Trajectory prediction can employ Kalman filtering, uniform motion models, exponential smoothing models, or other motion prediction methods, with the aim of narrowing the candidate association range and assisting in trajectory recovery after short-term occlusion or missed detection.

[0106] S4. Scale-Adaptive Local Group Relationship Graph Construction: In actual termite monitoring scenarios on dams, individual termites may be sparsely distributed or densely clustered near bait materials, bait sticks, or termite tunnel entrances. Globally matching all detected targets with all historical trajectories would not only significantly increase computational complexity but also introduce numerous invalid match candidates that are spatially distant and unlikely to belong to the same termite individual, significantly increasing the probability of false associations. Therefore, this invention employs a scale-adaptive local neighborhood strategy to construct a local population relationship graph, establishing candidate associations only within local neighborhoods. This reduces computational complexity while improving the accuracy of associations.

[0107] Figure 3 A schematic diagram illustrating the construction of the scale-adaptive local group relationship graph of this invention is provided. For example... Figure 3 As shown, the specific construction process is as follows: The current frame's detected target and historical trajectory prediction status are uniformly represented as graph nodes. The detected target node includes the detection box position, detection confidence, quality category, and appearance features; the trajectory prediction node includes the prediction position, prediction scale, historical quality status, appearance memory features, and trajectory lifecycle status.

[0108] For each graph node Based on the width of its corresponding detection bounding box or trajectory prediction box and height Define its scale. The calculation method is as follows: ; This scale measure, expressed as the square root of the bounding box area, comprehensively reflects the size of the termite target in the image. Larger-scale graph nodes typically correspond to termite individuals closer to the camera or of larger castes; smaller-scale graph nodes typically correspond to termite individuals farther away or of smaller castes.

[0109] Determine the target scale statistics for the current frame based on the scale of all detected targets in the current frame. The aforementioned It can detect the mean, median, truncated mean, or quantile of target scale quantities for the current frame, reflecting the overall scale level of termite targets in the current frame. Different statistical calculation methods can be used to adapt to the scale distribution characteristics of different scenarios: the mean is suitable for scenarios with relatively uniform scale distribution, the median is suitable for scenarios with a small number of extremely large or small scale outliers, and the truncated mean or quantile is suitable for scenarios where the influence of outlier scales needs to be removed.

[0110] For each graph node Calculate its local target density The calculation method is as follows: ; in, Indicates the first Local target density of each graph node To preset the initial statistical radius, With the node in the graph as the center and a radius of , The number of graph nodes within a given range. Local target density reflects the spatial aggregation of termite targets around that graph node: when termites densely cluster near the attractant material, The value is relatively large; when termites are dispersed, The value is relatively small. This density information is an important basis for subsequently determining the neighborhood radius and matching weight.

[0111] Based on the aforementioned scale parameters, target scale statistics, and local target density, the scale-adaptive neighborhood radius of the graph node is determined. The calculation method is as follows: ; in, 、 、 The neighborhood adjustment coefficient is used to balance the contributions of the overall scale level, individual scale, and local density to the neighborhood radius. Each coefficient can be calibrated experimentally according to the actual scenario; ln represents the natural logarithm function. This serves as a lower bound for the neighborhood radius to prevent potential matches from being missed due to an excessively small neighborhood radius. The upper limit of the neighborhood radius is used to prevent the neighborhood radius from being too large, which would introduce too many invalid candidates; clip represents the clipping function, which restricts the calculated value to a certain range. to Within the range.

[0112] The design of this scale-adaptive neighborhood radius takes into account the following principles: for scenes with a large overall scale ( For larger graph nodes, the neighborhood radius increases accordingly to cover the possible activity range of the termite target; for individual graph nodes with larger scales ( Larger targets have a larger neighborhood radius because their positional changes and activity ranges are relatively large on the image; for regions with high local target density ( (relatively large), using the logarithmic function ln(1+ Compressive adjustments are made to avoid excessive expansion of the neighborhood radius in dense areas, which could introduce too many irrelevant targets, while ensuring that relevant targets that may be occluded or interact with are still covered in dense areas.

[0113] After determining the scale-adaptive neighborhood radius of each graph node, graph edges are established between the nodes. An edge is established when the center distance between two graph nodes is less than or greater than the larger of their scale-adaptive neighborhood radii.

[0114] in: , Let be any two distinct graph nodes in a local group relationship graph. For the first The first frame The set of edges in a local group relationship graph. , No. Frame-in-Graph Nodes , The center coordinates of the corresponding bounding box , For graph nodes , In the Scale-adaptive neighborhood radius in the frame.

[0115] This connection method, constrained by center distance and adaptive neighborhood radius, ensures that each termite target only establishes connections with other targets within its local neighborhood that may be associated with it. The edge features of the graph include one or more of the following: spatial distance features, scale similarity features, intersection-union ratio features, movement direction consistency features, caste consistency features, apparent similarity features, and relative positional relationship features. These edge features collectively characterize the spatial adjacency relationships, scale distribution relationships, caste distribution relationships, and relative positional structural relationships between adjacent termite targets.

[0116] By constructing a local colony relationship graph, the spatial adjacency relationships between individual termites and the local colony structure can be explicitly modeled as a graph structure. In densely occluded scenarios, even if two termite individuals are highly similar in appearance, their relative positional relationships with surrounding individuals and differences in colony structure can be used to improve the ability to distinguish them. For example, multiple worker termites gathered near bait material may look similar, but the composition of their neighboring individuals may differ. The graph structure can help determine whether they belong to the same termite individual.

[0117] S5. Calculation of candidate association features and matching cost: Based on the local group relationship graph constructed in step S4, various matching costs are calculated for candidate association pairs between the current frame detection target and historical trajectories. The candidate association pairs are comprehensively evaluated from multiple dimensions such as spatial location, motion trend, appearance features, grade attributes and local group structure.

[0118] Specifically, for the detection target With historical trajectory For the candidate association pairs formed, calculate the following matching cost.

[0119] Spatial matching cost , used to characterize the normalized distance between the center point of the detected target and the center point of the trajectory prediction.

[0120] ; in, Indicates the first The center coordinates of the bounding box of each detected target; Indicates the first The predicted center coordinates of the historical trajectory in the current frame; and These represent the scale of the detected target and the trajectory prediction box, respectively; This indicates a preset positive number to prevent the denominator from being zero; This means that the calculation result will be limited to the range of 0 to 1.

[0121] This cost measures the proximity between the spatial location of the detected target and the predicted trajectory location: the closer the distance, the smaller the spatial matching cost, indicating a higher spatial consistency between the detected target and the trajectory. Normalization can eliminate the influence of image resolution variations on the distance scale, making this cost comparable across different image resolutions.

[0122] Motion matching cost It is used to characterize the motion deviation between the detected target position and the trajectory prediction position.

[0123] ; in, The Mahalanobis distance between the observed position and scale of the detected target and the predicted trajectory state; Mahalanobis distance normalization threshold or motion gating threshold.

[0124] This cost measures whether the actual position of the detected target matches the prediction of the trajectory motion model: the smaller the motion deviation, the smaller the motion matching cost, indicating a higher degree to which the detected target conforms to the trajectory motion trend. The motion matching cost can be calculated by combining factors such as the smoothness of the trajectory's historical motion state, the consistency of the motion direction, and the rate of change of the motion velocity, rather than relying on a single prediction model.

[0125] Apparent matching cost It is used to characterize the degree of dissimilarity between the apparent features of the detected target and the apparent memory features of the trajectory.

[0126] ; in, Indicates the middle The apparent feature vector of each detected target; Indicates the first The apparent memory feature vector preserved by each historical trajectory; This indicates a preset positive number to prevent the denominator from being zero.

[0127] This cost can be calculated using methods such as cosine distance, Euclidean distance, or feature similarity transformation values. The trajectory appearance memory features can be maintained using exponential moving average, weighted moving average, or other online update methods, continuously incorporating new appearance information as the trajectory is updated, while maintaining an appropriate memory of historical appearance information.

[0128] Grade matching cost It is used to characterize the degree of inconsistency between the target quality category and the historical quality status of the trajectory.

[0129] ; in, This represents the probability vector of the grade category of the currently detected target. The historical grade probability distribution up to the end of the historical trajectory. This represents the degree of similarity between the current detected grade distribution and the historical grade distribution of the trajectory.

[0130] This cost can be calculated based on the current detection's caste confidence level and the historical caste probability distribution of the trajectory. For candidate association pairs with consistent caste identification results and high caste confidence, the caste matching cost is low; for candidate association pairs with inconsistent caste identification results or low caste confidence, the caste matching cost is high. By introducing the caste matching cost, erroneous cross-caste associations can be effectively avoided, such as avoiding matching worker ant detection results with soldier ant historical trajectories.

[0131] The cost of matching graph relationships It is used to characterize the structural differences between the local group relationships where the detected target is located and the local group relationships where the historical trajectory is located.

[0132] ; in, For the current frame number A vector describing the relationship between each detected target in the local group relationship graph; For the first The relationship description vector of each historical trajectory in the local group relationship graph of trajectory prediction.

[0133] Specifically, we can construct relationship vectors between the detected target and its neighboring graph nodes, as well as historical relationship vectors between historical trajectories and their neighboring graph nodes. Then, we can measure the difference between these two relationship vectors using methods such as cosine distance or Euclidean distance. The lower the graph relationship matching cost, the more similar the local population structure of the detected target in the current frame is to the local population structure of the historical trajectory in historical frames, and the greater the probability that the two targets are the same termite individual.

[0134] S6. Calculation of candidate pair dynamic fusion weights: The activity of termites on dams exhibits significant dynamic changes, and the reliability of matching modalities varies with changes in scene and trajectory states. When termite targets are sparsely distributed and detection confidence is high, spatial location and motion prediction can typically provide reliable matching criteria. However, when targets are densely clustered, severely occluded, or detection confidence is low, relying solely on spatial location can easily lead to false associations. In such cases, the appearance weights are adaptively adjusted based on appearance feature quality, and the roles of reliable modalities such as caste consistency and local colony structure are appropriately enhanced. Therefore, this invention dynamically calculates the fusion weights of each matching modality for each candidate association pair based on the scene and trajectory states, rather than applying fixed weights to all detected targets and trajectories.

[0135] Figure 4 A flowchart of the candidate pair-level dynamic fusion weights and adaptive multimodal matching of this invention is provided. For example... Figure 4 As shown, the specific process is as follows.

[0136] For the detection target With historical trajectory For each candidate association pair, the reliability index of each matching mode is calculated.

[0137] Space reliability This reflects the reliability of spatial location information in the matching process. It is positively correlated with the detection confidence level, that is, the higher the detection confidence level, the more reliable the position of the detection box, and the greater the reference value of spatial matching; The degree of occlusion is negatively correlated with the detection target's occlusion level; that is, the more severe the occlusion, the more likely the detection box position will shift, and the lower the reliability of spatial matching. The occlusion level can be calculated based on the maximum intersection-union ratio (CIU) between the current detection target and its neighboring targets, the proportion of local overlap area, or the local density.

[0138] Motion reliability This reflects the reliability of motion prediction in matching. The number of consecutive successful trajectory matching frames is positively correlated with the stability of trajectory prediction. The more consecutive successful matching frames, the more accurate the trajectory motion state estimation and the greater the reference value of motion prediction. When the trajectory is in the temporary loss recovery phase, the uncertainty of motion prediction increases due to the lack of updates for a period of time, and the reliability of motion prediction decreases accordingly.

[0139] Apparent reliability This reflects the credibility of appearance features in matching. The quality of the apparent features of the detected target and historical trajectory, the degree of occlusion of the detected target, and the density of the detected target are all relevant factors. When the target is clear and unobstructed, the extracted apparent features are of high quality and the confidence of the apparent matching is high. However, when the targets are dense and mutually occluded, the apparent features are easily mixed with information from neighboring targets, and the apparent reliability is reduced.

[0140] Grade Reliability This reflects the credibility of grade category information in the matching process. It is positively correlated with the confidence level of the detected grade and the stability of the historical grade distribution of the trajectory. The higher the confidence level of the detected grade, the more reliable the grade identification result; the more stable the historical grade distribution of the trajectory, that is, the more consistent the grade category of the trajectory in multiple consecutive frames, the stronger the reference value of the grade features.

[0141] Reliability of Graph Relationships This reflects the credibility of local group structure information in the matching process. The reliability of graph relationships is related to the local target density and the number of neighboring nodes. In densely populated areas, the local group structure information is rich, the ability to distinguish graph relationships is stronger, and the reliability of graph relationships is higher. In sparsely populated areas, the number of neighboring nodes is smaller, the graph relationship structure is simpler, and the reliability of graph relationships is correspondingly lower.

[0142] Based on the reliability of each matched mode, the dynamic fusion weight of each matched mode is determined according to the following formula: ; in, Indicates candidate association pairs In the Dynamic fusion weights under various matching modalities Indicates candidate association pairs In the Reliability scoring under various matching modes.

[0143] Pick , , , or These correspond to spatial matching, motion matching, appearance matching, grade matching, and graph relation matching, respectively. The set of matching modes; the denominator is the sum of the reliability of all matching modes plus a preset positive constant. , This is used to prevent the denominator from being zero. Through the above normalization process, the sum of the dynamic fusion weights of each matching mode is made to be 1.

[0144] In this way, the present invention does not use a fixed set of weights to match all detected targets and historical trajectories. Instead, for each candidate association pair, it adaptively generates fusion weights for each modality based on its current scene state and trajectory state. When targets are sparse and clearly visible, spatial and motion matching weights are higher; when targets are dense and severely occluded, appearance and graph relation matching weights are increased; when the grade recognition results are stable and reliable, grade matching plays a greater role in reducing cross-grade false associations.

[0145] S7. Adaptive Multimodal Correlation Cost Matrix Generation: Let the current frame have a total of There are [number] detection targets, and the historical trajectory set contains [number]... Trajectory. After obtaining the matching cost and corresponding dynamic fusion weights for each matching modality, for the current frame's trajectory. The first detection target and the first Candidate association pairs composed of historical trajectories are used to calculate the comprehensive association cost according to the following formula, based on spatial matching cost, motion matching cost, apparent matching cost, grade matching cost, and graph relationship matching cost, and their corresponding dynamic fusion weights. : ; in, , 、 、 、 These are spatial matching cost, motion matching cost, appearance matching cost, grade matching cost, and graph relation matching cost, respectively. 、 、 、 and These are the dynamic fusion weights corresponding to each matching modality, and they satisfy the following: ; The lower the overall correlation cost, the greater the likelihood that the current detection target and the historical trajectory belong to the same termite individual.

[0146] To eliminate obviously unreasonable candidate association pairs, a gating indicator is set. When a candidate association pair simultaneously satisfies the following conditions: spatial distance does not exceed the spatial gating threshold, motion deviation does not exceed the motion gating threshold, grade inconsistency does not exceed the grade gating threshold, and comprehensive association cost does not exceed the association cost threshold, let... Otherwise, Its valid conditions are expressed as follows: ; in, This represents the spatial distance between the detected target and the predicted trajectory location. Indicates motion deviation; , , and These represent the spatial gating threshold, motion gating threshold, grade gating threshold, and comprehensive correlation cost threshold, respectively.

[0147] Based on the gating results, determine the element values ​​of the candidate association pairs in the final association cost matrix: ; in, Indicates the first The first frame The first detection target and the first The final associated cost of a historical trajectory after gating; This is a preset maximum value that is greater than the cost of all valid associations. When At that time, the matrix elements retain the original comprehensive correlation cost. ;when When, the matrix elements are set to This indicates that no association is allowed between the detected target and its historical trajectory. In actual calculations, this matrix element can also be directly marked as invalid.

[0148] The adaptive multimodal correlation cost matrix is ​​composed of the gated correlation costs between all detected targets and historical trajectories. ; in, Indicates the first The adaptive multimodal association cost matrix of a frame; Indicates the number of targets detected in the current frame; This indicates the number of historical trajectories associated with the current frame data.

[0149] The matrix can be expanded as follows: ; Wherein, the first of the matrix The line corresponds to the current frame number The first detection target, the first Column corresponding to the first Historical trajectory, matrix elements This represents the final association cost after dynamic weight fusion and gating processing.

[0150] S8. Hierarchical Data Association and Trajectory Update: The adaptive multimodal association cost matrix generated in step S7 This completes the data association between the detected target in the current frame and the historical trajectory. To balance the accuracy of the data association with the ability to recover the trajectory after short-term occlusion, the data association process is divided into three layers: a first layer of strict matching, a second layer of recovery matching, and a third layer of unmatched target processing. Figure 5 As shown.

[0151] First-level strict matching: Based on the detection confidence threshold The current frame's detected targets are divided into a set of high-confidence detected targets: ; in, Indicates the first The set of high-confidence detection targets in the frame; Indicates the first The first frame One detection target; Indicates the first The detection confidence of each target. This indicates the high-confidence detection threshold.

[0152] Trajectories whose lifecycle state is active are selected from the historical trajectory set to form the active trajectory set: ; in, Indicates the first The set of historical trajectories that are active at the end of the frame and participate in the first-level strict matching of the current frame; Indicates the first The historical trajectory in the first The complete trajectory state of the frame; Indicates the first The life cycle status of a historical trajectory; This indicates an active state, meaning the trajectory has been effectively confirmed and is in a normal, continuous tracking state.

[0153] A one-to-one matching is performed between the high-confidence target set and the active trajectory set based on the association cost matrix. This matching can be expressed as: ; in, This represents the matrix of optimal matching results obtained from the first level of strict matching; This represents the matching decision matrix to be solved; Indicates the first The first detection target and the first The final association cost of historical trajectories after dynamic weight fusion and gating processing; Denotes the binary matching decision variable, when the first... The first detection target and the first The value is 1 when a historical trajectory is successfully matched, and 0 otherwise.

[0154] One-to-one matching should satisfy: ; in, Indicates the first The first detection target and the first The historical trajectory was successfully matched. This indicates that the two did not match.

[0155] Candidate association pairs are allowed to participate in the first-level matching when their association cost satisfies the following formula: ; in, This is the threshold for strict matching at the first layer. For association costs greater than... Alternatively, candidate association pairs marked as invalid in step S7 will not be matched.

[0156] The first layer of matching is mainly used to detect termite targets with high confidence and stable historical status. It prioritizes the accuracy of data association by using strict matching thresholds to reduce the probability of false associations and identity switching.

[0157] Second-level recovery matching: After the first layer of matching is completed, the unmatched high-confidence detected targets, medium- and low-confidence detected targets, and candidate detected targets located within the local search area of ​​the temporarily lost trajectory in the first layer are combined to form the recovered detected target set: ; in, Indicates the first The set of recovery detection targets in the frame that participate in the second-layer recovery matching; This represents the set of unmatched detected targets in the first layer; Indicates the first The first frame One detection target; and These represent the minimum and maximum detection confidence thresholds, respectively. Indicates the first The detection confidence level of each target.

[0158] The unmatched active trajectories in the first layer and the trajectories in a temporarily lost state are combined to form a set of recovered trajectories: ; in, This represents the set of recovery trajectories that participate in the second-level recovery matching of the current frame; This represents the set of unmatched active trajectories in the first layer; Indicates the first The historical trajectory in the first The complete trajectory state at the end of the frame; Indicates the first The life cycle status of a historical trajectory; This indicates a temporary loss of track data due to short-term occlusion or missed detection, but it is still retained for subsequent recovery of the association.

[0159] For the The recovered trajectory searches for candidate targets only near its predicted location. Let the trajectory prediction center be... The detection target center is Then the candidate associations satisfy: ; in, The scale-adaptive neighborhood radius is determined in step S4.

[0160] Among candidate association pairs that satisfy the local neighborhood condition and the corresponding gating condition, a one-to-one matching is performed again based on the comprehensive association cost. The second layer uses a relatively relaxed recovery matching threshold: ; That is, the first layer uses a stricter threshold. To ensure matching accuracy, a more lenient threshold is used in the second layer. Improve the ability to recover trajectories after short-term occlusion, missed detection, and dense interactions. Successfully recovered temporarily lost trajectories retain their original unique trajectory numbers and are not re-established.

[0161] Third-layer unmatched target handling: After the first and second layer matching, for the detection targets that are still not matched, they are initialized as candidate trajectories and passed to step S9 for candidate confirmation.

[0162] For historical trajectories that are still not matched, based on their original lifecycle status, the number of consecutive unmatched frames, and the trajectory confidence, they are passed to step S9 for candidate deletion, temporary retention, or termination.

[0163] Therefore, step S8 is mainly responsible for determining the matching relationship between the detection target and the historical trajectory, and step S9 performs the complete life cycle state transition, specifically candidate confirmation, active maintenance, temporary loss recovery and termination deletion.

[0164] Track update after successful matching For historical trajectories that have been successfully matched in the first or second layer, the trajectory prediction state is corrected based on the position and scale observation results of the corresponding detected targets: ; in, Indicates the first The trajectory in the first The frame's motion state after correction; Indicates the first The predicted state of the trajectory in the current frame. Indicates the first The location and scale observations of the detection target Represents the observation matrix. This represents the state correction coefficient or Kalman gain.

[0165] The apparent memory features of the trajectory and the probability distribution of historical grades are updated as follows: ; in, Indicates the first The trajectory in the first Apparent memory features after frame update; Indicates the first The trajectory in the first Historical grade probability distribution after frame update; This indicates the original apparent memory characteristics of the trajectory. This represents the apparent features of the currently matched and detected target. Indicates the coefficient for retaining apparent historical information; This represents the probability distribution of the original historical grade of the trajectory. This represents the probability distribution of the grade of the current detection target. This represents the retention coefficient for historical information on grade.

[0166] The trajectory confidence is updated based on the current detection confidence and matching quality: ; in: Indicates the first The trajectory in the first Frame matching updated trajectory confidence; This represents the original confidence level of the trajectory. This indicates the detection confidence level of the current target. This represents the normalized overall cost of association, and its value ranges from 0 to 1. This represents the trajectory confidence smoothing coefficient.

[0167] After a successful match, the lifecycle state of the trajectory remains unchanged or is restored to an active state: ; in: Indicates the first The trajectory in the first The lifecycle state after frame update; This indicates an active state, meaning the trajectory has been effectively confirmed and is in a normal, continuous tracking state.

[0168] Through the three-layer data association method described above, the first layer, strict matching, ensures the accuracy of association for high-confidence targets; the second layer, restorative matching, improves trajectory recovery capabilities under conditions of occlusion, missed detection, and dense interaction; and the third layer processes unmatched detected targets and historical trajectories, completing candidate trajectory establishment, temporary loss retention, and termination judgment. Each layer of matching can be completed using Hungarian matching, greedy matching, minimum cost matching, or other one-to-one data association methods.

[0169] S9. Trajectory Lifecycle Management and Grade-Based Count Correction: Trajectory lifecycle management is a crucial step in achieving stable and reliable graded counting in this invention. By implementing closed-loop management of the entire process of trajectory selection, confirmation, activity, temporary loss, and termination, the impact of false detections, missed detections, and duplicate detections on the counting results is effectively suppressed.

[0170] Figure 6 A flowchart of the trajectory lifecycle management of this invention is provided. For example... Figure 6 As shown, trajectory lifecycle management includes four stages: candidate confirmation, active maintenance, temporary loss recovery, and termination deletion. It follows the trajectory status representation defined in step S3. The trajectory in the first The frame's lifecycle state satisfies: These correspond to candidate state, active state, temporarily lost state, and terminated state, respectively.

[0171] 1. Candidate Confirmation Stage: For detected targets that are still not matched in step S8, candidate trajectories are established, but they are not immediately included in the effective termite count. The positional continuity of the candidate trajectories is represented as follows: ; in, Indicates the first The candidate trajectory in the 1st Frame and the Normalized center position change between frames The center position of the trajectory, This indicates the center position of the bounding box in the previous frame. and The width and height of the bounding box in the previous frame. The preset positive number is used. When the candidate trajectory satisfies: ; in, Indicates the first Candidate trajectories up to the [number]th The number of frames that were successfully matched consecutively. Indicates the first The candidate trajectory in the 1st Frame belongs to the Historical smoothed probability of termite caste.

[0172] Candidate trajectories are converted into active trajectories and assigned unique trajectory numbers. Among these, To confirm the frame count threshold, This represents the average detection confidence during the candidate period. To detect the confidence threshold, For positional continuity threshold, This is the consistency threshold for the grade. If a candidate trajectory does not meet the above conditions within the maximum number of confirmed frames, it is judged as a false detection and deleted.

[0173] 2. Active maintenance phase: For the active trajectories that are successfully matched in step S8, update their motion state, apparent memory features, historical grade probability distribution, trajectory confidence, number of consecutive successfully matched frames, and number of consecutive unmatched frames as described in step S8, and maintain their active state: ; The active trajectories continue to use the original unique trajectory number. For active trajectories that did not match in step S8, the number of consecutive unmatched frames is incremented by one. ; The number of consecutive successful matches will be reset to zero. At the same time, the trajectory is changed to a temporarily lost state: .

[0174] 3. Temporary loss recovery phase: For temporarily lost trajectories caused by short-term occlusion, missed detection, or dense interaction, their predicted state, apparent memory features, historical grade probability distribution, trajectory confidence, and unique trajectory number are retained within a preset maximum number of temporarily lost frames.

[0175] The trajectory was temporarily lost in the subsequent episodes. The predicted motion state in a frame can be represented as: ; in, Indicates the first The temporarily lost trajectory is in the first Predicted motion state in the frame; Represents the state transition matrix of Power; This indicates the most recent valid update of the motion state before the trajectory entered the temporary loss state; The number of consecutive frames where the trajectory is temporarily lost. Let be the state transition matrix.

[0176] To accommodate prediction uncertainties, the local search radius of the temporarily lost trajectory can be appropriately increased with the duration of the temporary loss: ; in, Indicates the first The temporarily lost trajectory is in the first Local recovery search radius in the frame; This indicates the initial recovery search radius when the trajectory enters a temporary loss state; To maximize the search radius, This is a search range adjustment factor; This indicates the number of consecutive frames lost in the trajectory.

[0177] When the subsequently detected target is located within the search area and satisfies: ; in, This represents the maximum associated cost threshold allowed for matching the recovery of temporarily lost trajectories.

[0178] When the temporarily lost trajectory is within the maximum number of temporarily lost frames, and a successful re-match with the target detected in subsequent frames is achieved through the second-layer matching in step S8, it is restored to an active state. And reset the number of consecutive unmatched frames to zero. Once the track is successfully restored, it will continue to use the original unique track number and will not re-establish the track, thereby reducing duplicate track establishment and duplicate counting caused by occlusion recurrence.

[0179] 4. Termination of deletion phase: When a temporarily lost trajectory meets any of the following conditions, it is converted to a terminated state. : ; in, The maximum number of temporarily lost frames. Indicates the first The trajectory in the first Frame trajectory confidence, The confidence threshold for trajectory termination. The degree of abnormality in trajectory movement, This is the threshold for abnormal movement. This indicates a logical OR.

[0180] Termination trajectories are no longer included in the data association of subsequent frames and the current termite count, but their unique trajectory number, final grade, activity duration, and historical trajectory information can be retained for the purpose of accumulating the number of confirmed trajectories and analyzing historical termite populations.

[0181] 5. Grade-based counting correction: For a confirmed trajectory, its current smoothing grade is determined based on the historical grade probability distribution updated in step S8: ; in, Indicates the first The trajectory in the first The termite caste is determined by the frame; Indicates the first The trajectory in the first Frame belongs to the The historical smoothed probability of each termite caste.

[0182] For the The termite caste, number one Current number of termites in the frame Represented as: ; in, Indicates the first The first frame The current number of termites for each termite caste; Indicates characteristic functions; Indicates the first The number of consecutive unmatched frames for a trajectory; This indicates the maximum number of frames that can be kept in place to participate in the current statistics for temporarily lost trajectories; Indicates the first The current grade of the trajectory is determined after smoothing with historical grade probabilities. Active trajectories and those in a short-term lost state with no more than [number missing] consecutive unmatched frames are also included. The trajectory is included in the current quantity statistics. Each trajectory is evaluated individually. When a trajectory is active, or temporarily lost but the number of consecutive unmatched frames has not exceeded [a certain threshold], it is considered active. And the smoothness grade of the trajectory is the first. When the class is used, the trajectory is included in the first class. The current number of termites.

[0183] Through the above-mentioned trajectory lifecycle management and grade-based counting correction, this invention performs individual counting based on the unique trajectory number of the confirmed trajectory, restores the original trajectory number for termite targets that reappear after short-term occlusion or missed detection, performs historical probability smoothing for short-term fluctuations in grade category, and suppresses invalid candidate trajectories and long-term failed trajectories, thereby reducing the impact of missed detection, duplicate track establishment, duplicate detection, and single-frame grade misclassification on the grade-based continuous monitoring results.

[0184] Figure 7 A schematic diagram of the ant population dynamic counting and ant population output of this invention is provided. Based on the full lifecycle management of the trajectory, tiered counting correction is performed. The counting method of this invention does not simply count the number of detection boxes in each frame, but rather counts individuals based on the unique trajectory number of the confirmed trajectory, fundamentally avoiding the counting fluctuations caused by missed or duplicate detections in traditional detection box counting methods.

[0185] The specific strategies for graded counting correction include: Individual counting based on unique trajectory numbers: As long as a confirmed trajectory has not been terminated, it is counted as the same termite individual in trajectory statistics, regardless of how many times it is detected in consecutive frames. This completely eliminates the problem of duplicate counting caused by repeated detection of the same target in consecutive frames.

[0186] Track number restoration after occlusion: For termite targets that are temporarily lost due to short-term occlusion or missed detection and then reappear, if they successfully match the lost track, the original unique track number is restored and no new track is established, thereby avoiding duplicate track establishment and duplicate counting caused by occlusion.

[0187] Historical Class Probability Smoothing: For trajectories where the class category fluctuates briefly in a few frames, a smoothing correction is performed based on the historical class probability distribution of the trajectory. For example, if a trajectory is identified as a worker ant with high confidence in most historical frames, but is misidentified as a nymph or larva only in a single frame, the system will not immediately change the class of the trajectory within that single frame. Instead, it will combine the class identification results and class probability distribution of multiple consecutive frames to make a smooth determination, thereby avoiding drastic fluctuations in class counts caused by misclassification in a single frame.

[0188] Invalid trajectory suppression: Trajectories with a duration less than the minimum number of valid frames, abnormal motion states such as sudden position changes, abnormal speeds, or a correlation confidence level that is consistently below the invalid threshold are identified as invalid trajectories and removed during counting to reduce the impact of background interference and false detections on the counting.

[0189] S10. Monitoring Results Output: The final output of the continuous monitoring results of termites in the dikes by class includes the number of frames, changes in the number of time windows, the cumulative number of confirmed trajectories, the duration of activity of a single trajectory, termite activity paths, distribution of activity hotspots, trend of termite infestation, and abnormal growth alerts for different termite classes.

[0190] The monitoring system can provide auxiliary information for assessing termite risk based on the number and activity characteristics of termite castes: when the number of worker termites increases continuously, it indicates increased foraging, nest building, or tunneling activities; when soldier termites are concentrated in a certain area, it indicates that the area may be close to the termite colony's defensive activity area; when the number of nymphs or larvae increases, it indicates that the termite colony's expansion potential is enhanced; when multiple castes of termites increase simultaneously and their activity duration is prolonged, it indicates a high risk of termite activity in the monitored area; when the termite activity area changes from a localized, point-like distribution to a continuous, strip-like distribution, it indicates an increased possibility of tunnel expansion.

[0191] The monitoring results can be output to on-site display terminals, dam safety monitoring platforms, or remote management systems, providing data support for dam safety management personnel's termite control decisions.

[0192] Example 2: This embodiment provides a continuous monitoring system for termites in dikes by classification. The system includes a video acquisition module, a classification detection module, a trajectory prediction module, a local colony relationship graph construction module, an adaptive multimodal matching module, a trajectory lifecycle management module, a classification counting module, and a monitoring output module, which are used to execute the above-described continuous monitoring method for termites in dikes by classification.

[0193] The video acquisition module is used to acquire video images of the termite activity area on the dam. This module may include a camera, supplementary lighting, a mounting bracket, an observation window for the trapping box, and a communication unit. The camera can be installed inside the termite trapping device on the dam, or near the dam slope, dam shoulder, or suspected termite infestation area, transmitting video images to the processing equipment via wired or wireless means.

[0194] The termite classification detection module is used to detect termite targets in the current frame image and outputs detection bounding boxes, detection confidence scores, termite caste categories, and target appearance features. This module integrates a backbone feature extraction unit, a morphological perception feature enhancement unit (including orientation perception branch, scale perception branch, and local texture perception branch), a context-guided compact reconstruction unit, and a detection output unit, enabling accurate detection of small termite targets and fine-grained caste identification against the complex background of dams.

[0195] The trajectory prediction module is used to predict the position and scale of a trajectory in the current frame based on the status of historical trajectories, thus obtaining the predicted status of historical trajectories. This module stores the number, position, velocity, scale, grade status, appearance features, and lifecycle status information of each trajectory.

[0196] The local population graph construction module is used to uniformly represent the detected target in the current frame and the historical trajectory prediction state as graph nodes, and construct a scale-adaptive local population graph. This module determines the scale-adaptive neighborhood radius based on the scale of the graph nodes, the scale statistics of the target in the current frame, and the local target density, and establishes graph edges within the local neighborhood to provide local population structure information for subsequent matching.

[0197] The adaptive multimodal matching module calculates dynamic fusion weights for each matching modality based on the detected target density, occlusion level, detection confidence, and trajectory lifecycle state, generating an adaptive multimodal association cost matrix and completing the data association between the detected target and historical trajectories. This module integrates five modalities: spatial matching, motion matching, appearance matching, quality matching, and graph relation matching, and achieves adaptive adaptation to scene and trajectory states through dynamic weights at the candidate pair level.

[0198] The trajectory lifecycle management module is used for candidate confirmation, active maintenance, temporary loss recovery, and termination deletion of trajectories. This module can suppress direct trajectory establishment due to false detection in a single frame, support trajectory recovery after short-term occlusion, reduce identity switching, and clear long-term invalid trajectories.

[0199] The termite class counting module is used to perform class-based counting correction based on the unique trajectory number of the confirmed trajectory and the historical class status, and outputs the real-time number of termites of different classes such as worker ants, soldier ants, nymphs, and larvae, as well as the changes in the number of windows.

[0200] The monitoring output module is used to output the continuous monitoring results of termites in the dam by grade, including the change curve of termite population by grade, activity duration, trajectory distribution map, distribution of activity hotspots, termite infestation trend and abnormal alert information.

[0201] Project deployment method: This invention can be applied to various scenarios, such as continuous monitoring of termite activity in reservoir earth-rock dams, video monitoring of termite traps on dams, dynamic observation of suspected termite tunnel areas on dams, evaluation of the effects of termite control before and after treatment, intelligent inspection of biological hazards on dams, and data access to water conservancy project safety monitoring platforms.

[0202] In practical engineering applications, flexible deployment methods can be selected based on the network, power supply, and equipment conditions at the monitoring site. For monitoring points with wired networks and stable power supplies, the video streams acquired by the video acquisition devices can be transmitted in real time to a server or cloud for centralized processing and analysis. For remote dam sites with poor network conditions, edge computing devices can be used to deploy the various processing modules of this invention, achieving a combination of real-time inference, local data caching, and scheduled uploading, ensuring stable system operation even under weak network or network outage conditions. Edge computing devices can be industrial control computers, edge smart boxes, or embedded development boards with GPU, NPU, or embedded AI acceleration capabilities.

Claims

1. A method for continuous monitoring of termites in dikes by grade, characterized in that... Includes the following steps: S1: Obtain video images of the termite activity area on the dam to obtain the current frame image; S2: Input the current frame image into the termite detection model to obtain the current frame detection result, which includes the detection bounding box, detection confidence, termite caste category and target appearance features; S3: Predict the position and scale of the historical trajectory in the current frame based on the state of the historical trajectory to obtain the predicted state of the historical trajectory. S4: Represent the current frame's detected target and the historical trajectory prediction status as graph nodes. Based on the scale of the graph nodes, the current frame's target scale statistics, and the local target density, determine the scale-adaptive neighborhood radius, and construct a local group relationship graph within the local area defined by the scale-adaptive neighborhood radius. S5: Based on the local group relationship graph, calculate the spatial matching cost, motion matching cost, appearance matching cost, grade matching cost and graph relationship matching cost between the detected target in the current frame and the historical trajectory, respectively; S6: Calculate the dynamic fusion weights of each matching modality based on the current frame's detected target density, detected target occlusion degree, detection confidence, and trajectory lifecycle state; S7: Generate an adaptive multimodal association cost matrix based on spatial matching cost, motion matching cost, appearance matching cost, grade matching cost, graph relationship matching cost, and the corresponding dynamic fusion weights; S8: Based on the adaptive multimodal association cost matrix, the target detected in the current frame is associated with historical trajectories, and the trajectory status is updated according to the association results; S9: Perform full lifecycle management of trajectories, including candidate confirmation, active maintenance, temporary loss recovery and termination deletion, and perform grade-based counting correction based on the unique trajectory number of the confirmed trajectory and its historical grade status; S10: Output the results of continuous monitoring of termite classification in the dikes.

2. The method for continuous monitoring of termites in dikes by grade according to claim 1, characterized in that: In step S2, the processing procedure of the termite detection model by grade includes: S2.1: Extract multi-scale visual features of the current frame image using the backbone feature extraction unit: ； in, This indicates the backbone feature extraction operation. This represents the first layer of visual features. Represents the second layer of visual features. Indicates the first Layered visual features; This represents the preprocessed image of the current frame obtained in step S1; S2.2: Enhance the direction, scale and local texture of multi-scale visual features using a morphology-aware feature enhancement unit to obtain morphology-enhanced features, wherein the morphology-aware feature enhancement unit includes a direction-aware branch, a scale-aware branch and a local texture-aware branch. The processing procedure of the morphological feature enhancement unit is represented as follows: ； in: , and These represent orientation-aware features, scale-aware features, and local texture-aware features, respectively. Indicates the first The basic visual features obtained from the frame image after backbone feature extraction and feature alignment processing; This indicates the direction-aware feature extraction operation, which is used to enhance information on the long axis direction of the termite's body, the bending direction of its posture, and the direction of local edges; This represents a scale-aware feature extraction operation, used to enhance information on individual termites at different scales, head size, and overall outline. This represents a local texture-aware feature extraction operation, used to enhance local texture information of the termite's head, thorax, abdomen, and body surface; The features extracted from the three branches are fused to obtain the morphological enhancement features: ； in: Indicates the first The morphological enhancement features obtained by fusing orientation-aware, scale-aware, and local texture-aware features of a frame image; in, , and These are the fusion weights for direction, scale, and local texture features, respectively, and they satisfy: ； S2.3: Context-guided compact reorganization units are used to aggregate, compress, and reorganize local background features, adjacent target distribution features, and multi-scale context features. The reorganized context features are then fused with morphological enhancement features to obtain fused features; the details are as follows: ； in, Indicates the first The context reassembly features are obtained after the frame undergoes context feature aggregation, channel compression, and compact reassembly. Indicates local background features, Indicates the distribution characteristics of adjacent targets. Represents multi-scale contextual features, This represents context feature aggregation, channel compression, and compact reassembly operations; The feature fusion unit integrates morphological enhancement features, contextual reconstruction features, and shallow, mid-level, and deep semantic features: ； in, This indicates feature concatenation, convolutional weighting, or attention fusion operations. Indicates the final fusion characteristics; S2.4: Input the fused features into the detection output unit, and output the detection bounding box, detection confidence score, termite caste category, and target appearance features: The detection output unit performs bounding box regression, grade classification, confidence assessment, and appearance feature extraction based on fused features, and outputs the detection result for the current frame: ； in, Indicates the first The complete set of termite target detection results for the frame; Indicates the detection output unit; Indicates the first The final fusion features of the frames; This indicates the number of termite targets detected in the current frame; Indicates the first The detection bounding box of each target; Indicates the confidence level of the detection; This represents the probability distribution of grade categories; Describes the target appearance feature, the target appearance feature Used in subsequent steps to detect the appearance matching between the target and historical trajectories; No. The caste category of a termite target is represented as follows: ； in, Indicates the first The trajectory in the first The current termite class is determined after the frame is smoothed by historical class probabilities; Indicates the first The termite target belongs to the first The probability of each grade category; This indicates that the grade category with the highest historical grade probability is selected as the current smoothing grade of the trajectory.

3. The method for continuous monitoring of termites in dikes by grade according to claim 2, characterized in that: S3 includes: for the already established... For each termite trajectory, based on its historical location, target scale, movement speed, appearance characteristics, historical caste status, and life cycle status, the position and scale of the trajectory in the current frame are predicted to obtain the historical trajectory prediction status. In the In the frame, the first The state of a trajectory is represented as follows: ； in: Indicates the first The historical trajectory in the first The complete trajectory state at the end of the frame; A unique trajectory number; The trajectory motion state; For trajectory apparent memory features; This represents the probability distribution of historical grades. This refers to the lifecycle state; and These represent the number of consecutive successfully matched frames and the number of consecutive unmatched frames, respectively. The trajectory motion state is represented as follows: ； in, Indicates the first The trajectory in the first The motion state vector of the frame, The coordinates of the center of the trajectory bounding box. For the width and height of the bounding box, For the rate of change of position, The rate of change of scale; A uniform motion model is used to predict the trajectory state: ； in, The predicted state for the current frame is represented by the state transition matrix. Represented as: ； In the formula, The time interval between adjacent video frames. It is a fourth-order identity matrix; Represents a fourth-order zero matrix; Obtain the trajectory prediction bounding box for the current frame from the prediction state: ； In the formula, Indicates the first The historical trajectory in the first Predicted bounding boxes in the frame; and These represent the x and y coordinates of the predicted bounding box center point, respectively. and These represent the width and height of the predicted bounding box, respectively. Step S3 is based only on the first The historical trajectory in the first The motion state of the frame, and its position in the first frame. The position and scale in the frame are predicted to obtain the trajectory prediction state; Finally, the first The predicted state of the trajectory in the current frame is represented as follows: ； In the formula, Indicates the first The trajectory in the first The complete trajectory state of the frame; Indicates a unique trajectory number; Indicates the trajectory motion state; This represents the apparent memory features of the trajectory; Represents the probability distribution of historical grades; Indicates the trajectory lifecycle state; and These represent the number of consecutive successfully matched frames and the number of consecutive unmatched frames, respectively.

4. The method for continuous monitoring of termites in dikes by grade according to claim 3, characterized in that: S4 includes: S4.1: For each graph node Based on the width of its corresponding detection bounding box or trajectory prediction box and height Calculate the scale of the nodes in the graph. ; S4.2: Determine the target scale statistics for the current frame based on the scale of all detected targets in the current frame. The The mean, median, truncated mean, or quantile of the target scale in the current frame; S4.3: For each graph node Calculate its local target density: ； in: To preset the initial statistical radius, With the node in the graph as the center and a radius of , The number of graph nodes within the specified range; S4.4: Determine the scale-adaptive neighborhood radius of the graph node: ； in: , , The neighborhood adjustment coefficient, This is the lower bound of the neighborhood radius. The upper limit of the neighborhood radius, Represent the natural logarithm function; S4.5: Based on the scale-adaptive neighborhood radius of each graph node, establish graph edges between graph nodes to form a local group relationship graph; the local group relationship graph is constructed as follows: when the center distance between two graph nodes is less than or greater than the larger of their scale-adaptive neighborhood radii, a graph edge is established, that is: ； in: , For any two distinct graph nodes in a local group relationship graph; For the first The first frame The set of edges in a local group relationship graph; , No. Frame-in-Graph Nodes , The center coordinates of the corresponding bounding box; , For graph nodes , In the Scale-adaptive neighborhood radius in the frame; It means "if and only if"; This represents the L2 norm, used to calculate the Euclidean distance between the centers of the bounding boxes corresponding to two graph nodes; Indicates a node in the graph Graph Nodes The larger value is selected from the corresponding scale-adaptive neighborhood radius; This indicates the type identifier of the local group relationship diagram; and ;when When, it represents the local group relationship graph formed by the detected targets in the current frame, when At this time, it represents a local group relationship diagram of trajectory prediction composed of historical trajectory prediction states.

5. The method for continuous monitoring of termites in dikes by grade according to claim 4, characterized in that: S5 includes: 5.1: Calculate the detection target With historical trajectory Spatial matching cost between The spatial matching cost Used to characterize the normalized distance between the center point of the detected target and the center point of the trajectory prediction; ； in, Indicates the first The center coordinates of the bounding box of each detected target; Indicates the first The predicted center coordinates of the historical trajectory in the current frame; and These represent the scale of the detected target and the trajectory prediction box, respectively; This indicates a preset positive number to prevent the denominator from being zero; This indicates that the calculation result will be limited to the range of 0 to 1; 5.2: Calculate the detection target With historical trajectory Motion matching cost between The motion matching cost Used to characterize the motion deviation between the detected target position and the trajectory prediction position; Motion matching cost , used to characterize the motion deviation between the detected target position and the trajectory prediction position; ； in, The Mahalanobis distance between the observed position and scale of the detected target and the predicted trajectory state; Mahalanobis distance normalization threshold or motion gating threshold; 5.3: Calculate the detection target With historical trajectory Apparent matching cost between The apparent matching cost Used to characterize the degree of dissimilarity between the apparent features of the detected target and the apparent memory features of the trajectory; ； in, Indicates the middle The apparent feature vector of each detected target; Indicates the first The apparent memory feature vector preserved by each historical trajectory; This indicates a preset positive number to prevent the denominator from being zero; 5.4: Calculate the detection target With historical trajectory The cost of matching grades between them The grade matching cost Used to characterize the degree of inconsistency between the target quality category and the historical quality status of the trajectory; ； in, This represents the probability vector of the grade category of the currently detected target. The historical grade probability distribution up to the end of the historical trajectory. This represents the degree of similarity between the current detected grade distribution and the historical grade distribution of the trajectory. 5.5: Calculate the detection target With historical trajectory Graph relationship matching cost The graph relation matching cost Used to characterize the structural differences between the local group relationships where the detected target is located and the local group relationships where the historical trajectory is located; ； in, For the current frame number A vector describing the relationship between each detected target in the local group relationship graph; For the first The relationship description vector of each historical trajectory in the local group relationship graph of trajectory prediction.

6. The method for continuous monitoring of termites in dikes by grade according to claim 5, characterized in that: In step S6, the target is detected. With historical trajectory The constructed candidate association pairs, computational space reliability Motion reliability Apparent reliability Grade reliability Reliability of graph relationships ; Among them, spatial reliability It is positively correlated with detection confidence and negatively correlated with the degree of target occlusion. Motion reliability Related to the number of consecutive successful trajectory matches or the status of temporary loss recovery; Apparent reliability Related to the quality of the target's appearance features and the density of the detected targets; Grade Reliability It is related to the confidence level of the detection grade and the stability of the historical grade distribution of the trajectory; Reliability of Graph Relationships Related to local target density and the number of neighboring nodes; The dynamic fusion weights for each matched modality are determined using the following formula: ； in, This represents the dynamic fusion weight of the candidate association pair (i,j) in the k-th matching modality; k can be s, m, a, c, or g, and K is the set of matching modalities. It is a preset positive constant.

7. The method for continuous monitoring of termites in dikes by grade according to claim 6, characterized in that: In S7, it is assumed that the current frame has a total of There are [number] detection targets, and the historical trajectory set contains [number]... Trajectory; after obtaining the matching cost and corresponding dynamic fusion weights for each matching modality, for the current frame's trajectory; The first detection target and the first Candidate association pairs composed of historical trajectories are used to calculate the comprehensive association cost according to the following formula, based on spatial matching cost, motion matching cost, apparent matching cost, grade matching cost, and graph relationship matching cost, and their corresponding dynamic fusion weights. : ； in, , 、 、 、 These are spatial matching cost, motion matching cost, appearance matching cost, grade matching cost, and graph relation matching cost, respectively. 、 、 、 and These are the dynamic fusion weights corresponding to each matching modality, and they satisfy the following: ； The lower the overall correlation cost, the greater the likelihood that the current detection target and the historical trajectory belong to the same termite individual; To eliminate obviously unreasonable candidate association pairs, a gating indicator is set. When a candidate association pair simultaneously satisfies the following conditions: spatial distance does not exceed the spatial gating threshold, motion deviation does not exceed the motion gating threshold, grade inconsistency does not exceed the grade gating threshold, and comprehensive association cost does not exceed the association cost threshold, let... Otherwise, Its valid conditions are expressed as follows: ； in, This represents the spatial distance between the detected target and the predicted trajectory location. Indicates motion deviation; , , and These represent the spatial gating threshold, motion gating threshold, grade gating threshold, and comprehensive correlation cost threshold, respectively. Based on the gating results, determine the element values ​​of the candidate association pairs in the final association cost matrix: ； in, Indicates the first The first frame The first detection target and the first The final associated cost of a historical trajectory after gating processing; It is a preset maximum value that is greater than the cost of all valid associations; when At that time, the matrix elements retain the original comprehensive correlation cost. ;when When, the matrix elements are set to This indicates that no association is allowed between the detected target and the historical trajectory; in actual calculations, this matrix element can also be directly marked as invalid. The adaptive multimodal correlation cost matrix is ​​composed of the gated correlation costs between all detected targets and historical trajectories. ； in, Indicates the first The adaptive multimodal association cost matrix of a frame; Indicates the number of targets detected in the current frame; This indicates the number of historical trajectories associated with the current frame data.

8. The method for continuous monitoring of termites in dikes by grade according to claim 7, characterized in that: In step S8, the adaptive multimodal association cost matrix generated in step S7 is used. The data association between the target detected in the current frame and the historical trajectory is completed. In order to balance the accuracy of the data association and the trajectory recovery capability after short-term occlusion, the data association process is divided into three layers: strict matching, recovery matching, and unmatched target processing. (1) First-level strict matching: Based on the detection confidence threshold The current frame's detected targets are divided into a set of high-confidence detected targets: ； in, Indicates the first The set of high-confidence detection targets in the frame; Indicates the first The first frame One detection target; Indicates the first The detection confidence of each target. Indicates the high-confidence detection threshold; Trajectories whose lifecycle state is active are selected from the historical trajectory set to form the active trajectory set: ； in, Indicates the first The set of historical trajectories that are active at the end of the frame and participate in the first-level strict matching of the current frame; Indicates the first The historical trajectory in the first The complete trajectory state of the frame; Indicates the first The life cycle status of a historical trajectory; This indicates an active state, meaning the trajectory has been effectively confirmed and is in a normal, continuous tracking state; A one-to-one matching is performed between the high-confidence target set and the active trajectory set based on the association cost matrix; the matching can be expressed as: ； in, This represents the matrix of optimal matching results obtained from the first level of strict matching; This represents the matching decision matrix to be solved; Indicates the first The first detection target and the first The final association cost of historical trajectories after dynamic weight fusion and gating processing; Denotes the binary matching decision variable, when the first... The first detection target and the first The value is 1 if a historical trajectory is successfully matched, and 0 otherwise. One-to-one matching should satisfy: ； in, Indicates the first The first detection target and the first The historical trajectory was successfully matched. This indicates that the two did not match; Candidate association pairs are allowed to participate in the first-level matching when their association cost satisfies the following formula: ； in, The threshold for strict matching at the first layer; for association costs greater than Alternatively, candidate association pairs marked as invalid in step S7 will not be matched. (2) Second-level recovery matching: After the first layer of matching is completed, the unmatched high-confidence detected targets, medium- and low-confidence detected targets, and candidate detected targets located within the local search area of ​​the temporarily lost trajectory in the first layer are combined to form the recovered detected target set: ； in, Indicates the first The set of recovery detection targets in the frame that participate in the second-layer recovery matching; This represents the set of unmatched detected targets in the first layer; Indicates the first The first frame One detection target; and These represent the minimum and maximum detection confidence thresholds, respectively. Indicates the first Detection confidence of each target; The unmatched active trajectories in the first layer and the trajectories in a temporarily lost state are combined to form a set of recovered trajectories: ； in, This represents the set of recovery trajectories that participate in the second-level recovery matching of the current frame; This represents the set of unmatched active trajectories in the first layer; Indicates the first The historical trajectory in the first The complete trajectory state at the end of the frame; Indicates the first The life cycle status of a historical trajectory; This indicates a temporary loss of trajectory status, where the trajectory failed to match due to short-term occlusion or missed detection, but is still retained for subsequent recovery of association. For the The recovered trajectory searches for candidate targets only near its predicted location; let the trajectory prediction center be... The detection target center is Then the candidate associations satisfy the following: ； in, The scale-adaptive neighborhood radius determined in step S4; Among the candidate association pairs that satisfy the local neighborhood condition and the corresponding gating condition, a one-to-one matching is performed again based on the comprehensive association cost; the second layer adopts a relatively relaxed recovery matching threshold: ； That is, the first layer uses a stricter threshold. To ensure matching accuracy, a more lenient threshold is used in the second layer. Improve the ability to recover trajectories after short-term occlusion, missed detection, and dense interactions; successfully recovered temporarily lost trajectories retain their original unique trajectory numbers and are not re-established. (3) Handling of unmatched targets in the third layer: After the first and second layers of matching, for the detected targets that are still not matched, they are initialized as candidate trajectories and passed to step S9 for candidate confirmation. For historical trajectories that are still not matched, based on their original lifecycle status, number of consecutive unmatched frames and trajectory confidence, they are passed to step S9 for candidate deletion, temporary retention or termination judgment. Track update after successful matching: For historical trajectories that have been successfully matched in the first or second layer, the trajectory prediction state is corrected based on the position and scale observation results of the corresponding detected targets: ； in, Indicates the first The trajectory in the first The frame's motion state after correction; Indicates the first The predicted state of the trajectory in the current frame. Indicates the first The location and scale observations of the detection target Represents the observation matrix. This represents the state correction coefficient or Kalman gain; The apparent memory features of the trajectory and the probability distribution of historical grades are updated as follows: ； in, Indicates the first The trajectory in the first Apparent memory features after frame update; Indicates the first The trajectory in the first Historical grade probability distribution after frame update; This indicates the original apparent memory characteristics of the trajectory. This represents the apparent features of the currently matched and detected target. Indicates the coefficient for retaining apparent historical information; This represents the probability distribution of the original historical grade of the trajectory. This represents the probability distribution of the grade of the current detection target. Indicates the retention coefficient of historical grade information; The trajectory confidence is updated based on the current detection confidence and matching quality: ； in: Indicates the first The trajectory in the first Frame matching updated trajectory confidence; This represents the original confidence level of the trajectory. This indicates the detection confidence level of the current target. This represents the normalized overall cost of association, and its value ranges from 0 to 1. Indicates the trajectory confidence smoothing coefficient; After a successful match, the lifecycle state of the trajectory remains unchanged or is restored to an active state: ； in: Indicates the first The trajectory in the first The lifecycle state after frame update; This indicates an active state, meaning the trajectory has been effectively confirmed and is in a normal, continuous tracking state.

9. The method for continuous monitoring of termites in dikes by grade according to claim 8, characterized in that: S9 includes: 9.1 Candidate Confirmation: Unmatched detection targets are established as candidate trajectories. When a candidate trajectory meets the conditions of position continuity, detection confidence level and grade consistency within a preset number of consecutive frames, it is confirmed as an active trajectory and assigned a unique trajectory number. 9.2 Active Maintenance: Update the position, scale, motion state, apparent memory features, historical grade status, and trajectory confidence of continuously matched trajectories; 9.3 Temporary Loss Recovery: For unmatched trajectories caused by occlusion or missed detection, their prediction state is retained within the maximum number of temporary loss frames, allowing them to be re-associated with the target detected in subsequent frames. After successful re-association, the original unique trajectory number is retained. 9.4 Termination of Deletion: When the number of consecutive unmatched frames of a temporarily lost trajectory exceeds the maximum temporary loss threshold, or the recovery confidence is lower than the termination threshold, it is put into the termination state; 9.5, Grade-based counting correction: Individual counting is performed based on the unique trajectory number of the confirmed trajectory. The original trajectory number is restored for termite targets that reappear after short-term occlusion or missed detection. Historical grade probability smoothing is performed on trajectories with short-term fluctuations in grade category, and invalid trajectories are suppressed.

10. A continuous monitoring system for termites in dikes, characterized in that: The system includes: The video acquisition module is used to acquire video images of the termite activity area on the dam. The termite classification detection module is used to detect termite targets in the current frame image and outputs the detection bounding box, detection confidence, termite classification, and target appearance features. The trajectory prediction module is used to predict the position and scale of the historical trajectory in the current frame based on the state of the historical trajectory, and obtain the predicted state of the historical trajectory. The local group relationship graph construction module is used to uniformly represent the current frame detected target and the historical trajectory prediction state as graph nodes, and construct a scale-adaptive local group relationship graph; The adaptive multimodal matching module is used to calculate the dynamic fusion weights of each matching modality based on the current frame's detected target density, detected target occlusion degree, detection confidence, and trajectory lifecycle state, generate an adaptive multimodal association cost matrix, and complete the data association between the detected target and historical trajectories. The trajectory lifecycle management module is used to perform candidate confirmation, active maintenance, temporary loss recovery, and termination deletion of trajectories; The grade-based counting module is used to perform grade-based counting correction based on the unique trajectory number of the confirmed trajectory and the historical grade status. The monitoring output module is used to output the continuous monitoring results of termite classification in the dike.

Images