Crocodile polyculture behavior abnormity early warning identification system

Abstract

The invention discloses a crocodile polyculture behavior abnormity early warning identification system, and relates to the field of behavior identification. The system comprises an acquisition module, a feature extraction module, an optimization module and an abnormity early warning module. The acquisition module is used for acquiring and preprocessing crocodile behavior data in a polyculture scene; the feature extraction module is used for extracting visual features, language cue word features and auxiliary features of the preprocessed crocodile behavior data, and fusing the three features to generate a multi-modal feature matrix; the optimization module is used for sequentially performing decomposition and pseudo label generation processing on the multi-modal feature matrix to obtain a pseudo label weight matrix, and performing bidirectional enhancement processing to obtain an enhanced multi-modal feature matrix; and the abnormity early warning module is used for calculating and matching abnormal behavior characteristics through mixed similarity based on the enhanced multi-modal characteristic matrix, the pseudo label weight matrix and the attribute label characteristic matrix, identifying abnormal categories and generating graded early warning.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of behavior recognition, in particular to a crocodile mixed culture behavior abnormality early warning recognition system. BACKGROUND

[0002] The crocodile mixed culture behavior abnormality early warning recognition system is an intelligent system for monitoring individual and group behavior patterns in crocodile culture environment and realizing early warning through intelligent analysis and identification of abnormal states such as disease symptoms, stress response, aggressive behavior, etc. Its core value lies in protecting crocodile welfare, reducing breeding risks and improving production efficiency. Crocodiles are high-value breeding species, and their behavior abnormalities are often early signals of health problems or environmental deterioration. Traditional manual observation methods have the defects of strong subjectivity, delayed response and difficulty in all-weather coverage, while timely and accurate abnormality early warning can effectively reduce mortality and avoid group disease outbreaks, which is crucial for large-scale breeding.

[0003] In the prior art, computer vision-based behavior detection methods have been applied in the field of aquaculture. A mixed culture fish group behavior detection method, device and system based on YOLOv3-Lite disclosed in Chinese patent CN112634202B obtains fish group images under different environmental conditions through an underwater image acquisition device, and after image enhancement, noise reduction and augmentation, etc. Preprocessing, an improved YOLOv3-Lite network model is used for training and detection, realizing the rapid identification of fish group behavior.

[0004] This scheme improves the detection efficiency through lightweight network design, and improves the small target detection effect through multi-scale feature fusion, and has made significant progress in fish group behavior monitoring. However, this technology still has obvious limitations. Its detection dimension is single, mainly relying on visual image data, without integrating environmental parameters, physiological indicators and population structure information and other multi-source heterogeneous data, which is difficult to fully understand the complex behavior patterns of living organisms. Its model lacks robustness to complex scenes such as blur and occlusion, and lacks a fine processing mechanism for uncertain samples, resulting in a high false detection rate in dynamic mixed culture environment. These defects make it difficult for the existing technology to be directly applied to crocodile mixed culture scenarios, and there is an urgent need for a new solution that integrates multi-source data, has feature optimization capability and supports hierarchical early warning. SUMMARY

[0005] The technical problem solved by the present application is that the existing technology has a single detection dimension, mainly relies on visual image data, does not integrate environmental parameters, physiological indicators and population structure information and other multi-source heterogeneous data, and is difficult to fully understand the complex behavior patterns of living organisms. Its model lacks robustness to complex scenes such as blur and occlusion, and lacks a fine processing mechanism for uncertain samples, resulting in a high false detection rate in dynamic mixed culture environment.

[0006] To solve the above technical problems, the application provides the following technical solutions: a crocodile mixed breeding behavior anomaly early warning identification system, comprising a collection module, a feature extraction module, an optimization module, and an anomaly early warning module; The collection module is used for collecting crocodile behavior data in a mixed breeding scene and performing preprocessing; The feature extraction module is used for extracting visual features, language prompt word features, and auxiliary features of the preprocessed crocodile behavior data, and fusing the visual features, language prompt word features, and auxiliary features to generate a multi-modal feature matrix; The optimization module is used for sequentially performing decomposition and pseudo-label generation processing on the multi-modal feature matrix, obtaining a pseudo-label weight matrix, and obtaining an enhanced multi-modal feature matrix through bidirectional enhancement processing based on the pseudo-label weight matrix; The anomaly early warning module matches abnormal behavior features and generates graded early warnings through hybrid similarity calculation based on the enhanced multi-modal feature matrix, the pseudo-label weight matrix, and an attribute label feature matrix.

[0007] Preferably, the data types of the crocodile behavior data include video data, environmental data, physiological data, and population data; The preprocessing includes video data preprocessing and non-video data preprocessing; The video data preprocessing includes frame extraction, noise reduction, enhancement, and format unification on original videos; The non-video data preprocessing includes normalization of environmental data, encoding of physiological data into numerical features, conversion of population data into structured vectors, and synchronous generation of a crocodile attribute label set.

[0008] Preferably, the feature extraction module is used for extracting visual features, language prompt word features, and auxiliary features; Extracting the visual features includes using an encoder to extract a visual feature matrix of frame images; Extracting the language prompt word features includes constructing a three-level prompt word library and converting it into a language feature matrix; The preprocessed environmental, physiological, and population features are respectively encoded into auxiliary feature matrices; The auxiliary feature matrices, the visual feature matrix, and the language feature matrix are fused through weighted summation to generate a multi-modal fusion feature matrix.

[0009] Preferably, the decomposition of the multi-modal feature matrix includes: The multi-modal feature matrix and the language prompt word feature matrix are constructed into a joint matrix, and collective matrix decomposition is performed on the joint matrix, and the objective function mathematical expression of the decomposition is: ; ; wherein the objective function is to minimize the weighted reconstruction error, is 1, 2, is an omics weight, is a weight regulation parameter, is a transposed matrix of the interpolated multi-modal feature matrix, is a transposed matrix of the language prompt word feature matrix, is a latent factor matrix, is a unified coefficient matrix, is a Frobenius norm; The objective function is solved by the Lagrange multiplier method, and the latent factor matrix and the unified coefficient matrix are obtained by iterative updating until the objective function converges.

[0010] Preferably, generating the pseudo label specifically includes: The first n columns of the unified coefficient matrix are intercepted to obtain a unified latent representation matrix, and the last m columns are intercepted as a language latent representation matrix; Each column of the unified latent representation matrix and the language latent representation matrix is normalized to obtain a normalized unified latent representation matrix and a normalized language latent representation matrix, a similarity matrix is calculated through matrix multiplication, for each row of the similarity matrix, the maximum value of the row and the column number where the maximum value is located are found, and a determination and coding are performed according to a preset similarity threshold; If all the maximum values of the similarities are less than a first preset similarity threshold, it is determined to be normal, and an initial pseudo label normal is assigned; If all the maximum values of the similarities are greater than a second preset similarity threshold, it is determined to be abnormal, and the category corresponding to the language prompt word with the highest similarity is taken as an initial pseudo label; If all the maximum values of the similarities are greater than or equal to the first preset similarity threshold and less than or equal to the second preset similarity threshold, it is determined to be uncertain; The initial pseudo labels determined to be normal and abnormal are numerically coded and converted into an initial pseudo label vector.

[0011] Preferably, generating the pseudo label specifically includes: An initial pseudo label vector is added to the decomposition objective function of the unified coefficient matrix as a constraint term to construct a new optimization objective function; the mathematical expression of the optimization objective function is: ; wherein, is a minimum value of the optimization objective function, is a pseudo label constraint weight, is a pseudo label of the i th data point, is a linear projection matrix, is the i th column of the unified latent representation matrix, is a pseudo label constraint term; The objective function is solved by alternating least squares method, and the process is iterated cyclically until the preset number of iterations is reached, resulting in the optimized latent factor matrix, the optimized unified latent representation matrix, the optimized linear projection matrix, and the optimized pseudo-label. Based on the optimized unified latent representation matrix, its similarity with the language prompt word feature matrix is ​​recalculated, and the similarity is normalized. The normalized result is used as the confidence level to form a pseudo-label weight matrix.

[0012] Preferably, generating the attribute label feature matrix specifically includes: Encode the crocodile attribute tag set into a 24-dimensional basic attribute feature matrix; The coding rules include: the breed uses a 4-dimensional monothermic coding, the age uses an 8-dimensional segmented coding, the sex uses a 2-dimensional monothermic coding, and the kinship, weight class, skin damage class, reproductive status and environmental adaptability use the corresponding dimension normalization or segmented coding respectively. The basic attribute feature matrix and the pseudo-label weight matrix are fused to generate the final attribute label feature matrix, which specifically includes: The 24-dimensional basic attribute feature vectors are concatenated with the corresponding row vectors in the pseudo-label weight matrix to form high-dimensional feature vectors. The high-dimensional feature vectors are then sorted according to the time series to form the final attribute label feature matrix.

[0013] Preferably, the bidirectional enhancement process specifically includes: Columns identified as uncertain features are selected from the unified latent representation matrix. Based on the confidence level in the pseudo-label weight matrix, the columns are divided into two groups. The confidence levels are sorted in descending order. The first K items are denoted as the set to be enhanced in the abnormal direction, and the rest are denoted as the set to be enhanced in the normal direction. By analyzing the similarity matrix, the constraint region is determined, the contribution of each feature dimension to the similarity calculation result is calculated, and the feature dimensions with the highest contribution and the highest preset proportion are selected as the constraint region with the greatest impact on the classification result. Gaussian noise is injected into the constrained region for enhancement. For features to be enhanced in the abnormal direction, the enhancement intensity is proportional to the difference between 1 and the confidence level. For features to be enhanced in the normal direction, the enhancement intensity is proportional to the confidence level. In the bidirectional enhancement process, a graph Laplacian constraint is introduced. After reaching the preset maximum number of iterations, the enhanced multimodal feature matrix is ​​obtained.

[0014] Preferably, the hybrid similarity calculation includes first-level similarity and second-level similarity; The calculation process of the first-level similarity is to calculate the cosine similarity between the enhanced multimodal feature matrix and the language prompt word feature matrix, and to use the pseudo-label weight matrix to weight the feature dimensions corresponding to high confidence. The calculation process of the second-level similarity is as follows: based on the pseudo-label weight matrix and the attribute label feature matrix, a weighted average attribute feature vector is calculated for each anomaly category as the attribute prototype of the corresponding category, wherein the weight of the weighted average is the confidence of all data points belonging to the corresponding category. Calculate the cosine similarity between the attribute label features of the current frame and the attribute prototypes of each category. The cosine similarity is the second-order similarity. The first-level similarity and the second-level similarity are weighted and summed according to preset weights to generate the final mixed similarity.

[0015] Preferably, generating tiered early warnings includes: For each frame of video data, its mixed similarity is compared with that of all abnormal behavior sub-cue words. The abnormal category corresponding to the sub-cue word with the highest similarity is determined as the abnormal category of the frame. Within a continuous time segment, the highest mixed similarity values ​​of all frames are summed to obtain a total anomaly score. This total score is compared with preset thresholds for various anomalies. If the threshold is exceeded, the occurrence of that type of anomaly event is confirmed. Based on the confirmed anomaly event type and severity, a graded warning is generated. When the total abnormal score is between the event-specific threshold and 1.2 times it, the system will trigger a low-risk yellow alert; When the total abnormal score is between 1.2 and 1.5 times the event-specific threshold, the system will trigger a medium-risk orange alert. When the total abnormal score is greater than 1.5 times the event’s specific threshold, the system will trigger a high-risk orange alert.

[0016] The beneficial effects of this invention are as follows: This invention achieves high-precision and robust identification and early warning of abnormal behavior in crocodiles in mixed-species environments. The optimization module effectively integrates heterogeneous information from visual, linguistic, and auxiliary features by performing collective matrix decomposition on the multimodal feature matrix, constructing a unified latent representation space. Furthermore, through an iteratively optimized pseudo-label generation mechanism, a pseudo-label weight matrix with confidence rating is generated, improving the representation ability of unknown samples. The optimization module introduces a bidirectional enhancement processing strategy to actively identify and target ambiguous samples judged as uncertain. By analyzing confidence levels, these samples are differentiated in terms of feature enhancement, moving towards either abnormal or normal behavior, thereby reducing misjudgments and missed judgments caused by feature ambiguity or sample sparsity, and enhancing the system's robustness in complex mixed-species environments. The anomaly early warning module utilizes the optimization module's... The output, through the construction of a multi-layered hybrid similarity calculation system, achieves comprehensive judgment of abnormal behavior. This module not only performs core matching based on enhanced multimodal features and language prompts, but also introduces pseudo-label weights for confidence weighting. This makes early warning decisions no longer rely solely on a single behavioral manifestation, but deeply integrates multi-dimensional information such as behavioral semantics, occurrence probability, and individual biological background. This multi-dimensional fusion decision-making mechanism enables the system to effectively distinguish different types of abnormal behavior and, combined with the cumulative analysis of time series, ultimately generate graded early warnings. This not only avoids false alarms caused by instantaneous interference, but also achieves differentiated responses to abnormal events of different severity, providing aquaculture managers with accurate, reliable, and operable decision support, thus demonstrating outstanding application value in ensuring crocodile health and reducing aquaculture risks. Attached Figure Description

[0017] Figure 1 This is a basic flowchart of a crocodile mixed-species behavior abnormality early warning and identification system provided in one embodiment of the present invention. Detailed Implementation

[0018] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0019] Example, refer to Figure 1 As an embodiment of the present invention, an abnormal behavior warning and identification system for crocodiles in mixed breeding is provided, including a data acquisition module, a feature extraction module, an optimization module and an abnormal warning module; The data acquisition module is used to collect and preprocess crocodile behavior data in mixed-species environments. The feature extraction module is used to extract visual features, language cue word features, and auxiliary features from the preprocessed crocodile behavior data, and to fuse the visual features, language cue word features, and auxiliary features to generate a multimodal feature matrix; The optimization module is used to decompose the multimodal feature matrix and generate pseudo-labels sequentially to obtain the pseudo-label weight matrix. Based on the pseudo-label weight matrix, the enhanced multimodal feature matrix is ​​obtained through bidirectional enhancement processing. The anomaly warning module, based on the enhanced multimodal feature matrix, pseudo-label weight matrix, and attribute label feature matrix, uses hybrid similarity calculation to match abnormal behavior features, identify anomaly categories, and generate tiered warnings.

[0020] This invention constructs a complete framework for an early warning and identification system for abnormal behavior in mixed-species crocodile farming. By integrating four modules—data acquisition, feature extraction, optimization, and early warning—it achieves fully automated processing from raw multi-source data to final tiered early warning. Through the collaborative work between modules, this invention forms a data-driven, intelligent closed-loop monitoring system that can comprehensively, in real-time, and intelligently perceive and analyze the complex behavior of crocodiles in mixed-species environments. This significantly improves the accuracy and timeliness of abnormal behavior identification, providing strong technical support for the refined management and risk control of crocodile farming.

[0021] The data types of crocodile behavior data include: video data, environmental data, physiological data, and population data; Preprocessing includes video data preprocessing and non-video data preprocessing; Video data preprocessing includes frame extraction, noise reduction, enhancement, and format unification of the original video. Non-video data preprocessing includes normalizing environmental data, encoding physiological data into numerical features, converting population data into structured vectors, and simultaneously generating a crocodile attribute label set.

[0022] In one specific embodiment of the present invention, the video data covers the entire area of ​​the polyculture pond and individual behavior. The environmental data includes water temperature, air temperature, water quality parameters and light intensity. The water quality parameters include salinity, pH value and dissolved oxygen. The physiological data includes individual weight, skin damage score, fecal sample metabolomics data and wound infection status. The skin damage score is divided into levels, with higher scores indicating more severe damage. The population data includes the number of polyculture individuals, sex ratio, age structure, kinship and reproductive records. The reproductive records include the number of eggs laid and fertilization rate. Video data preprocessing includes: extracting frame images from the original video at fixed time intervals of 5s to 10s to generate image frame sequences; using Gaussian filtering to eliminate noise caused by water reflection and lens dust; performing histogram equalization on low-light frames; using the segmentation algorithm Mask R-CNN to segment the foreground of frames with occlusion; scaling the frame images to a uniform size; and uniformly encoding them into RGB format. Non-video data preprocessing includes: Z-score standardization of environmental data, encoding physiological data into numerical features, encoding skin damage scores into numerical features, converting weight data into rate of change; converting sex ratio and age structure into structured vectors, encoding reproductive records into the proportion of unfertilized eggs and embryo mortality rate, converting population data into structured vectors, and simultaneously generating a crocodile attribute tag set, which includes species, age, sex, and kinship. Align all preprocessed data using timestamps.

[0023] This invention establishes a comprehensive and multi-dimensional data acquisition foundation by defining four data types: video, environment, physiology, and population. This provides a high-quality, standardized data source for subsequent feature extraction and model analysis, effectively reducing the interference of noise and redundant information in the original data.

[0024] The feature extraction module is used to extract visual features, language cue word features, and auxiliary features; Extracting visual features involves using an encoder to extract the visual feature matrix of the frame image; Extracting language cue word features includes constructing a three-level cue word library and converting it into a language feature matrix; The preprocessed environmental, physiological, and population characteristics are encoded into auxiliary feature matrices, respectively. The auxiliary feature matrix is ​​fused with the visual feature matrix and the language feature matrix by weighted summation to generate a multimodal fusion feature matrix.

[0025] In a specific embodiment of the present invention, a 768-dimensional visual feature vector is extracted from the preprocessed frame image using a CLIPViT-L / 14 encoder. The visual feature matrix obtained by concatenating the visual feature vectors corresponding to all frames has a dimension of [missing information]. n is the total number of frames; Extracting language cue word features involves constructing a cue word library for abnormal crocodile behavior. This library is built using a three-level structure: category, attribute, and sub-cue words. Based on the CLIP text encoder, each sub-cue word is converted into a 768-dimensional language feature vector. All cue word language feature vectors are then concatenated to obtain a language feature matrix with dimensions of [missing information]. m is the number of sub-prompt words; The three-level cue word library is a hierarchical, structured set of cue words designed for identifying abnormal behavior in mixed-species crocodile colonies. Its core is constructed in a three-level progression based on abnormality category, core attributes, and sub-cue words, as shown in Table 1. The aim is to accurately align linguistic features with visual and auxiliary features, thereby improving the accuracy of abnormality identification. Table 1. Level 3 Hint Word Database The preprocessed environmental, physiological, and population features are encoded into 256-dimensional auxiliary feature vectors. The auxiliary feature vectors from n frames are concatenated to obtain an auxiliary feature matrix with dimensions [missing information]. n is the total number of frames; Since the dimensions of visual features, language features and auxiliary features are inconsistent, the dimensions of the auxiliary features need to be expanded first, and then mapped to 768 dimensions through a fully connected layer; The multimodal feature matrix is ​​generated using a weighted summation formula, specifically including: For each frame of image, the visual feature vector is calculated and the cosine similarity between the visual feature vector and the language feature vector corresponding to each sub-cue word is used for feature matching. If the cosine similarity is greater than 0.7, the matching is considered successful, confirming that the visual feature vector corresponds to the language feature vector. The visual feature vector, language feature vector and auxiliary feature vector corresponding to the frame are then fused according to the formula. The mathematical expression of the formula is: ; in, For multimodal feature vectors, It is a visual feature vector. It is the corresponding language feature vector. It is an auxiliary feature vector after dimensionality increase, with the same dimension as the visual feature vector and the language feature vector. The multimodal feature vectors are concatenated in the time order of the frames to obtain the multimodal feature matrix.

[0026] This invention provides a multimodal feature fusion strategy. By extracting visual, linguistic cue words, and auxiliary features separately and then weighting and fusing them, it comprehensively utilizes intuitive image information, expert prior knowledge of linguistic cue words, and contextual information such as environmental and physiological information. This fusion method breaks through the limitations of a single data source, making feature representation richer and more comprehensive, and laying a solid foundation for accurately identifying complex and abnormal behaviors.

[0027] Decomposition of the multimodal feature matrix includes: The multimodal feature matrix and the language prompt word feature matrix are constructed into a joint matrix. Collective matrix decomposition is then performed on the joint matrix, and the objective function of the decomposition is expressed as follows: ; ; The objective function is to minimize the weighted reconstruction error. The values ​​are 1 and 2. For omics weights, and The sum of is 1. This is the weight adjustment parameter, with a value of 1.2. This is the transpose of the interpolated multimodal eigenvalue matrix. This is the transpose of the feature matrix of language prompt words. For the latent factor matrix, To unify the coefficient matrix, It is the Frobenius norm; The objective function is solved using the Lagrange multiplier method, and iterative updates are performed until the objective function converges, thereby obtaining the latent factor matrix and the unified coefficient matrix.

[0028] In one specific embodiment of the present invention, the convergence condition of the objective function is until the number of iterations reaches 100; when When the value is 1, the corresponding This is the transpose of the interpolated multimodal eigenvalue matrix. The dimension is 768 represents the multimodal fusion feature dimension, and n represents the number of frames. For dimension A dense matrix with full column rank; when When the value is 2, the corresponding This is the transpose of the feature matrix of language prompt words. The dimension is 768 represents the language feature dimension, and m represents the number of sub-prompt words. For dimension A dense matrix with full column rank; The unified coefficient matrix represents the coordinates of each frame or each cue word in the latent space. Each element V(k,i) represents the coordinate value of the i-th data point on the k-th latent factor, with dimension 1. ; Extracting the first n columns of the corresponding multimodal feature matrix from the unified coefficient matrix (i.e., extracting the first n columns of the corresponding n frames from the V matrix) yields the unified latent representation matrix. Each element represents the coordinates of the i-th frame on the k-th latent factor, representing the low-dimensional features of the video frame in the shared latent space, used for subsequent anomaly detection. The dimension is... .

[0029] This invention employs collective matrix factorization (CMF) technology to reduce the dimensionality of multimodal features and mine latent factors. By constructing a joint matrix and setting a weighted reconstruction objective function, it effectively maps visual features and language cue word features to a shared latent space, achieving efficient data compression and denoising. This enables the system to analyze the relationship between behavior and language description at a lower dimensional level, providing a more stable and discriminative feature foundation for subsequent pseudo-label generation and anomaly matching.

[0030] Generating pseudo-tags specifically includes: Extracting the first n columns of the unified coefficient matrix yields the unified latent representation matrix, and the remaining m columns form the language latent representation matrix. Normalize each column of the unified latent representation matrix and the language latent representation matrix to obtain the normalized unified latent representation matrix and the language latent representation matrix. Calculate the similarity matrix by matrix multiplication. For each row of the similarity matrix, find the maximum value of the row and the column number it belongs to. Make judgments and encode according to the preset similarity threshold. If the maximum value of all similarities is less than the first preset similarity threshold, it is judged as normal and the initial pseudo-label is assigned as normal. If the maximum value of all similarities is greater than the second preset similarity threshold, it is judged as an anomaly, and the category corresponding to the language prompt word with the highest similarity is used as the initial pseudo-label; If the maximum value of all similarities is greater than or equal to the first preset similarity threshold and less than or equal to the second preset similarity threshold, it is determined to be uncertain; The initial pseudo-labels that are determined to be normal or abnormal are numerically encoded and converted into initial pseudo-label vectors.

[0031] This invention utilizes the similarity between the unified latent representation obtained from matrix factorization and the latent representation of language prompt words to achieve automatic pre-classification without manual annotation. By setting three judgment states—normal, abnormal, and uncertain—it demonstrates high practicality and robustness. This invention filters out fuzzy samples, avoiding the negative impact of early incorrect labels on subsequent optimization. This judgment and encoding method based on similarity thresholds builds an initial knowledge structure for the system, providing a clear starting point and goal for subsequent iterative optimization and bidirectional enhancement.

[0032] In one specific embodiment of the present invention, the first n columns are extracted from the unified coefficient matrix as the unified latent representation matrix, and the last m columns are extracted as the language latent representation matrix. L2 normalization is performed on each column vector of both matrices. The unified latent representation matrix is ​​transposed so that each row is a feature vector of a video frame. Then, the similarity between the transposed unified latent representation matrix and the language latent representation matrix is ​​calculated by matrix multiplication to obtain a similarity matrix with a dimension of n×m. Traverse each row of the similarity matrix to find its maximum similarity value and the corresponding language prompt word; If the maximum similarity is less than 0.3, it is determined to be normal, and its concept category is normally encoded as a 6-dimensional numerical vector [0,0,0,0,0,1].

[0033] If the maximum similarity is higher than 0.7, it is determined to be an anomaly, and its concept category is encoded as the corresponding 6-dimensional one-hot vector [1,0,0,0,0,0].

[0034] If the maximum similarity is between 0.3 and 0.7, it is determined to be uncertain. These samples will be ignored in subsequent steps. After this step, all samples that are determined to be normal or abnormal have obtained a corresponding initial pseudo-label vector.

[0035] Generating pseudo-tags specifically includes: A new optimization objective function is constructed by adding an initial pseudo-label vector as a constraint term to the objective function of the unified coefficient matrix decomposition; the mathematical expression of the optimization objective function is: ; in, To optimize the minimum value of the objective function, The pseudo-label constraint weight has a value of 0.05. This is the pseudo-label for the i-th data point. It is a linear projection matrix. To unify the i-th column of the latent representation matrix, These are pseudo-label constraint terms; The objective function is solved by alternating least squares method, and the process is iterated cyclically until the preset number of iterations is reached, resulting in the optimized latent factor matrix, the optimized unified latent representation matrix, the optimized linear projection matrix, and the optimized pseudo-label. Based on the optimized unified latent representation matrix, its similarity with the language prompt word feature matrix is ​​recalculated, and the similarity is normalized. The normalized result is used as the confidence level to form a pseudo-label weight matrix.

[0036] In a specific embodiment of the present invention, the initial pseudo-label vector obtained in the previous step is substituted into a new optimization objective function. This function consists of two parts: a reconstruction term of collective matrix factorization and a pseudo-label constraint term. The pseudo-label constraint term is constructed by calculating the difference between the prediction result of a linear projection matrix on the latent representation and the initial pseudo-label vector. The weight of the pseudo-label constraint term is determined to be 0.05 through grid search. The new optimization objective function is solved by alternating least squares method and iterated 100 times to obtain an optimized and more discriminative unified latent representation matrix.

[0037] Based on the optimized unified latent representation matrix, the similarity calculation process is performed again to obtain a new similarity matrix. Softmax normalization is performed on each row of the new similarity matrix so that the sum of the values ​​in each row is 1. The result is the final pseudo-label weight matrix. Each value in this matrix represents the confidence level of the corresponding video frame belonging to a specific anomaly category.

[0038] This invention introduces the initially generated pseudo-labels as soft constraints into the optimization objective function of matrix factorization. In this process, the system actively adjusts the latent space to make the feature representations as consistent as possible with the existing pseudo-labels. This feedback mechanism enables the latent representations to accurately distinguish different behavior categories. The optimized pseudo-label weight matrix quantifies the reliability of each classification judgment and significantly enhances the system's ability to handle fuzzy and boundary samples.

[0039] The generation of the attribute label feature matrix specifically includes: Encode the crocodile attribute tag set into a 24-dimensional basic attribute feature matrix; The coding rules include: the breed uses a 4-dimensional monothermic coding, the age uses an 8-dimensional segmented coding, the sex uses a 2-dimensional monothermic coding, and the kinship, weight class, skin damage class, reproductive status and environmental adaptability use the corresponding dimension normalization or segmented coding respectively. The basic attribute feature matrix and the pseudo-label weight matrix are fused to generate the final attribute label feature matrix, which specifically includes: The 24-dimensional basic attribute feature vectors are concatenated with the corresponding row vectors in the pseudo-label weight matrix to form high-dimensional feature vectors. The high-dimensional feature vectors are then sorted according to the time series to form the final attribute label feature matrix.

[0040] In a specific embodiment of the present invention, the crocodile attribute tag set synchronously generated by the acquisition module is encoded into a basic attribute feature matrix. The crocodile attribute tag set includes breed, age, sex, and kinship. The encoding rules include: breed is encoded using 4-dimensional one-hot encoding, age is encoded using 8-dimensional segmented encoding, sex is encoded using 2-dimensional one-hot encoding, and kinship is encoded using 10-dimensional normalized coefficients, generating a 24-dimensional basic attribute feature vector corresponding to the main observation object in each frame of data. Dynamic anomaly tendency features are extracted from the pseudo-label weight matrix obtained from the optimization module. Each row of the pseudo-label weight matrix represents an anomaly category, each column represents a frame of data, and the matrix elements represent the confidence that the i-th frame belongs to the c-th anomaly. This matrix reflects the system's real-time assessment of the probability of various abnormal behaviors of the crocodile group within the current observation period. The basic attribute feature matrix and the pseudo-label weight matrix are fused to generate the final attribute label feature matrix. Specifically, for the i-th frame of data, its corresponding 24-dimensional basic attribute feature vector is concatenated with the i-th column of the pseudo-label weight matrix, which contains a vector containing the confidence scores of all anomaly categories, to form a high-dimensional feature vector that integrates static individual attributes and dynamic anomaly tendencies. The fused feature vectors of all frames are arranged in chronological order to form the final attribute label feature matrix. The dimension of this matrix is ​​(24+C)×n, where C is the total number of anomaly categories and n is the total number of frames. The encoding rules are as follows: The variety is 4-dimensional unique thermal coding; Age is encoded in 8 dimensions, ranging from childhood [1,0,...0] to old age [0,...0,1]. Sex is encoded by a 2D one-hot code: male [1,0], female [0,1]. Kinship is represented by a 10-dimensional normalized coefficient; Weight class is coded in 5 dimensions based on weight change rate. Skin damage severity is determined by 3-dimensional unique thermal coding; Reproductive status is encoded in 6 dimensions; Environmental adaptability is characterized by 26 normalized dimensions; The matrix elements are the k-dimensional attribute feature values ​​of the crocodile corresponding to the i-th frame.

[0041] This invention deeply integrates behavioral confidence with static inherent attributes of crocodiles to construct an attribute label feature matrix. Traditional behavioral analysis often ignores individual differences, while this solution concatenates 24 basic attributes such as species, age, gender, and kinship with a pseudo-label weight matrix, making anomaly identification no longer an isolated pattern matching, but a comprehensive judgment based on individual background. This reduces false alarms and false negatives caused by individual differences, making the early warning results more accurate and reliable.

[0042] The bidirectional enhancement process specifically includes: Columns identified as uncertain features are selected from the unified latent representation matrix. Based on the confidence level in the pseudo-label weight matrix, the columns are divided into two groups. The confidence levels are sorted in descending order. The first K columns are denoted as the set to be enhanced in the abnormal direction, and the remaining columns are denoted as the set to be enhanced in the normal direction. By analyzing the similarity matrix, the constraint region is determined, the contribution of each feature dimension to the similarity calculation result is calculated, and the feature dimensions with the highest contribution and the highest preset proportion are selected as the constraint region with the greatest impact on the classification result. Gaussian noise is injected into the constrained region for enhancement. For features to be enhanced in the abnormal direction, the enhancement intensity is proportional to the difference between 1 and the confidence level. For features to be enhanced in the normal direction, the enhancement intensity is proportional to the confidence level. In the bidirectional enhancement process, a graph Laplacian constraint is introduced. After reaching the preset maximum number of iterations, the enhanced multimodal feature matrix is ​​obtained.

[0043] In one specific embodiment of the present invention, columns corresponding to data judged as uncertain are selected from the unified latent representation matrix. Based on the confidence levels of these uncertain data in the pseudo-label weight matrix, the data is divided into two groups. First, the confidence levels are sorted in descending order. The first K columns are denoted as the set to be enhanced in the abnormal direction, and the remaining columns are denoted as the set to be enhanced in the normal direction, where K is a natural number greater than 5. Using the generated similarity matrix (n×m) and the corresponding initial pseudo-labels, a set of data samples determined to be normal and a set of data samples determined to be abnormal are separated. For each feature dimension q in the unified latent representation matrix (200×n), q is from 1 to... For a natural number 200, calculate the average value of this dimension on the set of abnormal data and the average value on the set of normal data. The absolute value of the difference between the two average values ​​is taken as the contribution score of this dimension. The larger the absolute value of the contribution score, the more important the feature dimension is in distinguishing between normal and abnormal behavior. Sort the contribution scores of all 200 dimensions in descending order, select the top 30% of dimensions, i.e., the top 60 dimensions, as the constraint region, and generate a 200-dimensional mask vector. The positions corresponding to these 60 key dimensions are 1, and the remaining 140 positions are 0. This mask vector defines the feature range to be perturbed in subsequent enhancement operations.

[0044] Gaussian noise is injected into the constrained region for enhancement. For the set to be enhanced in the anomaly direction, enhancement is performed according to the anomaly direction enhancement objective function formula. The mathematical expression of the anomaly direction enhancement objective function formula is: ; in, Let i be the enhanced feature vector of the i-th sample. The original, unenhanced feature vector of the i-th sample comes from the unified latent representation V. It represents the confidence level that sample i belongs to the anomaly category c. It is a constrained region. For Gaussian noise, a 768-dimensional vector randomly sampled from a Gaussian distribution with mean 0 and variance σ²; For the set to be augmented in the normal direction, augmentation is performed according to the objective function formula for normal direction augmentation. The mathematical expression of the objective function formula for normal direction augmentation is: ; In the bidirectional enhancement process, a graph Laplacian constraint is introduced to ensure that after the features are modified, the originally similar features still maintain their proximity relationship in space. After reaching the maximum number of iterations, the enhanced latent representation is mapped back to 768 dimensions through the generation of enhanced connection layers to obtain the enhanced multimodal feature matrix. The graph Laplace constraint calculates the similarity between all samples. If two samples are very similar, i.e., the cosine similarity is >0.6, an edge is drawn between them to form a graph. The non-smoothness score is calculated through the constraint term. If two originally similar samples become dissimilar after enhancement, the score will increase. This penalty term is added to the overall optimization objective function, and the optimization objective function is to be minimized. The specific process of the Graph Laplace constraint includes: Construct an adjacency matrix A. If the cosine similarity between two samples is greater than 0.6, then A(i,j) = 1, considered as a neighborhood feature; otherwise, A(i,j) = 0. Construct a degree matrix, the mathematical expression of which is: ; in, For degree matrix, The neighborhood matrix; Calculate the graph Laplacian matrix by subtracting the adjacency matrix from the degree matrix, and add constraint terms to the enhancement objective function. Constraints This is a manifold regularization term. Its core function is to ensure that, after bidirectional feature enhancement, samples that were originally similar in the feature space remain similar after enhancement, preventing the enhancement process from destroying the intrinsic structure of the data and ensuring that similar features maintain their neighborhood relationships. To enhance the feature matrix, The graph Laplacian matrix is ​​the difference between the degree matrix and the neighborhood matrix. The trace operator calculates the sum of the diagonal elements of a square matrix. The constraint weight is set to 0.01. The maximum number of iterations is set to 20. After the iteration stops, the enhanced multimodal feature matrix is ​​generated. The element in row i and column k of the enhanced multimodal feature matrix is ​​the enhanced feature of the k-th dimension of frame i, which combines the results of interpolation, decomposition and enhancement. The enhanced multimodal feature matrix is ​​obtained by bidirectionally enhancing the unified latent representation matrix and then mapping it back to the original dimensions through a fully connected layer. This improves the model's ability to discriminate uncertain samples and enhances the robustness of the features. The dimension is [dimension value missing]. It is from Latent space mapping back The original feature space.

[0045] This invention groups uncertain samples by confidence level and sets opposite augmentation directions. This bidirectional augmentation operation makes the model more clearly distinguish between normal and abnormal samples. By analyzing the contribution of features, the constraint region is determined, and Gaussian noise with intensity related to confidence level is injected. This data augmentation method allows the system to focus on the features most important to the classification decision and learns the robustness boundary of features through controllable perturbation. By introducing graph Laplace constraints, the local neighborhood structure between data points is maintained during the augmentation process, avoiding the augmentation operation from destroying the inherent manifold of the data, and generating high-quality, high-information augmented samples.

[0046] Hybrid similarity calculation includes primary similarity and secondary similarity; The calculation process of first-level similarity is to calculate the cosine similarity between the enhanced multimodal feature matrix and the language prompt word feature matrix, and to use the pseudo-label weight matrix to weight the feature dimensions corresponding to high confidence. The calculation process of second-order similarity is as follows: Based on the pseudo-label weight matrix and the attribute label feature matrix, a weighted average attribute feature vector is calculated for each anomaly category as the attribute prototype of the corresponding category. The weight of the weighted average is the confidence of all data points belonging to the corresponding category. Calculate the cosine similarity between the attribute label features of the current frame and the attribute prototypes of each category. This similarity is the second-order similarity. The first-level similarity and the second-level similarity are weighted and summed according to preset weights to generate the final mixed similarity.

[0047] In a specific embodiment of the present invention, the first-level similarity is used to measure the degree of matching between the visual behavior and the linguistic semantic description of the current frame. The i-th row of the enhanced multimodal feature matrix, i.e., the 768-dimensional feature vector of the i-th frame, is taken, and the cosine similarity between the feature vector and the feature vector of each sub-prompt word in the linguistic feature matrix is ​​calculated to obtain an m-dimensional similarity vector. Since the three-level prompt word library in Table 1 has a hierarchical structure, each sub-prompt word belongs to a first-level anomaly category, with a total of c categories. The m-dimensional similarity vector is aggregated into a C-dimensional first-level similarity vector. The aggregation rule is: for each category c, the maximum value of the similarity of all sub-prompt words under it is taken. Second-order similarity is used to measure the degree of matching between the individual attributes of the crocodiles observed in the current frame and various high-incidence groups of anomalies. Based on the pseudo-label weight matrix and the attribute label feature matrix, an attribute prototype is calculated for each anomaly category c. The attribute prototype is a 24-dimensional vector representing the typical individual attributes of this type of anomaly. The calculation process is a weighted average, and the mathematical expression for the calculation process is as follows: ; in, For attribute prototypes, For the first Frame belongs to the Weights of class exceptions For the first The 24-dimensional basic attribute vector of a frame. Indicates all frames , will the Frame belongs to the Weight of class exceptions With the 24-dimensional fundamental attribute vector of this frame Perform element-wise multiplication, then sum the result vectors of all frames to obtain the weighted attribute sum vector. Indicates all frames Summation of the first Frame belongs to the Weight of class exceptions The total weights are obtained; Extract the 24-dimensional basic attribute vector of the current frame i and calculate... With each category attribute prototype The cosine similarity between them yields a C-dimensional second-order similarity vector. The basic attribute vector reflects the correlation between the background of the individual currently experiencing an anomaly and various anomalies. The first-level similarity and the second-level similarity are weighted and summed to obtain the final mixed similarity vector. The weight of the first-level similarity is 0.6 because it directly reflects the behavior itself and is the main basis for judgment. The weight of the second-level similarity is 0.4, which serves as important auxiliary verification information. Individual biological background is incorporated into the decision-making process, effectively reducing misjudgments caused by individual differences.

[0048] Generating tiered early warnings includes: For each frame of video data, its mixed similarity is compared with that of all abnormal behavior sub-cue words. The abnormal category corresponding to the sub-cue word with the highest similarity is determined as the abnormal category of the frame. Within a continuous time segment, the highest mixed similarity values ​​of all frames are summed to obtain a total anomaly score. This total score is then compared with preset thresholds for various anomalies. If the score exceeds the threshold, the occurrence of that type of anomaly is confirmed. Based on the confirmed anomaly type and severity, a graded warning is generated.

[0049] In a specific embodiment of the present invention, for each frame of video data i, its final C-dimensional mixed similarity vector is obtained, the maximum value in the vector and its corresponding category are found, which is determined to be the abnormal category of the i-th frame. A 30-second time window is set, and within the window, the highest mixed similarity values ​​of all frames are accumulated to obtain a total abnormal score. At the same time, the occurrence frequency of each predicted category within the window is counted, and the total score is assigned to the abnormal category with the most occurrences. The total abnormal score is compared with the preset specific thresholds for each type of abnormality, and the thresholds are set based on historical data and expert experience. Tiered early warning includes: When the total abnormal score is between the event-specific threshold and 1.2 times it, the system will trigger a low-risk yellow alert; When the total abnormal score is between 1.2 and 1.5 times the event-specific threshold, the system will trigger a medium-risk orange alert. When the total abnormal score is greater than 1.5 times the event’s specific threshold, the system will trigger a high-risk orange alert.

[0050] This invention achieves high-precision and robust identification and early warning of abnormal behavior in crocodiles in mixed-species environments. The optimization module effectively integrates heterogeneous information from visual, linguistic, and auxiliary features by performing collective matrix decomposition on the multimodal feature matrix, constructing a unified latent representation space. Furthermore, an iteratively optimized pseudo-label generation mechanism generates a pseudo-label weight matrix with confidence ratings, enhancing the representation ability of unknown samples. The optimization module introduces a bidirectional enhancement processing strategy, actively identifying and targeting ambiguous samples deemed uncertain. By analyzing confidence levels, these samples are differentiated in terms of feature enhancement, shifting towards either abnormal or normal behavior, thereby reducing misjudgments and missed judgments caused by feature ambiguity or sample sparsity, and enhancing the system's robustness in complex mixed-species environments. The anomaly early warning module utilizes the output of the optimization module... A multi-layered hybrid similarity calculation system was constructed to achieve comprehensive judgment of abnormal behavior. This module not only performs core matching based on enhanced multimodal features and language prompts, but also introduces pseudo-label weights for confidence weighting. This makes early warning decisions no longer rely solely on a single behavioral manifestation, but deeply integrates multi-dimensional information such as behavioral semantics, occurrence probability, and individual biological background. This multi-dimensional fusion decision-making mechanism enables the system to effectively distinguish different types of abnormal behavior and, combined with cumulative analysis of time series, ultimately generate graded early warnings. This not only avoids false alarms caused by instantaneous interference, but also enables differentiated responses to abnormal events of different severity. It provides aquaculture managers with accurate, reliable, and operable decision support, thus demonstrating outstanding application value in ensuring crocodile health and reducing aquaculture risks.

[0051] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

Claims

1. A system for early warning and identification of abnormal behavior in mixed-species crocodile farming, characterized in that: It includes a data acquisition module, a feature extraction module, an optimization module, and an anomaly warning module; The acquisition module is used to collect and preprocess crocodile behavior data in mixed-species scenarios. The feature extraction module is used to extract visual features, language cue word features, and auxiliary features from the preprocessed crocodile behavior data, and to fuse the visual features, language cue word features, and auxiliary features to generate a multimodal feature matrix; The optimization module is used to decompose and generate pseudo-labels sequentially on the multimodal feature matrix to obtain a pseudo-label weight matrix, and to obtain an enhanced multimodal feature matrix based on the pseudo-label weight matrix through bidirectional enhancement processing. The anomaly warning module is based on the enhanced multimodal feature matrix, pseudo-label weight matrix and attribute label feature matrix. It uses hybrid similarity calculation to match abnormal behavior features and generate hierarchical warnings.

2. The crocodile mixed-species behavior abnormality early warning and identification system as described in claim 1, characterized in that, The data types of the crocodile behavior data include: video data, environmental data, physiological data, and population data; The preprocessing includes video data preprocessing and non-video data preprocessing; The video data preprocessing includes frame extraction, noise reduction, enhancement, and format unification of the original video. The non-video data preprocessing includes normalizing environmental data, encoding physiological data into numerical features, converting population data into structured vectors, and simultaneously generating a crocodile attribute tag set.

3. The crocodile mixed-species behavior abnormality early warning and identification system as described in claim 2, characterized in that, The feature extraction module is used to extract visual features, language cue word features, and auxiliary features; Extracting the visual features includes using an encoder to extract the visual feature matrix of the frame image; Extracting the language cue word features includes constructing a three-level cue word library and converting it into a language feature matrix; The preprocessed environmental, physiological, and population characteristics are encoded into auxiliary feature matrices, respectively. The auxiliary feature matrix is ​​fused with the visual feature matrix and the language feature matrix by weighted summation to generate a multimodal fusion feature matrix.

4. The crocodile mixed-species behavior abnormality early warning and identification system as described in claim 3, characterized in that, Decomposing the multimodal feature matrix includes: The multimodal feature matrix and the language prompt word feature matrix are constructed into a joint matrix. Collective matrix decomposition is then performed on the joint matrix, and the objective function of the decomposition is expressed as follows: ； ； The objective function is to minimize the weighted reconstruction error. The values ​​are 1 and 2. For omics weights, The weighting parameter is used when t equals 1. Let be the transpose of the interpolated multimodal eigenvalue matrix. When t equals 2, This is the transpose of the feature matrix of language prompt words. For the latent factor matrix, To unify the coefficient matrix, It is the Frobenius norm; The objective function is solved using the Lagrange multiplier method, and iterative updates are performed until the objective function converges, thereby obtaining the latent factor matrix and the unified coefficient matrix.

5. The crocodile mixed-species behavior abnormality early warning and identification system as described in claim 4, characterized in that, Generating pseudo-tags specifically includes: Extracting the first n columns of the unified coefficient matrix yields the unified latent representation matrix, and the remaining m columns form the language latent representation matrix. Normalize each column of the unified latent representation matrix and the language latent representation matrix to obtain the normalized unified latent representation matrix and the language latent representation matrix. Calculate the similarity matrix by matrix multiplication. For each row of the similarity matrix, find the maximum value of the row and the column number it belongs to. Make judgments and encode according to the preset similarity threshold. If the maximum value of all similarities is less than the first preset similarity threshold, it is judged as normal and the initial pseudo-label is assigned as normal. If the maximum value of all similarities is greater than the second preset similarity threshold, it is judged as an anomaly, and the category corresponding to the language prompt word with the highest similarity is used as the initial pseudo-label; If the maximum value of all similarities is greater than or equal to the first preset similarity threshold and less than or equal to the second preset similarity threshold, it is determined to be uncertain; The initial pseudo-labels that are determined to be normal or abnormal are numerically encoded and converted into initial pseudo-label vectors.

6. The crocodile mixed-species behavior abnormality early warning and identification system as described in claim 5, characterized in that, Generating pseudo-tags specifically includes: A new optimization objective function is constructed by adding an initial pseudo-label vector as a constraint term to the objective function of the decomposition of the unified coefficient matrix; The mathematical expression for the objective function is: ； in, To optimize the minimum value of the objective function, Weights are constrained by pseudo-labels. The pseudo-label for the i-th data point. It is a linear projection matrix. To unify the i-th column of the latent representation matrix, These are pseudo-label constraint terms; The objective function is solved by alternating least squares method, and the process is iterated cyclically until the preset number of iterations is reached, resulting in the optimized latent factor matrix, the optimized unified latent representation matrix, the optimized linear projection matrix, and the optimized pseudo-label. Based on the optimized unified latent representation matrix, its similarity with the language prompt word feature matrix is ​​recalculated, and the similarity is normalized. The normalized result is used as the confidence level to form a pseudo-label weight matrix.

7. The crocodile mixed-species behavior abnormality early warning and identification system as described in claim 6, characterized in that, The generation of the attribute label feature matrix specifically includes: Encode the crocodile attribute tag set into a 24-dimensional basic attribute feature matrix; The coding rules include: the breed uses a 4-dimensional monothermic coding, the age uses an 8-dimensional segmented coding, the sex uses a 2-dimensional monothermic coding, and the kinship, weight class, skin damage class, reproductive status and environmental adaptability use the corresponding dimension normalization or segmented coding respectively. The basic attribute feature matrix and the pseudo-label weight matrix are fused to generate the final attribute label feature matrix, which specifically includes: The 24-dimensional basic attribute feature vectors are concatenated with the corresponding row vectors in the pseudo-label weight matrix to form high-dimensional feature vectors. The high-dimensional feature vectors are then sorted according to the time series to form the final attribute label feature matrix.

8. The crocodile mixed-species behavior abnormality early warning and identification system as described in claim 7, characterized in that, The bidirectional enhancement process specifically includes: Columns identified as uncertain features are selected from the unified latent representation matrix. Based on the confidence level in the pseudo-label weight matrix, the columns are divided into two groups. The confidence levels are sorted in descending order. The first K columns are denoted as the set to be enhanced in the abnormal direction, and the remaining columns are denoted as the set to be enhanced in the normal direction. Calculate the contribution of each feature dimension to the similarity calculation result, and select the feature dimensions with the highest contribution and a preset proportion as the constraint regions that have the greatest impact on the classification result; Gaussian noise is injected into the constrained region for enhancement. For features to be enhanced in the abnormal direction, the enhancement intensity is proportional to the difference between 1 and the confidence level. For features to be enhanced in the normal direction, the enhancement intensity is proportional to the confidence level. In the bidirectional enhancement process, a graph Laplacian constraint is introduced. After reaching the preset maximum number of iterations, the enhanced multimodal feature matrix is ​​obtained.

9. The crocodile mixed-species behavior abnormality early warning and identification system as described in claim 8, characterized in that, The hybrid similarity calculation includes primary similarity and secondary similarity; The calculation process of the first-level similarity is to calculate the cosine similarity between the enhanced multimodal feature matrix and the language prompt word feature matrix, and to use the pseudo-label weight matrix to weight the feature dimensions corresponding to high confidence. The calculation process of the second-level similarity is as follows: based on the pseudo-label weight matrix and the attribute label feature matrix, a weighted average attribute feature vector is calculated for each anomaly category as the attribute prototype of the corresponding category, wherein the weight of the weighted average is the confidence of all data points belonging to the corresponding category. Calculate the cosine similarity between the attribute label features of the current frame and the attribute prototypes of each category. The cosine similarity is the second-order similarity. The first-level similarity and the second-level similarity are weighted and summed according to preset weights to generate the final mixed similarity.

10. The crocodile mixed-species behavior abnormality early warning and identification system as described in claim 9, characterized in that, Generating tiered early warnings includes: For each frame of video data, its mixed similarity is compared with that of all abnormal behavior sub-cue words. The abnormal category corresponding to the sub-cue word with the highest similarity is determined as the abnormal category of the frame. Within a continuous time segment, the highest mixed similarity values ​​of all frames are summed to obtain a total anomaly score. This total score is compared with preset thresholds for various anomalies. If the threshold is exceeded, the occurrence of that type of anomaly event is confirmed. Based on the confirmed anomaly event type and severity, a graded warning is generated. When the total abnormal score is between the event-specific threshold and 1.2 times it, the system will trigger a low-risk yellow alert; When the total abnormal score is between 1.2 and 1.5 times the event-specific threshold, the system will trigger a medium-risk orange alert. When the total abnormal score is greater than 1.5 times the event’s specific threshold, the system will trigger a high-risk orange alert.

Images