Aquaculture disease prediction method based on multi-modal data fusion

Abstract

The invention discloses an aquaculture disease prediction method based on multi-modal data fusion, and relates to the technical field of aquaculture, and the method mainly comprises the steps: collecting aquaculture data containing structured data and unstructured text data; inputting the structured data into a TabTransform model, carrying out column embedding and modeling of a context relationship between features through a multi-layer Transform encoder, and outputting a structured feature vector; inputting the unstructured text data into a pre-trained BERT encoder, and extracting an output vector marked by the CLS as a text semantic feature; and splicing the structured feature vector and the text semantic feature into a fusion feature, inputting the fusion feature into a full connection layer, and predicting the incidence probability of each target disease through a Sigmoid function. The prediction accuracy is improved, and the problems of single prediction dimension and weak generalization ability in the prior art are effectively solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of aquaculture, and in particular to an aquaculture disease prediction method based on multi-modal data fusion. BACKGROUND

[0002] In the field of parasite disease early warning in fish farming, the existing technology mainly relies on three types of methods, but these methods all have significant limitations. First, the traditional manual inspection and microscopic diagnosis method requires the aquaculture personnel to observe the fish body or water sample through a microscope to identify the pathogen. This method not only has low detection efficiency - the detection of a single sample takes more than 30 minutes, and the inspection cycle is usually more than 48 hours, resulting in a serious lack of real-time early warning capability; at the same time, it is highly dependent on the experience of professional personnel. According to the statistics of the Agricultural and Rural Department Fish Technology Platform, the manual misdiagnosis rate can reach 25%-40%, which is difficult to meet the monitoring needs of the kiloton-scale aquaculture farm. Second, the method based on traditional machine learning models (such as support vector machine or random forest) can use structured water quality parameters (such as temperature, salinity, pH value) for modeling, but only supports numerical variable input, and cannot handle text unstructured information such as "reduced feeding" and "white water color" in the breeding log, resulting in an environmental feature coverage rate of less than 60%. In addition, such models are sensitive to water quality fluctuations, and the accuracy decreases by more than 35% when applied across aquaculture farms, with weak generalization ability. Finally, the special model designed for a single pathogen needs to build a prediction system for each parasite, which not only increases the operation cost by 300%, but also cannot provide concurrent early warning for multiple pathogens due to the fragmentation of models, for example, when the Cryptocaryon irritans and blood cone are simultaneously out of control, the model cannot provide a coordinated risk assessment. SUMMARY

[0003] To solve the problem of low detection efficiency of artificial disease judgment in aquaculture and the inability of machine learning models to combine unstructured information, the present application proposes an aquaculture disease prediction method based on multi-modal data fusion, which includes the following steps: S1: Collect aquaculture data containing structured data and unstructured text data, the structured data is various target water quality parameters, and the unstructured text data is breeding logs and environmental description texts; S2: Input the structured data into the TabTransformer model, perform column embedding and context relationship modeling between features through multiple layers of Transformer encoder, and output the structured feature vector; S3: Input the unstructured text data into the pre-trained BERT encoder, and extract the output vector of the CLS marker as the text semantic feature; S4: Concatenate the structured feature vector and the text semantic feature into a fusion feature and input it into a fully connected layer to predict the incidence probability of each target disease through the Sigmoid function.

[0004] The application fully captures the multi-dimensional correlation of pathogen inducers in a complex aquaculture environment by fusing structured water quality parameters and unstructured text data, combining TabTransformer and BERT dual-channel modeling architecture, and compared with traditional single-modal models, the average prediction accuracy is improved, effectively solving the problem of single prediction dimension and weak generalization ability of the prior art.

[0005] Further, in the S1 step, the structured data includes at least three of temperature, salinity, pH value, dissolved oxygen, nitrite concentration and silicate concentration, and the unstructured text data includes at least one of aquaculture logs, field descriptions and expert records.

[0006] Further, in the S1 step, the structured data is input in CSV or Excel format and includes continuous variables and discrete classification variables.

[0007] Further, in the S3 step, the extracted semantic vector of the CLS label is specifically: inputting the unstructured text data into a BERT encoder, and taking the output vector at the position corresponding to the CLS label as the text semantic feature.

[0008] Further, in the S4 step, the full connection layer is a multi-label output structure, and the output dimension is , respectively corresponding to the incidence probability of each type of target disease, wherein is a real number space, n is the number of samples, and m is the total number of target diseases.

[0009] Further, in the S4 step, the model for predicting the incidence probability is trained by a Sigmoid function as follows: wherein, is the total loss, i is the number of target diseases, is a binary cross-entropy loss function, used to measure the difference between the predicted value of the model prediction probability value of the i-th target disease and the corresponding real label .

[0010] Further, after the S4 step, it further includes the step of: S5: dividing the risk level based on the positional relationship between the incidence probability of the target disease and the preset threshold interval; when the incidence probability of the target disease is below the preset threshold interval, it is classified as low risk; when the incidence probability of the target disease is within the preset threshold interval, it is classified as medium risk; When the probability of the target disease is above the preset threshold range, it is classified as high risk.

[0011] Furthermore, the method further comprises the steps of: S6: Display the prediction results and the variable influencing factors that affect the current prediction results through a visual interactive interface.

[0012] Compared with the prior art, the present invention has at least the following beneficial effects: (1) The present invention proposes a method for predicting aquaculture diseases based on multimodal data fusion. By integrating structured water quality parameters with unstructured text data and combining the TabTransformer and BERT dual-channel modeling architecture, it fully captures the multidimensional correlation of pathogen inducements in complex aquaculture environments. Compared with traditional single-modal models, the average prediction accuracy is improved, effectively solving the problems of single prediction dimension and weak generalization ability of existing technologies. (2) A multi-label Sigmoid output layer design is adopted, and a single model simultaneously outputs the incidence probability of multiple types of parasites. Compared with the traditional single pathogen independent modeling scheme, resource consumption is reduced and compatibility issues between models are avoided. It meets the real-time needs of the breeding site for concurrent early warning of multiple pathogens and significantly improves the practicality and deployment efficiency of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is a step diagram of a global measurement and control task scheduling method based on resource conflict optimization. DETAILED DESCRIPTION

[0014] The following are specific embodiments of the present invention and the accompanying drawings to further describe the technical solutions of the present invention, but the present invention is not limited to these embodiments.

[0015] In order to enable those skilled in the art to more clearly understand the technical solutions and advantages of the present invention, the implementation process of the present invention is described in detail below in combination with specific breeding scenarios. This embodiment takes the intelligent early warning of three types of parasites, Cryptocaryon irritans, Trypanosoma sanguineus and Benedenia (i.e., the total number of target disease categories m=3) in a certain marine fish farm as the application target, and demonstrates how the present invention can achieve end-to-end prediction through the collaboration of multimodal data fusion, deep learning modeling and interactive platform. It should be noted that this embodiment is only an exemplary description and does not constitute a limitation on the scope of protection of the present invention. Figure 1 As shown, the present invention proposes a method for predicting aquaculture diseases based on multimodal data fusion, comprising the steps of:

[0016] S1: Collect aquaculture data including structured data and unstructured text data. The structured data are various target water quality parameters, and the unstructured text data are aquaculture logs and environmental description texts. S2: input structured data into the TabTransformer model, perform column embedding and context relationship modeling between features through multi-layer Transformer encoder, and output structured feature vector; S3: input unstructured text data into the pre-trained BERT encoder, and extract the output vector of the CLS mark as the text semantic feature; S4: concatenate the structured feature vector and the text semantic feature into a fusion feature and input it into a fully connected layer to predict the incidence probability of each target disease through the Sigmoid function; S5: divide the risk level based on the position relationship between the incidence probability of the target disease and the preset threshold interval; S6: display the prediction results and the variable influence factors affecting the current prediction results through a visual interactive interface.

[0017] The present application obtains multi-source heterogeneous information in the aquaculture environment through an integrated data acquisition process. In terms of structured data, the system extracts target water quality parameters from the aquaculture sensor network, water quality monitoring equipment and manual records, covering continuous variables such as temperature (unit: ℃), salinity (unit: ‰), pH value, dissolved oxygen (unit: mg / L), nitrite concentration and silicate concentration, as well as discrete classification variables such as aquaculture unit number and feeding cycle. After such data is input in CSV or Excel table format, standardization preprocessing needs to be performed: continuous variables adopt Z-score normalization to eliminate dimension difference (the calculation formula is ), wherein is the sample mean, is the standard deviation, is the normalized value, is the variable), and classification variables are converted into interpretable numerical vectors through one-hot encoding or embedding layer to retain the potential association between discrete features.

[0018] For unstructured text data, the system synchronously collects aquaculture logs, field observation records and technician description texts (such as “fish feeding amount suddenly decreased”, “pool water turbidity increased” and the like), and imports them in TXT or CSV format. The text processing process includes two stages of semantic cleaning and vectorization: firstly, special characters, stop words and irrelevant symbols are removed to retain the core description phrases; then the semantic representation ability of the pre-trained BERT model is used to map the cleaned text sequence into a fixed-dimensional semantic feature vector, specifically by extracting the 768-dimensional vector corresponding to the [CLS] mark in the BERT output layer as the global semantic representation of the whole text, providing high-dimensional input for subsequent multi-modal fusion.

[0019] To ensure the spatiotemporal consistency of multi-source data, the system establishes a dual indexing mechanism of timestamp and breeding unit ID to align structured data and text data sources, removes invalid samples with a missing rate exceeding 15%, and detects outliers based on the box plot rule for interpolation and repair, ultimately generating a standardized spatiotemporal paired data set for model training and inference.

[0020] In this area, the present invention overcomes the limitation of traditional models that only support numerical input by distinguishing between discrete / continuous structured parameters and free text through differentiated processing. BERT's [CLS] vector extraction mechanism transforms unstructured text into computable semantic features, filling a gap in the application of livestock logs in disease modeling. Furthermore, spatiotemporal alignment and outlier correction mechanisms ensure the reliability of multimodal input, avoiding prediction bias caused by data misalignment.

[0021] The standardized continuous variables and the encoded categorical variables together form the dimension The input matrix (where is a real number space, n is the number of samples, and d is the feature dimension), which is input into the TabTransformer module for column-level embedding and contextual relationship modeling. The process first maps each feature column into a high-dimensional vector space through a learnable column embedding layer, generating a dimension of The embedding representation (h is the hidden dimension) enables the discrete categorical variables and continuous variables to establish semantic associations in a unified vector space. Subsequently, the embedding vector is input into the multi-layer Transformer Encoder Stack, which uses the self-attention mechanism to dynamically capture the global dependencies between features, such as the synergistic effect of dissolved oxygen concentration and salinity changes, the regulatory effect of pH fluctuations on nitrite toxicity, and other complex nonlinear interaction patterns. Finally, the context-aware feature matrix output by the encoder stack is As a deep representation of structured data, it provides feature vectors with high information density for subsequent multimodal fusion.

[0022] In this process, the self-attention mechanism explicitly learns the dynamic relationships between parameters (for example, the physical and chemical law that "rising temperature leads to decreased dissolved oxygen saturation"), breaking away from the traditional model's heavy reliance on feature engineering. Furthermore, the column embedding layer is compatible with both continuous and categorical variables, avoiding the curse of dimensionality caused by one-hot encoding. Attention weights are used to visualize and trace key influencing factors (for example, the contribution of salinity mutations to parasite onset), meeting the transparency requirements of farm decision-making.

[0023] The preprocessed aquaculture log and environment description text (e.g., "fish aggregation on the water surface, feeding activity decreases" "floc sediment appears at the bottom of the tank") is converted into a token sequence and added with a [CLS] start token and a [SEP] separator token to adapt to the input specification of BERT. After inputting this sequence into the multi-layer Transformer encoder of BERT, the context semantic association between tokens (e.g., the potential causal chain between "feeding activity decreases" and parasitic stress) is analyzed through self-attention mechanism, and finally the hidden state vector of each token is generated in the output layer. Among them, the hidden state vector corresponding to the [CLS] token at the beginning of the sequence is extracted as the global semantic representation of the entire text, and its dimension is fixed to (taking the BERT-Base model as an example). This vector condenses the key semantic information in the text (such as disease precursor behavior, abnormal environmental phenomena), and serves as a unified feature representation of unstructured data, providing high-dimensional semantic input for subsequent fusion with structured features.

[0024] Compared with traditional text processing methods, the self-attention mechanism of BERT identifies the implicit causal chain in the text (e.g., "increased water surface foam → insufficient dissolved oxygen → susceptible to parasites"), breaking through the limitations of the bag-of-words model that only captures surface word frequency. By dynamically analyzing the true meaning of descriptive phrases (e.g., the specificity of "fish body rubbing the tank wall" in the context of parasitic infection), it avoids the misjudgment caused by semantic ambiguity in traditional methods. The fixed-dimension CLS vector (768 dimensions) ensures compatibility with the output structure of TabTransformer, enabling seamless alignment of heterogeneous data.

[0025] The structured feature matrix generated by TabTransformer is concatenated with the text semantic feature vector extracted by BERT in the feature dimension to form a fused feature matrix. After inputting this matrix into the fully connected layer (output dimension ), the cross-modal association between structured parameters and text semantics (e.g., "dissolved oxygen decreases" and the cooperative indicative meaning of "fish floating head" described in the log) is learned through nonlinear transformation, and finally the output layer activated by the three-node Sigmoid simultaneously generates the disease probabilities of Cryptobia, Haemoproteus, and Benedenia .

[0026] To optimize the robustness of multi-label prediction, the model training uses a weighted loss function with class imbalance awareness: where is the total loss, i is the number of the target disease, is the binary cross-entropy loss function, True label of the i-th pathogen (0 for non-disease, 1 for disease), Category weights are dynamically adjusted. This design improves the recognition sensitivity of rare parasites (such as blood trypanosomes with small sample size), and avoids model bias towards high sample size categories.

[0027] The present application identifies implicit risk patterns that cannot be captured by a single data type by fusing complementary information such as pH abnormalities + "fish body mucus increase" text. The three-way Sigmoid layer independently calculates the disease probability of each type of parasite, avoiding the maintenance burden of multiple independent models, and amplifying the loss contribution of rare diseases through weight to solve the modeling bias of uneven distribution of pathogens in actual data.

[0028] In addition, the present application converts the pathogen disease probability output by the deep learning model into an operable three-level risk label. For each target parasite, the system presets a dynamic probability threshold interval for automatic classification: when the disease probability value is less than 0.5, it is marked as low risk, corresponding to regular monitoring recommendations; when the probability value is between 0.5 and 0.8, it is marked as medium risk, triggering early warning instructions for water quality parameter optimization and observation frequency improvement; when the probability value exceeds 0.8, it is marked as high risk, and the emergency process of isolation treatment or medicated bath intervention is forcibly started. This classification mechanism calibrates the threshold boundary by combining historical disease data and expert knowledge base, for example, appropriately relaxing the high-risk threshold for blood trypanosomes with long incubation period to 0.85, to ensure the biological rationality of risk decision-making. The final generated risk label and the original probability together constitute the structured output, forming a direct mapping chain from the prediction result to the aquaculture operation.

[0029] At the same time, the system converts the prediction results into multi-dimensional interactive views through the visualization platform built by the Shiny framework. The risk matrix heat map presents the global risk distribution state with red, yellow and green color blocks, with the aquaculture unit as the row and the pathogen as the column. Users can view detailed prediction data by clicking on the color blocks. The key influence factor analysis module quantifies the contribution of each input variable (such as dissolved oxygen numerical anomaly or log keyword "fish body white spot") to the prediction result based on the attention weight of TabTransformer and the gradient backpropagation of the fully connected layer, and highlights the top five risk driving factors in descending order bar chart. At the same time, the projection of the fusion feature space to the two-dimensional plane by UMAP dimensionality reduction algorithm reveals the spatial aggregation pattern of high-risk samples in the cluster scatter plot, which helps to identify the potential disease outbreak trend. Users can dynamically adjust the confidence threshold through the slider to refresh all views in real time; one-key export function generates a CSV report containing time stamp, pathogen probability, risk level and main influence factors, which directly interfaces with the aquaculture farm management system for intervention measure scheduling.

[0030] In summary, the present invention proposes a method for predicting aquaculture diseases based on multimodal data fusion. By integrating structured water quality parameters with unstructured text data and combining the TabTransformer and BERT dual-channel modeling architecture, it fully captures the multidimensional correlation of pathogen inducements in complex aquaculture environments. Compared with traditional single-modal models, the average prediction accuracy is improved, effectively solving the problems of single prediction dimension and weak generalization ability of existing technologies.

[0031] Using a multi-label Sigmoid output layer design, a single model simultaneously outputs the incidence probability of multiple types of parasites. Compared with the traditional independent modeling scheme of a single pathogen, this reduces resource consumption and avoids compatibility issues between models. It meets the real-time needs of breeding sites for concurrent early warning of multiple pathogens and significantly improves the practicality and deployment efficiency of the system.

[0032] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative position relationship, movement status, etc. between the various components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0033] In addition, in the present invention, descriptions such as "first," "second," and "one" are for descriptive purposes only and should not be understood to indicate or imply their relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include at least one of such features. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0034] In the present invention, unless otherwise specified or limited, the terms "connection" and "fixation" should be understood in a broad sense. For example, "fixation" can mean fixed connection, detachable connection, or integration; mechanical connection or electrical connection; direct connection or indirect connection through an intermediate medium; internal communication between two elements or interaction between two elements, unless otherwise specified. Those skilled in the art will be able to understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0035] In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that ordinary technicians in this field can implement it. When the combination of technical solutions is mutually contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

Claims

1. A method for predicting aquaculture diseases based on multimodal data fusion, characterized in that: Including steps: S1: Collect aquaculture data including structured data and unstructured text data. The structured data are various target water quality parameters, and the unstructured text data are aquaculture logs and environmental description texts. S2: Input the structured data into the TabTransformer model, perform column embedding and contextual relationship modeling between features through a multi-layer Transformer encoder, and output a structured feature vector; S3: Input the unstructured text data into the pre-trained BERT encoder and extract the output vector of the CLS tag as the text semantic feature; S4: The structured feature vector and text semantic features are concatenated into fusion features and input into the fully connected layer, and the incidence probability of each target disease is predicted through the Sigmoid function.

2. The aquaculture disease prediction method based on multimodal data fusion according to claim 1, characterized in that: In step S1, the structured data includes at least three water quality parameters of temperature, salinity, pH value, dissolved oxygen, nitrite concentration, and silicate concentration, and the unstructured text data includes at least one of aquaculture logs, on-site descriptions, and expert records.

3. The aquaculture disease prediction method based on multimodal data fusion according to claim 1, characterized in that: In the S1 step, the structured data is input in CSV or Excel format and includes continuous variables and discrete categorical variables.

4. The aquaculture disease prediction method based on multimodal data fusion according to claim 1, characterized in that: In the step S3, the semantic vector of the CLS tag is extracted as follows: unstructured text data is input into the BERT encoder, and the output vector at the position corresponding to the CLS tag is taken as the text semantic feature.

5. The aquaculture disease prediction method based on multimodal data fusion according to claim 1, characterized in that: In the S4 step, the fully connected layer has a multi-label output structure, and the output dimension is , respectively corresponding to the incidence probability of each category of target disease, where is a real number space, n is the number of samples, and m is the total number of target disease categories.

6. The aquaculture disease prediction method based on multimodal data fusion according to claim 5, characterized in that: In step S4, the model for predicting the probability of disease by the Sigmoid function is trained using the following loss function: Where, is the total loss, i is the number of the target disease, is the dynamically adjusted category weight, is a binary cross entropy loss function, which is used to measure the predicted probability value of the model for the i-th target disease. and the corresponding true label The difference between.

7. The aquaculture disease prediction method based on multimodal data fusion according to claim 1, characterized in that: After the S4 step, the following steps are further included: S5: Risk levels are divided based on the relationship between the probability of occurrence of the target disease and the preset threshold interval; When the probability of the target disease is below the preset threshold range, it is classified as low risk; When the probability of the target disease is within the preset threshold range, it is classified as medium risk; When the probability of the target disease is above the preset threshold range, it is classified as high risk.

8. The aquaculture disease prediction method based on multimodal data fusion according to claim 7, characterized in that: Also includes the steps: S6: Display the prediction results and the variable influencing factors that affect the current prediction results through a visual interactive interface.

Images