Hilly area pig disease multi-modal digital twin early warning system

Abstract

The application discloses a hilly area pig epidemic disease multi-modal digital twin early warning system, relates to the technical field of livestock epidemic disease monitoring, wherein the early warning method comprises the following steps: synchronously collecting the body surface temperature distribution, the movement posture sequence, the respiratory tract acoustic signal and the pig house environment parameters of live pigs, and forming standardized multi-modal feature data after pretreatment; inputting the standardized multi-modal feature data into a multi-modal joint feature extraction network with a lightweight Transform architecture, and outputting fusion feature representation; combining three-dimensional terrain data and meteorological data in the hilly area, and performing joint analysis through a space-time dynamics epidemic evolution model to obtain group transmission risk assessment results; generating early warning information according to the assessment results; and incrementally updating model parameters based on veterinarian diagnosis feedback data, so that the information dimension relied on by early warning determination simultaneously covers two aspects of the physiological state of live pigs and the pig house environment state, and the probability of false judgment caused by single monitoring index fluctuation in the hilly area is reduced.

Description

Technical Field

[0001] This invention relates to the field of livestock disease monitoring technology, specifically to a multimodal digital twin early warning system for swine diseases in hilly areas. Background Technology

[0002] Pig farming in hilly areas faces challenges in disease control due to the complex temperature and humidity environment and the need for enclosed or semi-enclosed pigsties. The undulating terrain of hilly areas leads to significant differences in microclimate conditions, complex pathogen transmission routes, and the structure of local valleys and ridges can alter airflow direction, exacerbating the accumulation or diffusion of aerosols in specific areas.

[0003] Current disease monitoring methods mainly rely on manual inspections and single sensor data collection, which limits the dimensions of monitoring the physiological state of individual pigs and cannot simultaneously acquire multi-dimensional physiological information such as body temperature, posture, and acoustics. At the same time, existing methods lack the ability to jointly model hilly terrain and meteorological factors, making it difficult to simulate the actual diffusion patterns of pathogens under the constraints of hilly microclimates.

[0004] The aforementioned shortcomings lead to the following problems with existing methods in hilly pig farming scenarios: single monitoring indicators are prone to misjudgment due to environmental fluctuations in hilly areas; disease transmission risk assessments cannot reflect the true spread trend of pathogens in hilly terrain; when network signal coverage in hilly areas is unstable, early warning processing relying on remote servers suffers from response delays; and purely data-driven methods lack generalization ability when the historical disease sample size is limited. These problems make it difficult to identify early outbreaks of diseases in hilly pig farming scenarios in a timely manner. Summary of the Invention

[0005] This invention provides a multimodal digital twin early warning system for swine diseases in hilly areas, which solves the technical problems in related technologies that cannot integrate multimodal biological signals with hilly terrain and environmental data, making it difficult to conduct accurate spatiotemporal risk assessment of swine disease transmission.

[0006] This invention discloses a multimodal digital twin early warning method for swine diseases in hilly areas, comprising the following steps: Simultaneously acquiring body surface temperature distribution data, movement posture sequence data, and respiratory acoustic data of the target pig herd using multimodal acquisition devices deployed in pigsties, and acquiring environmental parameter data within the pigsties using an environmental sensor array; aligning the data from each channel according to a unified timestamp to form a multimodal raw dataset; preprocessing and standardizing the multimodal raw dataset to obtain standardized multimodal feature data; inputting the standardized multimodal feature data into a multimodal joint feature extraction network to output a fused feature representation. The multimodal joint feature extraction network adopts a lightweight Transformer architecture. It integrates feature representations and hilly area geographical environment data into a spatiotemporal dynamic disease evolution model for joint analysis to obtain disease transmission risk assessment results. The spatiotemporal dynamic disease evolution model includes pathogen environmental survival estimation unit, aerosol diffusion simulation unit, host susceptibility assessment unit, and herd immunity status aggregation unit. Based on the disease transmission risk assessment results, the herd transmission risk level is compared with a preset warning threshold. When the herd transmission risk level reaches or exceeds the warning threshold, a warning message is generated.

[0007] Furthermore, the original multimodal dataset is preprocessed and standardized to obtain standardized multimodal feature data, including: spatial interpolation and outlier removal of body surface temperature distribution data to obtain the body surface temperature distribution map of each pig at each collection time, wherein the spatial interpolation adopts a linear interpolation algorithm; extracting the trajectory coordinates of joint points from the motion posture sequence data to generate a posture time-series vector; performing spectral transformation on the respiratory acoustic data to extract the Mel frequency cepstral coefficients and obtain an acoustic feature vector; performing statistical aggregation of environmental parameter data within a time window to obtain an environmental state vector, which includes the mean and rate of change of each environmental parameter within a preset time window; and concatenating the body surface temperature distribution map, posture time-series vector, acoustic feature vector, and environmental state vector after Z-score standardization to form standardized multimodal feature data.

[0008] Furthermore, the process of spectral transformation of respiratory acoustic data also includes adaptive filtering of background noise in the pigsty environment. Adaptive filtering includes: pre-recording background noise samples in the pigsty under empty conditions, and using the spectral characteristics of the background noise samples as the noise baseline; in real-time acoustic data processing, using spectral subtraction to subtract the frequency band energy corresponding to the noise baseline from the original acoustic spectrum, and retaining the effective frequency band components of the pigs' respiratory tract sound production.

[0009] Furthermore, the lightweight Transformer architecture is a variant of the standard Transformer's self-attention layer with parameter compression. It replaces the fully connected weight matrix in the standard self-attention layer with a low-rank decomposition form to complete inference operations on the NPU chip of the edge diagnostic terminal.

[0010] Furthermore, the pathogen environmental survival estimation unit receives temperature and humidity data from the environmental parameter data, and combines them with the pre-injected pathogen microorganism environmental survival curve parameters to calculate the survival probability of the target pathogen under the current environmental conditions in the following manner: Based on the initial survival probability baseline value of the pathogen, the exposure time calculated from the release of the pathogen is exponentially decayed using the decay rate coefficient related to the ambient temperature and relative humidity to obtain the pathogen survival probability distribution and output the pathogen survival probability map; wherein, the decay rate coefficient is obtained by looking up the environmental survival curve parameter table determined by veterinary epidemiological experiments.

[0011] Furthermore, the aerosol diffusion simulation unit receives pathogen survival probability maps, high-precision three-dimensional topographic data of hilly areas, and real-time meteorological data. It uses the hydrodynamic diffusion equation to numerically solve on a discretized three-dimensional spatial grid, simulating the spatial diffusion trajectory and concentration decay process of pathogen-containing aerosols under the constraints of hilly terrain, and outputs the pathogen exposure concentration distribution of each spatial grid node at each time. Among them, the three-dimensional topographic data includes the surface elevation, slope aspect, and valley orientation of the area where the pigsty is located, and the three-dimensional topographic data serves as the boundary condition of the aerosol diffusion simulation unit.

[0012] Furthermore, the host susceptibility assessment unit receives fusion feature representations and age information of individual pigs. It weights the fusion feature representations using an adjustable weight vector and then performs a difference operation with the susceptibility threshold parameter for the corresponding age group. The result is then mapped using a Sigmoid function to output an individual susceptibility score. The herd immunity status aggregation unit receives the susceptibility scores and pathogen exposure concentration distributions of each individual. It multiplies the susceptibility scores of each individual in the same or adjacent pig houses with the pathogen exposure concentration at the corresponding spatial location and takes the average to obtain a herd transmission risk score. During the aggregation process, historical vaccination records are also incorporated. The time interval between each pig's most recent vaccination is used to perform an exponential decay calculation using the vaccine efficacy decay rate constant to obtain an immune decay coefficient. The product of the immune decay coefficient and the individual susceptibility score replaces the individual susceptibility score in the aggregation calculation of the herd transmission risk score.

[0013] Furthermore, after acquiring the fusion feature representation, the process also includes: arranging the fusion feature representations from multiple consecutive acquisition times in chronological order to form a feature representation time series; inputting the feature representation time series into a time series coding layer, which is a single-layer gated recurrent unit, and outputting dynamic fusion features containing time series evolution information; and using the dynamic fusion features to replace the fusion feature representations in subsequent steps for analysis.

[0014] Furthermore, the method also includes the step of updating parameters of the spatiotemporal dynamic disease evolution model based on veterinary diagnosis feedback data: acquiring veterinary diagnosis feedback data, which includes at least one of serological test results, PCR nucleic acid test results, and pathological autopsy records; comparing the warning level and abnormal indicators in the warning information with the veterinary diagnosis feedback data to obtain warning accuracy labels, and forming training sample pairs with the warning accuracy labels and the standardized multimodal feature data at the corresponding time; marking the number of detection methods for each veterinary diagnosis feedback data, and when the number of detection methods included in the veterinary diagnosis feedback data of the same warning event reaches a preset threshold, marking the corresponding training sample pair as a high-confidence sample, and only high-confidence samples participate in the parameter update calculation; and using an incremental learning method, correcting only the parameter deviation caused by the newly added training sample pairs while retaining the existing parameters, and performing gradient updates on the adjustable parameters of the host susceptibility assessment unit and the herd immunity status aggregation unit.

[0015] This invention discloses a multimodal digital twin early warning system for swine diseases in hilly areas, used to execute the aforementioned multimodal digital twin early warning method for swine diseases in hilly areas. The system includes: a multimodal data acquisition module, used to simultaneously acquire body surface temperature distribution data, movement posture sequence data, and respiratory acoustic data of the target pig herd through multimodal acquisition devices deployed in pigsties, and to acquire environmental parameter data within the pigsties through an environmental sensor group, aligning the data from each channel according to a unified timestamp to form a multimodal raw dataset; a preprocessing module, used to preprocess and standardize the multimodal raw dataset to obtain standardized multimodal feature data; and a feature extraction module, used to input the standardized multimodal feature data into a multimodal joint feature extraction module. The system comprises a feature extraction network that outputs fused feature representations. The multimodal joint feature extraction network adopts a lightweight Transformer architecture. A risk assessment module is used to input the fused feature representations and hilly area geographical environment data into a spatiotemporal dynamic disease evolution model for joint analysis, obtaining disease transmission risk assessment results. The spatiotemporal dynamic disease evolution model includes a pathogen environmental survival estimation unit, an aerosol diffusion simulation unit, a host susceptibility assessment unit, and a herd immunity status aggregation unit. An early warning generation module is used to compare the herd transmission risk level with a preset early warning threshold based on the disease transmission risk assessment results. When the herd transmission risk level reaches or exceeds the early warning threshold, an early warning message is generated.

[0016] The present invention has the following beneficial effects.

[0017] By simultaneously collecting data from four channels—body surface temperature distribution, movement posture sequence, respiratory acoustic signal, and pig house environmental parameters—and performing time alignment and standardized fusion, the information dimensions used for early warning judgments cover both the individual physiological state of pigs and the environmental state of pig houses, reducing the probability of misjudgment caused by fluctuations in the hilly area environment due to a single monitoring indicator.

[0018] By employing a lightweight Transformer architecture in the multimodal joint feature extraction network, the feature extraction process can be completed independently on the NPU chip of the edge diagnostic terminal. The early warning processing does not depend on the real-time connection of the remote data processing server, making the early warning response delay controllable under unstable network signal conditions in hilly areas.

[0019] By incorporating prior knowledge such as pathogen survival curve parameters, three-dimensional topographic constraints in hilly areas, and age-based susceptibility thresholds into the spatiotemporal dynamic disease evolution model, the disease transmission risk assessment can reflect the actual diffusion patterns of pathogens in the hilly microclimate, thus compensating for the shortcomings of purely data-driven methods in terms of insufficient generalization ability when the sample size of historical diseases is limited.

[0020] By using incremental parameter updates and confidence screening mechanisms based on veterinary diagnostic feedback data, the spatiotemporal dynamics disease evolution model can gradually adapt to seasonal changes in pig herd structure and environmental conditions during continuous operation, while reducing the interference of noise feedback on the stability of model parameters. Attached Figure Description

[0021] Figure 1 This is a flowchart of the multimodal joint early warning method for swine diseases in hilly areas provided in an embodiment of the present invention; Figure 2 This is a schematic diagram comparing the surface temperature and acoustic anomaly scores of a typical individual, as provided in an embodiment of the present invention. Figure 3 This is a schematic diagram of the distribution of typical individual standardized multimodal feature data provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the deviation of typical individual dynamic fusion features provided in the embodiments of the present invention; Figure 5 This is a schematic diagram comparing the typical individual immune attenuation coefficient with the corrected score provided in an embodiment of the present invention; Figure 6 This is a schematic diagram illustrating the relationship between typical individual susceptibility scores and pathogen exposure concentrations provided in an embodiment of the present invention; Figure 7 This is a schematic diagram illustrating the relationship between the number of days since the last vaccination and the immune attenuation coefficient, provided in an embodiment of the present invention. Figure 8This is a schematic diagram illustrating the change in the probability of pathogen survival in the environment with exposure time, provided in an embodiment of the present invention. Figure 9 This is a schematic diagram of a typical individual multidimensional risk index heatmap provided in an embodiment of the present invention. Detailed Implementation

[0022] Example 1 Pig farming in hilly areas generally faces challenges in disease control due to the complex temperature and humidity environment and the presence of enclosed or semi-enclosed pig houses. Traditional monitoring methods rely on manual inspections, which suffer from slow response times and insufficient coverage. Furthermore, the undulating terrain of hilly areas leads to significant differences in microclimate conditions and complex pathogen transmission pathways, making it difficult to identify early-stage outbreaks of disease based on conventional experience. The essence of these problems lies in the lack of an automated early warning system that can integrate multimodal sensing data and combine it with the unique topographical and meteorological factors of hilly areas to analyze disease transmission trends.

[0023] The hardware environment involved in this implementation includes: a multimodal acquisition device deployed in a hilly pigsty, which integrates an infrared thermal imaging module, a depth vision module, and a directional acoustic array module for simultaneously acquiring the surface temperature distribution, movement posture sequence, and respiratory acoustic signals of pigs; an edge diagnostic terminal deployed locally in the pigsty, which has a built-in micro NPU computing chip to support independent operation of the pre-recognition algorithm offline; and a remote data processing server for running spatiotemporal dynamics analysis and updating model parameters. An environmental sensor group is synchronously configured in the pigsty to collect environmental parameters such as temperature, humidity, ammonia concentration, and ventilation volume.

[0024] Please see Figure 1 , Figure 2 and Figure 3 A multimodal digital twin early warning method for swine diseases in hilly areas includes the following steps: Step 1: Obtain multimodal health monitoring data and environmental parameter data for pigs.

[0025] Multimodal data acquisition devices deployed in pigsties are used to synchronously acquire surface temperature distribution data, movement posture sequence data, and respiratory acoustic data of the target pig herd at a preset acquisition frequency. Simultaneously, environmental sensor arrays acquire data on temperature, humidity, ammonia concentration, and ventilation volume within the pigsty. The data from all these channels are aligned using a unified timestamp to form a time-stamped multimodal raw dataset.

[0026] It should be noted that the above-mentioned data acquisition frequencies are set according to the required monitoring accuracy. In the standard monitoring mode, the data acquisition frequency for body surface temperature distribution and movement posture sequence data is once per minute per pig; respiratory acoustic data is acquired using continuous streaming, segmented into fixed-duration windows. In the high-alert mode, the acquisition frequency for each channel is increased to several times that of the standard mode.

[0027] Step 2: Preprocess the original multimodal dataset to obtain standardized multimodal feature data.

[0028] Step 2 may specifically include: Step S201 involves spatial interpolation and outlier removal of the body surface temperature distribution data to obtain the body surface temperature distribution map for each pig at each acquisition time. Joint trajectory coordinates are extracted from the motion posture sequence data to generate a posture time-series vector. The spatial interpolation uses a linear interpolation algorithm, with the input being the temperature value and spatial coordinates of each infrared thermal imaging sampling point, and the output being an estimated temperature distribution value in continuous space.

[0029] Step S202 involves performing a spectral transformation on the respiratory acoustic data, extracting the Mel frequency cepstral coefficients, and obtaining the acoustic feature vector. During the spectral transformation process, adaptive filtering is performed on equipment operation noise and ventilation system noise in the hilly pigsty environment.

[0030] It should be noted that the adaptive filtering method in step S202 is as follows: Background noise samples from the pigsty are pre-recorded in an empty state, and the spectral characteristics of these samples are used as the noise baseline. In real-time acoustic data processing, spectral subtraction is used to subtract the frequency band energy corresponding to the noise baseline from the original acoustic spectrum, retaining the effective frequency band components of the pigs' respiratory tract sounds. This adaptive filtering method is well-suited to addressing the persistent background noise generated by the high-load operation of ventilation equipment in hilly pigsties due to terrain obstruction.

[0031] Step S203: Perform statistical aggregation on the environmental parameter data within a time window to obtain an environmental state vector. The environmental state vector includes the mean and rate of change of each environmental parameter within a preset time window.

[0032] Step S204 involves performing Z-score standardization on the body surface temperature distribution map, posture time-series vector, acoustic feature vector, and environmental state vector to eliminate the influence of different physical dimensions of the data from each channel on subsequent splicing operations. The standardized data from each channel are then spliced ​​together to form standardized multimodal feature data.

[0033] Step 3: Input the standardized multimodal feature data into the multimodal joint feature extraction network and output the fused feature representation.

[0034] The aforementioned multimodal joint feature extraction network employs a lightweight Transformer architecture. The input layer of the network receives standardized multimodal feature data, while the output layer is a fully connected layer, outputting a fixed-dimensional fused feature representation. This fused feature representation encodes the correlation information of four channels—body temperature, posture, acoustics, and environment—at the same time point. During training, the labeled health status categories for each channel are used as supervision signals, and the network parameters are updated using the cross-entropy loss function and the Adam optimization algorithm.

[0035] It should be noted that the aforementioned lightweight Transformer architecture refers to a variant of the standard Transformer's self-attention layer with parameter compression. The lightweight Transformer architecture replaces the fully connected weight matrix in the standard self-attention layer with a low-rank decomposition form to reduce computational complexity, enabling inference operations to be performed on the NPU chip of the edge diagnostic terminal.

[0036] Please see Figure 4 In this embodiment of the application, to ensure that the fused feature representation can reflect the trend of individual pig states changing over time, the following processing is included in addition to step 3: The fused feature representations from multiple consecutive acquisition times are arranged in chronological order to form a feature representation time series. This feature representation time series is input to a time-series coding layer, which is a single-layer gated recurrent unit. Its input is the feature representation time series, and its output is a dynamic fused feature containing time-series evolution information. In subsequent steps, the dynamic fused feature replaces the fused feature representation in the analysis.

[0037] Step 4: Based on the spatiotemporal dynamics disease evolution model, conduct joint analysis of the integrated feature representation and hilly area geographical environment data to obtain the disease transmission risk assessment results.

[0038] The above spatiotemporal dynamics disease evolution model is an unscaled model, and its constituent units and the data transfer relationships between them are as follows: The spatiotemporal dynamics disease evolution model includes a pathogen environmental survival estimation unit, an aerosol diffusion simulation unit, a host susceptibility assessment unit, and a herd immunity status aggregation unit.

[0039] The pathogen survival estimation unit receives temperature and humidity data from the environmental state vector, combines this data with pre-injected pathogen survival curve parameters, calculates the survival probability distribution of the target pathogen in various areas inside and outside the pigsty under the current environmental conditions, and outputs a pathogen survival probability map. Specifically, let the current environmental temperature be... Relative humidity is Pathogen survival probability Calculated by the following formula: ; in, This serves as the baseline value for the initial survival probability of the pathogen. Represents an exponential function. The duration of exposure is calculated from the time the pathogen is released. To be in harmony with ambient temperature and relative humidity The relevant decay rate coefficients were obtained from a table of environmental survival curve parameters determined by veterinary epidemiological experiments. These environmental survival curve parameters are biological constants determined by veterinary epidemiological experiments and incorporated into the spatiotemporal dynamic disease evolution model.

[0040] The aerosol diffusion simulation unit receives pathogen survival probability maps, high-precision three-dimensional topographic data of hilly areas, and real-time meteorological data. It uses the hydrodynamic diffusion equation to perform numerical solutions on a discretized three-dimensional spatial grid, simulating the spatial diffusion trajectory and concentration decay process of pathogen-containing aerosols under the constraint of hilly terrain, and outputs the pathogen exposure concentration distribution of each spatial grid node at each time.

[0041] It should be noted that the aforementioned three-dimensional topographic data includes geographical features such as surface elevation, slope aspect, and valley orientation of the area where the pigsty is located. The valley and ridge structures of hilly terrain can alter airflow direction and speed, causing aerosols to accumulate or accelerate their diffusion in local areas. Therefore, it is necessary to use three-dimensional topographic data as the boundary condition for aerosol diffusion simulation units.

[0042] The host susceptibility assessment unit receives fused feature representations and age information of individual pigs. Based on preset susceptibility threshold parameters corresponding to different age groups, it compares the current health status of each pig with the susceptibility threshold and outputs an individual susceptibility score. Specifically, let the... The fusion characteristics of the first pig are characterized as The susceptibility threshold parameter corresponding to its age is Individual susceptibility score Calculated by the following formula: ; in, It is an adjustable weight vector. Indicates transpose. For the Sigmoid function, For each pig, assign an individual number. The immune response levels of pigs of different ages to specific pathogens were determined based on veterinary clinical statistics.

[0043] The herd immunity status aggregation unit receives the susceptibility scores and pathogen exposure concentration distributions of each individual. It weights and aggregates the susceptibility scores of individuals within the same or adjacent pig houses, combining this with the pathogen exposure concentrations at the corresponding spatial locations to calculate the herd-level disease transmission risk assessment results. Specifically, assuming there are a total of [number missing] individuals within the pig house... The first pig, the first The pathogen exposure concentration at the current spatial location of the first pig is: Group transmission risk score Calculated by the following formula: ; in, This represents the total number of pigs in the pigsty. For the first Individual susceptibility score of the first pig. For the first The pathogen exposure concentration at the corresponding location of each pig. The disease transmission risk assessment results include the group transmission risk level and the estimated risk window period, where the group transmission risk level is determined by… The risk window period is obtained by mapping the data after comparing it with the preset graded threshold range. The estimated risk window period is calculated from the pathogen exposure concentration distribution time series output by the aerosol diffusion simulation unit.

[0044] The adjustable parameters in the host susceptibility assessment unit and the herd immunity status aggregation unit are trained using labeled historical disease event records, supervised by early warning accuracy labels, and updated using the mean squared error loss function and gradient descent optimization algorithm.

[0045] Please see Figure 4 and Figure 5 In this embodiment of the application, to improve the aggregation accuracy of the herd immunity status aggregation unit, historical vaccination record data is introduced during the aggregation process. The difference between the time of each pig's most recent vaccination with a specific vaccine and the current time is mapped to an immune decay coefficient. The immune decay coefficient is then multiplied by the individual susceptibility score before being included in the aggregation. Specifically, let the first... The time interval between the first pig's most recent vaccination is Immune attenuation coefficient Calculated by the following formula: ; in, Represents an exponential function. This is the constant representing the rate of decline in vaccine immunization efficacy, determined by historical vaccination statistics. For the first The time interval between the pig's most recent vaccination. and product Alternative Risk score for group transmission The aggregation operation is performed. Through processing, the aggregation result can reflect the actual situation of vaccine immunization efficacy decaying over time.

[0046] It should be noted that, The range of values ​​is :when hour, This indicates complete immune efficacy; Increase A monotonous decrease reflects a continuous decline in immune efficacy. Due to... ,product The numerical range and Similarly, there is no need to score the risk of group transmission. The calculation formula has been adjusted accordingly.

[0047] Step 5: Generate early warning information based on the results of the disease transmission risk assessment.

[0048] The risk level of group transmission is compared with a preset warning threshold. When the risk level of group transmission reaches or exceeds the warning threshold, a warning message is generated, including the warning level, the number of pig houses involved, the estimated risk window period, and the main abnormal indicators that triggered the warning. The warning message is then sent to a remote data processing server and the on-site management terminal.

[0049] When the risk level of group transmission does not reach the warning threshold, the current fusion characteristic representation and the result of the epidemic transmission risk assessment are stored in the historical record for subsequent time series analysis.

[0050] It should be noted that the aforementioned warning thresholds are divided into multiple levels according to the risk level. Low-level warnings trigger an increase in monitoring frequency, medium-level warnings trigger on-site veterinary verification, and high-level warnings trigger emergency intervention procedures such as containment and sampling. The risk level range corresponding to each level is determined based on historical disease data statistics during system initialization.

[0051] Please see Figure 6 and Figure 7 In this embodiment, to ensure the real-time performance of early warnings in network outage scenarios, step 5 executes local early warning determination in parallel on the edge diagnostic terminal. The edge diagnostic terminal utilizes an NPU chip to run a lightweight Transformer architecture and simplified risk assessment rules, independently completing the entire processing flow from feature extraction to early warning determination after receiving the multimodal raw dataset. Once the communication link between the edge diagnostic terminal and the remote data processing server is restored, the locally accumulated early warning information and the multimodal raw dataset are uploaded in batches to the remote data processing server for synchronization. This processing method ensures that the early warning response delay is controllable even under unstable network signal conditions in hilly areas.

[0052] Step 6: Update the parameters of the spatiotemporal dynamic disease evolution model based on veterinary diagnosis feedback data.

[0053] For the warning information generated in step 5, after the veterinarian completes the on-site verification, obtain the veterinary diagnosis feedback data. The veterinary diagnosis feedback data includes at least one of the following: serological test results, PCR nucleic acid test results, and pathological autopsy records.

[0054] The warning levels and abnormal indicators in the early warning information are compared with veterinary diagnostic feedback data to obtain early warning accuracy labels. These accuracy labels are then used to create training sample pairs with standardized multimodal feature data at the corresponding time points. Using these training sample pairs, the adjustable parameters of the host susceptibility assessment unit and the herd immunity status aggregation unit in the spatiotemporal dynamics disease evolution model are updated using gradients.

[0055] It should be noted that the above parameter updates are performed using an incremental learning approach, meaning that only the parameter biases caused by newly added training samples are corrected while retaining existing parameters. This incremental learning approach avoids the computational overhead of full retraining and allows the spatiotemporal dynamics disease evolution model to continuously adapt to seasonal changes in the hilly area environment and pig herd structure.

[0056] Please see Figure 8 and Figure 9 In this embodiment, to avoid parameter shifts in the spatiotemporal dynamics disease evolution model caused by a small number of misdiagnosed reports, confidence screening of training sample pairs is performed before parameter updates. Specifically, the number of detection methods is marked for each veterinary diagnosis feedback data. When the number of detection methods included in the veterinary diagnosis feedback data for the same early warning event reaches a preset threshold, the corresponding training sample pair is marked as a high-confidence sample. Only high-confidence samples participate in parameter update calculations. Through screening, the interference of false positives or false negatives that may exist from a single detection method on the parameters of the spatiotemporal dynamics disease evolution model is reduced.

[0057] Technical effects of this embodiment This implementation method simultaneously acquires data from four channels: pig body surface temperature, movement posture, respiratory acoustics, and environmental parameters. It then performs time alignment and standardized fusion of the data from each channel, ensuring that the information dimensions used for early warning judgment cover both the individual physiological state of the pig and the environmental state of the pigsty. This overcomes the problem of misjudgment caused by fluctuations in the hilly area environment due to single monitoring indicators. By employing a lightweight Transformer architecture in the multimodal joint feature extraction network, the feature extraction process can be completed on the NPU chip of the edge diagnostic terminal. This eliminates the need for real-time connection to a remote data processing server for early warning processing, overcoming the problem of early warning delay caused by unstable network signal coverage in the hilly area. By injecting prior biological and geographical knowledge, such as pathogen survival curve parameters, three-dimensional terrain constraints in the hilly area, and age-based susceptibility thresholds, into the spatiotemporal dynamic disease evolution model, the disease transmission risk assessment can reflect the actual diffusion patterns of pathogens in the hilly area microclimate, overcoming the insufficient generalization ability of purely data-driven methods with limited sample sizes. By using incremental parameter updates and confidence level screening based on veterinary diagnostic feedback data, the spatiotemporal dynamics disease evolution model can gradually adapt to changes in pig herd structure and environmental conditions during continuous operation, while avoiding the impact of noise feedback on the stability of the spatiotemporal dynamics disease evolution model.

[0058] A pig farm in a hilly area (hereinafter referred to as "Farm A") is located in a mountainous area with significant topographical undulations. It has three pig houses (House A, House B, and House C) with a total stock of approximately 180 pigs. In the autumn of 20XX, the temperature difference between day and night increased, and swine respiratory diseases entered their peak season. Farm A deployed multimodal data acquisition equipment and environmental sensor arrays in each pig house. An edge diagnostic terminal was installed in the power distribution room of House B, and the remote data processing server was located in the farm's management center. From 08:00 to 08:05 on a certain day, the system executed a complete disease early warning process for the pigs in House B (a total of 20 pigs, numbered 1 to 20).

[0059] At 08:00, the multimodal data acquisition equipment completed a synchronous data collection for the pig herd in house B. Body surface temperature distribution data and movement posture sequence data were collected once per minute per pig, while respiratory acoustic data were collected in 30-second windows. The environmental sensor group simultaneously recorded the house temperature, humidity, ammonia concentration, and ventilation volume. Data from all channels were aligned with a unified timestamp of 08:00:00 to form the original multimodal dataset. The original data collected for pigs 3, 7, 12, and 18 are shown below.

[0060] Table 1. Original multimodal data collected from typical individuals in House B (timestamp: 08:00:00, Autumn 20XX)

[0061] Linear spatial interpolation is performed on the surface temperature distribution data of each individual body to remove abnormal pixels caused by equipment noise, generating a continuous temperature distribution map. Simultaneously, the trajectory coordinates of joints captured by the depth vision module are extracted to form a posture temporal vector. A spectral transformation is performed over a 30-second acoustic window, using a pre-recorded background noise spectrum of the empty building as a baseline. Continuous noise from the ventilation equipment is filtered out using spectral subtraction, and Mel-frequency cepstral coefficients are extracted to generate an acoustic feature vector. Using the past 5 minutes as a time window, the mean and rate of change of environmental parameters are statistically aggregated to generate an environmental state vector. The four types of features are Z-score standardized and then concatenated to form standardized multimodal feature data.

[0062] Table 2. Standardized multimodal feature data of typical individuals (after Z-score processing)

[0063] Both pigs 3 and 12 showed significant positive shifts in body surface temperature and acoustic characteristics, and negative shifts in movement posture characteristics, suggesting that these two pigs exhibited a combination of abnormalities, including elevated body temperature, reduced movement, and enhanced abnormal acoustic signals in the respiratory tract.

[0064] Standardized multimodal feature data is input into a multimodal joint feature extraction network running within the edge diagnostic terminal. The multimodal joint feature extraction network employs a lightweight Transformer architecture with low-rank decomposition self-attention, and inference is performed on the NPU chip. The fused feature representations from five consecutive acquisition times before 08:00 (07:56 to 08:00) are arranged chronologically and input into a single-layer gated recurrent unit, outputting dynamic fused features containing temporal evolution information. The dynamic fused features of pigs 3 and 12 continuously deviate from the healthy state reference range, indicating that their abnormal state is not a single, occasional fluctuation, but rather shows a continuous development trend over the past 5 minutes.

[0065] Table 3 Summary of typical individual dynamic fusion features output

[0066] The remote data processing server invokes the spatiotemporal dynamics disease evolution model to perform joint analysis of the four units in sequence.

[0067] The pathogen survival estimation unit receives the current building temperature. ℃, relative humidity The attenuation rate coefficient of porcine reproductive and respiratory syndrome virus (PRRSV) under these conditions was obtained by referring to the table. Initial survival probability baseline value Exposure duration Minutes, substitute into the formula Calculated This indicates that under the current low temperature and high humidity conditions, the survival probability of pathogens is still at a moderately low level, but they have not yet been rapidly inactivated.

[0068] The aerosol diffusion simulation unit inputs pathogen survival probability maps, three-dimensional topographic data of the hilly slope where pig house B is located (including the northern ridge and eastern valley structure), and real-time meteorological data (northeast wind 1.8 m / s). It then solves the hydrodynamic diffusion equations on a discrete three-dimensional grid, outputting the pathogen exposure concentration distribution for each grid node. Simulation results show that, guided by the eastern valley topography, the aerosol concentration in the eastern area of ​​pig house B is significantly higher than in the western area, with higher exposure concentrations observed in the eastern pens where pigs 3 and 12 are located.

[0069] The host susceptibility assessment unit combines dynamic fusion characteristics and information on the age of each pig, according to the formula. Calculate individual susceptibility scores, where For the Sigmoid function, For the weight vector, For the first Dynamic fusion feature vector of a pig, The susceptibility threshold parameters were determined based on an age-group susceptibility threshold parameter table. Historical vaccination records were also incorporated, and the results were calculated using a formula. Calculate the immune attenuation coefficient, where (Unit: 1 / day) represents the decay rate parameter. For the first The number of days since the last vaccination for each pig will be... Alternative Participate in subsequent aggregations.

[0070] Table 4. Typical individual susceptibility scores and immune attenuation correction results.

[0071] The herd immunity status aggregation unit weighted and aggregated the corrected scores and corresponding pathogen exposure concentrations of all 20 pigs in stall B, according to the formula. Calculate the risk score for group transmission, where The total number of heads in the group, the final result is Based on preset grading thresholds, It falls into the medium-risk range (0.25 to 0.40). Aerosol diffusion time-series forecasts indicate that if current weather conditions continue, the peak pathogen exposure concentration will spread towards the C-axis within the next 2 to 4 hours.

[0072] The risk level of group transmission is medium, reaching the medium-level warning threshold. The system generates a warning message, which includes: warning level is medium, the affected pig house is House B, the estimated risk window period is from 08:00 to 12:00 on the same day, and the main abnormal indicators triggering the warning are abnormally high body surface temperature and abnormal respiratory acoustic signals in pigs 3 and 12. The warning message is simultaneously pushed to the remote data processing server and the on-site management terminal, triggering the veterinarian on-site verification process.

[0073] During this period, due to a brief interruption of the network signal in the hilly area (lasting about 12 minutes), the edge diagnostic terminal independently completed feature extraction and local early warning determination from 08:01 to 08:12 using the NPU chip, and uploaded the accumulated data in batches after communication was restored to ensure that there was no delay in the early warning information.

[0074] Table 5 Summary of Early Warning Information Output

[0075] The veterinarian completed the on-site verification at 10:30 AM that day, simultaneously collecting serum and nasal swab samples from pigs 3 and 12. Both serological and PCR nucleic acid tests were performed, a total of two tests, reaching the preset threshold (threshold is 2). The corresponding training sample pair was marked as high-confidence samples. Serological results showed positive PRRSV antibodies, and PCR results confirmed positive viral nucleic acid, consistent with the intermediate warning judgment. The warning accuracy label was marked as true positive.

[0076] Using high-confidence training sample pairs, incremental gradient updates were performed on the adjustable parameters of the host susceptibility assessment unit and the herd immunity status aggregation unit. A mean squared error loss function and gradient descent algorithm were employed to correct only the parameter biases caused by new samples, while retaining existing parameters. After the update, the model's accuracy in assessing the susceptibility of 92- to 105-day-old pigs under low-temperature and high-humidity autumn conditions was corrected.

[0077] Table 6. Veterinary Diagnostic Feedback Data and Confidence Level Screening Results

[0078] Throughout the process, data begins with multimodal raw acquisition in step 1, undergoes standardization in step 2 to form comparable feature data, and is then compressed into dynamic fusion features by a lightweight network in step 3. Next, a spatiotemporal dynamic model in step 4, combined with hilly terrain constraints and immune attenuation correction, transforms individual-level physiological abnormality signals into spatialized risk scores at the population level. Finally, specific and actionable early warning information is generated in step 5. Veterinary feedback data in step 6 is incrementally fed back to the model parameters, creating a closed loop in the data flow. The data connections between each step are clear; abnormal signals from pigs 3 and 12 are present throughout the entire process, with corresponding data at each step, ensuring the integrity and consistency of the data flow.

[0079] The embodiments of the present invention have been described above. However, the embodiments are not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make more equivalent embodiments based on the guidance of the present embodiments, and all of them fall within the protection scope of the present embodiments.

Claims

1. A multimodal digital twin early warning method for swine diseases in hilly areas, characterized in that, Includes the following steps: By deploying multimodal acquisition devices in pig houses, the body surface temperature distribution data, movement posture sequence data and respiratory acoustic data of the target pig herd are acquired simultaneously. Environmental parameter data in the pig house are acquired through an environmental sensor group. The data from each channel are aligned with a unified timestamp to form a multimodal raw dataset. The original multimodal dataset is preprocessed and standardized to obtain standardized multimodal feature data; Standardized multimodal feature data is input into a multimodal joint feature extraction network, which outputs a fused feature representation. The multimodal joint feature extraction network adopts a lightweight Transformer architecture. The fusion of feature representation and hilly area geographical environment data is input into the spatiotemporal dynamic disease evolution model for joint analysis to obtain the disease transmission risk assessment results. The spatiotemporal dynamic disease evolution model includes pathogen environmental survival estimation unit, aerosol diffusion simulation unit, host susceptibility assessment unit, and herd immunity status aggregation unit. Based on the results of the disease transmission risk assessment, the risk level of group transmission is compared with the preset warning threshold. When the risk level of group transmission reaches or exceeds the warning threshold, a warning message is generated.

2. The multimodal digital twin early warning method for swine diseases in hilly areas according to claim 1, characterized in that, The original multimodal dataset is preprocessed and standardized to obtain standardized multimodal feature data, including: Spatial interpolation and outlier removal were performed on the body surface temperature distribution data to obtain the body surface temperature distribution map of each pig at each collection time. The spatial interpolation used a linear interpolation algorithm. The trajectory coordinates of the joint points were extracted from the motion posture sequence data to generate the posture time-series vector. Spectral transformation is performed on respiratory acoustic data to extract Mel frequency cepstral coefficients and obtain acoustic feature vectors; Perform statistical aggregation on environmental parameter data within a time window to obtain an environmental state vector. The environmental state vector contains the mean and rate of change of each environmental parameter within a preset time window. The surface temperature distribution map, posture time-series vector, acoustic feature vector, and environmental state vector are Z-score standardized and then concatenated to form standardized multimodal feature data.

3. The multimodal digital twin early warning method for swine diseases in hilly areas according to claim 2, characterized in that, The process of spectral transformation of respiratory acoustic data also includes adaptive filtering of background noise in the pigsty environment. Adaptive filtering includes: Background noise samples in the pigsty under empty conditions were pre-recorded, and the spectral characteristics of the background noise samples were used as the noise baseline. In real-time acoustic data processing, spectral subtraction is used to remove the frequency band energy corresponding to the noise baseline from the original acoustic spectrum, while retaining the effective frequency band components of the pig's respiratory tract sound production.

4. The multimodal digital twin early warning method for swine diseases in hilly areas according to claim 1, characterized in that, The lightweight Transformer architecture is a variant of the standard Transformer's self-attention layer with parameter compression. It replaces the fully connected weight matrix in the standard self-attention layer with a low-rank decomposition form to perform inference operations on the NPU chip of the edge diagnostic terminal.

5. The multimodal digital twin early warning method for swine diseases in hilly areas according to claim 1, characterized in that, The pathogen environmental survival estimation unit receives temperature and humidity data from environmental parameter data, and combines this with pre-injected pathogen microorganism environmental survival curve parameters to calculate the survival probability of the target pathogen under the current environmental conditions in the following manner: Based on the baseline value of the initial survival probability of the pathogen, the exposure time calculated from the release of the pathogen is exponentially decayed using the decay rate coefficient related to the ambient temperature and relative humidity to obtain the pathogen survival probability distribution and output the pathogen survival probability map. The attenuation rate coefficient was obtained by referring to the environmental survival curve parameter table determined by veterinary epidemiological experiments.

6. The multimodal digital twin early warning method for swine diseases in hilly areas according to claim 5, characterized in that, The aerosol diffusion simulation unit receives pathogen survival probability maps, high-precision three-dimensional topographic data of hilly areas, and real-time meteorological data. It uses the hydrodynamic diffusion equation to perform numerical solutions on a discretized three-dimensional spatial grid, simulating the spatial diffusion trajectory and concentration decay process of pathogen-containing aerosols under the constraint of hilly terrain, and outputs the pathogen exposure concentration distribution of each spatial grid node at each time. The three-dimensional terrain data includes the surface elevation, slope aspect, and valley orientation of the area where the pigsty is located. The three-dimensional terrain data serves as the boundary condition for the aerosol diffusion simulation unit.

7. The multimodal digital twin early warning method for swine diseases in hilly areas according to claim 6, characterized in that, The host susceptibility assessment unit receives the fusion feature representation and the age information of individual pigs. After weighting the fusion feature representation with an adjustable weight vector, it performs a difference operation with the susceptibility threshold parameter of the corresponding age group, and then outputs the individual susceptibility score through the Sigmoid function mapping. The herd immunity status aggregation unit receives the susceptibility scores and pathogen exposure concentration distributions of each individual. It then performs an individual-by-individual product operation on the susceptibility scores of each individual in the same or adjacent pig houses and the pathogen exposure concentration at the corresponding spatial location, and takes the average value to obtain the herd transmission risk score. In the aggregation process, historical vaccination record data is also introduced. The time interval between each pig's most recent vaccination is used to perform an exponential decay calculation using the vaccine immunization efficacy decay rate constant to obtain the immunization decay coefficient. The product of the immunization decay coefficient and the individual susceptibility score is used to replace the individual susceptibility score in the aggregation calculation of the group transmission risk score.

8. The multimodal digital twin early warning method for swine diseases in hilly areas according to claim 1, characterized in that, After obtaining the fusion feature representation, the following is also included: The fused feature representations from multiple consecutive acquisition times are arranged in chronological order to form a feature representation time series; The feature representation is input temporally into the temporal coding layer, which is a single-layer gated recurrent unit, and outputs dynamic fused features containing temporal evolution information. In subsequent steps, dynamic fusion features are used instead of fusion feature representations in the analysis.

9. The multimodal digital twin early warning method for swine diseases in hilly areas according to claim 1, characterized in that, It also includes the step of updating the parameters of the spatiotemporal dynamic disease evolution model based on veterinary diagnosis feedback data: Obtain veterinary diagnostic feedback data, which includes at least one of the following: serological test results, PCR nucleic acid test results, and pathological autopsy records; The warning level and abnormal indicators in the warning information are compared with the veterinary diagnosis feedback data to obtain the warning accuracy label. The warning accuracy label and the standardized multimodal feature data at the corresponding time are used to form a training sample pair. For each veterinary diagnosis feedback data, the number of detection methods is marked. When the number of detection methods included in the veterinary diagnosis feedback data of the same early warning event reaches a preset threshold, the corresponding training sample pair is marked as a high-confidence sample, and only the high-confidence sample participates in the parameter update calculation. An incremental learning approach is adopted, which corrects the parameter bias caused by newly added training sample pairs while retaining the existing parameters, and performs gradient updates on the adjustable parameters of the host susceptibility assessment unit and the herd immunity status aggregation unit.

10. A multimodal digital twin early warning system for swine diseases in hilly areas, used to execute the multimodal digital twin early warning method for swine diseases in hilly areas according to any one of claims 1 to 9, characterized in that, include: The multimodal data acquisition module is used to simultaneously acquire surface temperature distribution data, movement posture sequence data and respiratory acoustic data of the target pig herd through multimodal acquisition devices deployed in the pig house, and acquire environmental parameter data in the pig house through environmental sensor group, and align the data of each channel according to a unified timestamp to form a multimodal raw dataset; The preprocessing module is used to preprocess and standardize the original multimodal dataset to obtain standardized multimodal feature data; The feature extraction module is used to input standardized multimodal feature data into the multimodal joint feature extraction network and output fused feature representation. The multimodal joint feature extraction network adopts a lightweight Transformer architecture. The risk assessment module is used to input the fusion feature representation and hilly area geographical environment data into the spatiotemporal dynamic disease evolution model for joint analysis to obtain the disease transmission risk assessment results. The spatiotemporal dynamic disease evolution model includes a pathogen environmental survival estimation unit, an aerosol diffusion simulation unit, a host susceptibility assessment unit, and a herd immunity status aggregation unit. The early warning generation module is used to compare the risk level of group transmission with the preset early warning threshold based on the risk assessment results of disease transmission. When the risk level of group transmission reaches or exceeds the early warning threshold, an early warning message is generated.

Images