Pet dog health monitoring necklace based on multi-modal data fusion and early warning method

Abstract

The invention relates to the technical field of pet health monitoring, and discloses a pet dog health monitoring necklace based on multi-modal data fusion and an early warning method, and the method comprises the steps: collecting multi-modal original data with aligned time sequences; performing anomaly correction on the multi-modal original data, and outputting a corrected multi-modal data set; performing multi-modal feature layered extraction according to the corrected multi-modal data set, wherein the multi-modal features comprise physiological-environment features, behavior time sequence features and discrete behavior features; carrying out multi-modal feature fusion under an attention mechanism to obtain a fusion feature vector; inputting a preset classification model, and outputting a health state label of the pet dog according to a fuzzy logic rule optimization decision; and performing health state early warning in combination with preset early warning output logic. According to the pet dog health monitoring system and method, the comprehensiveness, accuracy and real-time performance of pet dog health monitoring are effectively improved through integration of physiological, behavior and environment multi-mode data of the pet dog and through preprocessing, layered feature extraction, attention mechanism fusion and intelligent recognition real-time grading early warning.

Description

Technical Field

[0001] This invention relates to the field of pet health monitoring technology, and in particular to a pet dog health monitoring collar and early warning method based on multimodal data fusion. Background Technology

[0002] As dogs play an increasingly important role as companion animals, their health management has become a core concern for pet-owning families. Since dogs cannot directly express physical discomfort verbally, early health abnormalities are often difficult to detect in time. Delayed intervention can lead to mild conditions developing into severe illnesses, increasing the difficulty and cost of treatment.

[0003] Existing pet health monitoring methods have significant limitations: traditional methods rely on the owner's subjective observation, which is greatly affected by differences in experience and easily misses early signals such as subtle changes in gait and abnormal breathing rhythms; single-sensor devices can only collect physiological indicators (such as heart rate and body temperature) or behavioral data (such as activity level), resulting in limited data dimensions and failing to reflect the correlation between physiological state, behavioral patterns, and environmental factors. For example, elevated body temperature may be related to excessive heat in the environment or disease infection, which is difficult to distinguish based on a single data point; while some systems attempt to collect multiple data points, they lack effective fusion algorithms, simply splicing or analyzing the data independently, failing to explore the complementary value of multimodal data, leading to a high rate of false alarms; furthermore, most solutions rely on cloud computing, which lacks real-time capability and cannot meet the need for immediate response to health abnormalities.

[0004] Therefore, there is an urgent need for technology that can integrate multi-dimensional data and achieve accurate analysis and real-time early warning through intelligent fusion algorithms to solve the problems of insufficient comprehensiveness, low accuracy, and delayed response of existing monitoring methods, and to provide more reliable protection for the health of pet dogs. Summary of the Invention

[0005] This invention provides a pet dog health monitoring collar and early warning method based on multimodal data fusion. By integrating multimodal data on pet dogs' physiology, behavior and environment, and through preprocessing, hierarchical feature extraction, attention mechanism fusion and intelligent recognition, it provides real-time graded early warning, effectively improving the comprehensiveness, accuracy and real-time nature of pet dog health monitoring.

[0006] This invention provides a method for monitoring and early warning of pet dog health based on multimodal data fusion, comprising:

[0007] S1. Collect physiological data and behavioral data of the pet dog, as well as environmental data of the environment in which the pet dog is located, and synchronize the physiological data, behavioral data, and environmental data in time to obtain time-aligned multimodal raw data;

[0008] S2. Perform anomaly correction on the original multimodal data and output the corrected multimodal dataset;

[0009] S3. Perform multimodal feature layer extraction based on the modified multimodal dataset, including physiological-environment feature extraction, behavioral temporal feature extraction, and discrete behavioral feature extraction;

[0010] S4. The physiological-environmental features, behavioral temporal features, and discrete behavioral features are fused using a multimodal feature fusion mechanism under attention to obtain a fused feature vector;

[0011] S5. Input the fused feature vector into a preset classification model, and optimize the decision output of the pet dog's health status label according to preset fuzzy logic rules; wherein, the health status label includes normal, warning, and abnormal;

[0012] S6. Provide a health status warning based on the pet dog's health status tag and the preset warning output logic.

[0013] Furthermore, S1 specifically includes:

[0014] S101. Set the sampling frequency of the physiological sensor group, behavioral sensor group and environmental sensor group, use a sliding time window mechanism to achieve data synchronization, and ensure the alignment of the timestamps of the multi-sensor data through clock synchronization.

[0015] S102. Collect multimodal raw data of the pet dog, and perform format standardization processing on the collected multimodal raw data to convert it into a set format. The set format includes sensor type, collection timestamp, data value, and device ID.

[0016] S103. Generate a multimodal raw dataset D = {HR(t), T(t), R(t), G(t), A(t), E(t), H(t), L(t)}, where t is the time variable, and T(t), R(t), G(t), A(t), E(t), H(t), and L(t) are the heart rate, body temperature, respiratory rate, gait cycle, activity intensity, eating behavior, ambient temperature and humidity, and light intensity in the multimodal raw data, respectively.

[0017] Furthermore, S2 specifically includes:

[0018] S201. Use the box plot method to identify outliers in the multimodal raw dataset D, calculate the quartiles of each modality, including the lower quartile q1 and the upper quartile q3, and calculate the interquartile range Δ = q3 - q1. Define the outlier detection boundary as L. down =q1-1.5Δ,L up =q3+1.5Δ, when the data value When this happens, mark it as an outlier;

[0019] S202. The PSO-RBF neural network algorithm is used to correct the labeled outliers, specifically as follows:

[0020] A radial basis function (RBF) neural network is constructed, with the input layer consisting of adjacent normal data within the window containing outliers, and the output layer consisting of the corrected data values.

[0021] The center vector c of the RBF network is optimized using the Particle Swarm Optimization (PSO) algorithm. i Width σ i and weight w i The optimization objective is to minimize the prediction error. Among them, y k For the true value, Here, m represents the network prediction value, and m is the number of training samples.

[0022] The outliers are replaced with the output values ​​of the optimized PSO-RBF model to generate the corrected multimodal dataset D′.

[0023] Furthermore, S3 specifically includes:

[0024] S301. A stacked autoencoder is used to learn features from the physiological data HR, T, R and environmental data H, L in the corrected multimodal dataset D′; wherein the autoencoder contains 3 hidden layers, and the reconstruction error is minimized through unsupervised training. Where, x i For input data, To reconstruct the data, N is the number of samples; the output of the intermediate layer is taken as the physiological-environmental feature vector F1;

[0025] S302. The behavioral temporal data G and A in the corrected multimodal dataset D′ are processed using a bidirectional long short-term memory network (BiLSTM). The BiLSTM includes a forward LSTM layer and a backward LSTM layer. It captures the dynamic patterns of behavior through temporal modeling and outputs a behavioral temporal feature vector F2.

[0026] S303. Perform statistical analysis on the eating behavior E in the corrected multimodal dataset D′, extract statistics including daily eating frequency, duration of a single eating session, and standard deviation of the eating interval, and construct a discrete behavioral feature vector F3.

[0027] Furthermore, S4 specifically includes:

[0028] S401. The physiological-environmental feature vector F1, behavioral temporal feature vector F2, and discrete behavioral feature vector F3 are transformed to the same high-dimensional space through a linear mapping layer; wherein, the mapping formula is:

[0029]

[0030] Among them, W k Let b be the mapping weight matrix. k It is the bias vector;

[0031] S402. Construct an attention mechanism model based on the aforementioned physiological-environmental features. Let Q be the query vector, and Q be the behavioral features. Calculate the weights of each feature, given the key vectors K1 and K2 and the value vectors V1 and V2:

[0032]

[0033] Where β is the trainable scaling factor, d is the feature dimension, and QK T It is the dot product of vectors;

[0034] S403. Generate the fusion feature vector F = α1·V1 + α2·V2.

[0035] Furthermore, S5 specifically includes:

[0036] S501. Define health status labels and their triggering conditions. The health status labels include normal (N), warning (W), and abnormal (A). Among them, a normal status is defined when all multimodal features are within the historical normal range, a warning status is defined when a single modal feature exceeds the threshold but does not meet the pathological standard, and an abnormal status is defined when at least two modal features exceed the pathological threshold.

[0037] S502. A Deep Belief Network (DBN) is used as the classification model. The input layer is a fused feature vector F, the hidden layers contain 64 and 32 neurons, and the output layer is the probability distribution of health status labels. The classification model uses a cross-entropy loss function combined with a regularization term. The loss function expression is:

[0038]

[0039] Among them, L ce For cross-entropy loss, y i For real labels, For the predicted probability, M is the number of samples, and λ1 and λ2 are regularization coefficients;

[0040] S503. Introduce fuzzy logic rules to optimize decision-making, specifically including:

[0041] Conservative rule: only when If at least two modal features are abnormal, it is judged as abnormal A;

[0042] Mild rules: When Furthermore, when a single modal feature is abnormal, it is determined as a warning (W);

[0043] Radical rule: When physiological characteristics HR and T exceed the pathological threshold and If so, it is directly judged as an anomaly A.

[0044] Furthermore, in S6, the warning output logic is as follows:

[0045] If the status is normal (N), the terminal device records the data and does not trigger an alert.

[0046] If the status is Warning W, the terminal device will push a text prompt, including abnormal characteristics and suggestions;

[0047] If the status is abnormal (A), an audible and visual alarm will be triggered, and the terminal device will push an emergency warning and synchronize the data to the cloud-based veterinary service platform.

[0048] This invention also provides a pet dog health monitoring collar based on multimodal data fusion. Based on the pet dog health monitoring and early warning method based on multimodal data fusion as described above, it includes a main control module, a sensor module, a communication module, an edge computing module, and a collar carrier. The main control module, sensor module, communication module, and edge computing module are all integrated in the collar carrier. The main control module is connected to the sensor module, communication module, and edge computing module respectively. The communication module is also connected to a terminal device.

[0049] The main control module uses a low-power microprocessor to coordinate the timing synchronization and preliminary data processing of the sensor module, communication module and edge computing module.

[0050] The sensor module includes a physiological sensor group, a behavioral sensor group, and an environmental sensor group. The physiological sensor group integrates a PPG optical heart rate sensor, a contact body temperature sensor, and a respiratory rate sensor to collect the pet dog's heart rate (HR), body temperature (T), and respiratory rate (R). The behavioral sensor group integrates a triaxial accelerometer and a gyroscope to capture the pet dog's gait period (G), activity intensity (A), and eating behavior (E). The environmental sensor group integrates a temperature and humidity sensor and a light sensor to record the ambient temperature and humidity (H) and light intensity (L).

[0051] The communication module is used to transmit the collected multimodal raw data to the terminal device;

[0052] The edge computing module is used to run steps S1-S7 in real time to achieve early warning of the pet dog's health status.

[0053] The present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described method.

[0054] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the above-described method.

[0055] The beneficial effects of this invention are as follows:

[0056] This invention integrates physiological indicators, behavioral characteristics, and environmental information of pet dogs, breaking through the limitations of single data dimensions to achieve comprehensive capture of health-related information. Through synchronous data preprocessing and outlier correction, it ensures data integrity and reliability, providing a high-quality foundation for subsequent analysis. By fusing hierarchical feature extraction and attention mechanisms, it effectively uncovers the intrinsic relationships between multimodal data, improving the effectiveness of feature representation. Combining a health status recognition model with fuzzy logic rules improves the accuracy of health status judgment, precisely distinguishing between normal, warning, and abnormal states. Relying on edge computing, it achieves real-time monitoring and tiered early warning output, ensuring immediate response to health anomalies. This not only promptly alerts to early potential health risks but also provides rapid intervention support for emergency situations, comprehensively improving the comprehensiveness, accuracy, and real-time nature of pet dog health monitoring, providing reliable technical support for pet dog health management. Attached Figure Description

[0057] Figure 1 This is a flowchart illustrating the pet dog health monitoring and early warning method based on multimodal data fusion according to the present invention.

[0058] Figure 2 This is a schematic diagram of the structure of the pet dog health monitoring collar based on multimodal data fusion according to the present invention.

[0059] Figure 3 This is a schematic diagram of the internal structure of the computer device in this invention.

[0060] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0061] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0062] like Figure 1 As shown, this invention provides a method for monitoring and early warning of pet dog health based on multimodal data fusion, including:

[0063] S1. Collect physiological and behavioral data of the pet dog, as well as environmental data of the pet dog's environment, and synchronize the physiological, behavioral, and environmental data in time to obtain time-aligned multimodal raw data; specifically including:

[0064] S101. Set the sampling frequencies of the physiological sensor group, behavioral sensor group, and environmental sensor group (e.g., 100Hz, 50Hz, and 20Hz respectively). Use a sliding time window mechanism to achieve data synchronization. Set the window length to 10s and the window overlap rate to 50%. Ensure the alignment of the timestamps of the multi-sensor data through the clock synchronization function of the main control module (clock error ≤ 1ms).

[0065] S102. Collect multimodal raw data of the pet dog and perform format standardization processing on the collected multimodal raw data to convert it into a set format (such as JSON format). The set format includes sensor type, collection timestamp, data value, and device ID.

[0066] S103. Generate a multimodal raw dataset D = {HR(t), T(t), R(t), G(t), A(t), E(t), H(t0, L(t)}, where t is the time variable (unit: s), and T(t), R(t), G(t), A(t), E(t), H(t0), and L(t) represent heart rate, body temperature, respiratory rate, gait cycle, activity intensity, eating behavior, ambient temperature and humidity, and light intensity, respectively, from the multimodal raw data. Finally, transmit the raw dataset D to local storage on the terminal device and cloud backup via the communication module, storing it in a time-series database format.

[0067] S2. Perform anomaly correction on the original multimodal data and output the corrected multimodal dataset; specifically including:

[0068] S201. Use the box-plot method to identify outliers in the multimodal raw dataset D. Calculate the quartiles of each modality, including the lower quartile q1 and the upper quartile q3, and calculate the interquartile range Δ = q3 - q1. Define the outlier detection boundary as L. down =q1-1.5Δ,L up =q3+1.5Δ, when the data value When this happens, mark it as an outlier;

[0069] S202. The PSO-RBF neural network algorithm is used to correct the labeled outliers, specifically as follows:

[0070] A radial basis function (RBF) neural network is constructed. The input layer consists of adjacent normal data (dimension n=5) of the window where outliers are located, and the output layer consists of the corrected data values.

[0071] The center vector c of the RBF network is optimized using the Particle Swarm Optimization (PSO) algorithm. i Width σ i and weight w i The optimization objective is to minimize the prediction error. Among them, yk For the true value, Here, m represents the network prediction value, and m is the number of training samples.

[0072] The outliers are input into the optimized PSO-RBF model, and the output values ​​are used to replace the original outliers to generate the corrected multimodal dataset D′.

[0073] S3. Perform multimodal feature layering extraction based on the modified multimodal dataset, including physiological-environment feature extraction, behavioral temporal feature extraction, and discrete behavioral feature extraction; specifically including:

[0074] S301, Extract physiological-environmental feature vector F1

[0075] A stacked autoencoder was used to learn features from the corrected multimodal dataset D′, specifically physiological data (HR, T, R) and environmental data (H, L). The autoencoder contained three hidden layers, and the reconstruction error was minimized through unsupervised training. Where, x i For input data, To reconstruct the data, N is the number of samples; the output of the intermediate layer is taken as the physiological-environmental feature vector F1; specifically,

[0076] Using physiological data (heart rate HR, body temperature T, respiratory rate R) and environmental data (ambient temperature and humidity H, light intensity L) from the corrected multimodal dataset D′ as input, feature learning is performed through a stacked autoencoder, specifically including:

[0077] (1) Model construction: The stacked autoencoder consists of an input layer, three hidden layers, and an output layer. The input layer has the same dimension as the input data (5-dimensional, corresponding to 5 types of physiological-environmental parameters); the number of neurons in the hidden layers is 128, 64, and 32 respectively, and nonlinearity is introduced through the ReLU activation function; the output layer has the same dimension as the input layer (5-dimensional) and is used to reconstruct the input data.

[0078] (2) Unsupervised training: The model is trained in an unsupervised manner with the goal of minimizing the reconstruction error. The formula for calculating the reconstruction error is:

[0079]

[0080] Where, x i For the i-th input sample (physiological-environmental data window), The model reconstructs the output, where N is the total number of training samples. The weights and biases of each layer are iteratively optimized using the backpropagation algorithm until the reconstruction error converges.

[0081] (3) Feature output: After training, the output layer is removed and the output of the intermediate hidden layer (64 neurons) is taken as the physiological-environment feature vector F1 with a dimension of 64. This vector contains abstract features such as heart rate variability, body temperature fluctuation trend, and the correlation between environmental factors and physiological state.

[0082] S302, Extract behavioral temporal feature vector F2

[0083] A bidirectional long short-term memory (BiLSTM) network is used to process the behavioral temporal data G and A in the corrected multimodal dataset D′. The BiLSTM network comprises forward LSTM layers and backward LSTM layers, capturing dynamic behavioral patterns through temporal modeling and outputting a behavioral temporal feature vector F2. Specifically,

[0084] For the behavioral temporal data (gait period G, activity intensity A) in the corrected multimodal dataset D′, a bidirectional long short-term memory network (BiLSTM) is used to capture temporal dynamic features, specifically including:

[0085] (1) Data preprocessing: The time series data of gait period G and activity intensity A are divided into continuous subsequences by sliding window. The length of each subsequence is the number of sampling points in the window (based on a sampling frequency of 50Hz, a 10s window contains 500 data points), forming a two-dimensional input matrix (number of samples × sequence length × 2, where 2 corresponds to the two parameters G and A).

[0086] (2) BiLSTM model construction: The model consists of a forward LSTM layer and a backward LSTM layer, each containing 32 neurons. The forward LSTM layer processes the subsequence in chronological order (from the beginning to the end of the window) to capture historical behavioral patterns; the backward LSTM layer processes the subsequence in reverse chronological order (from the end to the beginning of the window) to capture future behavioral trends.

[0087] (3) Feature fusion and output: The outputs of the forward and backward LSTM layers are fused by splicing (32+32=64 dimensions), and output as a behavioral temporal feature vector F2 with a dimension of 64 through a fully connected layer (64 neurons, ReLU activation). This vector contains temporal information such as gait stability, activity intensity change rate, and behavioral mode switching features.

[0088] S303. Perform statistical analysis on the feeding behavior E in the corrected multimodal dataset D′, extract statistics including daily feeding frequency, duration of a single feeding, and standard deviation of feeding intervals, to construct a discrete behavioral feature vector F3; specifically,

[0089] For the discrete behavioral data (eating behavior E) in the corrected multimodal dataset D′, quantitative features are extracted through statistical analysis, specifically including:

[0090] (1) Behavioral event labeling: Based on the abrupt change characteristics of accelerometer and gyroscope data (such as the frequency of head-down movement and the amplitude of neck posture changes), the start and end timestamps of a single feeding event are identified and labeled in the corrected multimodal dataset D′, generating a feeding event sequence E={e1,e2,…,e m}, where e k This represents the duration of the k-th meal.

[0091] (2) Statistical feature calculation: Time dimension statistics are performed on the feeding event sequence, and 16 features are extracted, including: total number of daily feedings, mean / maximum / minimum / standard deviation of single feeding duration, mean / standard deviation of adjacent feeding intervals, proportion of feedings in specific time periods of the day (such as morning / noon / evening), and deviation rate of feeding duration from the previous day.

[0092] (3) Feature vector construction: Arrange the above 16 statistical features in a preset order to form a discrete behavioral feature vector F3 with a dimension of 16. This vector reflects the regularity and abnormal fluctuations of the pet dog's eating behavior.

[0093] S4. The physiological-environmental features, behavioral temporal features, and discrete behavioral features are fused using a multimodal feature fusion mechanism under an attention mechanism to obtain a fused feature vector; specifically including:

[0094] S401. The physiological-environmental feature vector F1 (64-dimensional), the behavioral temporal feature vector F2 (64-dimensional), and the discrete behavioral feature vector F3 (64-dimensional) are transformed to the same high-dimensional space (128-dimensional) through a linear mapping layer; wherein, the mapping formula is:

[0095]

[0096] Among them, W k The trainable linear mapping weight matrix (dimension 128×dim(F)) k ), dim(F k (64, 64, 16 respectively), b k The bias vector (128 dimensions); These are the mapped physiological-environmental characteristics, behavioral temporal characteristics, and discrete behavioral characteristics, respectively.

[0097] S402. Definition of Core Parameters and Dynamic Weight Calculation of Attention Mechanism

[0098] (1) Construct an attention model guided by physiological-environmental features, and define the core vector roles as follows:

[0099] Query vector (Q): Retrieved physiological-environmental features after mapping As a query vector, it is directly related to the core physiological state of pet dogs (such as heart rate and body temperature), and is the basis for health judgment.

[0100] Key vector (K): Takes the mapped behavioral temporal features respectively Discrete behavioral characteristics As key vectors K1 and K2, they are used to characterize the strength of the association between behavioral characteristics and physiological states;

[0101] Value vector (V): corresponds one-to-one with the key vector, taking... As value vectors V1 and V2, they are the actual information carriers of behavioral characteristics.

[0102] (2) Measure the similarity between the query vector and the key vector using the vector inner product, and adjust the weight sensitivity using a trainable scaling factor. The specific steps are as follows:

[0103] Similarity calculation: Calculate the inner product of the query vector Q and the key vectors K1 and K2 to quantify the correlation between physiological and behavioral features. Among them, K T For vector transpose, a larger inner product indicates a higher degree of correlation.

[0104] Scaling and Normalization: A scaling factor β (initially 1.0, dynamically adjusted during model training) is introduced to enhance weight discriminative power. The similarity is normalized to weights using the softmax function.

[0105]

[0106] α2=1-α1

[0107] Where d = 128 is the dimension of the high-dimensional space. This is used to avoid gradient vanishing due to excessively large inner product results; α1 and α2 are the attention weights for temporal and discrete behavioral features, respectively, satisfying α1 + α2 = 1.

[0108] S403. Based on the calculated attention weights, the value vectors are weighted and summed to generate the final fused feature vector F = α1·V1 + α2·V2.

[0109] The fusion process dynamically adjusts the weights (e.g., when a pet dog has an abnormal body temperature, the correlation between physiological feature Q and behavioral temporal feature K1 increases, and α1 automatically increases), so that the fused feature F can retain the core information of physiological-environmental features and highlight the behavioral features most relevant to the current health status, providing a more discriminative input for health status identification.

[0110] S5. Input the fused feature vector into a preset classification model, and optimize the decision according to preset fuzzy logic rules to output the health status label of the pet dog; wherein, the health status label includes normal, warning, and abnormal; specifically including:

[0111] S501. Define health status labels and their triggering conditions. The health status labels include Normal (N), Warning (W), and Abnormal (A). Normal is defined when all multimodal features are within the historical normal range (thresholds determined through statistical analysis of healthy samples); Warning is defined when a single modal feature exceeds the threshold but does not meet pathological standards; and Abnormal is defined when at least two modal features exceed the pathological threshold (referencing pet medical standards). Specifically,

[0112] Obtain the fusion feature vector F, where each feature corresponds to the fusion result of multimodal data over a continuous monitoring period (e.g., 1 hour), forming a sample set {F1, F2, ..., F...}. N}, where N is the total number of samples. Based on the pet dog's health records (including veterinary diagnostic reports and owner observation logs) and physiological-behavioral baseline thresholds (determined through statistical analysis of historical data from healthy pet dogs), each sample is labeled:

[0113] Normal (N): Physiological indicators (heart rate, body temperature, etc.) and behavioral characteristics (activity intensity, eating patterns, etc.) corresponding to fusion characteristic F are all within the healthy baseline range, and there are no abnormalities diagnosed by veterinary physicians;

[0114] Warning (W): A single modality characteristic (such as a sudden drop in activity intensity of more than 30% but normal physiological indicators, or a body temperature slightly higher than the baseline but no abnormal behavior) is outside the healthy range but does not meet the pathological criteria;

[0115] Abnormal (A): At least two modal characteristics (such as persistently high heart rate + sudden decrease in food intake, or elevated body temperature + abnormal respiratory rate) exceed the pathological threshold (refer to the clinical standards of veterinary medicine), or are diagnosed by a veterinarian as being in a sick state.

[0116] S502. A Deep Belief Network (DBN) is used as the classification model. The input layer is a fused feature vector F (128 dimensions), the hidden layers contain 64 and 32 neurons, and the output layer is the probability distribution of three health status labels. The classification model uses a cross-entropy loss function combined with a regularization term. The loss function expression is:

[0117]

[0118] Among them, L ce For cross-entropy loss, y i For real labels, To predict probabilities, M is the sample size, and λ1 and λ2 are regularization coefficients; specifically,

[0119] (1) The deep belief network model adopts a multi-layer nonlinear structure, specifically including:

[0120] Input layer: Receives fused features F, with 128 neurons (consistent with the dimension of F);

[0121] Hidden layers: Contains two fully connected hidden layers, with 64 neurons in the first layer and 32 neurons in the second layer. Both layers use the LeakyReLU activation function (slope 0.01) to enhance the non-linear representation of features.

[0122] Output layer: 3 neurons, corresponding to three states: normal (N), warning (W), and abnormal (A). The softmax activation function is used to output the probability distribution for each state. in and

[0123] (2) Define the composite loss function: To balance classification accuracy and feature reconstruction consistency, a composite loss function is adopted, which includes cross-entropy loss, reconstruction error regularization term, and weight decay term:

[0124]

[0125] in, For cross-entropy loss, y i Let i be the true label (one-hot encoded form). M represents the predicted probability of the model, and M is the batch size of the samples used to optimize classification accuracy. For reconstruction error, The input features are reconstructed by the model through the hidden layer, and λ1 = 0.01 is the regularization coefficient used to constrain the stability of the feature representation. λ2 is the sum of the L2 norms of all network weight matrices, and λ2 = 0.001 is the weight decay coefficient used to prevent the model from overfitting.

[0126] (3) Divide the sample set into a training set (for parameter updates), a validation set (for hyperparameter tuning), and a test set (for final performance evaluation) in a 7:2:1 ratio. The Adam optimizer is used with an initial learning rate of 0.001, which decays to 0.5 every 50 epochs. The batch size is set to 32. The total number of training epochs is set to 200, and an early stopping strategy is adopted (training is terminated if the validation set loss does not decrease for 10 consecutive epochs).

[0127] A batch of samples is randomly drawn from the training set and input into the DBN model. The predicted probabilities are then calculated through forward propagation. and reconstruction features Calculate the composite loss L, and update the network weights W and bias b using the backpropagation algorithm (chain rule); evaluate the model performance (accuracy, F1 score) on the validation set every 10 epochs, and dynamically adjust the learning rate; after training, verify the model's generalization ability on the test set to ensure that the recognition accuracy of the three states of normal / warning / abnormal is ≥90%.

[0128] S503. To improve the model's ability to distinguish edge cases, fuzzy logic rules are introduced to correct the model output. The specific rules are as follows:

[0129] Conservative rule: If the model predicts anomaly probability If at least two modalities (such as physiological and behavioral) in the corresponding original features exceed the pathological threshold, then it is finally judged as abnormal (A);

[0130] Mild rule: If the model predicts the probability of a warning If only a single modality exceeds the health threshold (e.g., only abnormal activity level), it will be ultimately judged as a warning (W);

[0131] Radical rule: If core physiological characteristics (heart rate, body temperature) exceed pathological thresholds, and the model predicts an abnormal probability... It is then directly classified as an anomaly (A), and is not subject to other feature constraints.

[0132] Through the above training process, the model can output accurate health status judgment results based on the fused feature F, providing a reliable reasoning basis for real-time monitoring and early warning.

[0133] S6. Provide a health status warning based on the pet dog's health status tag and the preset warning output logic.

[0134] The edge computing module receives the corrected multimodal dataset D′, performs feature extraction and feature fusion sequentially, and inputs it into the deep belief network model to obtain the real-time health status.

[0135] Early warning output logic:

[0136] a. If the status is normal (N): The terminal device records data and does not trigger an alert;

[0137] b. If the status is Warning (W): The terminal device (mobile APP) pushes a text prompt (including abnormal characteristics and suggestions, such as the activity intensity has decreased by 40% compared with yesterday, and it is recommended to increase interaction);

[0138] c. If the status is abnormal (A): The hardware system triggers an audible and visual alarm (buzzer frequency 2kHz, LED flashing cycle 1s), and at the same time, the terminal device pushes an emergency warning (including real-time data and historical trend charts) and synchronizes the data to the cloud veterinary service platform.

[0139] This invention achieves real-time monitoring and graded early warning of pet dogs' health status through deep fusion of multimodal data and intelligent reasoning, thereby improving the accuracy of early identification of health abnormalities.

[0140] This invention also provides a pet dog health monitoring collar based on multimodal data fusion. Based on the pet dog health monitoring and early warning method based on multimodal data fusion as described above, it includes a main control module, a sensor module, a communication module, an edge computing module, and a collar carrier. The main control module, sensor module, communication module, and edge computing module are all integrated in the collar carrier. The main control module is connected to the sensor module, communication module, and edge computing module respectively. The communication module is also connected to a terminal device.

[0141] The main control module uses a low-power microprocessor to coordinate the timing synchronization and preliminary data processing of the sensor module, communication module and edge computing module.

[0142] The sensor module includes a physiological sensor group, a behavioral sensor group, and an environmental sensor group. The physiological sensor group integrates a PPG optical heart rate sensor, a contact body temperature sensor, and a respiratory rate sensor to collect the pet dog's heart rate (HR), body temperature (T), and respiratory rate (R). The behavioral sensor group integrates a triaxial accelerometer and a gyroscope to capture the pet dog's gait period (G), activity intensity (A), and eating behavior (E). The environmental sensor group integrates a temperature and humidity sensor and a light sensor to record the ambient temperature and humidity (H) and light intensity (L).

[0143] The communication module is used to transmit the collected multimodal raw data to the terminal device;

[0144] The edge computing module is used to run steps S1-S7 in real time to achieve early warning of the pet dog's health status.

[0145] The collar carrier is made of food-grade silicone, with a weight controlled between 15-20g (suitable for small to large dogs). The outer shell is waterproof (IP67) and tear-resistant, and a 200mAh lithium polymer battery is used.

[0146] like Figure 3 As shown, the present invention also provides a computer device, which may be a server, and its internal structure may be as follows: Figure 3As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides the environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores all data required for the process of a pet dog health monitoring and early warning method based on multimodal data fusion. The network interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements the pet dog health monitoring and early warning method based on multimodal data fusion.

[0147] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer equipment on which the present application is applied.

[0148] An embodiment of this application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements any one of the above-described methods for monitoring and early warning of pet dog health based on multimodal data fusion.

[0149] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media provided in this application and in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual-speed SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0150] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0151] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for monitoring and early warning of pet dog health based on multimodal data fusion, characterized in that, include: S1. Collect physiological data and behavioral data of the pet dog, as well as environmental data of the environment in which the pet dog is located, and synchronize the physiological data, behavioral data, and environmental data in time to obtain time-aligned multimodal raw data; S2. Perform anomaly correction on the original multimodal data and output the corrected multimodal dataset; S3. Perform multimodal feature layer extraction based on the modified multimodal dataset, including physiological-environment feature extraction, behavioral temporal feature extraction, and discrete behavioral feature extraction; S4. The physiological-environmental features, behavioral temporal features, and discrete behavioral features are fused using a multimodal feature fusion mechanism under attention to obtain a fused feature vector; S5. Input the fused feature vector into a preset classification model, and optimize the decision output of the pet dog's health status label according to preset fuzzy logic rules; wherein, the health status label includes normal, warning, and abnormal; S6. Provide a health status warning based on the pet dog's health status tag and the preset warning output logic.

2. The method for monitoring and early warning of pet dog health based on multimodal data fusion according to claim 1, characterized in that, S1 specifically includes: S101. Set the sampling frequency of the physiological sensor group, behavioral sensor group and environmental sensor group, use a sliding time window mechanism to achieve data synchronization, and ensure the alignment of the timestamps of the multi-sensor data through clock synchronization. S102. Collect multimodal raw data of the pet dog, and perform format standardization processing on the collected multimodal raw data to convert it into a set format. The set format includes sensor type, collection timestamp, data value, and device ID. S103. Generate a multimodal raw dataset D = {HR(t), T(t), R(t), G(t), A(t), E(t), H(t), L(t)}, where t is the time variable, and T(t), R(t), G(t), A(t), E(t), H(t), and L(t) are the heart rate, body temperature, respiratory rate, gait cycle, activity intensity, eating behavior, ambient temperature and humidity, and light intensity in the multimodal raw data, respectively.

3. The method for monitoring and early warning of pet dog health based on multimodal data fusion according to claim 2, characterized in that, S2 specifically includes: S201. Use the box plot method to identify outliers in the multimodal raw dataset D, calculate the quartiles of each modality, including the lower quartile q1 and the upper quartile q3, and calculate the interquartile range Δ = q3 - q1. Define the outlier detection boundary as L. down =q1-1.5Δ,L up =q3+1.5Δ, when the data value When this happens, mark it as an outlier; S202. The PSO-RBF neural network algorithm is used to correct the labeled outliers, specifically as follows: A radial basis function (RBF) neural network is constructed, with the input layer consisting of adjacent normal data within the window containing outliers, and the output layer consisting of the corrected data values. The center vector c of the RBF network is optimized using the Particle Swarm Optimization (PSO) algorithm. i Width σ i and weight w i The optimization objective is to minimize the prediction error. Among them, y k For the true value, Here, m represents the network prediction value, and m is the number of training samples. The outliers are replaced with the output values ​​of the optimized PSO-RBF model to generate the corrected multimodal dataset D′.

4. The method for monitoring and early warning of pet dog health based on multimodal data fusion according to claim 3, characterized in that, S3 specifically includes: S301. A stacked autoencoder is used to learn features from the physiological data HR, T, R and environmental data H, L in the corrected multimodal dataset D′; wherein the autoencoder contains 3 hidden layers, and the reconstruction error is minimized through unsupervised training. Where, x i For input data, To reconstruct the data, N is the number of samples; the output of the intermediate layer is taken as the physiological-environmental feature vector F1; S302. The behavioral temporal data G and A in the corrected multimodal dataset D′ are processed using a bidirectional long short-term memory network (BiLSTM). The BiLSTM includes a forward LSTM layer and a backward LSTM layer. It captures the dynamic patterns of behavior through temporal modeling and outputs a behavioral temporal feature vector F2. S303. Perform statistical analysis on the eating behavior E in the corrected multimodal dataset D′, extract statistics including daily eating frequency, duration of a single eating session, and standard deviation of the eating interval, and construct a discrete behavioral feature vector F3.

5. The method for monitoring and early warning of pet dog health based on multimodal data fusion according to claim 4, characterized in that, S4 specifically includes: S401. The physiological-environmental feature vector F1, behavioral temporal feature vector F2, and discrete behavioral feature vector F3 are transformed to the same high-dimensional space through a linear mapping layer; wherein, the mapping formula is: Among them, W k Let b be the mapping weight matrix. k It is the bias vector; S402. Construct an attention mechanism model based on the aforementioned physiological-environmental features. Let Q be the query vector, and Q be the behavioral features. Calculate the weights of each feature, given the key vectors K1 and K2 and the value vectors V1 and V2: α2＝1-α1 Where β is the trainable scaling factor, d is the feature dimension, and QK T It is the dot product of vectors; S403. Generate the fusion feature vector F = α1·V1 + α2·V2.

6. The method for monitoring and early warning of pet dog health based on multimodal data fusion according to claim 5, characterized in that, S5 specifically includes: S501. Define health status labels and their triggering conditions. The health status labels include normal (N), warning (W), and abnormal (A). Among them, a normal status is defined when all multimodal features are within the historical normal range, a warning status is defined when a single modal feature exceeds the threshold but does not meet the pathological standard, and an abnormal status is defined when at least two modal features exceed the pathological threshold. S502. A Deep Belief Network (DBN) is used as the classification model. The input layer is a fused feature vector F, the hidden layers contain 64 and 32 neurons, and the output layer is the probability distribution of health status labels. The classification model uses a cross-entropy loss function combined with a regularization term. The loss function expression is: Among them, L ce For cross-entropy loss, y i For real labels, For the predicted probability, M is the number of samples, and λ1 and λ2 are regularization coefficients; S503. Introduce fuzzy logic rules to optimize decision-making, specifically including: Conservative rule: only when If at least two modal features are abnormal, it is judged as abnormal A; Mild rules: When Furthermore, when a single modal feature is abnormal, it is determined as a warning (W); Radical rule: When physiological characteristics HR and T exceed the pathological threshold and If so, it is directly judged as an anomaly A.

7. The method for monitoring and early warning of pet dog health based on multimodal data fusion according to claim 6, characterized in that, In S6, the warning output logic is as follows: If the status is normal (N), the terminal device records the data and does not trigger an alert. If the status is Warning W, the terminal device will push a text prompt, including abnormal characteristics and suggestions; If the status is abnormal (A), an audible and visual alarm will be triggered, and the terminal device will push an emergency warning and synchronize the data to the cloud-based veterinary service platform.

8. A pet dog health monitoring collar based on multimodal data fusion, characterized in that, The pet dog health monitoring and early warning method based on multimodal data fusion according to any one of claims 1-7 includes a main control module, a sensor module, a communication module, an edge computing module, and a collar carrier. The main control module, sensor module, communication module, and edge computing module are all integrated in the collar carrier. The main control module is connected to the sensor module, communication module, and edge computing module respectively. The communication module is also connected to a terminal device. The main control module uses a low-power microprocessor to coordinate the timing synchronization and preliminary data processing of the sensor module, communication module and edge computing module. The sensor module includes a physiological sensor group, a behavioral sensor group, and an environmental sensor group. The physiological sensor group integrates a PPG optical heart rate sensor, a contact body temperature sensor, and a respiratory rate sensor to collect the pet dog's heart rate (HR), body temperature (T), and respiratory rate (R). The behavioral sensor group integrates a triaxial accelerometer and a gyroscope to capture the pet dog's gait period (G), activity intensity (A), and eating behavior (E). The environmental sensor group integrates a temperature and humidity sensor and a light sensor to record the ambient temperature and humidity (H) and light intensity (L). The communication module is used to transmit the collected multimodal raw data to the terminal device; The edge computing module is used to run steps S1-S7 in real time to achieve early warning of the pet dog's health status.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images