Multi-source data fusion system

Abstract

The invention discloses a multi-source data fusion system, which comprises a data acquisition module, a data processing module, a data fusion module, a wireless transmission module, a display and early warning module and a remote server, the data acquisition module is used for acquiring data information in a farm through camera equipment and a sensor; the data processing module is used for receiving and processing the collected data information; the data fusion module is used for fusing the data information processed by the data processing module, analyzing the fused data information and generating an analysis result; and the display and early warning module is used for displaying the analysis result and judging whether an abnormal condition exists or not. The accuracy of pig breeding monitoring is improved, and the misjudgment rate caused by data instability of a single sensor is reduced. Real-time monitoring and informatization management of pig breeding are achieved, and the breeding efficiency and the management level are improved. The system has good expansibility, and the types and the number of sensors can be increased or decreased according to actual requirements.

Description

Technical Field

[0001] The invention relates to the technical field of animal husbandry, and in particular to a multi-source data fusion system. Background Art

[0002] With the modernization of animal husbandry, the information management of pig farming has gradually become a trend. Traditional farming methods rely more on manual experience, and there are problems such as untimely information collection, low data accuracy, and low management efficiency. In order to improve the management efficiency of pig farming, sensor technology has been introduced into this field to monitor the physiological state of pigs and their environmental conditions in real time. Multi-sensor data fusion is to obtain the desired goals with higher probability and accuracy by combining multi-source information distributed in a certain space, and to evaluate events more accurately to obtain useful decision-making information.

[0003] As an important technology in the field of object detection, the object detection model (Faster RCNN) has been widely recognized for its efficiency and accuracy. In the field of object detection, traditional methods are usually divided into two stages: generating candidate regions and classifying these regions. However, these methods have a slow processing speed, which limits the possibility of real-time applications. R-CNN achieves object detection by extracting candidate regions of fixed size in an image and then classifying each candidate region. Although R-CNN performs well in accuracy, its processing speed is very slow. To solve this problem, the Faster R-CNN algorithm came into being.

[0004] However, due to the limitations of technical instability and environmental noise, a single sensor often leads to deviations between observed data and actual conditions, which in turn affects the accuracy of decision-making. Therefore, applying multi-sensor data fusion technology to pig farming monitoring can improve the reliability and accuracy of data, thereby optimizing pig farming management.

[0005] Regarding farm environment detection, the remote control device for farm environment disclosed in the prior art (publication number: CN205229867U) discloses remote collection of environmental information in the farm, but it can only monitor and improve the farming environment, and cannot perceive the health of the farmed organisms. When the farmed organisms have physiological or pathological problems, it is impossible to obtain the problem situation in time to treat the farmed organisms. Regarding the health perception of farmed organisms, a monitoring and early warning system based on multi-source big data animal breeding data mining (publication number: CN114463701B) disclosed in the prior art discloses judging and scoring the health status of farmed organisms by obtaining the vital signs information and sounds of the farmed organisms. However, it only collects and analyzes the position, body temperature, amount of exercise, etc. of the farmed organisms, and cannot accurately locate the range of the diseased parts of the farmed organisms. Manual judgment and positioning are required again. In addition, it is easily affected by the noise and instability of single sensor data, which leads to errors in health status analysis. The system lacks scalability and cannot achieve the effect of high-precision remote management. It is difficult to flexibly add new sensors or functional modules according to actual needs. The data processing method is relatively simple, mainly relying on the YoLov4 neural network, and does not fully utilize multi-source data information. Summary of the invention

[0006] In order to solve the above problems, the present invention provides a multi-source data fusion system, which is characterized by comprising a data acquisition module, a data processing module, a data fusion module, a wireless transmission module, a display and warning module, and a remote server;

[0007] The data acquisition module is used to collect data information in the farm through camera equipment and sensors; the data processing module is used to receive and process the collected data information, wherein the method for processing the data information includes: data denoising, data filtering, and data normalization; the data fusion module is used to fuse the data information processed by the data processing module for analysis and generate analysis results; the display and early warning module is used to display the analysis results and determine whether there are abnormal conditions; the wireless transmission module is used to upload the analysis results to a remote server for storage and send abnormal information to staff; the remote server is used to store and manage historical analysis results; and also includes a mobile terminal APP, which is used by management personnel to monitor the health status and environmental conditions of the farmed organisms in real time, and receive alarm information in time when the system detects abnormal conditions;

[0008] The data acquisition module specifically includes a physiological information acquisition submodule and an environmental information acquisition submodule;

[0009] The physiological information collection submodule is used to collect physiological information of the cultured organisms; the environmental information collection submodule is used to collect information about the culture environment; the physiological information collected by the physiological information collection submodule specifically includes: heart rate information, body temperature information, audio information, weight information, and body shape information; the information collected by the environmental information collection submodule specifically includes: environmental temperature information, environmental humidity information, ammonia concentration information, carbon dioxide concentration information, and mosquito quantity information; wherein the audio information is specifically the call information of the cultured organisms;

[0010] The body posture information is specifically collected by a camera device, and key points are extracted and analyzed;

[0011] The key points specifically include: the left ear of the farmed creature, the right ear of the farmed creature, the nose of the farmed creature, the center of the left forelimb scapula of the farmed creature, the intersection of the ulna and the brachial bone of the left forelimb of the farmed creature, the intersection of the wrist and the arm of the left forelimb of the farmed creature, the center of the left forelimb system of the farmed creature, the center of the right forelimb scapula of the farmed creature, the intersection of the ulna and the brachial bone of the right forelimb of the farmed creature, the intersection of the wrist and the arm of the right forelimb of the farmed creature, the center of the right forelimb system of the farmed creature, the left hind limb of the farmed creature, The position of the connection between the hip bone and femur of the left hind limb of the farmed organism, the junction of the femur and tibia of the left hind limb of the farmed organism, the connection between the fibula and sacrum of the left hind limb of the farmed organism, the center position of the left hind limb system of the farmed organism, the position of the connection between the hip bone and femur of the right hind limb of the farmed organism, the junction of the femur and tibia of the right hind limb of the farmed organism, the position of the connection between the fibula and sacrum of the right hind limb of the farmed organism, the center position of the right hind limb system of the farmed organism, the back of the farmed organism, the midpoint of the waist of the farmed organism, and the midpoint of the lower abdomen of the farmed organism.

[0012] Furthermore, the remote server includes a historical data analysis submodule, and the historical data analysis submodule is used to analyze the historical analysis result data and optimize the breeding management.

[0013] Furthermore, the display and warning module is provided with an audible and visual alarm and a threshold alarm mechanism. The audible and visual alarm is used to immediately sound an alarm to notify surrounding staff when the system detects an abnormal situation; the threshold alarm mechanism is used to push the alarm information to the mobile terminal of the manager through the wireless transmission module when the system identifies a situation that exceeds the set threshold.

[0014] Furthermore, the method for processing data information in the data processing module specifically includes: A1. Receiving the physiological information and breeding environment information of the farmed organisms collected by the data acquisition module; A2. Using the target detection model to perform target detection on the farmed organisms in the collected image data, marking the farmed organisms detected in each frame, and generating a corresponding bounding box; A3. Using a Kalman filter and a particle filter combined algorithm to track and detect the movement trajectory of the farmed organisms.

[0015] Furthermore, the Kalman filter and particle filter combination algorithm in A3 specifically includes: Kalman filter; particle filter;

[0016] The Kalman filter specifically includes:

[0017] ① Set the initial state vector of each farmed organism, where the state vector is defined as: In the formula, x and y represent the current position of the cultured organism, v x 、v y Both represent the speed of the cultured organisms;

[0018] ② The Kalman filter motion model is used to predict the state of the farmed organisms at the next moment. The specific calculation formula is: t+1 =AX t +Bu t +w t , where X t+1 Indicates the state of the cultivated organism at the next moment, X t is the state vector, A is the state transfer matrix, B is the control input matrix, u t represents the control input vector, w t represents the noise term;

[0019] ③ The new position information collected by the data acquisition module is used as the observation value Z t , and the state is updated through the Kalman gain K, and its calculation formula specifically includes: K=P t H T (HP t H T +R) -1 ;X t =X t +K(Z t -HX t ), where H is the observation matrix, which is used to transform the state vector X t Mapped to the observation space, R is the measurement noise covariance matrix, which is used to describe the size of the sensor measurement noise;

[0020] The particle filter specifically includes:

[0021] a) The initial position of each farmed organism is represented by a particle swarm, each particle is a hypothesis of a position, and a system dynamic model is established;

[0022] b) Based on the system dynamic model, each particle is randomly sampled and its position is updated to predict the state of the particle at the next moment. The calculation formula is: in, represents the state vector of the ith particle at time t+1, Represents the state transfer equation, which is used to describe the current state and control input u t How does the particle transfer to the next state? represents process noise;

[0023] c) According to the data Z collected by the data acquisition module t The weight of each particle is calculated based on the degree of match with the particle state: in Represents particles The degree of agreement with the observed values;

[0024] d) Resampling is performed according to the weights of the particles, particles with large weights and close to the true state are retained, particles with small weights are filtered out, and the final estimated position of each farmed organism is obtained by weighted averaging.

[0025] Furthermore, the analysis after fusion of the data information in the data fusion module is specifically: extracting features from the data information processed by the data processing module, and then fusing and analyzing the extracted feature information to generate analysis results.

[0026] Furthermore, the extracted characteristic information specifically includes: the change pattern of the heart rate of the farmed organisms, the fluctuation range of the body temperature of the farmed organisms, the effective audio information of the farmed organisms, the body shape information of the farmed organisms, the weight change of the farmed organisms, the body temperature of the farmed organisms, the ambient temperature, the ambient humidity, the ambient ammonia concentration, the ambient carbon dioxide concentration, and the number of mosquitoes in the environment.

[0027] Furthermore, the feature extraction of the body shape information of the cultured organisms is specifically: bottom-up analysis of the heat map of key points is used to extract image features and body shape key points; the feature extraction of the number of environmental mosquitoes is specifically: multiple groups of environmental photos are collected by a camera device and pre-processed to obtain a single-channel grayscale image, and then binarization and similarity calculation are performed and mathematical statistical analysis is used to obtain the number of environmental mosquitoes.

[0028] Furthermore, the data fusion of the extracted feature information is specifically: using an extended Kalman filter algorithm to fuse the feature information, wherein the extended Kalman filter algorithm specifically includes:

[0029] Nonlinear state update: The nonlinear equation is linearized by Taylor expansion to predict the state, where the calculation formula for state prediction is: X t+1 =f(X t ,u t )+w t , where X t+1 represents the state vector at time t+1, f(X t ,u t ) represents the state transfer function, which is a nonlinear transfer equation used to describe the transition from the current state X t and control input u t Calculate the relationship to the next state, control input u t Describes the external influence on the system state change, w t represents process noise;

[0030] Observation update: Linearize the observation equation through Taylor expansion: Z t =h(X t )+v t , and adjust the predicted state by updating the observed value, where Z t is the observation value vector at time t, representing the measurement result obtained from the data acquisition module, h(X t ) is a nonlinear observation function used to describe the state vector X t Calculate the observed value Z t , v t is the observation noise, which represents the uncertainty generated in the observation process;

[0031] Through the weighted update mechanism of Kalman filtering, the data information is weighted and fused, and the weights are updated. The fused estimate for: Among them, w i is the weight of the collected data i, Z i is the observed value of the collected data i; the update of the weight is determined by the confidence of the collected data, which is determined by the residual Calculation, the weight update calculation formula is: In the formula, ε is a constant.

[0032] The present invention provides a multi-source data fusion system, which has the following beneficial effects:

[0033] The present invention improves the accuracy of pig farming monitoring and reduces the misjudgment rate caused by the instability of single sensor data. It realizes real-time monitoring and information management of pig farming, and improves farming efficiency and management level. The system has good scalability and can increase or decrease the type and number of sensors according to actual needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0035] Figure 1 A block diagram of the system structure provided by the present invention;

[0036] Figure 2 This is a schematic diagram of the system structure provided by the present invention. DETAILED DESCRIPTION

[0037] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0038] The following is a detailed description of the implementation method of the present invention in conjunction with the accompanying drawings. Only some embodiments are described, not all embodiments. For the purpose of clarity, representations and descriptions that are not related to the present invention are omitted in the drawings and descriptions.

[0039] In order to have a clearer understanding of the technical features, purposes and beneficial effects of the present invention, the technical solution of the present invention is now described in detail below. Obviously, the implementation cases described are part of the embodiments of the present invention, not all of the embodiments, and cannot be understood as limiting the scope of the implementation of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the protection scope of the present invention.

[0040] like Figure 1 , Figure 2As shown, the present invention provides a multi-source data fusion system, including a data acquisition module, a data processing module, a data fusion module, a wireless transmission module, a display and early warning module, and a remote server; the data acquisition module is used to collect data information in a farm through a camera and a sensor; the data processing module is used to receive and process the collected data information, wherein the method for processing the data information includes: data denoising, data filtering, and data normalization; the data fusion module is used to fuse the data information processed by the data processing module and analyze it, and generate an analysis result; the display and early warning module is used to display the analysis result and determine whether there is an abnormal situation; the wireless transmission module is used to upload the analysis result to the remote server for storage, and send the abnormal information to the staff; the remote server is used to store and manage historical analysis results.

[0041] It also includes a mobile APP, which is used by management personnel to monitor the health status and environmental conditions of the cultured organisms in real time, and receive alarm information in time when the system detects an abnormal situation.

[0042] Among them, the data acquisition module specifically includes a physiological information acquisition submodule and an environmental information acquisition submodule; the physiological information acquisition submodule is used to collect physiological information of the cultured organisms; the environmental information acquisition submodule is used to collect information on the culture environment.

[0043] The physiological information collected by the physiological information collection submodule specifically includes: heart rate information, body temperature information, audio information, weight information, and body shape information; the information collected by the environmental information collection submodule specifically includes: environmental temperature information, environmental humidity information, ammonia concentration information, carbon dioxide concentration information, and mosquito quantity information. Among them, the audio information specifically refers to the call information of the farmed organisms.

[0044] The body posture information is collected through camera equipment, and the key points are extracted and analyzed:

[0045] The key points include: the left ear of the farmed creature, the right ear of the farmed creature, the nose of the farmed creature, the center of the scapula of the left forelimb of the farmed creature, the intersection of the ulna and arm bone of the left forelimb of the farmed creature, the intersection of the wrist and arm of the left forelimb of the farmed creature, the center of the left forelimb system of the farmed creature, the center of the scapula of the right forelimb of the farmed creature, the intersection of the ulna and arm bone of the right forelimb of the farmed creature, the intersection of the wrist and arm of the right forelimb of the farmed creature, the center of the right forelimb system of the farmed creature, the left hind limb of the farmed creature The location of the connection between the hip bone and the femur, the junction of the femur and tibia of the left hind limb of the farmed creature, the connection between the fibula and the sacrum of the left hind limb of the farmed creature, the center of the left hind limb system of the farmed creature, the location of the connection between the hip bone and the femur of the right hind limb of the farmed creature, the junction of the femur and tibia of the right hind limb of the farmed creature, the location of the connection between the fibula and the sacrum of the right hind limb of the farmed creature, the center of the right hind limb system of the farmed creature, the back of the farmed creature, the midpoint of the waist of the farmed creature, and the midpoint of the lower abdomen of the farmed creature.

[0046] The remote server includes a historical data analysis submodule, which is used to analyze the historical analysis result data and optimize the breeding management.

[0047] The display and warning module is equipped with an audible and visual alarm and a threshold alarm mechanism. The audible and visual alarm is used to immediately sound an alarm to notify surrounding staff when the system detects an abnormal situation; the threshold alarm mechanism is used to push the alarm information to the mobile terminal of the manager through the wireless transmission module when the system identifies a situation that exceeds the set threshold.

[0048] The method for processing data information in the data processing module specifically includes: A1. Receiving the physiological information and breeding environment information of the farmed organisms collected by the data acquisition module; A2. Using the target detection model to detect the farmed organisms in the collected image data, marking the farmed organisms detected in each frame, and generating the corresponding bounding box; A3. Using the Kalman filter and particle filter combined algorithm to track and detect the movement trajectory of the farmed organisms.

[0049] The Kalman filter and particle filter combination algorithm in A3 specifically includes: Kalman filter; particle filter;

[0050] The Kalman filter specifically includes:

[0051] ① Set the initial state vector of each farmed organism, where the state vector is defined as: In the formula, x and y represent the current position of the cultured organism, v x 、v y Both represent the speed of the cultured organisms.

[0052] ② The Kalman filter motion model is used to predict the state of the farmed organisms at the next moment. The specific calculation formula is: t+1 =AX t +Bu t +w t , where X t+1 Indicates the state of the cultivated organism at the next moment, X t is the state vector, A is the state transfer matrix, B is the control input matrix, u t represents the control input vector, w t represents the noise term.

[0053] ③ The new position information collected by the data acquisition module is used as the observation value Z t , and the state is updated through the Kalman gain K, and its calculation formula specifically includes: K=P t H T (HP t H T +R) -1 ;X t =X t +K(Z t -HX t ), where H is the observation matrix, which is used to transform the state vector X t Mapped to the observation space, R is the measurement noise covariance matrix, which is used to describe the size of the sensor measurement noise.

[0054] The particle filter specifically includes:

[0055] a) The initial position of each farmed organism is represented by a particle swarm, each particle is a hypothesis of a position, and a system dynamic model is established.

[0056] b) Based on the system dynamic model, each particle is randomly sampled and its position is updated to predict the state of the particle at the next moment. The calculation formula is: in, represents the state vector of the ith particle at time t+1, Represents the state transfer equation, which is used to describe the current state and control input u t How does the particle transfer to the next state? represents process noise.

[0057] c) According to the data Z collected by the data acquisition module t The weight of each particle is calculated based on the degree of match with the particle state: in Represents particles The degree of agreement with the observed values.

[0058] d) Resampling is performed according to the weights of the particles, particles with large weights and close to the true state are retained, particles with small weights are filtered out, and the final estimated position of each farmed organism is obtained by weighted averaging.

[0059] After completing multi-target tracking, the health status of the farmed organisms is determined by estimating their movement status, body shape changes and other physiological information. Physiological information such as heart rate, body temperature and behavioral characteristics can be collected by other sensors. Based on the tracking results of Kalman filtering or particle filtering, the physiological data collected by the sensor (heart rate, body temperature, weight changes, etc.)

[0060] Movement pattern analysis: Analyze the movement pattern of the farmed organisms based on their movement trajectory, such as whether there are abnormal behaviors (such as long periods of stillness, abnormally intense activities). Body posture information extraction: Use key points in the image to extract the body posture information of the farmed organisms and determine their posture changes, such as whether there are standing, lying on the side, etc. Heart rate / body temperature analysis: Compare the heart rate, body temperature and other data collected by the sensor with the normal range to estimate the health status.

[0061] The data fusion module performs analysis after fusion of data information, specifically: extracting features from the data information processed by the data processing module, and then fusing and analyzing the extracted feature information to generate analysis results.

[0062] Among them, the extracted characteristic information specifically includes: the changing pattern of the heart rate of the farmed organisms, the fluctuation range of the body temperature of the farmed organisms, the effective audio information of the farmed organisms, the body shape information of the farmed organisms, the weight changes of the farmed organisms, the body temperature of the farmed organisms, the ambient temperature, the ambient humidity, the ambient ammonia concentration, the ambient carbon dioxide concentration, and the number of mosquitoes in the environment.

[0063] The feature extraction of the body shape information of the cultured organisms is specifically as follows: bottom-up analysis is performed on the heat map of key points to extract image features and body shape key points; the feature extraction of the number of environmental mosquitoes is specifically as follows: multiple groups of environmental photos are collected by a camera device and pre-processed to obtain a single-channel grayscale image, and then binarization and similarity calculation are performed to obtain the number of environmental mosquitoes using mathematical statistical analysis.

[0064] The data fusion of the extracted feature information is specifically: using the extended Kalman filter algorithm to fuse the feature information, wherein the extended Kalman filter algorithm specifically includes:

[0065] Nonlinear state update: The nonlinear equation is linearized by Taylor expansion to predict the state, where the calculation formula for state prediction is: X t+1 =f(X t ,u t )+w t, where X t+1 represents the state vector at time t+1, f(X t ,u t ) represents the state transfer function, which is a nonlinear transfer equation used to describe the transition from the current state X t and control input u t Calculate the relationship to the next state, control input u t Describes the external influence on the system state change, w t represents process noise;

[0066] Observation update: Linearize the observation equation through Taylor expansion: Z t =h(X t )+v t , and adjust the predicted state by updating the observed value, where Z t is the observation value vector at time t, representing the measurement result obtained from the data acquisition module, h(X t ) is a nonlinear observation function used to describe the state vector X t Calculate the observed value Z t , v t is the observation noise, which represents the uncertainty generated in the observation process;

[0067] Through the weighted update mechanism of Kalman filtering, the data information is weighted and fused, and the weights are updated. The fused estimate for: Among them, w i is the weight of the collected data i, Z i is the observed value of the collected data i; the update of the weight is determined by the confidence of the collected data, which is determined by the residual Calculation, the weight update calculation formula is: In the formula, ε is a constant.

[0068] When the sensor collects signals, there is uncertainty data. To solve this problem, multiple input variables (such as body temperature, heart rate, motion status, etc.) are set, and each input variable is fuzzified through a fuzzy membership function.

[0069] Assume that the fuzzy set of input variable x is "low", "medium", and "high", and its fuzzy membership function is:

[0070]

[0071] Among them, x1, x2, and x3 are threshold points. Decisions are made through fuzzy reasoning rules, and the health level of the farmed organisms is finally output. Multimodal data fusion and tracking are performed through target detection model + Kalman filter + particle filter + extended Kalman filter. Then, the observation values ​​of different sensors are weighted and fused using extended Kalman filter, and decisions are made through fuzzy reasoning rules. Finally, the health status of the farmed organisms is output, which improves the accuracy of health status estimation.

[0072] Example: Install multiple sensors in a pig farm, including temperature sensors, humidity sensors, heart rate sensors, audio and video sensors, ammonia concentration sensors, etc. The data collected by the sensors is transmitted to the central processing system through a wireless network, and Kalman filtering is used for data fusion processing, and the processed data is uploaded to a remote server. Farm managers monitor the health status and environmental conditions of pigs in real time through mobile APP, and receive alarm information in time when the system detects abnormalities.

[0073] The physiological information collection module includes a heart rate sensor, a body temperature sensor, an audio sensor, a weight sensor and a camera. The heart rate sensor is used to detect the heart rate of the pig, the body temperature sensor is used to detect the body temperature of the pig, the audio sensor is used to collect the pig's call signal, the weight sensor is used to monitor the weight change of the pig, and the camera is used to identify the pig's body shape.

[0074] Among them, in terms of pig call recognition, the collected pig call signals are first input into the data processing module. After preprocessing, multiple segments of valid audio signals can be obtained. Then the valid audio signals are input into the feature extraction module. After MFCC feature parameter extraction, time regularization and valid segment standardization, a 10*9*48 feature matrix can be obtained. Based on the feature matrix, the 3D deep residual neural network can be used to identify the type of pig calls and output them, realizing non-contact, real-time and intelligent pig call and health monitoring; in terms of pig posture recognition, the pig behavior recognition technology based on neural network is adopted, and the bottom-up method is used to comprehensively analyze the heatmap of key points to extract image features and posture key points. Based on the pig skeleton, it can be used as the main reference for unique identification. For the image of the target pig, 22 key points of the whole body are extracted, and the coordinate information of the key points can be obtained: the left ear of the pig, the right ear of the pig, the front end of the pig nose, the center of the scapula of the left forelimb of the pig, the intersection of the ulna and arm of the left forelimb of the pig, the intersection of the wrist and arm of the left forelimb of the pig, the center of the left forelimb system of the pig, the center of the scapula of the right forelimb of the pig, the intersection of the ulna and arm of the right forelimb of the pig, the intersection of the wrist and arm of the right forelimb of the pig, the center of the right forelimb system of the pig, the position of the connection between the hip bone and femur of the left hind limb of the pig, the junction point of the femur and tibia of the left hind limb of the pig, the connection between the fibula and the hip bone of the left hind limb of the pig, the center position of the left hind limb system of the pig, the position of the connection between the hip bone and femur of the right hind limb of the pig, the junction point of the femur and tibia of the right hind limb of the pig, the position of the connection between the fibula and the hip bone of the right hind limb of the pig, the center position of the right hind limb system of the pig, the midpoint of the pig back, the waist and the midpoint of the pig's lower abdomen.

[0075] Environmental information collection modules are deployed at different locations in the farm, including temperature sensors, humidity sensors, ammonia concentration sensors, carbon dioxide concentration sensors and cameras. The temperature sensor is used to monitor the temperature of the breeding environment, the humidity sensor is used to monitor the humidity level, the ammonia concentration sensor is used to detect the ammonia concentration in the air, the carbon dioxide concentration sensor is used to detect the carbon dioxide concentration in the air, and the camera is used to identify mosquitoes and count their number.

[0076] In terms of mosquito identification, a fixed camera is used to collect 20 pictures in a group, and each group of pictures is input into the data processing module. After picture preprocessing, an enhanced single-channel grayscale image is obtained, and then binarization and similarity calculation are performed. Based on the similarity value, mathematical statistical analysis can be used to obtain the number of mosquitoes.

[0077] The data collected by each sensor is transmitted to the data processing module through the wireless communication module. The data processing module integrates the data preprocessing module, feature extraction module and data fusion module. In the data preprocessing module, the collected data is denoised to filter out abnormal data caused by sensor errors and environmental noise. The feature extraction module extracts useful feature parameters from the preprocessed data, such as the fluctuation trend of heart rate, changes in body temperature, fluctuations in environmental humidity, etc. Subsequently, the data fusion module uses Kalman filtering to fuse the output data of each sensor to generate more accurate health status and environmental status information.

[0078] The data processed by the data processing module is transmitted to the remote server through the communication interface, so that the farm managers can view the health status of the pigs and the monitoring data of the breeding environment in real time on the mobile APP. In addition, the display screen included in the display and alarm module will display the current monitoring data. If the system detects an abnormal situation, such as the pig's body temperature exceeds the normal range or the ammonia concentration exceeds the standard, the sound and light alarm will immediately sound an alarm. The threshold alarm mechanism set in the system can automatically identify situations that exceed the set value, and promptly push alarm information to the manager's mobile APP through the wireless network, so that the manager can take immediate measures to avoid losses caused by pig health problems or environmental deterioration.

[0079] The present invention improves the accuracy of pig farming monitoring and reduces the misjudgment rate caused by the instability of single sensor data. It realizes real-time monitoring and information management of pig farming, and improves farming efficiency and management level. The system has good scalability and can increase or decrease the type and number of sensors according to actual needs.

[0080] The above is only a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be modified within the scope of the concept described herein through the above teachings or the technology or knowledge of the relevant field. The changes and modifications made by those skilled in the art shall not deviate from the spirit and scope of the present invention, and shall be within the scope of protection of the claims attached to the present invention.

Claims

1. A multi-source data fusion system, characterized in that: It includes data acquisition module, data processing module, data fusion module, wireless transmission module, display and warning module, and remote server; The data acquisition module is used to collect data information in the farm through camera equipment and sensors; The data processing module is used to receive and process the collected data information, wherein the method of processing the data information includes: data denoising, data filtering, and data normalization; The data fusion module is used to fuse the data information processed by the data processing module for analysis and generate analysis results; The display and warning module is used to display the analysis results and determine whether there are abnormal conditions; The wireless transmission module is used to upload the analysis results to the remote server for storage, and send abnormal information to the staff; The remote server is used to store and manage historical analysis results; It also includes a mobile APP, which is used by managers to monitor the health status and environmental conditions of the farmed organisms in real time and receive alarm information in a timely manner when the system detects an abnormal situation; The data acquisition module specifically includes a physiological information acquisition submodule and an environmental information acquisition submodule; The physiological information collection submodule is used to collect physiological information of the cultured organisms; the environmental information collection submodule is used to collect information about the culture environment; The physiological information collected by the physiological information collection submodule specifically includes: heart rate information, body temperature information, audio information, weight information, and body shape information; the information collected by the environmental information collection submodule specifically includes: environmental temperature information, environmental humidity information, ammonia concentration information, carbon dioxide concentration information, and mosquito quantity information; wherein the audio information specifically includes the call information of the cultured organisms; The body posture information is specifically collected by a camera device, and key points are extracted and analyzed; The key points specifically include: the left ear of the farmed creature, the right ear of the farmed creature, the nose of the farmed creature, the center of the left forelimb scapula of the farmed creature, the intersection of the ulna and the brachial bone of the left forelimb of the farmed creature, the intersection of the wrist and the arm of the left forelimb of the farmed creature, the center of the left forelimb system of the farmed creature, the center of the right forelimb scapula of the farmed creature, the intersection of the ulna and the brachial bone of the right forelimb of the farmed creature, the intersection of the wrist and the arm of the right forelimb of the farmed creature, the center of the right forelimb system of the farmed creature, the left hind limb of the farmed creature, The position of the connection between the hip bone and femur of the left hind limb of the farmed organism, the junction of the femur and tibia of the left hind limb of the farmed organism, the connection between the fibula and sacrum of the left hind limb of the farmed organism, the center position of the left hind limb system of the farmed organism, the position of the connection between the hip bone and femur of the right hind limb of the farmed organism, the junction of the femur and tibia of the right hind limb of the farmed organism, the position of the connection between the fibula and sacrum of the right hind limb of the farmed organism, the center position of the right hind limb system of the farmed organism, the back of the farmed organism, the midpoint of the waist of the farmed organism, and the midpoint of the lower abdomen of the farmed organism.

2. The multi-source data fusion system according to claim 1, characterized in that: The remote server includes a historical data analysis submodule, which is used to analyze the historical analysis result data and optimize the breeding management.

3. The multi-source data fusion system according to claim 1, characterized in that: The display and early warning module is provided with an audible and visual alarm and a threshold alarm mechanism. The audible and visual alarm is used to immediately sound an alarm to notify surrounding staff when the system detects an abnormal situation; the threshold alarm mechanism is used to push the alarm information to the mobile terminal of the manager through the wireless transmission module when the system identifies a situation that exceeds the set threshold.

4. The multi-source data fusion system according to claim 1, characterized in that: The method for processing data information in the data processing module specifically includes: A1. Receive the physiological information of the cultured organisms and the culture environment information collected by the data collection module; A2. Use the target detection model to detect the farmed organisms in the collected image data, mark the farmed organisms detected in each frame, and generate a corresponding bounding box; A3. Use Kalman filter and particle filter combined algorithm to track and detect the movement trajectory of the cultured organisms.

5. The multi-source data fusion system according to claim 4, characterized in that: The Kalman filter and particle filter combination algorithm in A3 specifically includes: Kalman filter; particle filter; The Kalman filter specifically includes: ① Set the initial state vector of each farmed organism, where the state vector is defined as: In the formula, x and y represent the current position of the cultured organism, v x 、v y Both represent the speed of the cultured organisms; ② The Kalman filter motion model is used to predict the state of the farmed organisms at the next moment. The specific calculation formula is: t+1 =AX t +Bu t +w t , where X t+1 Indicates the state of the cultivated organism at the next moment, X t is the state vector, A is the state transfer matrix, B is the control input matrix, u t represents the control input vector, w t represents the noise term; ③ The new position information collected by the data acquisition module is used as the observation value Z t , and the state is updated through the Kalman gain K, and its calculation formula specifically includes: K=P t H T (HP t H T +R) -1 ;X t =X t +K(Z t -HX t ), where H is the observation matrix, which is used to transform the state vector X t Mapped to the observation space, R is the measurement noise covariance matrix, which is used to describe the size of the sensor measurement noise; The particle filter specifically includes: a) The initial position of each farmed organism is represented by a particle swarm, each particle is a hypothesis of a position, and a system dynamic model is established; b) Based on the system dynamic model, each particle is randomly sampled and its position is updated to predict the state of the particle at the next moment. The calculation formula is: in, represents the state vector of the ith particle at time t+1, Represents the state transfer equation, which is used to describe the current state and control input u t How does the particle transfer to the next state? represents process noise; c) According to the data Z collected by the data acquisition module t The weight of each particle is calculated based on the degree of match with the particle state: in Represents particles The degree of agreement with the observed values; d) Resampling is performed according to the weights of the particles, particles with large weights and close to the true state are retained, particles with small weights are filtered out, and the final estimated position of each farmed organism is obtained by weighted averaging.

6. The multi-source data fusion system according to claim 1, characterized in that: The analysis after fusion of the data information in the data fusion module is specifically: extracting features from the data information processed by the data processing module, and then fusing and analyzing the extracted feature information to generate analysis results.

7. The multi-source data fusion system according to claim 6, characterized in that: The extracted characteristic information specifically includes: the change pattern of the heart rate of the farmed organism, the fluctuation range of the body temperature of the farmed organism, the effective audio information of the farmed organism, the body shape information of the farmed organism, the weight change of the farmed organism, the body temperature of the farmed organism, the ambient temperature, the ambient humidity, the ambient ammonia concentration, the ambient carbon dioxide concentration, and the number of mosquitoes in the environment.

8. The multi-source data fusion system according to claim 7, characterized in that: The feature extraction of the body shape information of the cultured organisms is specifically as follows: bottom-up analysis is performed on the heat map of key points to extract image features and body shape key points; The feature extraction of the number of mosquitoes in the environment is specifically as follows: multiple groups of environmental photos are collected by a camera device and pre-processed to obtain a single-channel grayscale image, and then binarization and similarity calculation are performed and mathematical statistical analysis is used to obtain the number of mosquitoes in the environment.

9. The multi-source data fusion system according to claim 6, characterized in that: The data fusion of the extracted feature information is specifically: using an extended Kalman filter algorithm to fuse the feature information, wherein the extended Kalman filter algorithm specifically includes: Nonlinear state update: The nonlinear equation is linearized by Taylor expansion to predict the state, where the calculation formula for state prediction is: X t+1 =f(X t ,u t )+w t , where X t+1 represents the state vector at time t+1, f(X t ,u t ) represents the state transfer function, which is a nonlinear transfer equation used to describe the transition from the current state X t and control input u t Calculate the relationship to the next state, control input u t Describes the external influence on the system state change, w t represents process noise; Observation update: Linearize the observation equation through Taylor expansion: Z t =h(X t )+v t , and adjust the predicted state by updating the observed value, where Z t is the observation value vector at time t, representing the measurement result obtained from the data acquisition module, h(X t ) is a nonlinear observation function used to describe the state vector X t Calculate the observed value Z t , v t is the observation noise, which represents the uncertainty generated in the observation process; Through the weighted update mechanism of Kalman filtering, the data information is weighted and fused, and the weights are updated. The fused estimate for: Among them, w i is the weight of the collected data i, Z i is the observed value of the collected data i; the update of the weight is determined by the confidence of the collected data, which is determined by the residual Calculation, the weight update calculation formula is: In the formula, ε is a constant.

Images