Poultry hatching intelligent management system based on multi-modal data fusion

Abstract

The application discloses a kind of poultry hatching intelligent management system based on multi-modal data fusion, through environmental monitoring module acquisition temperature, humidity, CO2 concentration and so on multidimensional parameter, after sliding average filtering pretreatment, combine variety specificity environment parameter library, pass through time sequence dynamic neural network and construct multi-parameter coupling model, output hatching window span time;Sound recognition module uses multi-source noise adaptive suppression algorithm to extract calling signal, after time domain, frequency domain feature extraction, through CNN+LSTM model accurately identifies the hatching stage;Image processing module acquires infrared image, after denoising, enhancement pretreatment, utilize YOLO model detection chick, statistics hatching quantity and proportion;Hatching management module is based on the output of the first three, through reinforcement learning dynamic weight distribution algorithm, adapt to different stages of data value in whole cycle of incubation, intelligently identify earliest hatching poultry, accurately predict hatching time, optimize batch push row.

Description

Technical Field

[0001] This invention relates to the field of poultry hatching management, and specifically to an intelligent poultry hatching management system based on multimodal data fusion. Background Technology

[0002] Current poultry hatching management technologies still face significant technical bottlenecks and application shortcomings. On the one hand, the industry almost universally relies on manual inspection to monitor hatching. This involves farmers observing chicks hatching at fixed times and locations, counting hatched chicks, assessing their condition, and adjusting the temperature and humidity of incubation equipment. This method is not only labor-intensive and costly, but also susceptible to subjective experience, fatigue, and ambient light, leading to misjudgments and missed detections. It fails to achieve real-time, precise, and dynamic monitoring of the hatching process. On the other hand, existing technologies lack a deep understanding of the differences in data value across different stages of the hatching cycle and specific processing mechanisms. This results in insufficient accuracy in hatching time prediction and a lack of scientific rigor in batch chick selection, hindering significant improvements in hatching efficiency, quality, and farming profitability. Therefore, a technological solution is urgently needed to address these issues and achieve efficient, precise, and intelligent management of the poultry hatching process. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention proposes an intelligent poultry hatching management system based on multimodal data fusion. By acquiring multimodal data and using a reinforcement learning dynamic weight allocation algorithm, the system adapts the data value of different stages throughout the incubation cycle, intelligently identifies the earliest hatching chicks, accurately predicts hatching time, and optimizes batch scheduling.

[0004] The technical solution to achieve the purpose of this invention is as follows:

[0005] A poultry hatching intelligent management system based on multimodal data fusion includes:

[0006] The environmental monitoring module includes a data acquisition unit and a hatching prediction unit. The data acquisition unit is used to collect multi-dimensional environmental parameters of poultry incubators at multiple consecutive time points and perform data preprocessing. The hatching prediction unit calls the corresponding breed-specific environmental parameter library according to the poultry breed, inputs the multi-dimensional environmental parameters into a multi-parameter coupling model built based on a time-series dynamic neural network, analyzes the dynamic coupling effect between various environmental parameters and the breed-specific influence, constructs a high-precision and breed-adaptive poultry chick hatching window prediction model, and outputs the hatching window span time.

[0007] The sound recognition module includes a sound acquisition unit and a call recognition unit. The sound acquisition unit is used to acquire the sound signals of poultry chicks and extract the call sound signals from the sound signals using a multi-source noise adaptive suppression algorithm. The call recognition unit extracts the call sound features from the call sound signals and obtains the call recognition result by accurately recognizing the call of poultry chicks through a call recognition model.

[0008] The image processing module includes an image acquisition unit and an image recognition unit; the image acquisition unit is used to acquire infrared images inside the poultry incubator; the image recognition unit is used to perform target detection on the chicks in the infrared images and calculate the number of chicks hatched.

[0009] The hatching management module uses the output of the poultry chick hatching window prediction model, the call recognition results, and the number and proportion of hatching chicks to make intelligent decisions using a multimodal data dynamic weight allocation algorithm based on reinforcement learning. It identifies the earliest hatching chicks in the large incubator, accurately predicts the hatching time, and rationally arranges the hatching batches of chicks based on the prediction results and the status of the chicks, thereby significantly improving the hatching efficiency, quality, and subsequent breeding benefits of poultry.

[0010] Furthermore, the environmental monitoring module collects data from the poultry incubator via the data acquisition unit during continuous operation. Multi-dimensional environmental parameters at each time point, including temperature, humidity, and CO2 concentration, for the first... Multi-dimensional environmental parameters at each time point are obtained by concatenating the three environmental parameters of temperature, humidity, and CO2 concentration. The multidimensional environmental characteristics at each point in time are denoted as follows: Then continuous The multidimensional environmental features corresponding to the multidimensional environmental parameters at each time point are denoted as follows: Subsequently, a moving average filtering algorithm was used to preprocess the multi-dimensional environmental features to obtain smooth environmental features. This reduces the impact of random noise on the data and highlights the trends in data changes; the hatching prediction unit calls up the corresponding breed-specific environmental parameter library based on the poultry breed, smoothing environmental characteristics. Inputting a multi-parameter coupled model based on a temporal dynamic neural network, this study analyzes the dynamic coupling effects among various environmental parameters and their influence on breed specificity. A high-precision, breed-adaptive poultry chick hatching window prediction model is then constructed, and the hatching window span time is output. ;

[0011] Specifically, for in Multidimensional environmental features collected at various time points Let the window size of the moving average filtering algorithm be... When processing multi-dimensional environmental features using a sliding window, a simple moving average method is used to improve data computation speed and reduce computational resource consumption. Each data point within the window is assigned the same weight value of 1. Therefore, after window processing, the [number of data points]... The smooth environmental characteristics at each time point are denoted as follows: ;

[0012] Specifically, due to the significant differences in the genetic characteristics of different poultry breeds, their requirements for the incubation environment vary considerably. The hatching prediction unit retrieves the corresponding breed-specific environmental parameter database based on the poultry breed in the incubator, obtains the optimal temperature, humidity, and CO2 concentration for that breed during incubation, and then synthesizes these parameters to obtain the optimal environmental characteristics, denoted as [missing information]. At the same time, the baseline value of the incubation period for this poultry breed is obtained and recorded as follows: Smooth environmental features Optimal environmental characteristics and incubation cycle benchmark Inputting a multi-parameter coupled model based on a temporal dynamic neural network, this study analyzes the dynamic coupling effects among various environmental parameters and their influence on breed specificity. A high-precision, breed-adaptive poultry chick hatching window prediction model is then constructed, and the hatching window span time is output. ;

[0013] The multi-parameter coupled model based on a temporal dynamic neural network consists of a TCN convolutional layer, a GRU layer, and an influence factor output layer; firstly, the optimal environmental features are... and smooth environmental features The input feature is obtained by concatenating each element in the input. The input features are then fed into the TCN convolutional layer to capture local features and global trends of parameter changes over time.

[0014] Then, the first Output characteristics at time points The input is fed into the GRU layer to capture long-term dependencies between time-series data, and the data is updated through update and reset gates. The hidden state at a given time point;

[0015] The smooth environment features are processed through the aforementioned TCN convolutional layer and GRU layer. After that, the first Hidden state of time ,Will The input is fed into the impact factor output layer to obtain the impact factor output of the multi-parameter coupled model. Subsequently, the incubation cycle benchmark value will be set. and impact factor Multiplication is used to calculate the hatching window span time. ;

[0016] Furthermore, the sound recognition module includes a sound acquisition unit and a call recognition unit; the sound acquisition unit is used to acquire the sound signals of poultry chicks and extract the call sound signals from the sound signals using a multi-source noise adaptive suppression algorithm; the call recognition unit extracts the call sound features from the call sound signals and obtains the call recognition result by accurately recognizing the call of poultry chicks through a call recognition model.

[0017] Specifically, the sound acquisition unit collects the sound signals of poultry chicks, and then uses a multi-source noise adaptive suppression algorithm to extract the chirping sound signals from the sound signals. Considering that the sound signals collected by the sound acquisition unit not only contain the chirping sounds of the chicks, but also various noise interferences, the total number of sounds collected in the sound signals is denoted as follows: Each independent signal Individual signals are mixed during acquisition to obtain a mixed sound signal. ;

[0018] Independent component analysis (ICA) was used to separate the chirping sounds of chicks from the collected sound signals. The separation matrix is ​​denoted as follows. Considering that the negative entropy of an independent signal is maximized, negative entropy is chosen as the objective function, and the separation matrix is ​​optimized. Minimize the objective function; then use natural gradient descent to iteratively update the separation matrix by minimizing the objective function. Includes the following steps:

[0019] First, initialize the separation matrix. With identity matrix As the initial separation matrix; for the th In the next iteration, the mixed audio signal is separated by a separation matrix. Separation is performed to obtain the separation signal. Then, the gradient of the objective function is calculated in this iteration, and the separation matrix is ​​updated using the calculated gradient. After multiple iterations, the maximum number of iterations is reached, or the change between the updated and unupdated separation matrices is very small. Stop iterating when the Frobenius norm is reached, and output the final separation matrix. ;

[0020] Based on the final separation matrix For mixed sound signals The final separation signal is obtained through processing. For the final separated signal Each chick's call is determined by an independent signal, based on its species. Fourier transform is then used to analyze the signal frequency. Each independent signal is processed to calculate its corresponding frequency, and then... The individual signals are filtered to obtain the final chirping sound signal, which is denoted as [signature]. ;

[0021] Specifically, the call recognition unit extracts call sound features from the call sound signal and uses the call recognition model to accurately identify the call of poultry chicks and obtain the call recognition result;

[0022] First, send the chirping sound signal Divided into The chirping sound features are extracted from each subframe. For example, for the first subframe... Extract the corresponding temporal features from each subframe. This includes the peak amplitude, average amplitude, duration, interval, and number of calls of the chirping sound signal; among which, the peak amplitude and average amplitude reflect the volume of the chirping, the duration reflects the duration of a single chirping, the interval reflects the time difference between two chirping, and the number of chirping reflects the frequency of chirping per unit time.

[0023] Subsequently, the first The chirping sound signal corresponding to each subframe Converting time-domain signal to subframe frequency-domain signal Then from the subframe frequency domain signal Extracting frequency domain features This includes center frequency, frequency bandwidth, and peak spectral density; among which, center frequency reflects the main frequency components of the chirping signal, frequency bandwidth reflects the frequency distribution range of the chirping signal, and peak spectral density reflects the energy concentration of the frequency domain signal.

[0024] From the first The chirping sound signal corresponding to each subframe Extracting temporal features from subframes and subframe frequency domain features Then, the time-domain features and frequency-domain features of the subframe are normalized to eliminate the dimensional differences between different feature parameters. Finally, the time-domain features and frequency-domain features of the subframe are concatenated to obtain the first... The subframe chirping feature vector of each subframe is denoted as... Extract the total amount in the above manner The subframe chirping feature vectors of each subframe are used to form the chirping sound signal. The corresponding call feature vector is denoted as The corresponding dimension is ;

[0025] The call recognition model is constructed by combining a convolutional neural network (CNN) and a long short-term memory network (LSTM), and includes two convolutional layers, an LSTM layer, and an output layer.

[0026] Both convolutional layers are 1D convolutional layers, with the first convolutional layer used to extract the call feature vector. The spatial local dependencies in the image are then analyzed, followed by a max-pooling layer for dimensionality reduction while preserving key features. This result is then fed into a second convolutional layer to further refine spatial feature extraction, which also undergoes a max-pooling layer for further dimensionality reduction to improve computational efficiency. The intermediate features obtained after these two convolutional layers are denoted as […]. ;

[0027] intermediate features The input is fed into the LSTM layer, which utilizes both forward and backward temporal features to capture the chirping change trend more comprehensively.

[0028] First, the forget gate controls whether the long-term memory of the previous subframe is retained. Then, the input gate controls whether the candidate long-term memory of the current subframe is written into the current subframe's long-term memory. Next, the candidate long-term memory of the current subframe is calculated and updated to store key temporal features. Finally, the output gate is used to obtain the hidden features of the current subframe.

[0029] The intermediate features obtained through the LSTM layer are denoted as follows: Corresponding hidden features Then, hide the features. The input is fed into the output layer, and the call recognition result is obtained through two fully connected layers;

[0030] Furthermore, the image processing module includes an image acquisition unit and an image recognition unit; the image acquisition unit is used to acquire infrared images inside the poultry incubator; the image recognition unit is used to perform target detection on the chicks in the infrared images and calculate the number of chicks hatched.

[0031] Specifically, the image acquisition unit acquires infrared images of the poultry incubator in real time and performs image preprocessing on the acquired infrared images to improve image quality; image preprocessing includes image denoising and enhancement.

[0032] For image denoising, considering that infrared images are subject to various noise interferences during acquisition, affecting the accuracy of subsequent target detection and feature extraction, it is necessary to denoise the acquired infrared images. First, median filtering is used to process impulse noise. For each pixel in the infrared image, its... The median of all pixel values ​​in the neighborhood is used to replace the original gray value of the pixel to ignore outliers and retain the gray value features of surrounding normal pixels. Based on median filtering, a Gaussian function is used to perform weighted smoothing on the infrared image, with higher weights for pixels closer to the current pixel and lower weights for pixels farther away. In addition to pixel smoothing, regions of abrupt changes in pixel gray value are preserved to avoid contour blurring caused by over-smoothing, ultimately resulting in a denoised infrared image.

[0033] For image enhancement, considering that the chicks, unhatched eggs, and the inner wall material of the poultry incubator have similar thermal radiation, their grayscale values ​​are quite similar in the acquired infrared images, making them difficult to distinguish quickly. Therefore, enhancement processing is needed to amplify the difference between the target and the background. A histogram equalization algorithm is used to adjust the grayscale distribution of the denoised infrared image. First, the grayscale values ​​of all pixels in the denoised infrared image are statistically analyzed and a grayscale histogram is constructed. Then, based on the cumulative distribution function of grayscale, the cumulative proportion is calculated and adjusted according to the cumulative proportion. A new gray value is assigned to each original gray value, and the original gray values ​​in the infrared denoising infrared image are replaced with the new gray values ​​to obtain the enhanced infrared image.

[0034] Specifically, the image recognition unit is used to detect chicks in the infrared image and calculate the number of chicks; based on the preprocessed enhanced infrared image, the YOLO target detection model is used to detect chicks, and the number of chicks in multiple consecutive frames of enhanced infrared images is statistically analyzed to calculate the current number of chicks and calculate the hatching ratio in combination with the total number of hatching eggs in the poultry incubator.

[0035] Furthermore, the hatching management module uses a multimodal data dynamic weight allocation algorithm based on reinforcement learning to make intelligent decisions, identify the earliest hatching chicks in the large incubator, accurately predict the hatching time, and rationally arrange the hatching batches of chicks based on the prediction results and the status of the chicks, thereby significantly improving the hatching efficiency, quality and subsequent breeding benefits of poultry.

[0036] First, the environmental monitoring module outputs the hatching window span time, and the sound recognition module outputs the chirping recognition results; the image processing module outputs the number of chicks and the hatching ratio; then, the current incubation stage is determined by analyzing the chirping recognition results, and the reliability of each modality's data is evaluated. The reliability of the hatching window span time is set as an influencing factor, the reliability of the chirping recognition results is set as the probability of the corresponding category, and the reliability of the number of chicks and the hatching ratio is set as the probability output by the YOLO object detection model; based on... - The greedy strategy selects weight combinations, prioritizing the weight scheme with the best past decision-making performance, and completes multimodal data fusion and core decision execution according to dynamic weights; considering that the importance of each modality of data varies at different stages of the incubation cycle, before hatching, the multi-dimensional environmental parameters collected by the environmental monitoring module are the main focus, highlighting the impact of the environment on the hatching window; during hatching, the weight of sound signals is increased, relying on the characteristics of chirping sounds to capture the start of hatching; after hatching, the weight of image data is increased, and decisions are optimized based on the number and proportion of hatching chicks; after the decision is executed, the comprehensive reward value is calculated based on the accuracy of hatching time prediction and hatching ratio prediction, and the weight allocation rules for each state are optimized based on the reward results.

[0037] Compared with existing technologies, this invention collects multi-dimensional parameters such as temperature, humidity, and CO2 concentration through an environmental monitoring module. After preprocessing with a moving average filter, it combines a variety-specific environmental parameter library and constructs a multi-parameter coupled model using a time-series dynamic neural network to output the hatching window span. The sound recognition module uses a multi-source noise adaptive suppression algorithm to extract chirping signals. After time-domain and frequency-domain feature extraction, it accurately identifies the hatching stage using a CNN+LSTM model. The image processing module collects infrared images, performs denoising and enhancement preprocessing, and uses a YOLO model to detect chicks and count the number and proportion of hatched chicks. The hatching management module, based on the outputs of the first three modules, uses a reinforcement learning dynamic weight allocation algorithm to adapt the data value of different stages of the incubation cycle, intelligently identifies the earliest hatched chicks, accurately predicts the hatching time, and optimizes batch scheduling. Attached Figure Description

[0038] Figure 1 This is a module diagram of an intelligent poultry hatching management system based on multimodal data fusion.

[0039] Figure 2 Here is a flowchart of the environmental monitoring module;

[0040] Figure 3 A flowchart for separating the chirping sounds of chicks;

[0041] Figure 4 A flowchart for generating chirping recognition results for the chirping recognition model. Detailed Implementation

[0042] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0043] like Figure 1 As shown in the figure, a specific embodiment of the present invention discloses an intelligent management system for poultry hatching based on multimodal data fusion, including: an environmental monitoring module, a sound recognition module, an image processing module, and a hatching management module;

[0044] The environmental monitoring module includes a data acquisition unit and a hatching prediction unit. The data acquisition unit is used to collect multi-dimensional environmental parameters of poultry incubators at multiple consecutive time points and perform data preprocessing. The hatching prediction unit calls the corresponding breed-specific environmental parameter library according to the poultry breed, inputs the multi-dimensional environmental parameters into a multi-parameter coupling model built based on a time-series dynamic neural network, analyzes the dynamic coupling effect between various environmental parameters and the breed-specific influence, constructs a high-precision and breed-adaptive poultry chick hatching window prediction model, and outputs the hatching window span time.

[0045] The sound recognition module includes a sound acquisition unit and a call recognition unit. The sound acquisition unit is used to acquire the sound signals of poultry chicks and extract the call sound signals from the sound signals using a multi-source noise adaptive suppression algorithm. The call recognition unit extracts the call sound features from the call sound signals and obtains the call recognition result by accurately recognizing the call of poultry chicks through a call recognition model.

[0046] The image processing module includes an image acquisition unit and an image recognition unit; the image acquisition unit is used to acquire infrared images inside the poultry incubator; the image recognition unit is used to perform target detection on the chicks in the infrared images and calculate the number of chicks hatched.

[0047] The hatching management module uses the output of the poultry chick hatching window prediction model, the call recognition results, and the number and proportion of hatching chicks to make intelligent decisions using a multimodal data dynamic weight allocation algorithm based on reinforcement learning. It identifies the earliest hatching chicks in the large incubator, accurately predicts the hatching time, and rationally arranges the hatching batches of chicks based on the prediction results and the status of the chicks, thereby significantly improving the hatching efficiency, quality, and subsequent breeding benefits of poultry.

[0048] Furthermore, such as Figure 2 As shown, the environmental monitoring module collects data from the poultry incubator via the data acquisition unit during continuous operation. Multi-dimensional environmental parameters at each time point, including temperature, humidity, and CO2 concentration, for the first... Multi-dimensional environmental parameters at each time point are obtained by concatenating the three environmental parameters of temperature, humidity, and CO2 concentration. The multidimensional environmental characteristics at each point in time are denoted as follows: Then continuous The multidimensional environmental features corresponding to the multidimensional environmental parameters at each time point are denoted as follows: Subsequently, a moving average filtering algorithm was used to preprocess the multi-dimensional environmental features to obtain smooth environmental features. This reduces the impact of random noise on the data and highlights the trends in data changes; the hatching prediction unit calls up the corresponding breed-specific environmental parameter library based on the poultry breed, smoothing environmental characteristics. Inputting a multi-parameter coupled model based on a temporal dynamic neural network, this study analyzes the dynamic coupling effects among various environmental parameters and their influence on breed specificity. A high-precision, breed-adaptive poultry chick hatching window prediction model is then constructed, and the hatching window span time is output. ;

[0049] Specifically, for in Multidimensional environmental features collected at various time points Let the window size of the moving average filtering algorithm be... When processing multi-dimensional environmental features using a sliding window, a simple moving average method is used to improve data computation speed and reduce computational resource consumption. Each data point within the window is assigned the same weight value of 1. Therefore, after window processing, the [number of data points]... Smooth environmental characteristics at each point in time The calculation method is as follows:

[0050] ,

[0051] Among them, when When the value is less than or equal to 0, the corresponding The value is set to 0; The specific value is selected based on the environment of the poultry incubator. When the value is large, the moving average filtering algorithm can effectively suppress noisy data and achieve excellent data smoothing, but it can also cause data lag, making it difficult to accurately reflect changes in the true data; when When the value is small, the data lag is small, and it can quickly track the changing trend of real data, but it cannot effectively remove spikes from the data;

[0052] Specifically, due to the significant differences in the genetic characteristics of different poultry breeds, their requirements for the incubation environment vary considerably. The hatching prediction unit retrieves the corresponding breed-specific environmental parameter database based on the poultry breed in the incubator, obtains the optimal temperature, humidity, and CO2 concentration for that breed during incubation, and then synthesizes these parameters to obtain the optimal environmental characteristics, denoted as [missing information]. At the same time, the baseline value of the incubation period for this poultry breed is obtained and recorded as follows: Smooth environmental features Optimal environmental characteristics and incubation cycle benchmark Inputting a multi-parameter coupled model based on a temporal dynamic neural network, this study analyzes the dynamic coupling effects among various environmental parameters and their influence on breed specificity. A high-precision, breed-adaptive poultry chick hatching window prediction model is then constructed, and the hatching window span time is output. ;

[0053] The multi-parameter coupled model based on a temporal dynamic neural network consists of a TCN convolutional layer, a GRU layer, and an influence factor output layer; firstly, the optimal environmental features are... and smooth environmental features The input feature is obtained by concatenating each element in the input. The calculation formula is as follows:

[0054] ;

[0055] The input features are then fed into the TCN convolutional layer to capture local features and global trends of parameter changes over time. The calculation formula is as follows:

[0056] ,

[0057] in, For the first The input features at each time point correspond to the output features in the TCN convolutional layer. This refers to the size of the convolution kernel in the TCN convolutional layer. The weights of the convolution kernel, For the first Input features at specific time points, The bias matrix, It is the ReLU activation function; this calculation formula reflects the calculation of the first... When considering the output features at a given time point, it depends only on the first... The timeframe and previous timeframes do not depend on future information;

[0058] Then, the first Output characteristics at time points The input is fed into the GRU layer to capture long-term dependencies between time-series data, and the data is updated through update and reset gates. The hidden state at a given time point is calculated using the following formula:

[0059] ,

[0060] ,

[0061] ,

[0062] ,

[0063] in, It updates the output of the gate. It is the output of the reset door. It is the first Candidate hidden states at a given time point It is the first Hidden state after time update. It is the hidden state from the previous moment. , , These are the update gate, the reset gate, and the weight matrix of the candidate hidden state. , , These are the corresponding bias terms. For element-wise multiplication;

[0064] The smooth environment features are processed through the aforementioned TCN convolutional layer and GRU layer. After that, the first Hidden state of time ,Will The input is fed into the impact factor output layer to obtain the impact factor output of the multi-parameter coupled model. The calculation formula is as follows:

[0065] ,

[0066] in, This is the output layer weight matrix. This is the output layer bias term; subsequently, the incubation cycle baseline value is used. and impact factor Multiplication is used to calculate the hatching window span time. ;

[0067] Furthermore, the sound recognition module includes a sound acquisition unit and a call recognition unit; the sound acquisition unit is used to acquire the sound signals of poultry chicks and extract the call sound signals from the sound signals using a multi-source noise adaptive suppression algorithm; the call recognition unit extracts the call sound features from the call sound signals and obtains the call recognition result by accurately recognizing the call of poultry chicks through a call recognition model.

[0068] Specifically, the sound acquisition unit collects the sound signals of poultry chicks, and then uses a multi-source noise adaptive suppression algorithm to extract the chirping sound signals from the sound signals. Considering that the sound signals collected by the sound acquisition unit not only contain the chirping sounds of the chicks, but also various noise interferences, the total number of sounds collected in the sound signals is denoted as follows: Each independent signal Individual signals are mixed during acquisition to obtain a mixed sound signal. ;

[0069] like Figure 3 As shown, the chirping sounds of chicks are separated from the collected sound signal using the independent component analysis algorithm. The separation matrix is ​​denoted as... Considering that the negative entropy of an independent signal is maximized, negative entropy is chosen as the objective function, and the separation matrix is ​​optimized. Minimize the objective function, the specific formula of which is as follows:

[0070] ,

[0071] in, Represents mixed sound signals The first result obtained after processing the separation matrix A separate signal, Represents the Gaussian kernel function. The mathematical expectation is the average of the data across all dimensions of the signal. Separation matrix The determinant is then used; subsequently, the natural gradient descent method is employed to minimize the objective function and iteratively update the separation matrix. Includes the following steps:

[0072] First, initialize the separation matrix. With identity matrix As the initial separation matrix; for the th In the next iteration, the mixed audio signal is separated by a separation matrix. Separation is performed to obtain the separation signal. The calculation formula is as follows:

[0073] ;

[0074] Then, the gradient of the objective function in this iteration is calculated using the following formula:

[0075] ,

[0076] in, for The derivative of the objective function is then used to update the separation matrix, as shown in the following formula:

[0077] ;

[0078] After multiple iterations, the maximum number of iterations is reached, or the change between the updated and unupdated separation matrices is very small. Stop iterating when the Frobenius norm is reached, and output the final separation matrix. ;

[0079] Based on the final separation matrix For mixed sound signals The final separation signal is obtained through processing. For the final separated signal Each chick's call is determined by an independent signal, based on its species. Fourier transform is then used to analyze the signal frequency. Each independent signal is processed to calculate its corresponding frequency, and then... The individual signals are filtered to obtain the final chirping sound signal, which is denoted as [signature]. ;

[0080] Specifically, such as Figure 4 As shown, the call recognition unit extracts call sound features from the call sound signal and uses the call recognition model to accurately recognize the call of poultry chicks and obtain the call recognition result;

[0081] First, send the chirping sound signal Divided into The chirping sound features are extracted from each subframe. For example, for the first subframe... Extract the corresponding temporal features from each subframe. This includes the peak amplitude, average amplitude, duration, interval, and number of calls of the chirping sound signal; among which, the peak amplitude and average amplitude reflect the volume of the chirping, the duration reflects the duration of a single chirping, the interval reflects the time difference between two chirping, and the number of chirping reflects the frequency of chirping per unit time.

[0082] Subsequently, the first The chirping sound signal corresponding to each subframe Converting time-domain signal to subframe frequency-domain signal The calculation formula is as follows:

[0083] ,

[0084] in, The imaginary unit, Indicates the first The frequency at each sampling point This indicates the sampling frequency; subsequently, the subframe frequency domain signal... Extracting frequency domain features This includes center frequency, frequency bandwidth, and peak spectral density; among which, center frequency reflects the main frequency components of the chirping signal, frequency bandwidth reflects the frequency distribution range of the chirping signal, and peak spectral density reflects the energy concentration of the frequency domain signal.

[0085] From the first The chirping sound signal corresponding to each subframe Extracting temporal features from subframes and subframe frequency domain features Then, the time-domain features and frequency-domain features of the subframe are normalized to eliminate the dimensional differences between different feature parameters. Finally, the time-domain features and frequency-domain features of the subframe are concatenated to obtain the first... The subframe chirping feature vector of each subframe is denoted as... Extract the total amount in the above manner The subframe chirping feature vectors of each subframe are used to form the chirping sound signal. The corresponding call feature vector is denoted as The corresponding dimension is ;

[0086] The call recognition model is constructed by combining a convolutional neural network (CNN) and a long short-term memory network (LSTM), and includes two convolutional layers, an LSTM layer, and an output layer.

[0087] Both convolutional layers are 1D convolutional layers, with the first convolutional layer used to extract the call feature vector. The spatial local dependencies in the image are then analyzed, followed by a max-pooling layer for dimensionality reduction while preserving key features. This result is then fed into a second convolutional layer to further refine spatial feature extraction, which also undergoes a max-pooling layer for further dimensionality reduction to improve computational efficiency. The intermediate features obtained after these two convolutional layers are denoted as […]. ;

[0088] intermediate features The input is fed into the LSTM layer, which utilizes both forward and backward temporal features to capture the chirping change trend more comprehensively.

[0089] First, the long-term memory of the previous subframe is controlled by the forget gate. The calculation formula is as follows:

[0090] ,

[0091] Then, the input gate controls whether the candidate long-term memory of the current subframe is written into the long-term memory of the current subframe. The calculation formula is as follows:

[0092] ,

[0093] Then, the candidate long-term memory for the current subframe is calculated and updated to store key temporal features. The calculation formula is as follows:

[0094] ,

[0095] ,

[0096] Finally, the hidden features of the current subframe are obtained through the output gate, and the calculation formula is as follows:

[0097] ,

[0098] ,

[0099] in, Indicates the current subframe, Indicates the previous subframe. The forget gate output of the current subframe is used to control the long-term memory of the previous subframe. Should it be retained? The input gate output of the current subframe is used to control the candidate long-term memory of the current subframe. Whether to write to the long-term memory of the current subframe Indicates the candidate long-term memory for the current subframe. This represents the long-term memory of the updated current subframe. The output of the current subframe's output gate is used to control the long-term memory of the current subframe. Whether to output the hidden features of the current subframe. It is the hidden feature of the current subframe. The current subframe is the input feature to the LSTM layer. , , , These are the forget gate weight matrix, input gate weight matrix, candidate long memory weight matrix, and output gate weight matrix. , , , These are the forget gate bias vector, input gate bias vector, candidate long-term memory bias vector, and output gate bias vector, respectively. Represents the hyperbolic tangent activation function;

[0100] The intermediate features obtained through the LSTM layer are denoted as follows: Corresponding hidden features Then, hide the features. The input is fed into the output layer, and the call recognition result is obtained through two fully connected layers. The calculation formula is as follows:

[0101] ,

[0102] ,

[0103] in, , Here are the weight matrix and bias vector of the first fully connected layer. , Here are the weight matrix and bias vector for the second fully connected layer. This is the 3D vector output by the output layer. The collected chirping sound signal belongs to the first... The probability of a class Indicates before hatching. Indicates that the shell is hatching. This indicates that after hatching, the class with the highest probability is selected as the chirping recognition result;

[0104] Furthermore, the image processing module includes an image acquisition unit and an image recognition unit; the image acquisition unit is used to acquire infrared images inside the poultry incubator; the image recognition unit is used to perform target detection on the chicks in the infrared images and calculate the number of chicks hatched.

[0105] Specifically, the image acquisition unit acquires infrared images of the poultry incubator in real time and performs image preprocessing on the acquired infrared images to improve image quality; image preprocessing includes image denoising and enhancement.

[0106] For image denoising, considering that infrared images are subject to various noise interferences during acquisition, affecting the accuracy of subsequent target detection and feature extraction, it is necessary to denoise the acquired infrared images. First, median filtering is used to process impulse noise. For each pixel in the infrared image, its... The median of all pixel values ​​in the neighborhood is used to replace the original gray value of the pixel to ignore outliers and retain the gray value features of surrounding normal pixels. Based on median filtering, a Gaussian function is used to perform weighted smoothing on the infrared image, with higher weights for pixels closer to the current pixel and lower weights for pixels farther away. In addition to pixel smoothing, regions of abrupt changes in pixel gray value are preserved to avoid contour blurring caused by over-smoothing, ultimately resulting in a denoised infrared image.

[0107] For image enhancement, considering that the chicks, unhatched eggs, and the inner wall material of the poultry incubator have similar thermal radiation, their grayscale values ​​are quite similar in the acquired infrared images, making them difficult to distinguish quickly. Therefore, enhancement processing is needed to amplify the difference between the target and the background. A histogram equalization algorithm is used to adjust the grayscale distribution of the denoised infrared image. First, the grayscale values ​​of all pixels in the denoised infrared image are statistically analyzed and a grayscale histogram is constructed. Then, based on the cumulative distribution function of grayscale, the cumulative proportion is calculated and adjusted according to the cumulative proportion. A new gray value is assigned to each original gray value, and the original gray values ​​in the infrared denoising infrared image are replaced with the new gray values ​​to obtain the enhanced infrared image.

[0108] Specifically, the image recognition unit is used to detect chicks in the infrared image and calculate the number of chicks; based on the preprocessed enhanced infrared image, the YOLO target detection model is used to detect chicks, and the number of chicks in multiple consecutive frames of enhanced infrared images is statistically analyzed to calculate the current number of chicks and calculate the hatching ratio in combination with the total number of hatching eggs in the poultry incubator.

[0109] Furthermore, the hatching management module uses a multimodal data dynamic weight allocation algorithm based on reinforcement learning to make intelligent decisions, identify the earliest hatching chicks in the large incubator, accurately predict the hatching time, and rationally arrange the hatching batches of chicks based on the prediction results and the status of the chicks, thereby significantly improving the hatching efficiency, quality and subsequent breeding benefits of poultry.

[0110] First, the environmental monitoring module outputs the hatching window span time, and the sound recognition module outputs the chirping recognition results; the image processing module outputs the number of chicks and the hatching ratio; then, the current incubation stage is determined by analyzing the chirping recognition results, and the reliability of each modality's data is evaluated. The reliability of the hatching window span time is set as an influencing factor, the reliability of the chirping recognition results is set as the probability of the corresponding category, and the reliability of the number of chicks and the hatching ratio is set as the probability output by the YOLO object detection model; based on... - A greedy strategy selects weight combinations, prioritizing the weight scheme with the best past decision-making performance, and dynamically weights to complete multimodal data fusion and core decision execution. Considering that the importance of each modality of data varies at different stages of the incubation cycle, before hatching, multi-dimensional environmental parameters collected by the environmental monitoring module are used as the main focus, highlighting the impact of the environment on the hatching window. During hatching, the weight of sound signals is increased, relying on the characteristics of chirping sounds to capture the start of hatching. After hatching, the weight of image data is increased, and decisions are optimized based on the number and proportion of hatching chicks. After the decision is executed, a comprehensive reward value is calculated based on the accuracy of hatching time prediction and hatching ratio prediction, and the weight allocation rules for each state are optimized based on the reward results.

[0111] Furthermore, the specific parameter settings for the temporal dynamic neural network, call recognition model, and YOLO object detection model used in this system are as follows:

[0112] For the temporal dynamic neural network, the kernel size of the TCN convolutional layer is set to 3, and the number of convolutional kernels is set to 128; the hidden layer dimension of the GRU layer is set to 128; the input dimension of the output layer is set to 128, and the output dimension is set to 1.

[0113] For the chirping recognition model, the first convolutional layer in the CNN convolutional layer has 64 kernels and a kernel size of 5; the max pooling layer corresponding to the first convolutional layer has a kernel size of 2; the second convolutional layer has 128 kernels and a kernel size of 3; the max pooling layer corresponding to the second convolutional layer has a kernel size of 2; the hidden layer dimension of the LSTM layer is set to 128; the input dimension of the first fully connected layer in the output layer is set to 128 and the output dimension is set to 64; the input dimension of the second fully connected layer is set to 64 and the output dimension is set to 3.

[0114] For the YOLO object detection model, the YOLOv5s version was used, and the input image size was set to 640x640.

[0115] This invention discloses an intelligent poultry hatching management system based on multimodal data fusion. The system collects multi-dimensional parameters such as temperature, humidity, and CO2 concentration through an environmental monitoring module. After preprocessing with a moving average filter, and combining this with a breed-specific environmental parameter library, a multi-parameter coupled model is constructed using a temporal dynamic neural network to output the hatching window span. A sound recognition module extracts chirping signals using a multi-source noise adaptive suppression algorithm. After time-domain and frequency-domain feature extraction, a CNN+LSTM model accurately identifies the hatching stage. An image processing module collects infrared images, performs denoising and enhancement preprocessing, and uses a YOLO model to detect chicks and statistically analyze the number and proportion of hatched chicks. Based on the outputs of the first three modules, a hatching management module uses a reinforcement learning dynamic weight allocation algorithm to adapt the data value of different stages throughout the incubation cycle, intelligently identifying the earliest hatching chicks, accurately predicting hatching time, and optimizing batch scheduling.

[0116] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A poultry hatching intelligent management system based on multimodal data fusion, characterized in that, include: The environmental monitoring module collects and preprocesses multi-dimensional environmental parameters, calls up the variety-specific environmental parameter library, and outputs the hatching window span time through a multi-parameter coupling model. The sound recognition module collects sound signals and extracts chirping sound signals. After extracting the chirping sound features, it obtains the chirping recognition result through the chirping recognition model. The image processing module collects infrared images and performs preprocessing, performs target detection on the chicks, and calculates the number and proportion of chicks. The chick management module makes intelligent decisions based on the output results of the above three modules, using a reinforcement learning multimodal data dynamic weight allocation algorithm.

2. The intelligent management system for poultry hatching based on multimodal data fusion as described in claim 1, characterized in that, The operation method of the hatching management module includes: The hatching management module determines the current incubation stage, assesses the reliability of data from each modality, selects weight combinations based on a greedy strategy, integrates multimodal data according to dynamic weights, predicts hatching time, and arranges the release of hatching batches of chicks.

3. The intelligent management system for poultry hatching based on multimodal data fusion as described in claim 2, characterized in that, The operation method of the environmental monitoring module includes: The environmental monitoring module collects temperature, humidity, and CO2 concentration parameters at multiple consecutive time points through the data acquisition unit, splices them to form multi-dimensional environmental characteristics, and inputs them into a multi-parameter coupled model based on a time-series dynamic neural network after preprocessing. Combined with a variety-specific environmental parameter library, it outputs the hatching window span time.

4. The intelligent management system for poultry hatching based on multimodal data fusion as described in claim 3, characterized in that, The operation method of the voice recognition module includes: The sound recognition module collects sound signals through the sound acquisition unit, extracts the chirping sound signals using a multi-source noise adaptive suppression algorithm, extracts time-domain and frequency-domain features and performs normalization processing, and obtains the chirping recognition result through a chirping recognition model constructed by combining CNN and LSTM.

5. The intelligent management system for poultry hatching based on multimodal data fusion as described in claim 4, characterized in that, The operation method of the image processing module includes: The image processing module acquires infrared images through the image acquisition unit, performs noise reduction and enhancement preprocessing on the infrared images, uses the YOLO target detection model to detect chicks, counts the number of chicks in multiple consecutive frames of images, and calculates the chick ratio.

6. The intelligent management system for poultry hatching based on multimodal data fusion as described in claim 3, characterized in that, The preprocessing method for the multi-dimensional environmental features includes: The environmental monitoring module uses a moving average filtering algorithm to preprocess multi-dimensional environmental features to obtain smooth environmental features. It then calls a variety-specific environmental parameter library to obtain the optimal environmental features and incubation cycle benchmark values, which are then input into a multi-parameter coupling model.

7. The intelligent management system for poultry hatching based on multimodal data fusion as described in claim 4, characterized in that, The method for processing the chirping sound signal includes: The sound recognition module uses independent component analysis algorithm to separate the chirping sound signal from the mixed sound signal, divides the chirping sound signal into subframes, extracts the peak amplitude, average amplitude, center frequency and other features of each subframe, and splices them to form a chirping feature vector.

8. The intelligent management system for poultry hatching based on multimodal data fusion as described in claim 5, characterized in that, The infrared image preprocessing method includes: The image processing module uses median filtering to process impulse noise in infrared images, combines it with Gaussian filtering for weighted smoothing, and adjusts the grayscale distribution of infrared images through histogram equalization algorithm to achieve image enhancement.

9. A poultry hatching intelligent management system based on multimodal data fusion as described in claim 2, characterized in that, The method for calculating the reliability of each modal data includes: The hatching management module determines the incubation stage based on the chirping recognition results. The influencing factor is set as the credibility of the hatching window span time, the probability of the chirping sound signal corresponding to the category is set as the credibility of the chirping recognition results, and the probability output by the YOLO target detection model is set as the credibility of the number of hatched chicks and the hatching ratio.

10. A poultry hatching intelligent management system based on multimodal data fusion as described in claim 2, characterized in that, The method for adjusting the dynamic weights includes: Before hatching, the weight of multi-dimensional environmental parameters collected in the environmental monitoring module is increased; during hatching, the weight of sound signals is increased; and after hatching, the weight of image data is increased. Decisions are optimized based on the number and proportion of hatching chicks. After the decision is executed, a comprehensive reward value is calculated based on the accuracy of hatching time prediction and the accuracy of hatching proportion prediction. The weight allocation rules for each state are optimized based on the reward results.

Images