Feed production control method for identifying mixing state based on image monitoring

Abstract

The invention relates to the technical field of image monitoring and recognition of feed production, and discloses a feed production control method based on image monitoring and recognition of a mixed state, and the method comprises the steps: collecting multi-dimensional sensor data in a feed production process, carrying out the analysis and feature extraction of the collected sound spectrum data through an acoustic signal processing technology, and carrying out the recognition of a mixed state; a gas sensor and a gas analysis technology are utilized to measure and analyze collected smell component data, a non-contact sensing technology is utilized to collect electromagnetic spectrum data in feed production, a computer vision technology and a deep learning method are utilized to carry out feature extraction and target detection on collected image data, and a target detection result is obtained. And fusing object identification and segmentation results with acoustic features, smell features and electromagnetic spectrum data, and establishing a multi-modal information model of a feed mixing state. By collecting multi-dimensional sensor data and applying an image monitoring and recognition technology, the mixing state in the feed production process is accurately monitored and recognized, and the feed production efficiency and quality are improved.

Description

Technical Field

[0001] This invention relates to the field of image monitoring and recognition in feed production technology, specifically to a feed production control method based on image monitoring and recognition of mixed states. Background Technology

[0002] In traditional feed production, the mixing process is a crucial step. Mixed feed is a primary compound feed made by simply processing and mixing various feed ingredients. It mainly considers nutritional indicators such as energy, protein, calcium, and phosphorus. It is common in many rural areas and can be used to feed animals directly. It is more effective than regular feed and promotes faster growth, but animals are more prone to disease and have poor resistance.

[0003] Traditional methods rely mainly on human experience and manual sampling to determine the mixing state of feed. However, this method is not only time-consuming and inaccurate, but also results in low feed production quality and efficiency, and easily leads to waste of resources and increased production costs. Therefore, it is necessary to design corresponding technical solutions to address these issues. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a feed production control method based on image monitoring and identification of mixing states. This method solves the technical problems of relying mainly on human experience and manual sampling to determine the mixing state of feed, which is not only time-consuming and inaccurate, but also results in low feed production quality and efficiency, as well as resource waste and increased production costs.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a feed production control method based on image monitoring and identification of mixed states, the method comprising the following steps: S1. Collect multi-dimensional sensor data during the feed production process, including sound spectrum data, odor component data, and electromagnetic spectrum data; S2. Using acoustic signal processing technology, the collected sound spectrum data is analyzed and features are extracted to obtain acoustic features related to the feed mixing state. S3. Utilize gas sensors and gas analysis technology to measure and analyze the collected odor component data in order to obtain the odor characteristics during the feed mixing process; S4. Using non-contact sensing technology, electromagnetic spectrum data are collected during the feed production process to obtain information on the temperature, humidity and density of the materials. S5. Using computer vision technology and deep learning methods, feature extraction and target detection are performed on the collected image data to achieve object recognition and segmentation during the feed mixing process; S6. The object recognition and segmentation results are fused with acoustic features, odor features and electromagnetic spectrum data to establish a multimodal information model of feed mixing state, which is used to describe the degree and quality of feed mixing. S7. Based on the multimodal information model, machine learning algorithms are used to classify and evaluate the state, divide the mixed state of the feed into multiple levels or continuous values, and generate corresponding control signals. S8. Adjust the operating parameters of the feed production equipment according to the control signal to achieve automatic control and optimization in the feed production process; S9. Continuous monitoring and feedback control improves the efficiency and quality of feed production by updating multimodal information models and optimization algorithms in real time.

[0006] Preferably, in step S2, the acoustic signal processing technology includes nonlinear dynamics analysis methods, nonnegative matrix decomposition methods, variants of Fourier transform methods, and feature selection methods based on swarm intelligence. By applying the theory and methods of nonlinear dynamics, and employing chaos analysis and complex network analysis, sound spectrum data is analyzed to reveal the nonlinear and dynamic characteristics of sound signals and identify dynamic patterns and interactions related to feed mixing state. By applying nonnegative matrix factorization, the sound spectrum data is decomposed into basic audio components and mixing coefficients. Through the analysis of basic audio components and mixing coefficients, features related to the feed mixing state are extracted. A variant of Fourier transform is used, employing wavelet transform or singular spectrum analysis to transform and analyze the sound spectrum data, capturing the characteristics of the sound signal at different frequencies and time scales, thereby extracting information related to the feed mixing state. By applying swarm intelligence algorithms, such as genetic algorithms or particle swarm optimization algorithms, features in sound spectrum data are selected and optimized. Through an automated feature selection process, the features most relevant to the feed mixing state are found, thereby improving the performance of classification and evaluation.

[0007] Preferably, in step S3, the gas sensor and gas analysis technology include odor component measurement technology based on electronic olfaction, volatile organic compound component identification algorithm technology, odor analysis technology based on vibrational spectroscopy, odor analysis technology based on odor image processing, and odor identification technology based on electronic tongue and electronic nose. The odor component measurement technology based on electronic olfaction uses an electronic olfaction sensor, employing an electrochemical sensor or a surface plasmon sensor, to measure odor components. It simulates the human olfactory system and achieves accurate capture and analysis of odor characteristics during feed mixing by sensing the volatile organic compounds released by the odor components. The volatile organic compound (VOC) component identification algorithm utilizes machine learning and pattern recognition technologies, including feature selection, feature extraction, and classifier design steps, to achieve accurate analysis and identification of odor component data, and to identify and extract features related to feed mixing status from odor component data. The vibrational spectroscopy-based odor analysis technology uses vibrational spectroscopy, employing Raman or infrared spectroscopy, to analyze odor components. By measuring the vibrational frequency and spectral characteristics of gas molecules, it identifies and quantifies the composition and concentration of odor components, enabling high-precision measurement and analysis of odor characteristics during feed mixing. The odor analysis technology based on odor image processing utilizes odor image processing technology to perform image processing and analysis on odor component data, converting the odor component data into odor distribution maps or odor pattern maps, and extracting odor features related to the feed mixing state by analyzing the texture, color and shape features of the images. The odor identification technology based on electronic tongue and electronic nose uses the principles of electronic tongue and electronic nose to identify odors during the feed mixing process. The electronic tongue simulates a taste system to identify different odors by measuring the chemical properties of odor components, while the electronic nose simulates an olfaction system to identify and quantify odors by measuring the volatile organic compounds of odor components.

[0008] Preferably, in step S4, the non-contact sensing technology includes laser interferometric imaging technology, ultra-wideband radar technology, infrared spectral imaging technology, and microwave imaging technology. The laser interferometric imaging technology obtains electromagnetic spectrum data by measuring the difference in optical path length on the feed surface, projects laser light onto the feed surface using the laser interference phenomenon, and deduces the propagation and characteristics of electromagnetic waves in the feed by analyzing the changes in the interference pattern. Through the analysis and processing of the interference pattern, electromagnetic spectrum data during the feed mixing process is obtained, including the temperature, humidity and density information of the material. The ultra-wideband radar technology sends a series of wideband electromagnetic pulse signals and receives echo signals. By analyzing the amplitude and phase information of the echo signals, electromagnetic spectrum data during the feed mixing process can be obtained. The infrared spectral imaging technology obtains electromagnetic spectrum data by measuring the infrared radiation on the surface of the feed. Based on the infrared spectral characteristics emitted by the object, the feed is imaged by an infrared camera or infrared thermal imager, and the image is processed and analyzed to extract electromagnetic spectrum data related to the mixing state of the feed. The microwave imaging technology acquires electromagnetic spectrum data by measuring the propagation and scattering characteristics of microwaves in feed. By utilizing the penetration and reflection characteristics of microwaves, the transmitted and received microwave signals are analyzed and processed to acquire electromagnetic spectrum data during the feed mixing process.

[0009] Preferably, in step S5, the computer vision technology and deep learning method include photonic compressed sensing technology, optical coding array technology, photonic compressed sensing image sensor technology, deep learning decoding network technology and target detection and segmentation technology. The photonic compressed sensing technology is based on photonic principles and uses photonic devices to optically encode and decode images, thereby achieving efficient extraction of image information. The optical coding array technology encodes the acquired image data, encodes and compresses the image in the optical domain, and converts the image information into a sparse representation in the optical domain. The photonic compressed sensing image sensor technology is used to acquire optically encoded image data. It combines an optical encoding array and a high-sensitivity photonic detector to directly obtain optically encoded sparse image data. The deep learning decoding network technology is used to recover the features of the target image from optically encoded and sampled image data. It combines the powerful feature extraction and reconstruction capabilities of deep learning to restore high-quality image features from sparse optically encoded data. The target detection and segmentation technology identifies and segments objects during the feed mixing process. It combines a deep learning target detection model with the recovered image features to perform accurate target detection and segmentation, and obtains target information related to the feed mixing state.

[0010] Preferably, in step S6, the fusion method includes a multimodal feature fusion network, feature alignment and mapping, cross-modal feature fusion, multimodal information fusion, and the construction of a multimodal information model; The method for fusing the object recognition and segmentation results with acoustic features, odor features, and electromagnetic spectrum data specifically includes: Design a novel multimodal feature fusion network that receives object recognition and segmentation results, acoustic features, odor features, and electromagnetic spectrum data as input, and automatically learns the correlations between them; Since the scale, resolution, and representation of data from different sensors vary, feature alignment and mapping are performed. By using adversarial generative networks or self-attention mechanisms, features from different modalities are mapped to a unified feature space for subsequent fusion. After feature alignment and mapping, the spatial information of object recognition and segmentation results, the spectral information of acoustic features, the chemical composition of odor features, and the physical parameters of electromagnetic spectrum data are fused across modally. Deep neural networks and attention mechanisms are used to learn the weights and correlations between cross-modal features. The features obtained after cross-modal fusion are fused together, and fusion layers, attention mechanisms or graph convolutional neural network methods are used to capture the interaction and dependency between different modalities and generate fused multimodal information. The fused multimodal information is input into a machine learning model, and deep neural networks, support vector machines, or random forests are used to classify and evaluate the feed mixing state, learn the complex relationships between different modal information, and thus accurately describe the degree and quality of feed mixing.

[0011] Preferably, in step S6, the multimodal information model for establishing the feed mixing state is a multimodal graph neural network model based on deep reinforcement learning, including graph neural network, deep reinforcement learning method and reinforcement learning training and optimization, used to model and control the feed mixing state; Graph neural networks are used to construct graph structures representing object recognition and segmentation results, acoustic features, odor features and electromagnetic spectrum data. Each sensor data corresponds to a node in the graph, and nodes are connected by edges. The edges represent the correlation between different sensor data. The graph neural network transmits and aggregates information in the graph structure and captures the complex relationships between multimodal data. The optimal control strategy in the feed mixing process is learned by using deep reinforcement learning. A multimodal information model is used as the environment, and an agent is introduced. The agent observes the environmental state and performs actions to maximize the preset reward function. Reinforcement learning algorithms are used to train and optimize the agent. Through interaction with the environment, the agent gradually learns the optimal control strategy in the feed production process, thereby optimizing the feed mixing state.

[0012] Preferably, in step S7, the methods for state classification and evaluation using machine learning algorithms include generative adversarial networks, reinforcement learning training and evaluation, and joint training. The generative adversarial network (GAN) consists of a generator network and a discriminator network. The generator network receives input from a multimodal information model and generates a set of synthetic feed mixture state samples. The discriminator network receives real feed mixture state samples and samples generated by the generator and performs classification judgment. Through adversarial training between the generator and the discriminator, the GAN gradually improves the generator's generation ability, making it difficult to distinguish the generated samples from real samples. The reinforcement learning training and evaluation involves introducing an agent whose policy network accepts input from a multimodal information model and outputs classification or evaluation results corresponding to the feed mixing state. The agent is trained through interaction with the environment, which includes generator and discriminator networks. The agent's goal is to guide the learning of classification and evaluation tasks by using the differences between samples generated by the generator and real samples. The joint training involves jointly training the generator network, discriminator network, and agent policy network. During training, the generator aims to generate samples similar to real feed mixture samples, the discriminator aims to distinguish between real samples and generated samples, and the agent aims to learn strategies for accurately classifying and evaluating feed mixtures. Through joint training, the generator, discriminator, and agent influence each other, gradually improving the accuracy and effectiveness of classification and evaluation.

[0013] Preferably, in step S8, the adjustment of the operating parameters of the feed production equipment includes stirring speed, feeding amount, temperature, humidity, conveying speed, and additives. The stirring speed adjustment: Based on the evaluation results of the feed mixing state, the control signal indicates the adjustment of the stirring speed or the position of the stirrer to ensure that the feed is fully mixed; The feed quantity adjustment: Based on the classification results and evaluation information of the feed mixing state, the control signal indicates the adjustment of the feed quantity and frequency to ensure the accuracy and uniformity of the feed composition; The temperature control: Based on the evaluation results of the feed mixing state, the control signal is used to adjust the heating or cooling system of the feed production equipment to maintain a suitable temperature range, which is conducive to the mixing and reaction of feed components; The humidity control: Based on the classification results and evaluation information of the feed mixing state, the control signal instructs the adjustment of the humidity control system of the feed production equipment to ensure that the humidity of the feed meets the requirements; The conveying speed adjustment: Based on the evaluation results of the feed mixing state, the control signal is used to adjust the speed of the feed conveying equipment to control the flowability and uniformity of the feed; The additive control involves using control signals to adjust the amount and timing of feed additives based on the classification results and evaluation information of the feed mixing state, ensuring the accuracy of feed composition and the precision of the formulation.

[0014] Preferably, in step S9, the method for updating the multimodal information model and optimization algorithm includes evolutionary algorithms, multi-objective optimization, ensemble learning, and reinforcement learning; The evolutionary algorithm performs genetic operations and weeding out the weakest elements in the parameters of the multimodal information model to achieve real-time updates and optimization of the model. Through continuous iteration and evolution, the model adapts to changes and demands in the feed production process, achieving better performance and results. The multi-objective optimization algorithm takes into account the mixing uniformity and component accuracy of the feed mixture state, and optimizes the model parameters. By weighing and compromising between different objectives, it realizes the real-time updating and optimization of the multi-modal information model to adapt to different production needs and quality standards. The ensemble learning integrates multiple different multimodal information models. By fusing and integrating the prediction results of different models, and through dynamic model updates and selective integration, the model can be updated and optimized in real time.

[0015] Compared with existing technologies, the beneficial effects of this invention are as follows: By collecting multi-dimensional sensor data and applying image monitoring and recognition technology, the mixing state in the feed production process can be accurately monitored and identified. The system can automatically detect and identify the mixing state, providing key information about feed quality and consistency. By fusing acoustic features, odor features, electromagnetic spectrum data, and image data, a multimodal information model of feed mixing state is established, providing a more comprehensive and accurate description of feed mixing state. The complementary nature of different sensor data is fully utilized to improve the understanding and evaluation of mixing state. Based on the multimodal information model and machine learning algorithms, the feed mixing state can be divided into multiple levels or continuous values, generating corresponding control signals. These control signals can be used to adjust the operating parameters of feed production equipment, achieving automatic control and optimization. Through continuous monitoring and feedback control, dynamic adjustment and optimization of the feed production process can be achieved, improving production efficiency and quality. By updating the multimodal information model and optimization algorithm in real time, the system can continuously learn and improve its ability to identify and control feed mixing state. Over time, the model can adapt to changes in raw material characteristics and production environment in different batches, continuously improving the efficiency and quality of feed production. Detailed Implementation

[0016] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0017] This invention provides a technical solution: a feed production control method based on image monitoring and identification of mixed states, the method steps of which include the following: S1. Collect multi-dimensional sensor data during the feed production process, including sound spectrum data, odor component data, and electromagnetic spectrum data; S2. Using acoustic signal processing technology, the collected sound spectrum data is analyzed and features are extracted to obtain acoustic features related to the feed mixing state. S3. Utilize gas sensors and gas analysis technology to measure and analyze the collected odor component data in order to obtain the odor characteristics during the feed mixing process; S4. Using non-contact sensing technology, electromagnetic spectrum data are collected during the feed production process to obtain information on the temperature, humidity and density of the materials. S5. Using computer vision technology and deep learning methods, feature extraction and target detection are performed on the collected image data to achieve object recognition and segmentation during the feed mixing process; S6. The object recognition and segmentation results are fused with acoustic features, odor features and electromagnetic spectrum data to establish a multimodal information model of feed mixing state, which is used to describe the degree and quality of feed mixing. S7. Based on the multimodal information model, machine learning algorithms are used to classify and evaluate the state, divide the mixed state of the feed into multiple levels or continuous values, and generate corresponding control signals. S8. Adjust the operating parameters of the feed production equipment according to the control signal to achieve automatic control and optimization in the feed production process; S9. Continuous monitoring and feedback control improves the efficiency and quality of feed production by updating multimodal information models and optimization algorithms in real time.

[0018] "Control signals" are signals generated based on state classification and evaluation results, used to adjust the operating parameters of feed production equipment in order to achieve automatic control and optimization in the feed production process.

[0019] This method achieves multi-dimensional monitoring and control of feed mixing status by collecting sound spectrum data, odor component data, and electromagnetic spectrum data, and fusing them with image data.

[0020] In a further improvement, in step S2, the acoustic signal processing technology includes one or more of the following: nonlinear dynamics analysis method, nonnegative matrix decomposition method, variant of Fourier transform method, and feature selection method based on swarm intelligence. By applying the theory and methods of nonlinear dynamics, and employing chaos analysis and complex network analysis, sound spectrum data is analyzed to reveal the nonlinear and dynamic characteristics of sound signals and identify dynamic patterns and interactions related to feed mixing state. By applying nonnegative matrix factorization (NMF) technology, the sound spectrum data is decomposed into basic audio components and mixing coefficients. Through the analysis of basic audio components and mixing coefficients, features related to the feed mixing state are extracted. A variant of Fourier transform is used, employing wavelet transform or singular spectrum analysis to transform and analyze the sound spectrum data, capturing the characteristics of the sound signal at different frequencies and time scales, thereby extracting information related to the feed mixing state. By applying swarm intelligence algorithms, such as genetic algorithms or particle swarm optimization algorithms, features in sound spectrum data are selected and optimized. Through an automated feature selection process, the features most relevant to the feed mixing state are found, thereby improving the performance of classification and evaluation.

[0021] This type of acoustic signal processing technology can provide new methods for the analysis and feature extraction of sound spectrum data. It can surpass traditional frequency domain and time domain analysis to extract richer and more accurate acoustic features, thereby achieving precise identification and control of feed mixing state.

[0022] In a further improvement, in step S3, the gas sensor and gas analysis technology include odor component measurement technology based on electronic olfaction, volatile organic compound component identification algorithm technology, odor analysis technology based on vibrational spectroscopy, odor analysis technology based on odor image processing, and odor identification technology based on electronic tongue and electronic nose. The odor component measurement technology based on electronic olfaction uses an electronic olfaction sensor, employing an electrochemical sensor or a surface plasmon sensor, to measure odor components. It simulates the human olfactory system and achieves accurate capture and analysis of odor characteristics during feed mixing by sensing the volatile organic compounds released by the odor components. The volatile organic compound (VOC) component identification algorithm utilizes machine learning and pattern recognition technologies, including feature selection, feature extraction, and classifier design steps, to achieve accurate analysis and identification of odor component data, and to identify and extract features related to feed mixing status from odor component data. The vibrational spectroscopy-based odor analysis technology uses vibrational spectroscopy, employing Raman or infrared spectroscopy, to analyze odor components. By measuring the vibrational frequency and spectral characteristics of gas molecules, it identifies and quantifies the composition and concentration of odor components. When applied to feed mixing, it can achieve high-precision measurement and analysis of odor characteristics. The odor analysis technology based on odor image processing utilizes odor image processing technology to perform image processing and analysis on odor component data, converting the odor component data into odor distribution maps or odor pattern maps, and extracting odor features related to the feed mixing state by analyzing the texture, color and shape features of the images. The odor identification technology based on electronic tongue and electronic nose uses the principles of electronic tongue and electronic nose to identify odors during the feed mixing process. The electronic tongue simulates a taste system to identify different odors by measuring the chemical properties of odor components, while the electronic nose simulates an olfaction system to identify and quantify odors by measuring the volatile organic compounds of odor components.

[0023] These gas sensors and gas analysis technologies are used to measure and analyze the collected odor component data. They surpass traditional gas sensing and analysis technologies, enabling accurate capture, identification, and analysis of odor characteristics during feed mixing, thereby providing more comprehensive information and decision-making basis for feed production control and optimization.

[0024] In a further improvement, in step S4, the non-contact sensing technology includes laser interferometry imaging technology, ultra-wideband radar technology, infrared spectral imaging technology, and microwave imaging technology. The laser interferometric imaging technology obtains electromagnetic spectrum data by measuring the difference in optical path length on the feed surface, projects laser light onto the feed surface using the laser interference phenomenon, and deduces the propagation and characteristics of electromagnetic waves in the feed by analyzing the changes in the interference pattern. Through the analysis and processing of the interference pattern, electromagnetic spectrum data during the feed mixing process is obtained, including the temperature, humidity and density information of the material. The ultra-wideband radar technology sends a series of wideband electromagnetic pulse signals and receives echo signals. By analyzing the amplitude and phase information of the echo signals, electromagnetic spectrum data during the feed mixing process can be obtained. The ultra-wideband radar technology has high resolution and anti-interference capabilities, and can achieve accurate measurement and analysis of the electromagnetic wave characteristics in feed. The infrared spectral imaging technology obtains electromagnetic spectrum data by measuring the infrared radiation on the surface of the feed. Based on the infrared spectral characteristics emitted by the object, the feed is imaged by an infrared camera or infrared thermal imager, and the image is processed and analyzed to extract electromagnetic spectrum data related to the mixing state of the feed. The microwave imaging technology acquires electromagnetic spectrum data by measuring the propagation and scattering characteristics of microwaves in feed. By utilizing the penetration and reflection characteristics of microwaves, the transmitted and received microwave signals are analyzed and processed to acquire electromagnetic spectrum data during the feed mixing process.

[0025] Laser interferometric imaging, ultra-wideband radar, infrared spectral imaging, and microwave imaging technologies are used to collect electromagnetic spectrum data in the feed production process in a non-contact manner. By measuring and analyzing the propagation, interference, radiation, and scattering characteristics of electromagnetic waves in feed, accurate description and evaluation of the feed mixing state can be achieved, providing more comprehensive information and decision-making basis for feed production control and optimization.

[0026] In a further improvement, in step S5, the computer vision technology and deep learning method include photonic compressed sensing technology, optical coding array technology, photonic compressed sensing image sensor technology, deep learning decoding network technology and target detection and segmentation technology. The photonic compressed sensing technology is based on photonic principles and uses photonic devices to optically encode and decode images, thereby achieving efficient extraction of image information. The optical coding array technology encodes the acquired image data, encodes and compresses the image in the optical domain, and converts the image information into a sparse representation in the optical domain. The photonic compressed sensing image sensor technology is used to acquire optically encoded image data. It combines an optical encoding array and a high-sensitivity photonic detector to directly obtain optically encoded sparse image data. The deep learning decoding network technology is used to recover the features of the target image from optically encoded and sampled image data. It combines the powerful feature extraction and reconstruction capabilities of deep learning to restore high-quality image features from sparse optically encoded data. The target detection and segmentation technology identifies and segments objects during the feed mixing process. It combines a deep learning target detection model with the recovered image features to perform accurate target detection and segmentation, and obtains target information related to the feed mixing state.

[0027] This image feature extraction and target detection scheme based on photonic compressed sensing achieves efficient extraction of image data through optical encoding and decoding techniques, and combines it with a deep learning decoding network for feature recovery. This enables accurate detection and segmentation of objects during feed mixing, providing more accurate and comprehensive visual information for feed production control and improving the efficiency and quality of the feed production process.

[0028] In a further improvement, step S6 includes the following fusion methods: multimodal feature fusion network, feature alignment and mapping, cross-modal feature fusion, multimodal information fusion, and construction of a multimodal information model. Specific methods for fusing the object recognition and segmentation results with acoustic features, odor features, and electromagnetic spectrum data include: Design a novel multimodal feature fusion network that receives object recognition and segmentation results, acoustic features, odor features, and electromagnetic spectrum data as input, and automatically learns the correlations between them; Since different sensor data have different scales, resolutions, and representations, feature alignment and mapping are performed. By using generative adversarial networks (GANs) or self-attention mechanisms, features from different modalities are mapped into a unified feature space for subsequent fusion. After feature alignment and mapping, the spatial information of object recognition and segmentation results, the spectral information of acoustic features, the chemical composition of odor features, and the physical parameters of electromagnetic spectrum data are fused across modally. Deep neural networks and attention mechanisms are used to learn the weights and correlations between cross-modal features. By fusing cross-modal features, fusion layers, attention mechanisms, or graph convolutional neural networks can be used to effectively capture the interactions and dependencies between different modalities and generate fused multimodal information. The fused multimodal information is input into a machine learning model, and deep neural networks, support vector machines, or random forests are used to classify and evaluate the feed mixing state, learn the complex relationships between different modal information, and thus accurately describe the degree and quality of feed mixing.

[0029] This multimodal feature fusion method can effectively integrate object recognition and segmentation results with acoustic features, odor features and electromagnetic spectrum data, making full use of information from multiple sources to improve the understanding and control of feed mixing status.

[0030] In a further improvement, in step S6, the multimodal information model for establishing the feed mixing state is a multimodal graph neural network model based on deep reinforcement learning, including a graph neural network, a deep reinforcement learning method, and reinforcement learning training and optimization, used to model and control the feed mixing state; Graph neural networks are used to construct graph structures representing object recognition and segmentation results, acoustic features, odor features and electromagnetic spectrum data. Each sensor data corresponds to a node in the graph, and nodes are connected by edges. The edges represent the correlation between different sensor data. The graph neural network transmits and aggregates information in the graph structure, thereby capturing the complex relationships between multimodal data. The optimal control strategy in the feed mixing process is learned by using deep reinforcement learning. A multimodal information model is introduced as the environment, and an agent is introduced. The agent observes the environmental state (output of the multimodal information model) and performs actions (adjusting the operating parameters of the feed production equipment) to maximize the preset reward function (e.g., the optimization objectives of feed mixing degree and quality).

[0031] Reinforcement learning algorithms (such as deep Q-networks, policy gradients, etc.) are used to train and optimize the agent. Through interaction with the environment, the agent gradually learns the optimal control strategy in the feed production process to optimize the feed mixing state.

[0032] This multimodal graph neural network model based on deep reinforcement learning can represent data of different modalities in a graph structure and learn the optimal control strategy through reinforcement learning. Compared with traditional models, it combines graph neural networks and deep reinforcement learning to solve the modeling and control problems of feed mixing state.

[0033] Further improvements include, in step S7, the methods for state classification and evaluation using machine learning algorithms include generative adversarial networks, reinforcement learning training and evaluation, and joint training. The generative adversarial network (GAN) consists of a generator network and a discriminator network. The generator network receives input from a multimodal information model (including object recognition and segmentation results, acoustic features, odor features, and electromagnetic spectrum data) and generates a set of synthetic feed mixture state samples. The discriminator network receives real feed mixture state samples and samples generated by the generator and performs classification judgment. Through adversarial training between the generator and the discriminator, the GAN gradually improves the generator's generation ability, making its generated samples difficult to distinguish from real samples. The reinforcement learning training and evaluation involves introducing an agent whose policy network accepts input from a multimodal information model and outputs classification or evaluation results corresponding to the feed mixing state. The agent is trained through interaction with the environment, which includes generator and discriminator networks. The agent's goal is to guide the learning of classification and evaluation tasks by using the differences between samples generated by the generator and real samples. The joint training involves jointly training the generator network, discriminator network, and agent policy network. During training, the generator aims to generate samples similar to real feed mixture samples, the discriminator aims to distinguish between real samples and generated samples, and the agent aims to learn strategies for accurately classifying and evaluating feed mixtures. Through joint training, the generator, discriminator, and agent influence each other, gradually improving the accuracy and effectiveness of classification and evaluation.

[0034] The joint training framework based on GAN and reinforcement learning can learn the classification and evaluation of feed mixing states from multimodal information models by combining generative adversarial learning and reinforcement learning.

[0035] In a further improvement, in step S8, the adjustment of the operating parameters of the feed production equipment includes stirring speed, feeding amount, temperature, humidity, conveying speed, and additives. The stirring speed adjustment: Based on the evaluation results of the feed mixing state, the control signal can instruct the adjustment of the stirring speed or the position of the stirrer to ensure that the feed is fully mixed; The feed quantity adjustment: Based on the classification results and evaluation information of the feed mixing state, the control signal can indicate the adjustment of the feed quantity and frequency to ensure the accuracy and uniformity of the feed composition; Temperature control: Based on the evaluation results of the feed mixing state, the control signal can be used to adjust the heating or cooling system of the feed production equipment to maintain a suitable temperature range to facilitate the mixing and reaction of feed components; The humidity control: Based on the classification results and evaluation information of the feed mixing state, the control signal can instruct the adjustment of the humidity control system of the feed production equipment to ensure that the feed humidity meets the requirements; The conveying speed adjustment: Based on the evaluation results of the feed mixing state, the control signal can be used to adjust the speed of the feed conveying equipment in order to control the flowability and uniformity of the feed; The additive control: Based on the classification results and evaluation information of the feed mixing state, the control signal can indicate the adjustment of the dosage and timing of feed additives to ensure the accuracy of feed composition and the precision of the formulation.

[0036] To achieve automatic control and optimization of the feed production process, thereby improving the efficiency and quality of feed production.

[0037] Specifically, in step S9, the method for updating the multimodal information model and optimization algorithm includes evolutionary algorithms, multi-objective optimization, ensemble learning, and reinforcement learning; The evolutionary algorithm performs genetic operations and weeding out the weakest elements in the multimodal information model to achieve real-time updates and optimization of the model. Through continuous iteration and evolution, the model can adapt to changes and demands in the feed production process and achieve better performance and results. The multi-objective optimization algorithm takes into account the mixing uniformity and component accuracy of the feed mixture state, and optimizes the model parameters. By weighing and compromising between different objectives, it realizes the real-time updating and optimization of the multi-modal information model to adapt to different production needs and quality standards. The ensemble learning integrates multiple different multimodal information models. By fusing and integrating the prediction results of different models, it improves the accuracy and stability of feed mixing state. Through dynamic model updates and selective integration, it achieves real-time model updates and optimization.

[0038] The reinforcement learning technology establishes an agent that interacts with the feed production process. Through trial and error and reward mechanisms, the agent updates the parameters and strategies of the multimodal information model in real time to optimize the control and quality of feed mixing, thereby enabling the model to continuously learn and self-optimize, and adapt to the ever-changing production environment.

[0039] In summary, by collecting multi-dimensional sensor data and applying image monitoring and recognition technology, the mixing state in the feed production process can be accurately monitored and identified. The system can automatically detect and identify the mixing state, providing key information about feed quality and consistency. By fusing acoustic features, odor features, electromagnetic spectrum data, and image data, a multimodal information model of feed mixing state is established, providing a more comprehensive and accurate description of the feed mixing state. This fully utilizes the complementarity of different sensor data, improving the understanding and evaluation capabilities of the mixing state. Based on the multimodal information model and machine learning algorithms, the feed mixing state can be divided into multiple levels or continuous values, generating corresponding control signals. These control signals can be used to adjust the operating parameters of feed production equipment, achieving automatic control and optimization. Through continuous monitoring and feedback control, dynamic adjustment and optimization of the feed production process can be achieved, improving production efficiency and quality. By updating the multimodal information model and optimization algorithms in real time, the system can continuously learn and improve its ability to identify and control feed mixing states. Over time, the model can adapt to changes in raw material characteristics and production environment in different batches, continuously improving the efficiency and quality of feed production.

[0040] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or basic characteristics. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, it is intended that all variations falling within the meaning and scope of equivalents of the claims be included within the present invention.

[0041] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A feed production control method based on image monitoring and identification of mixed states, characterized in that, The method and steps include the following: S1. Collect multi-dimensional sensor data during the feed production process, including sound spectrum data, odor component data, and electromagnetic spectrum data; S2. Using acoustic signal processing technology, the collected sound spectrum data is analyzed and features are extracted to obtain acoustic features related to the feed mixing state. S3. Utilize gas sensors and gas analysis technology to measure and analyze the collected odor component data in order to obtain the odor characteristics during the feed mixing process; S4. Using non-contact sensing technology, electromagnetic spectrum data are collected during the feed production process to obtain information on the temperature, humidity and density of the materials. S5. Using computer vision technology and deep learning methods, feature extraction and target detection are performed on the collected image data to achieve object recognition and segmentation during the feed mixing process; S6. The object recognition and segmentation results are fused with acoustic features, odor features and electromagnetic spectrum data to establish a multimodal information model of feed mixing state, which is used to describe the degree and quality of feed mixing. S7. Based on the multimodal information model, machine learning algorithms are used to classify and evaluate the state, divide the mixed state of the feed into multiple levels or continuous values, and generate corresponding control signals. S8. Adjust the operating parameters of the feed production equipment according to the control signal to achieve automatic control and optimization in the feed production process; S9. Continuous monitoring and feedback control improves the efficiency and quality of feed production by updating multimodal information models and optimization algorithms in real time.

2. The feed production control method based on image monitoring and identification of mixed states according to claim 1, characterized in that: In step S2, the acoustic signal processing techniques include nonlinear dynamics analysis methods, nonnegative matrix decomposition methods, variants of Fourier transform methods, and feature selection methods based on swarm intelligence. By applying the theory and methods of nonlinear dynamics, and employing chaos analysis and complex network analysis, sound spectrum data is analyzed to reveal the nonlinear and dynamic characteristics of sound signals and identify dynamic patterns and interactions related to feed mixing state. By applying nonnegative matrix factorization, the sound spectrum data is decomposed into basic audio components and mixing coefficients. Through the analysis of basic audio components and mixing coefficients, features related to the feed mixing state are extracted. A variant of Fourier transform is used, employing wavelet transform or singular spectrum analysis to transform and analyze the sound spectrum data, capturing the characteristics of the sound signal at different frequencies and time scales, thereby extracting information related to the feed mixing state. By applying swarm intelligence algorithms, such as genetic algorithms or particle swarm optimization algorithms, features in sound spectrum data are selected and optimized. Through an automated feature selection process, the features most relevant to the feed mixing state are found, thereby improving the performance of classification and evaluation.

3. The feed production control method based on image monitoring and identification of mixed states according to claim 1, characterized in that: In step S3, the gas sensor and gas analysis technology include odor component measurement technology based on electronic olfaction, volatile organic compound component identification algorithm technology, odor analysis technology based on vibration spectrum, odor analysis technology based on odor image processing, and odor identification technology based on electronic tongue and electronic nose. The odor component measurement technology based on electronic olfaction uses an electronic olfaction sensor, employing an electrochemical sensor or a surface plasmon sensor, to measure odor components. It simulates the human olfactory system and achieves accurate capture and analysis of odor characteristics during feed mixing by sensing the volatile organic compounds released by the odor components. The volatile organic compound (VOC) component identification algorithm utilizes machine learning and pattern recognition technologies, including feature selection, feature extraction, and classifier design steps, to achieve accurate analysis and identification of odor component data, and to identify and extract features related to feed mixing status from odor component data. The vibrational spectroscopy-based odor analysis technology uses vibrational spectroscopy, employing Raman or infrared spectroscopy, to analyze odor components. By measuring the vibrational frequency and spectral characteristics of gas molecules, it identifies and quantifies the composition and concentration of odor components, enabling high-precision measurement and analysis of odor characteristics during feed mixing. The odor analysis technology based on odor image processing utilizes odor image processing technology to perform image processing and analysis on odor component data, converting the odor component data into odor distribution maps or odor pattern maps, and extracting odor features related to the feed mixing state by analyzing the texture, color and shape features of the images. The odor identification technology based on electronic tongue and electronic nose uses the principles of electronic tongue and electronic nose to identify odors during the feed mixing process. The electronic tongue simulates a taste system to identify different odors by measuring the chemical properties of odor components, while the electronic nose simulates an olfaction system to identify and quantify odors by measuring the volatile organic compounds of odor components.

4. The feed production control method based on image monitoring and identification of mixed states according to claim 1, characterized in that: In step S4, the non-contact sensing technology includes laser interferometric imaging technology, ultra-wideband radar technology, infrared spectral imaging technology, and microwave imaging technology. The laser interferometric imaging technology obtains electromagnetic spectrum data by measuring the difference in optical path length on the feed surface, projects laser light onto the feed surface using the laser interference phenomenon, and deduces the propagation and characteristics of electromagnetic waves in the feed by analyzing the changes in the interference pattern. Through the analysis and processing of the interference pattern, electromagnetic spectrum data during the feed mixing process is obtained, including the temperature, humidity and density information of the material. The ultra-wideband radar technology sends a series of wideband electromagnetic pulse signals and receives echo signals. By analyzing the amplitude and phase information of the echo signals, electromagnetic spectrum data during the feed mixing process can be obtained. The infrared spectral imaging technology obtains electromagnetic spectrum data by measuring the infrared radiation on the surface of the feed. Based on the infrared spectral characteristics emitted by the object, the feed is imaged by an infrared camera or infrared thermal imager, and the image is processed and analyzed to extract electromagnetic spectrum data related to the mixing state of the feed. The microwave imaging technology acquires electromagnetic spectrum data by measuring the propagation and scattering characteristics of microwaves in feed. By utilizing the penetration and reflection characteristics of microwaves, the transmitted and received microwave signals are analyzed and processed to acquire electromagnetic spectrum data during the feed mixing process.

5. The feed production control method based on image monitoring and identification of mixed states according to claim 1, characterized in that: In step S5, the computer vision technology and deep learning method include photonic compressed sensing technology, optical coding array technology, photonic compressed sensing image sensor technology, deep learning decoding network technology and target detection and segmentation technology. The photonic compressed sensing technology is based on photonic principles and uses photonic devices to optically encode and decode images, thereby achieving efficient extraction of image information. The optical coding array technology encodes the acquired image data, encodes and compresses the image in the optical domain, and converts the image information into a sparse representation in the optical domain. The photonic compressed sensing image sensor technology is used to acquire optically encoded image data. It combines an optical encoding array and a high-sensitivity photonic detector to directly obtain optically encoded sparse image data. The deep learning decoding network technology is used to recover the features of the target image from optically encoded and sampled image data. It combines the powerful feature extraction and reconstruction capabilities of deep learning to restore high-quality image features from sparse optically encoded data. The target detection and segmentation technology identifies and segments objects during the feed mixing process. It combines a deep learning target detection model with the recovered image features to perform accurate target detection and segmentation, and obtains target information related to the feed mixing state.

6. The feed production control method based on image monitoring and identification of mixed states according to claim 1, characterized in that: In step S6, the fusion method includes multimodal feature fusion network, feature alignment and mapping, cross-modal feature fusion, multimodal information fusion, and construction of multimodal information model; The method for fusing the object recognition and segmentation results with acoustic features, odor features, and electromagnetic spectrum data specifically includes: Design a novel multimodal feature fusion network that receives object recognition and segmentation results, acoustic features, odor features, and electromagnetic spectrum data as input, and automatically learns the correlations between them; Since the scale, resolution, and representation of data from different sensors vary, feature alignment and mapping are performed. By using adversarial generative networks or self-attention mechanisms, features from different modalities are mapped to a unified feature space for subsequent fusion. After feature alignment and mapping, the spatial information of object recognition and segmentation results, the spectral information of acoustic features, the chemical composition of odor features, and the physical parameters of electromagnetic spectrum data are fused across modally. Deep neural networks and attention mechanisms are used to learn the weights and correlations between cross-modal features. The features obtained after cross-modal fusion are fused together, and fusion layers, attention mechanisms or graph convolutional neural network methods are used to capture the interaction and dependency between different modalities and generate fused multimodal information. The fused multimodal information is input into a machine learning model, and deep neural networks, support vector machines, or random forests are used to classify and evaluate the feed mixing state, learn the complex relationships between different modal information, and thus accurately describe the degree and quality of feed mixing.

7. The feed production control method based on image monitoring and identification of mixed states according to claim 6, characterized in that: In step S6, the multimodal information model for establishing the feed mixing state is a multimodal graph neural network model based on deep reinforcement learning, which includes graph neural network, deep reinforcement learning method and reinforcement learning training and optimization, and is used to model and control the feed mixing state. Graph neural networks are used to construct graph structures representing object recognition and segmentation results, acoustic features, odor features and electromagnetic spectrum data. Each sensor data corresponds to a node in the graph, and nodes are connected by edges. The edges represent the correlation between different sensor data. The graph neural network transmits and aggregates information in the graph structure and captures the complex relationships between multimodal data. The optimal control strategy in the feed mixing process is learned by using deep reinforcement learning. A multimodal information model is used as the environment, and an agent is introduced. The agent observes the environmental state and performs actions to maximize the preset reward function. Reinforcement learning algorithms are used to train and optimize the agent. Through interaction with the environment, the agent gradually learns the optimal control strategy in the feed production process, thereby optimizing the feed mixing state.

8. The feed production control method based on image monitoring and identification of mixed states according to claim 1, characterized in that: In step S7, the methods for state classification and evaluation using machine learning algorithms include generative adversarial networks, reinforcement learning training and evaluation, and joint training. The generative adversarial network (GAN) consists of a generator network and a discriminator network. The generator network receives input from a multimodal information model and generates a set of synthetic feed mixture state samples. The discriminator network receives real feed mixture state samples and samples generated by the generator and performs classification judgment. Through adversarial training between the generator and the discriminator, the GAN gradually improves the generator's generation ability, making it difficult to distinguish the generated samples from real samples. The reinforcement learning training and evaluation involves introducing an agent whose policy network accepts input from a multimodal information model and outputs classification or evaluation results corresponding to the feed mixing state. The agent is trained through interaction with the environment, which includes generator and discriminator networks. The agent's goal is to guide the learning of classification and evaluation tasks by using the differences between samples generated by the generator and real samples. The joint training involves jointly training the generator network, discriminator network, and agent policy network. During training, the generator aims to generate samples similar to real feed mixture samples, the discriminator aims to distinguish between real samples and generated samples, and the agent aims to learn strategies for accurately classifying and evaluating feed mixtures. Through joint training, the generator, discriminator, and agent influence each other, gradually improving the accuracy and effectiveness of classification and evaluation.

9. The feed production control method based on image monitoring and identification of mixed states according to claim 1, characterized in that: In step S8, adjusting the operating parameters of the feed production equipment includes stirring speed, feeding amount, temperature, humidity, conveying speed, and additives. The stirring speed adjustment: Based on the evaluation results of the feed mixing state, the control signal indicates the adjustment of the stirring speed or the position of the stirrer to ensure that the feed is fully mixed; The feed quantity adjustment: Based on the classification results and evaluation information of the feed mixing state, the control signal indicates the adjustment of the feed quantity and frequency to ensure the accuracy and uniformity of the feed composition; The temperature control: Based on the evaluation results of the feed mixing state, the control signal is used to adjust the heating or cooling system of the feed production equipment to maintain a suitable temperature range, which is conducive to the mixing and reaction of feed components; The humidity control: Based on the classification results and evaluation information of the feed mixing state, the control signal instructs the adjustment of the humidity control system of the feed production equipment to ensure that the humidity of the feed meets the requirements; The conveying speed adjustment: Based on the evaluation results of the feed mixing state, the control signal is used to adjust the speed of the feed conveying equipment to control the flowability and uniformity of the feed; The additive control involves using control signals to adjust the amount and timing of feed additives based on the classification results and evaluation information of the feed mixing state, ensuring the accuracy of feed composition and the precision of the formulation.

10. The feed production control method based on image monitoring and identification of mixed states according to claim 1, characterized in that: In step S9, the methods for updating the multimodal information model and optimization algorithm include evolutionary algorithms, multi-objective optimization, ensemble learning, and reinforcement learning; The evolutionary algorithm performs genetic operations and weeding out the weakest elements in the parameters of the multimodal information model to achieve real-time updates and optimization of the model. Through continuous iteration and evolution, the model adapts to changes and demands in the feed production process, achieving better performance and results. The multi-objective optimization algorithm takes into account the mixing uniformity and component accuracy of the feed mixture state, and optimizes the model parameters. By weighing and compromising between different objectives, it realizes the real-time updating and optimization of the multi-modal information model to adapt to different production needs and quality standards. The ensemble learning integrates multiple different multimodal information models. By fusing and integrating the prediction results of different models, and through dynamic model updates and selective integration, the model can be updated and optimized in real time.