Intelligent feeding system and method based on multi-modal space-time fusion and self-evolution correction

Abstract

The invention relates to the technical field of intelligent aquaculture, and provides an intelligent feeding method based on multi-modal space-time fusion and self-evolution correction, which comprises the following steps: step 1, system initialization and multi-source parameter setting; step 2, synchronously acquiring multi-modal sensing data and reading a hardware state; 3, audio-visual signal enhancement processing and feature extraction: in order to accurately extract visual and auditory effective features corresponding to the ingestion activities of the fish schools, firstly eliminating environment and equipment interference, then quantifying the features and generating ingestion indexes; 4, multi-physics field decoupling and net metabolism oxygen consumption rate inversion are carried out; 5, performing multi-modal information fusion and safety gating based on the variety adaptation weight; step 6, closed-loop feeding control based on the final fusion index and the variety rhythm strategy; the method can adapt to environmental conditions of different regions, can accurately match physiological habits and feeding characteristics of different breeding varieties, and thoroughly avoids feeding deviation caused by region adaptation deficiency and variety habit misjudgment.

Description

Technical Field

[0001] This invention relates to the field of intelligent aquaculture technology, specifically to an intelligent feeding system and method based on multimodal spatiotemporal fusion and self-evolutionary correction. Background Technology

[0002] Feeding in aquaculture is a core technical operation in the aquaculture production process. It refers to the production behavior in which aquaculture personnel, based on the biological characteristics, growth stage, nutritional needs of the cultured organisms and the environmental parameters of the culture water, release appropriate feed into the culture water at fixed times, in fixed quantities, and at fixed locations to meet the feeding needs of the cultured organisms and promote their healthy and rapid growth.

[0003] Currently, there are three main types of traditional aquaculture feeding techniques: First, manual feeding, where farmers rely on their daily experience to judge the feeding status of the fish, the amount of feed, and the feeding rhythm. There are no unified quantitative standards, and it depends entirely on personal subjective judgment. Second, mechanical quantitative feeding, which uses a timer or a simple switch to control the feeder and feeds the fish at fixed times and in fixed amounts, without considering the fish's real-time feeding needs and environmental changes.

[0004] Traditional aquaculture feeding methods have significant drawbacks: manual feeding relies entirely on the subjective experience of the farmers, lacking standardized criteria for judging the feeding status of the fish, the amount of feed, and the feeding rhythm. This easily leads to overfeeding or underfeeding due to differences in experience, and is inefficient, making it unsuitable for large-scale aquaculture. Mechanical quantitative feeding, on the other hand, uses timers or simple switches to deliver feed at fixed times and in fixed amounts, lacking consideration for the real-time feeding needs of the fish and the dynamic changes in the aquaculture environment. This results in extremely poor flexibility, easily causing feed waste and water pollution, and may also affect fish growth due to untimely feeding, making it difficult to meet the basic requirements of precision aquaculture.

[0005] To this end, we propose an intelligent feeding system and method based on multimodal spatiotemporal fusion and self-evolutionary correction. Summary of the Invention

[0006] This invention proposes an intelligent feeding system and method based on multimodal spatiotemporal fusion and self-evolutionary correction. It solves the problems of related technologies, such as: manual feeding relying on subjective experience, lack of unified quantitative standards, low efficiency and inability to adapt to large-scale aquaculture; mechanical quantitative feeding with fixed-time dosage, lack of consideration for the real-time feeding needs of fish and environmental changes, poor flexibility and easy to cause feed waste and water pollution; and basic intelligent feeding relying on only a single perception dimension, lacking regional and species adaptation mechanisms, lacking core parameter self-updating ability, and unable to dynamically adapt to changes in the aquaculture environment and fish habits, thus making it difficult to achieve precise feeding, save aquaculture costs and protect the aquatic environment.

[0007] The technical solution of the present invention is as follows:

[0008] The intelligent feeding method based on multimodal spatiotemporal fusion and self-evolutionary correction includes the following steps:

[0009] Step 1: System Initialization and Multi-Source Parameter Setting

[0010] Step 2: Synchronous acquisition of multimodal sensing data and reading of hardware status;

[0011] Step 3: Audiovisual signal enhancement and feature extraction. To accurately extract the effective visual and auditory features corresponding to the feeding activities of fish, it is necessary to first eliminate environmental and equipment interference, then quantify the features and generate a feeding index. This is carried out in an orderly manner through the following steps:

[0012] (3-1) Aerator status judgment and dynamic interference mask loading;

[0013] (3-2) Calculation of pixel motion vector field using Farneback dense optical flow method;

[0014] (3-3) Calculation of water surface disturbance energy;

[0015] (3-4) Normalization of visual food intake index;

[0016] (3-5) Sound signal beamforming and aerator noise suppression;

[0017] (3-6) Spectral subtraction and noise suppression;

[0018] (3-7) Calculation and normalization of auditory feeding index;

[0019] Step 4: Multiphysics field decoupling and net metabolic oxygen consumption rate inversion;

[0020] Step 5: Multimodal information fusion and security gating based on variety adaptation weights;

[0021] Step Six: Closed-loop feeding control based on the final fusion index and variety rhythm strategy;

[0022] Step 7: Monitoring the feeding process and limiting the total amount of food fed;

[0023] Step 8: Self-evolutionary correction and self-calibration throughout the entire life cycle.

[0024] Preferably, in step one, the user inputs the geographical information and salinity value of the aquaculture water body through an interactive interface; the system loads the corresponding saturated dissolved oxygen calculation benchmark based on the input geographical information and salinity value; the user selects the current aquaculture species from the preset aquaculture species list, and the system loads the corresponding feature vector from the built-in species characteristic adaptation database based on the selected species.

[0025] Preferably, in step two, multi-source time-series data is collected simultaneously; the visual perception unit collects video image sequences of the water surface in the feeding area, filtering out specular reflections from the water surface using a linear polarization filter integrated in front of the lens; the auditory perception unit collects sound signals from the feeding area and its surroundings through a linear array composed of multiple microelectromechanical system microphones; and the environmental monitoring unit collects the dissolved oxygen concentration of the water in real time. Water temperature Real-time atmospheric pressure is collected via an onboard atmospheric pressure sensor. The execution linkage layer collects real-time operating status signals of the aerator. It also includes its operating current data; all collected data is timestamped and cached via edge computing terminals to form a spatiotemporally synchronized multimodal data stream.

[0026] As a preferred option, in order to accurately extract the effective visual and auditory features corresponding to the feeding activities of fish, step three requires first eliminating environmental and equipment interference, then quantifying the features and generating a feeding index.

[0027] (3-1) The edge computing terminal first reads the real-time status of the aerator. To determine whether the aerator is on, when the aerator is on, in order to eliminate the interference of water surface disturbance caused by the aerator on the visual signal, the system loads a preset dynamic interference mask in the image coordinate system. ;

[0028] In step (3-2), for pixels outside the mask region, the system uses the Farneback dense optical flow method to calculate the pixel's motion vector field. In other words, the motion speed and direction of each pixel in the image. The Farneback optical flow method is based on the changes in the gray values ​​of the image and derives the motion information of each pixel to obtain the displacement vector of each pixel.

[0029] In step (3-3), based on the pixel motion vector field calculated by the Farneback optical flow method, the system calculates the water surface disturbance energy. The specific calculation formula is as follows:

[0030]

[0031] in, This represents the energy of water surface disturbance. For masking on pixels The value at that location, It is the square of the motion vector field of the pixel, representing the perturbation energy at that location. When the value is zero, it indicates that the position is inside the mask, and the motion energy is not included in the disturbance energy calculation; if If the value is 1, then that location is a valid area, and the disturbance energy will be taken into account;

[0032] The calculated water surface disturbance energy is described in (3-4). Normalization yields the visual food intake index. The normalization process is accomplished using the following formula:

[0033]

[0034] Normalized visual food intake index The value is between 0 and 1. A larger value indicates more obvious surface disturbance and more active feeding activities of the fish.

[0035] As described in (3-5), for the audio signals received by the microphone array, the system uses a delay-sum beamforming algorithm for processing. The delay-sum beamforming algorithm adjusts the phase delay of each microphone channel... The signals received from multiple microphones are weighted and combined to form a beam pointing in a specific direction. The goal of beamforming is to focus the beam on the direction of the material's landing point. The signal after beamforming... Calculated using the following formula:

[0036]

[0037] in, It is the signal after beamforming. It is the first The signal received by each microphone These are the corresponding weighting coefficients. It is the first Phase delay of each microphone, It is time;

[0038] The signal after beamforming (3-6) Further spectral subtraction processing is performed to remove steady-state mechanical noise generated by aerators and other equipment;

[0039] As stated in (3-7), for the processed auditory signal, the system performs beamforming on the signal within the acoustic characteristic frequency band specified by the variety feature vector. To integrate and calculate the energy of the biological impact sound, the specific integration operation is as follows:

[0040]

[0041] in, It refers to the energy of the signal. The integration operation integrates the signal within a specified frequency band to calculate the total energy of the biological splashing sound, and the normalized auditory feeding index is obtained. It also ranges between 0 and 1. The larger the value, the stronger the activity of aquatic organisms, indicating that the fish are more active in feeding.

[0042] Preferably, in step four, the edge computing terminal invokes a multiphysics decoupling engine to calculate the saturated dissolved oxygen value after corrections for air pressure, temperature, and salinity. :

[0043]

[0044] in, It's atmospheric pressure. It is the saturated vapor pressure of water, which is related to temperature. Related, It is a temperature correction function, which represents the effect of temperature on dissolved oxygen; It is the salinity value of the water. It is the salinity correction constant, used to correct the effect of salinity on dissolved oxygen. 14.652 is an empirical constant, which refers to the constant value per unit temperature and unit pressure under standard conditions of 25°C, 101.3 kPa and 1 atm.

[0045] Based on the status of the aerator Oxygenation capacity of aerator and the basic oxygen demand of water bodies Invert the net metabolic oxygen consumption rate of the fish population The calculation formula is:

[0046]

[0047] in, It represents the rate of change of dissolved oxygen concentration over time, reflecting the rate at which oxygen is consumed in the water. It is a real-time measured dissolved oxygen concentration. This is the corrected saturated dissolved oxygen concentration. By comparing the measured changes in dissolved oxygen with the factors affecting oxygen consumption, the net metabolic oxygen consumption rate of the fish population can be deduced. This refers to the amount of oxygen consumed by the fish per unit of time. This value reflects the physiological activities and feeding needs of the fish, providing a basis for intelligent feeding.

[0048] Preferably, in step four, the calculated... After normalization, the chemical intake index is obtained. This is used to quantify the feeding behavior of fish schools. The normalization process is accomplished using the following formula:

[0049]

[0050] in, The biggest of all historical moments The chemical feeding index should be maintained between 0 and 1; a higher value indicates more active feeding behavior in the fish. It provides a feeding requirement index for fish populations based on metabolism and oxygen consumption.

[0051] Preferably, in step five, the edge computing terminal adjusts the weights in the variety feature vector loaded in step one according to the weight correction coefficients. , , Visual food intake index Auditory feeding index Chemical intake index Weighted fusion is performed to generate a preliminary fusion feeding index. :

[0052]

[0053] The fault safety management module is invoked to perform a safety status assessment and monitor atmospheric pressure. The rate of sudden drop, and checking the continuity and validity of data from each sensor;

[0054] If a Level 1 meteorological risk or a Level 2 sensor malfunction is detected, a downgraded safety threshold coefficient is generated. ;

[0055] The initial integration of the food intake index With security gating coefficient Multiplying them together yields the final fusion index used for feeding decisions. ,Right now .

[0056] Preferably, in step six, the system determines the final fusion index. Does it exceed the minimum investment threshold set for this variety? ;like If so, the feeder will remain stopped; if Then, according to the proportional gain coefficient Calculate the real-time duty cycle control signal of the feeding motor ;

[0057]

[0058] in The calculation result is the minimum start-up duty cycle. Limited to between 0% and 100%; the calculated The control signal is modulated according to the feeding rhythm control mode corresponding to the variety loaded in step one.

[0059] Preferably, in step seven, during the feeding process, the system adjusts the feeder's calibrated flow rate. and real-time duty cycle The cumulative feeding amount is calculated in real time using the integral formula:

[0060]

[0061] in, This refers to the start time of this feeding. This refers to the current moment, where the cumulative feeding amount mentioned above is compared with the maximum single feeding amount calculated based on the estimated weight of the fish in the pond and the water temperature. Compare; if the cumulative feeding amount reaches That is, satisfying At that time, regardless of the final fusion index If the value is too high or too low, the system will forcibly stop the current feeding.

[0062] Preferably, in step eight, during the set non-feeding period, the system will automatically start the self-calibration process to update the model data. First, the aerator is turned off, and the system analyzes and fits the trend of dissolved oxygen content in the water over time to update the basic data that reflects the rate of natural oxygen consumption in the water without fish.

[0063] The intelligent feeding system based on multimodal spatiotemporal fusion and self-evolutionary correction includes: a system initialization and parameter configuration module, which loads the regional saturated dissolved oxygen calculation benchmark and species feature vector; a multimodal data acquisition module, which simultaneously collects visual, auditory, environmental, and equipment status data; an audiovisual signal processing module, which extracts the visual and auditory feeding indices; a multiphysics field decoupling module, which inverts the net metabolic oxygen consumption rate of the fish population and generates a chemical feeding index; a multimodal fusion module, which combines safety gating to obtain the final fusion index; a closed-loop feeding control module, which executes feeding according to the feeding rhythm control mode; a safety monitoring module, which monitors the cumulative feeding amount and equipment status; and a self-evolutionary correction module, which updates core parameters during non-feeding periods to achieve scenario adaptation and long-term accurate feeding.

[0064] The working principle and beneficial effects of this invention are as follows:

[0065] 1. This invention, through system initialization and multi-source parameter setting in step one, constructs a dual adaptation system of regionalization and species adaptation, fundamentally solving the problem of insufficient adaptability of traditional feeding methods. The system loads a corresponding saturated dissolved oxygen calculation benchmark based on the geographical information of the aquaculture water body input by the user. Different regions have environmental differences such as air pressure and water temperature; this benchmark can accurately correct the impact of these differences on dissolved oxygen calculation, ensuring the accuracy of dissolved oxygen-related data calculations. Simultaneously, the system loads a unique species feature vector based on the aquaculture species selected by the user. This vector includes visual modality weight correction coefficients, auditory modality weight correction coefficients, and chemical modality weight correction coefficients. These coefficients provide a basis for subsequent multi-source parameter setting. Modal information fusion provides species-specific weighting criteria, allowing the fusion results to align with the varying sensitivities of different species to various sensory signals. The acoustic feature frequency bands in the vectors clearly define the capture range of the splashing sound of fish schools for that species. The feeding rhythm control mode directly determines the specific method of subsequent closed-loop feeding, corresponding to three forms: pulse feeding, continuous feeding, and intermittent drip irrigation feeding. This dual-adaptation configuration runs through the entire process of data processing, fusion decision-making, and feeding execution, enabling the system to not only adapt to environmental conditions in different regions but also accurately match the physiological habits and feeding characteristics of different aquaculture species, completely avoiding feeding deviations caused by lack of regional adaptation or misjudgment of species habits.

[0066] 2. This invention achieves the dual goals of precise execution and risk control, ensuring that the feeding process is both demand-oriented and safe and controllable. In step six, the system uses the comparison between the final fusion index and the initial feeding threshold as the decision-making basis, calculates the real-time duty cycle of the feeding motor through the proportional gain coefficient, and then modulates the control signal according to the feeding rhythm control mode corresponding to the species characteristic vector. Pulse feeding is used for upper-layer predatory fish, continuous feeding is used for middle and upper-layer temperate fish, and intermittent drip feeding is used for bottom or benthic organisms, thus achieving precise closed-loop control of feeding speed and rhythm; Step seven In this system, the cumulative feeding amount is calculated through real-time integration and compared with the maximum single feeding limit. Once the limit is reached, feeding is forcibly stopped, avoiding water pollution and fish indigestion caused by overfeeding. At the same time, the operating current of the feeder is continuously monitored. Once an abnormality is detected, the power is cut off and an alarm is triggered, effectively preventing safety risks caused by equipment failure. By combining multiple safety restrictions, the system ensures real-time matching between feeding execution and the fish's feeding needs, and comprehensively avoids potential risks such as overfeeding and equipment failure, thus ensuring the stability and safety of the aquaculture process.

[0067] 3. This invention effectively solves the problems of susceptibility to environmental interference and insufficient feature extraction accuracy in aquaculture scenarios by employing targeted interference suppression, precise feature quantization, and signal enhancement technologies. It provides high-quality, high-reliability visual and auditory feeding index support for subsequent multimodal information fusion. In terms of visual signal processing, the system first loads a dynamic interference mask based on the aerator's status, resetting the pixel weights of the water surface disturbance area generated by the aerator to zero, thus completely eliminating the interference of this area on visual analysis. Then, it utilizes Farneback... Dense optical flow method calculates the pixel motion vector field of the effective area outside the mask, accurately capturing the energy of water surface ripples caused by fish feeding. The visual feeding index, obtained after normalization, can intuitively and quantitatively reflect the feeding activity of the fish. In terms of auditory signal processing, a delayed summation beamforming algorithm is used to focus the beam on the direction of the feeding point, while null control is applied to the aerator's orientation to significantly suppress aerator noise interference. Spectral subtraction is then used to remove steady-state mechanical noise, and the bio-slapping sound energy is calculated by integration within the acoustic characteristic frequency band specified by the species feature vector. The normalized auditory feeding index further quantifies the intensity of the fish feeding behavior. The entire process achieves directional optimization of both visual and auditory key signals, and precise quantification of features through scientific algorithms, ensuring that both types of feeding indices are within a unified quantization range of 0-1. This preserves the core information of feeding states in different dimensions while eliminating deviations caused by interference signals and dimensional differences. Attached Figure Description

[0068] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0069] Figure 1 This is a flowchart of the main method steps of the present invention;

[0070] Figure 2 This is a flowchart of the multimodal perception and feature extraction process in steps two and three of this invention;

[0071] Figure 3 This is a diagram illustrating the multiphysics field decoupling and chemical feeding index calculation in step four of this invention.

[0072] Figure 4 This is the multimodal fusion and security gating decision graph in step five of this invention;

[0073] Figure 5 This is a flowchart illustrating the feeding control and safety monitoring process in steps six and seven of this invention. Detailed Implementation

[0074] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0075] Example 1

[0076] like Figures 1 to 5 As shown, this embodiment proposes an intelligent feeding method based on multimodal spatiotemporal fusion and self-evolutionary correction. Its features include: a system initialization and parameter configuration module, loading a localized saturated dissolved oxygen calculation benchmark and species feature vectors; a multimodal data acquisition module, simultaneously acquiring visual, auditory, environmental, and equipment status data; an audiovisual signal processing module, extracting visual and auditory feeding indices; a multiphysics decoupling module, inverting the net metabolic oxygen consumption rate of the fish population and generating a chemical feeding index; a multimodal fusion module, combining safety gating to obtain the final fusion index; a closed-loop feeding control module, executing feeding according to the feeding rhythm control mode; a safety monitoring module, monitoring the cumulative feeding amount and equipment status; and a self-evolutionary correction module, updating core parameters during non-feeding periods to achieve scenario adaptation and long-term accurate feeding. The method includes the following steps:

[0077] Step 1: System initialization and multi-source parameter setting. The user inputs the geographical information and salinity value of the aquaculture water body through the interactive interface; the system loads the corresponding saturated dissolved oxygen calculation benchmark based on the input geographical information and salinity value; the system loads a set of exclusive parameters, including visual modality weight correction coefficients, from the built-in species characteristic adaptation database based on the current aquaculture species selected by the user from the preset aquaculture species list. Auditory modal weighting correction coefficient Chemical mode weighting correction coefficient The acoustic characteristic frequency bands and feeding rhythm control modes, along with the exclusive parameter set, constitute the variety characteristic vector, completing the localization and variety adaptation configuration of the system's initial parameters.

[0078] Step Two: Synchronous Acquisition of Multimodal Sensing Data and Hardware Status Reading. During the feeding process, multi-source time-series data is acquired simultaneously; the visual sensing unit acquires video image sequences of the water surface in the feeding area, filtering out specular reflections from the water surface using a linear polarization filter integrated in front of the lens; the auditory sensing unit acquires sound signals from the feeding area and its surroundings through a linear array composed of multiple microelectromechanical system microphones; and the environmental monitoring unit collects the dissolved oxygen concentration of the water in real time. Water temperature Real-time atmospheric pressure is collected via an onboard atmospheric pressure sensor. The execution linkage layer collects real-time operating status signals of the aerator. It also includes its operating current data; all collected data is timestamped and cached via edge computing terminals to form a spatiotemporally synchronized multimodal data stream.

[0079] Step 3: Audiovisual signal enhancement and feature extraction. To accurately extract the effective visual and auditory features corresponding to the feeding activities of fish, it is necessary to first eliminate environmental and equipment interference, then quantify the features and generate a feeding index to provide reliable data support for subsequent multimodal fusion. This is carried out in an orderly manner through the following steps:

[0080] (3-1) Aerator status judgment and dynamic interference mask loading: The edge computing terminal first reads the real-time status of the aerator. To determine whether the aerator is on, when the aerator is on, in order to eliminate the interference of water surface disturbance caused by the aerator on the visual signal, the system loads a preset dynamic interference mask in the image coordinate system. The mask's function is to reset the pixel weights in the disturbed area caused by the aerator to zero, while keeping the pixel weights outside the mask at one. This prevents the image in the aerator-disturbed area from being analyzed in subsequent processing. The area inside the mask represents the water surface area disturbed by the aerator, while the area outside the mask represents the undisturbed water surface area. In this way, the mask effectively removes the disturbance caused by the aerator, providing a clean visual signal for subsequent motion analysis.

[0081] (3-2) The Farneback dense optical flow method is used to calculate the pixel motion vector field. For pixels outside the mask area, the system uses the Farneback dense optical flow method to calculate the pixel motion vector field. In other words, the motion vector of each pixel in an image is determined by the velocity and direction of its movement. The Farneback optical flow method, based on changes in image grayscale values, derives the motion information of each pixel to obtain its displacement vector. Specifically, the algorithm calculates the pixel grayscale changes between adjacent frames in an image sequence to obtain the displacement vector of each pixel. This motion vector field accurately describes the energy distribution of water surface disturbances, thus reflecting the water surface fluctuations caused by fish activity.

[0082] (3-3) Calculation of water surface disturbance energy: Based on the pixel motion vector field calculated by the Farneback optical flow method, the system calculates the water surface disturbance energy. The specific calculation formula is as follows:

[0083]

[0084] in, This represents the energy of water surface disturbance. For masking on pixels The value at that location, It is the square of the motion vector field of that pixel, representing the perturbation energy at that location. When the value is zero, it indicates that the position is inside the mask, and the motion energy is not included in the disturbance energy calculation; if If the value is 1, then that location is a valid area, and the disturbance energy will be taken into account. In this way, It can effectively describe the intensity of water surface disturbance;

[0085] (3-4) Normalize the visual feeding index and calculate the surface disturbance energy. Normalization yields the visual food intake index. The normalization process is accomplished using the following formula:

[0086]

[0087] Normalized visual food intake index Between 0 and 1, a larger value indicates more obvious surface disturbance, that is, more active feeding activity of the fish. This visual feeding index can reflect the feeding activity of the fish and provide an important basis for subsequent feeding decisions.

[0088] (3-5) Sound Signal Beamforming and Aerator Noise Suppression: For the sound signal received by the microphone array, the system uses a delay-sum beamforming algorithm. The delay-sum beamforming algorithm adjusts the phase delay of each microphone channel... The signals received from multiple microphones are weighted and combined to form a beam pointing in a specific direction. In this scheme, the goal of beamforming is to focus the beam on the direction of the material feeding point, while simultaneously controlling the aerator's orientation to eliminate noise interference from the aerator. The signal after beamforming... Calculated using the following formula:

[0089]

[0090] in, It is the signal after beamforming. It is the first The signal received by each microphone These are the corresponding weighting coefficients. It is the first Phase delay of each microphone, In terms of timing, through beamforming, the system can enhance the signal from the target direction while suppressing noise from other directions, ensuring that only sound signals related to fish activity are acquired;

[0091] (3-6) Spectral subtraction and noise suppression, signal after beamforming Further spectral subtraction is performed to remove steady-state mechanical noise generated by aerators and other equipment. The purpose of spectral subtraction is to improve the signal-to-noise ratio by reducing background noise, thereby more accurately capturing the activity signals of fish in the water. In this way, the system can more clearly identify the activity of fish and further improve the quality of the auditory signal.

[0092] (3-7) Auditory feeding index calculation and normalization: For the processed auditory signal, the system normalizes the beamformed signal within the acoustic characteristic frequency band Freq specified by the variety feature vector. Integrate the energy of the sound of the organism striking the water. The specific integration operation is as follows:

[0093]

[0094] in, It refers to the energy of the signal. The integration operation integrates the signal within a specified frequency band to calculate the total energy of the biological splashing sound, and the normalized auditory feeding index is obtained. It also ranges between 0 and 1. The larger the value, the stronger the activity of aquatic organisms, that is, the more active the feeding behavior of the fish. By calculating the auditory feeding index, the system can further quantify the feeding behavior of the fish and provide a more comprehensive reference for feeding decisions.

[0095] Step 4: Multiphysics decoupling and net metabolic oxygen consumption rate inversion. The edge computing terminal calls the multiphysics decoupling engine to calculate the saturated dissolved oxygen value after corrections for air pressure, temperature, and salinity. This value represents the maximum amount of oxygen that can dissolve in water, and it is crucial for the oxygen supply in the water, directly affecting the metabolism and feeding behavior of fish. The calculation formula is as follows:

[0096]

[0097] in, It's atmospheric pressure. It is the saturated vapor pressure of water, which is related to temperature. Related, It is a temperature correction function, which represents the effect of temperature on dissolved oxygen; It is the salinity value of the water. This is the salinity correction constant, used to correct for the effect of salinity on dissolved oxygen. The formula's function is to accurately calculate the saturation concentration of dissolved oxygen by taking environmental factors (air pressure, temperature, and salinity) into account. This provides accurate basic data for subsequent oxygen consumption inversion;

[0098] Based on the status of the aerator Oxygenation capacity of aerator and the basic oxygen demand of water bodies Invert the net metabolic oxygen consumption rate of the fish population The calculation formula is:

[0099]

[0100] in, It represents the rate of change of dissolved oxygen concentration over time, reflecting the rate at which oxygen is consumed in the water. It is a real-time measured dissolved oxygen concentration. This is the corrected saturated dissolved oxygen concentration; This refers to the working status of the aerator. It is the oxygenation coefficient of the aerator after aging correction, which indicates the oxygenation efficiency of the aerator; This is the basic oxygen consumption rate of a water body, representing the rate at which oxygen is consumed in the absence of fish. The formula is used to deduce the net metabolic oxygen consumption rate of fish by comparing measured changes in dissolved oxygen with factors influencing oxygen consumption. Oxygen consumption is the amount of oxygen consumed by a school of fish per unit time. This value reflects the physiological activities and feeding needs of the school of fish, thus providing a basis for intelligent feeding. Factors affecting oxygen consumption include aerators, fish metabolism, and basal oxygen consumption.

[0101] The calculated After normalization, the chemical intake index is obtained. This is used to quantify the feeding behavior of fish schools. The normalization process is accomplished using the following formula:

[0102]

[0103] in, The biggest of all historical moments The chemical feeding index should be maintained between 0 and 1; a higher value indicates more active feeding behavior in the fish. It provides a feeding requirement index for fish populations based on metabolism and oxygen consumption.

[0104] Step 5: Multimodal information fusion and security gating based on variety adaptation weights. The edge computing terminal adjusts the weights in the variety feature vector loaded in Step 1 according to the weights. , , Visual food intake index Auditory feeding index Chemical intake index Weighted fusion is performed to generate a preliminary fusion feeding index. ;

[0105]

[0106] Simultaneously, the fault safety management module is invoked to perform a safety status assessment and monitor atmospheric pressure. The rate of sudden drop, and checking the continuity and validity of data from each sensor;

[0107] If a Level 1 meteorological risk or a Level 2 sensor malfunction is detected, a downgraded safety threshold coefficient is generated. ;

[0108] The initial integration of the food intake index With security gating coefficient Multiplying them together yields the final fusion index used for feeding decisions. ,Right now .

[0109] Step Six: Based on the closed-loop feeding control of the final fusion index and the variety rhythm strategy, the system determines the final fusion index. Does it exceed the minimum investment threshold set for this variety? ;

[0110] like If so, the feeder will remain stopped.

[0111] like Then, according to the proportional gain coefficient Calculate the real-time duty cycle control signal of the feeding motor ;

[0112]

[0113] in The calculation result is the minimum start-up duty cycle. It is limited to between 0% and 100%;

[0114] The calculated The control signal is modulated according to the feeding rhythm control strategy Mode corresponding to this variety, which was loaded in step one.

[0115] Pulse feeding is used for surface-feeding predatory fish, continuous feeding is used for mid-to-upper-level temperate fish, and intermittent drip feeding is used for bottom-dwelling or benthic organisms. The feeder motor is driven by pulse width modulation to achieve real-time and precise closed-loop control of feeding speed and rhythm.

[0116] Conditions / Input operate control signals Feeding strategy Final Fusion Index <Minimum Investment Threshold The feeder remains stopped. No feeding Final Fusion Index ≥Starting threshold Calculate the real-time duty cycle control signal of the feeding motor according to Control the feeding speed and rhythm Feeding rhythm control strategy: Upper-level predatory fish Pulse feeding is used, with the feeder motor driven by pulse width modulation. Calculated Signal Rapid and concentrated feeding, targeting surface-feeding predatory fish. Feeding rhythm control strategy: for mid-to-upper-level gentle fish Continuous feeding is used, and the feeder motor is adjusted via real-time control signals. Calculated Signal Stable and consistent feeding is recommended for mid-to-upper-level water-dwelling, mild-mannered fish. Feeding rhythm control strategy: bottom or benthic organisms Intermittent drip feeding is used, and the intermittent feeding is controlled by adjusting the duty cycle of the signal. Calculated Signal Intermittent, precise feeding, targeting bottom-dwelling or benthic organisms.

[0117] Step 7: Monitoring the feeding process and limiting the total amount of feed. During the feeding process, the system monitors the feeder's calibrated flow rate. and real-time duty cycle The cumulative feeding amount is calculated in real time using integrals.

[0118] The integral formula is:

[0119]

[0120] The cumulative feeding amount is compared with the maximum single feeding amount calculated based on the estimated weight of the fish in the pond and the water temperature. Compare;

[0121] If the cumulative feeding amount reaches That is, satisfying At that time, regardless of the final fusion index If the current is too high or too low, the system will forcibly stop the feeding operation. The system will continuously monitor the operating current of the feeder from the execution linkage layer. If the current is detected to be abnormal and exceeds the set threshold, the power supply of the feeder will be cut off immediately and the actuator fault alarm will be triggered.

[0122] Step 8: Full life cycle self-evolution correction and self-calibration. During the set non-feeding period, the system will automatically start the self-calibration process to update the model data. First, the aerator is turned off. By analyzing the trend of dissolved oxygen content in the water decreasing over time and fitting the data, the system updates the basic data that reflects the natural oxygen consumption rate of the water body in the absence of fish. This provides a more accurate basis for subsequent accurate judgment of fish oxygen consumption and optimization of feeding decisions, which is more in line with the actual aquaculture environment.

[0123] The intelligent feeding system based on multimodal spatiotemporal fusion and self-evolutionary correction includes: a system initialization and parameter configuration module, which loads the regional saturated dissolved oxygen calculation benchmark and species feature vector; a multimodal data acquisition module, which simultaneously collects visual, auditory, environmental, and equipment status data; an audiovisual signal processing module, which extracts the visual and auditory feeding indices; a multiphysics field decoupling module, which inverts the net metabolic oxygen consumption rate of the fish population and generates a chemical feeding index; a multimodal fusion module, which combines safety gating to obtain the final fusion index; a closed-loop feeding control module, which executes feeding according to the feeding rhythm control mode; a safety monitoring module, which monitors the cumulative feeding amount and equipment status; and a self-evolutionary correction module, which updates core parameters during non-feeding periods to achieve scenario adaptation and long-term accurate feeding.

[0124] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An intelligent feeding method based on multimodal spatiotemporal fusion and self-evolutionary correction, characterized in that, Includes the following steps: Step 1: System Initialization and Multi-Source Parameter Setting Step 2: Synchronous acquisition of multimodal sensing data and reading of hardware status; Step 3: Audiovisual signal enhancement and feature extraction. To accurately extract the effective visual and auditory features corresponding to the feeding activities of fish, it is necessary to first eliminate environmental and equipment interference, then quantify the features and generate a feeding index. This is carried out in an orderly manner through the following steps: (3-1) Aerator status judgment and dynamic interference mask loading; (3-2) Calculation of pixel motion vector field using Farneback dense optical flow method; (3-3) Calculation of water surface disturbance energy; (3-4) Normalization of visual food intake index; (3-5) Sound signal beamforming and aerator noise suppression; (3-6) Spectral subtraction and noise suppression; (3-7) Calculation and normalization of auditory feeding index; Step 4: Multiphysics field decoupling and net metabolic oxygen consumption rate inversion; Step 5: Multimodal information fusion and security gating based on variety adaptation weights; Step Six: Closed-loop feeding control based on the final fusion index and variety rhythm strategy; Step 7: Monitoring the feeding process and limiting the total amount of food fed; Step 8: Self-evolutionary correction and self-calibration throughout the entire life cycle.

2. The intelligent feeding method based on multimodal spatiotemporal fusion and self-evolutionary correction according to claim 1, characterized in that, In step one, the user inputs the geographical information and salinity value of the aquaculture water body through the interactive interface; the system loads the corresponding saturated dissolved oxygen calculation benchmark based on the input geographical information and salinity value; the user selects the current aquaculture species from the preset aquaculture species list, and the system loads the corresponding feature vector from the built-in species characteristic adaptation database based on the selected species.

3. The intelligent feeding method based on multimodal spatiotemporal fusion and self-evolutionary correction according to claim 1, characterized in that, In step two, multi-source time-series data is collected simultaneously; the visual perception unit collects video image sequences of the water surface in the feeding area, filtering out specular reflections from the water surface using a linear polarization filter integrated in front of the lens; the auditory perception unit collects sound signals from the feeding area and its surroundings through a linear array composed of multiple microelectromechanical system microphones; and the environmental monitoring unit collects the dissolved oxygen concentration of the water in real time. Water temperature Real-time atmospheric pressure is collected via an onboard atmospheric pressure sensor. The execution linkage layer collects real-time operating status signals of the aerator. It also includes its operating current data; all collected data is timestamped and cached via edge computing terminals to form a spatiotemporally synchronized multimodal data stream.

4. The intelligent feeding method based on multimodal spatiotemporal fusion and self-evolutionary correction according to claim 1, characterized in that, In step three, to accurately extract the effective visual and auditory features corresponding to the feeding activities of fish, it is necessary to first eliminate environmental and equipment interference, then quantify the features and generate a feeding index. (3-1) The edge computing terminal first reads the real-time status of the aerator. To determine whether the aerator is on, when the aerator is on, in order to eliminate the interference of water surface disturbance caused by the aerator on the visual signal, the system loads a preset dynamic interference mask in the image coordinate system. ; In step (3-2), for pixels outside the mask region, the system uses the Farneback dense optical flow method to calculate the pixel's motion vector field. In other words, the motion speed and direction of each pixel in the image. The Farneback optical flow method is based on the changes in the gray values ​​of the image and derives the motion information of each pixel to obtain the displacement vector of each pixel. In step (3-3), based on the pixel motion vector field calculated by the Farneback optical flow method, the system calculates the water surface disturbance energy. The specific calculation formula is as follows: in, This represents the energy of water surface disturbance. For masking on pixels The value at that location, It is the square of the motion vector field of the pixel, representing the perturbation energy at that location. When the value is zero, it indicates that the position is inside the mask, and the motion energy is not included in the disturbance energy calculation; if If the value is 1, then that location is a valid area, and the disturbance energy will be taken into account; The calculated water surface disturbance energy is described in (3-4). Normalization yields the visual food intake index. The normalization process is accomplished using the following formula: Normalized visual food intake index The value is between 0 and 1. A larger value indicates more obvious surface disturbance and more active feeding activities of the fish. As described in (3-5), for the audio signals received by the microphone array, the system uses a delay-sum beamforming algorithm for processing. The delay-sum beamforming algorithm adjusts the phase delay of each microphone channel... The signals received from multiple microphones are weighted and combined to form a beam pointing in a specific direction. The goal of beamforming is to focus the beam on the direction of the material's landing point. The signal after beamforming... Calculated using the following formula: in, It is the signal after beamforming. It is the first The signal received by each microphone These are the corresponding weighting coefficients. It is the first Phase delay of each microphone, It is time; The signal after beamforming (3-6) Further spectral subtraction processing is performed to remove steady-state mechanical noise generated by aerators and other equipment; As stated in (3-7), for the processed auditory signal, the system performs beamforming on the signal within the acoustic characteristic frequency band specified by the variety feature vector. To integrate and calculate the energy of the biological impact sound, the specific integration operation is as follows: in, It refers to the energy of the signal. The integration operation integrates the signal within a specified frequency band to calculate the total energy of the biological splashing sound, and the normalized auditory feeding index is obtained. It also ranges between 0 and 1. The larger the value, the stronger the activity of aquatic organisms, indicating that the fish are more active in feeding.

5. The intelligent feeding method based on multimodal spatiotemporal fusion and self-evolutionary correction according to claim 1, characterized in that, In step four, the edge computing terminal invokes a multiphysics decoupling engine to calculate the saturated dissolved oxygen value after corrections for air pressure, temperature, and salinity. : in, It's atmospheric pressure. It is the saturated vapor pressure of water, which is related to temperature. Related, It is a temperature correction function, which represents the effect of temperature on dissolved oxygen; It is the salinity value of the water. It is the salinity correction constant, used to correct the effect of salinity on dissolved oxygen; Based on the status of the aerator Oxygenation capacity of aerator and the basic oxygen demand of water bodies Invert the net metabolic oxygen consumption rate of the fish population The calculation formula is: in, It represents the rate of change of dissolved oxygen concentration over time, reflecting the rate at which oxygen is consumed in the water. It is a real-time measured dissolved oxygen concentration. This is the corrected saturated dissolved oxygen concentration. By comparing the measured changes in dissolved oxygen with the factors affecting oxygen consumption, the net metabolic oxygen consumption rate of the fish population can be deduced. This refers to the amount of oxygen consumed by the fish per unit of time. This value reflects the physiological activities and feeding needs of the fish, providing a basis for intelligent feeding.

6. The intelligent feeding method based on multimodal spatiotemporal fusion and self-evolutionary correction according to claim 1, characterized in that, In step four, the calculated After normalization, the chemical intake index is obtained. This is used to quantify the feeding behavior of fish schools. The normalization process is accomplished using the following formula: in, The biggest of all historical moments The chemical feeding index should be maintained between 0 and 1; a higher value indicates more active feeding behavior in the fish. It provides a feeding requirement index for fish populations based on metabolism and oxygen consumption.

7. The intelligent feeding method based on multimodal spatiotemporal fusion and self-evolutionary correction according to claim 1, characterized in that, In step five, the edge computing terminal adjusts the weights in the variety feature vector loaded in step one according to the weight correction coefficients. , , Visual food intake index Auditory feeding index Chemical intake index Weighted fusion is performed to generate a preliminary fusion feeding index. : The fault safety management module is invoked to perform a safety status assessment and monitor atmospheric pressure. The rate of sudden drop, and checking the continuity and validity of data from each sensor; If a Level 1 meteorological risk or a Level 2 sensor malfunction is detected, a downgraded safety threshold coefficient is generated. ; The initial integration of the food intake index With security gating coefficient Multiplying them together yields the final fusion index used for feeding decisions. ,Right now .

8. The intelligent feeding method based on multimodal spatiotemporal fusion and self-evolutionary correction according to claim 7, characterized in that, In step six, the system determines the final fusion index. Does it exceed the minimum investment threshold set for this variety? ;like If so, the feeder will remain stopped; if Then, according to the proportional gain coefficient Calculate the real-time duty cycle control signal of the feeding motor ; in The calculation result is the minimum start-up duty cycle. Limited to between 0% and 100%; the calculated The control signal is modulated according to the feeding rhythm control mode corresponding to the variety loaded in step one.

9. The intelligent feeding method based on multimodal spatiotemporal fusion and self-evolutionary correction according to claim 7, characterized in that, In step seven, during the feeding process, the system calculates the feeder's calibrated flow rate. and real-time duty cycle The cumulative feeding amount is calculated in real time using an integral formula: in, This refers to the start time of this feeding. This refers to the current moment, where the cumulative feeding amount mentioned above is compared with the maximum single feeding amount calculated based on the estimated weight of the fish in the pond and the water temperature. Compare; if the cumulative feeding amount reaches That is, satisfying At that time, regardless of the final fusion index If the feed level is too high or too low, the system will forcibly stop the current feeding operation. In step eight, during the designated non-feeding period, the system will automatically initiate a self-calibration process to update the model data. First, the aerator will be turned off. Then, by analyzing and fitting the trend of dissolved oxygen content in the water decreasing over time, the basic data that reflects the rate at which the water naturally consumes oxygen in the absence of fish will be updated.

10. An intelligent feeding system based on multimodal spatiotemporal fusion and self-evolutionary correction, characterized in that, This includes a system initialization and parameter configuration module, which loads the regional saturated dissolved oxygen calculation benchmark and species feature vector; a multimodal data acquisition module, which simultaneously collects visual, auditory, environmental and equipment status data; and an audiovisual signal processing module, which extracts the visual feeding index and auditory feeding index. A multiphysics decoupling module inverts the net metabolic oxygen consumption rate of fish and generates a chemical feeding index. The multimodal fusion module, combined with safety gating, obtains the final fusion index; the closed-loop feeding control module executes feeding according to the feeding rhythm control mode. The safety monitoring module monitors the cumulative feeding amount and equipment status; The self-evolution correction module updates core parameters during non-feeding periods to achieve scenario adaptation and long-term accurate feeding.

Images