A Multidimensional Multi-Model Data-Based Method for Precision Feeding Correction in Aquaculture

Abstract

This invention relates to the field of intelligent fisheries and aquaculture automation technology, and particularly to a multi-dimensional, multi-modal data-based method for precise feeding correction in aquaculture. The method includes the following steps: S1, collecting photoelectric multi-modal data and water quality multi-modal data under the aquaculture environment, simultaneously recording fish body multi-modal data and feed multi-modal data, and generating standardized data through timestamp synchronization and spatial coordinate mapping; S2, inputting the standardized data into a multi-channel feature encoding neural network. In this invention, by constructing an adaptive weighting and multi-head attention fusion mechanism based on information entropy, the weight allocation of visual, water quality, and attribute modalities can be automatically adjusted according to environmental changes, effectively overcoming interference caused by water turbidity or sensor drift. Furthermore, by using dual prediction correction of feeding intensity and environmental state to correct the baseline feeding curve, precise dynamic matching of feeding strategies under complex aquaculture environments is achieved.

Description

Technical Field

[0001] This invention relates to the field of smart fisheries and aquaculture automation technology, and in particular to a multi-dimensional, multi-modal data-based method for precise feeding correction in aquaculture. Background Technology

[0002] With the rapid development of smart fishery technology, precision feeding has become a key link in reducing costs and increasing efficiency in modern aquaculture. The use of IoT sensors to collect multi-dimensional data such as water quality parameters and fish behavior, combined with artificial intelligence algorithms to realize intelligent feeding decisions, has gradually become a research hotspot in the industry.

[0003] However, existing technologies mostly rely on single-modal data or simple feature splicing and fixed weight fusion, lacking the ability to adaptively adjust to data quality fluctuations in complex dynamic environments. This makes them prone to decision-making biases when turbid water obscures visual features or when sensors drift. Furthermore, they often fail to comprehensively consider the coupling constraints between fish physiological needs and the carrying capacity of the aquatic environment, making it difficult to meet the robustness and accuracy of feeding control in actual production. Summary of the Invention

[0004] To overcome the above shortcomings, this invention provides a multi-dimensional, multi-modal data-based method for precise feeding correction in aquaculture. This method aims to improve upon existing technologies that rely heavily on single-modal data or simple feature splicing and fixed-weight fusion, lacking the ability to adaptively adjust to data quality fluctuations in complex dynamic environments.

[0005] This invention provides the following technical solution: a multi-dimensional, multi-modal data-based method for precise feeding correction in aquaculture, comprising the following steps: S1. Collect photoelectric multi-mode data and water quality multi-mode data in the aquaculture environment, and simultaneously input fish body multi-mode data and feed multi-mode data. After time stamp synchronization and spatial coordinate mapping, generate standardized data. S2. Input the standardized data into a multi-channel feature encoding neural network, and use the parallel branches in the network to perform three-dimensional convolution operations and sequence encoding on the photoelectric multi-mode data and water quality multi-mode data respectively. Combine the attribute data of fish body and bait to extract semantic features, and independently output high-dimensional visual space features, temporal evolution feature vectors and semantic embedding vectors. S3. Based on the distribution dispersion of the high-dimensional visual space features, temporal evolution feature vectors and semantic embedding vectors, generate modal confidence weights, and adaptively weight and fuse each feature vector accordingly, and use a multi-head attention mechanism to generate a global state feature vector. S4. Input the global state feature vector into the state prediction neural network, and output the feeding intensity index and the environmental state index through the fully connected layer. Calculate the deviation of the feeding intensity index and the environmental state index from their respective standard thresholds, correct the benchmark feeding curve, and generate dynamic correction parameters for the feeding strategy. S5. Send the dynamic correction parameters of the feeding strategy to the control system, adjust the preset feeding rate and total amount, and control the equipment to perform feeding.

[0006] Preferably, the photoelectric multimode data includes depth point clouds acquired by a binocular camera, spectral features acquired by a multispectral sensor, and infrared thermal imaging data and polarized light data.

[0007] Preferably, in step S1, the generation of standardized data through timestamp synchronization and spatial coordinate mapping specifically includes the following steps: Establish a unified system clock reference, take the acquisition frequency of binocular vision data as the trigger axis, and use linear interpolation or nearest neighbor matching algorithm to upsample low-frequency water quality multimodal data and other photoelectric multimodal data to obtain time-aligned multimodal data; A three-dimensional spatial coordinate system is constructed based on binocular vision depth data. Using a pre-calibrated camera intrinsic parameter matrix and extrinsic parameter rotation and translation matrix between different sensors, the pixels of infrared thermal imaging images, multispectral images, and polarized light images are projected and mapped onto the three-dimensional spatial coordinate system to construct a four-dimensional optoelectronic fusion map containing depth information, thermal information, spectral information, and polarization information. Encode or embed features of fish species, growth cycle, and activity level in fish body multi-modal data, and feed composition and feed density in feed multi-modal data to generate static attribute feature vectors. The water quality multi-model data and the four-dimensional photoelectric fusion map are standardized, and the processed data is combined with the static attribute feature vector to form a standardized dataset.

[0008] Preferably, in step S2, performing convolution operations and sequence encoding on the visual image and the sensor time-series data using parallel branches within the network specifically includes the following steps: A visual perception branch based on a three-dimensional convolutional neural network is constructed. The four-dimensional photoelectric fusion map in the standardized dataset is used as input. Multi-scale three-dimensional convolutional kernels are used to scan the spatial dimension and channel dimension of the image simultaneously to extract high-dimensional visual spatial features. A temporal analysis branch based on a temporal convolutional network is constructed. Water quality multi-model data in the standardized dataset is used as input. Historical data of oxygen content and temperature are sequence encoded through a gating mechanism to output a temporal evolution feature vector. An attribute mapping branch based on a multilayer perceptron is constructed. The static attribute feature vectors in the standardized dataset are input into a fully connected layer to perform nonlinear mapping and feature recombination on the physiological characteristics of fish species and the physical properties of bait, and to extract semantic embedding vectors. The high-dimensional visual space features, temporal evolution feature vectors, and semantic embedding vectors are used as the encoded output of independent modalities.

[0009] Preferably, the extraction of the semantic embedding vector specifically includes the following steps: Discrete fish species category labels are mapped to dense vectors using a pre-constructed entity embedding matrix. These dense vectors are then concatenated with normalized growth cycle values ​​and activity indices to construct the original biological feature set of the fish. The feed composition ratio data and feed density values ​​are input into a fully connected neural network to perform feature upsizing, and the physical property parameters are mapped to a high-dimensional feature space to construct the original physical feature set of the feed. The original biological feature set of the fish is input into the biological behavior reasoning subnetwork within the attribute mapping branch, and the biological behavior semantic vector is extracted through multi-layer nonlinear transformation; The original physical feature set of the bait is input into the sedimentation dynamics simulation subnetwork within the attribute mapping branch, sedimentation characteristic semantic vectors are extracted, and the biological habit semantic vectors are combined with the sedimentation characteristic semantic vectors to form the final semantic embedding vector.

[0010] Preferably, in step S3, generating the global state feature vector specifically includes the following steps: The high-dimensional visual space features, temporal evolution feature vectors, and semantic embedding vectors are respectively input into their respective feature projection layers, mapping features of different dimensions to a unified-dimensional latent feature space. The distribution dispersion of each modality feature in the latent feature space is calculated respectively. The distribution dispersion is characterized by the information entropy of the feature vector along the channel dimension, and the distribution dispersion is normalized to the real-time confidence weight of each modality through the activation function. The real-time confidence weights are used to perform dot product operations on the corresponding modal features to generate weighted high-dimensional visual spatial features, temporal evolution feature vectors, and semantic embedding vectors. The weighted modal features are concatenated into a multimodal feature sequence, which is then input into the multi-head self-attention mechanism module of the pre-trained large model. The cross-modal correlation matrix between visual, temporal, and attribute features is calculated, and the global state feature vector is output through weighted aggregation.

[0011] Preferably, in step S4, calculating the deviation of the two indices relative to the standard threshold and correcting the benchmark feeding curve specifically includes the following steps: The global state feature vector is input into the task-specific decoding layer of the state prediction neural network, and the feeding intensity index and the environmental state index are mapped and output respectively through fully connected operations. Read the preset standard feeding threshold and safe environment threshold, calculate the positive demand deviation between the predicted feeding intensity index and the standard feeding threshold, and the environmental pressure deviation between the environmental state index and the safe environment threshold; Based on the current number of days of aquaculture and the attributes of the fish species, the preset baseline feeding curve is invoked to query and obtain the theoretical basic feeding rate and theoretical basic total feeding amount at the current time point. A dynamic correction coefficient is constructed using the positive demand deviation and environmental pressure deviation. The theoretical basic feeding rate and the theoretical basic total feeding amount are weighted and modulated to generate dynamic correction parameters for the feeding strategy that include the final feeding rate and the final feeding duration.

[0012] Preferably, the positive demand deviation between the calculated and predicted feeding intensity index and the standard feeding threshold, and the environmental pressure deviation between the environmental state index and the safe environmental threshold, specifically include the following steps: Based on the current fish growth cycle and sunshine duration, the corresponding standard feeding intensity benchmark value and water quality safety threshold value are retrieved from the preset database; Subtract the standard food intake intensity benchmark value from the food intake intensity index to obtain the original difference. Use a linear rectification function to filter out negative values, retain the positive part and multiply it by a preset food intake conversion coefficient to obtain the positive demand deviation. The difference between the environmental state index and the water quality safety threshold is calculated. When the environmental state index is lower than the safety threshold, the difference is mapped to a nonlinear environmental pressure deviation using an exponential decay function. The positive demand deviation and environmental pressure deviation are normalized to output the final deviation parameters used for correction calculations.

[0013] Preferably, in step S5, adjusting the preset feeding rate and total amount, and controlling the equipment to perform feeding specifically includes the following steps: The dynamic correction parameters of the feeding strategy are analyzed, and the final feeding rate and final feeding duration contained therein are extracted and used as the current execution-level feeding instruction. By calling the bait density attribute in the bait multi-model data, a mass-volume conversion model is constructed, and the final feeding rate is converted into the target rotation speed of the feeder motor. The target rotation speed and the final feeding time are sent to the underlying control unit through the industrial control communication protocol, driving the feeding actuator to start feeding at the updated rate. The actual cumulative feeding amount is monitored in real time by a metering device installed at the feeder outlet. When the monitored cumulative feeding amount reaches the target total feeding amount determined by the final feeding rate and the final feeding time, a stop command is triggered to interrupt the motor operation.

[0014] The present invention has the following beneficial effects: 1. In this invention, by constructing an adaptive weighted and multi-head attention fusion mechanism based on information entropy, the weight allocation of visual, water quality and attribute modalities can be automatically adjusted according to environmental changes, effectively overcoming the interference caused by water turbidity or sensor drift. Furthermore, by using dual prediction and correction of feeding intensity and environmental state to correct the baseline feeding curve, precise dynamic matching of feeding strategies in complex aquaculture environments is achieved.

[0015] 2. In this invention, by establishing a unified clock reference and three-dimensional spatial projection mapping, strict spatiotemporal alignment of high-frequency visual data and low-frequency water quality data is achieved, and a four-dimensional optoelectronic fusion map containing depth, heat, spectrum and polarization information is constructed. This eliminates the physical dimension differences of multi-source heterogeneous data and provides high-fidelity, strongly coupled standardized input data for deep learning models.

[0016] 3. In this invention, a strategy of jointly modulating positive demand deviation and environmental pressure deviation is adopted. While ensuring the explosive feeding demand of fish, the exponential decay mechanism is used to automatically suppress feeding when the water quality deteriorates. Combined with closed-loop motor control based on the mass-volume conversion model, the risk of residual feed polluting the water quality is effectively avoided, and the balance between aquaculture benefits and ecological safety is ensured. Attached Figure Description

[0017] Figure 1 This is a flowchart of a multi-dimensional, multi-modal data-based method for precise feeding correction in aquaculture, as proposed in this invention. Detailed Implementation

[0018] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] This invention provides a multi-dimensional, multi-modal data-based method for precise feeding correction in aquaculture, such as... Figure 1 As shown, it includes the following steps: S1. Collect photoelectric multi-mode data and water quality multi-mode data in the aquaculture environment, and simultaneously input fish body multi-mode data and feed multi-mode data. After time stamp synchronization and spatial coordinate mapping, generate standardized data. Furthermore, the optoelectronic multimode data includes depth point clouds acquired through a binocular camera, spectral features acquired through a multispectral sensor, and infrared thermal imaging data and polarized light data.

[0020] Furthermore, in S1, the generation of standardized data through timestamp synchronization and spatial coordinate mapping specifically includes the following steps: Establish a unified system clock reference, take the acquisition frequency of binocular vision data as the trigger axis, and use linear interpolation or nearest neighbor matching algorithm to upsample low-frequency water quality multimodal data and other photoelectric multimodal data to obtain time-aligned multimodal data; A three-dimensional spatial coordinate system is constructed based on binocular vision depth data. Using the pre-calibrated camera intrinsic parameter matrix and the extrinsic parameter rotation and translation matrix between different sensors, the pixel points of infrared thermal imaging images, multispectral images and polarized light images are projected and mapped onto the three-dimensional spatial coordinate system to construct a four-dimensional optoelectronic fusion map containing depth information, thermal information, spectral information and polarization information. Encode or embed features of fish species, growth cycle, and activity level in fish body multi-modal data, and feed composition and feed density in feed multi-modal data to generate static attribute feature vectors. The water quality multi-model data and four-dimensional photoelectric fusion map were standardized, and the processed data were combined with the static attribute feature vector to form a standardized dataset.

[0021] Specifically, the data acquisition hardware is first deployed by setting up an optoelectronic acquisition terminal above the aquaculture area, which integrates a binocular camera, a multispectral sensor, an infrared thermal imager, and a polarized light camera. Dissolved oxygen sensors, pH sensors, and temperature sensors are then deployed at different depths underwater. At the same time, the genetic information of the fish species in the current aquaculture batch and the physicochemical properties of the feed used are pre-entered into the server database.

[0022] Subsequently, a unified system clock reference was established, using the frame trigger signal of the binocular camera as the master axis clock, setting the acquisition frequency to 30 Hz, and the acquisition frequency of the water quality sensor to 1 Hz. For water quality multi-mode data, including dissolved oxygen concentration, water temperature, and pH value, with sampling frequencies lower than the master axis clock, a linear interpolation algorithm was used to map them to the timestamps of the binocular vision data to achieve alignment in the time dimension. Assuming... At a certain moment of the main spindle clock, this moment corresponds to the two data acquisition times of the low-frequency sensor. and Between, the corresponding data values ​​are and Then the aligned data The calculation formula is as follows: ; In the formula, Indicates the time at the principal axis The virtual water quality data values ​​generated by interpolation For a moment The most recent actual sampling time, For a moment The most recent actual sampling time after that, and These are the actual physical quantity values ​​transmitted by the sensor at these two moments.

[0023] After time synchronization is completed, a four-dimensional optoelectronic fusion map is constructed, and the depth value of each pixel within the field of view is calculated based on the parallax principle of the left and right views of the binocular camera. This allows for the reconstruction of the three-dimensional point cloud coordinates of the scene. Using pre-calibrated intrinsic parameter matrices of each sensor and extrinsic rotation and translation matrices relative to the binocular camera coordinate system, the coordinates of the aforementioned three-dimensional point cloud are... The data are projected onto the two-dimensional imaging planes of infrared thermal imaging, multispectral imaging, and polarized light, respectively, to obtain the temperature value, spectral reflectance, and degree of polarization corresponding to the spatial point. The projection mapping process uses a pinhole camera model, and the calculation formula is as follows: ; In the formula, and For three-dimensional space points The x and y coordinates of pixels mapped onto the infrared, multispectral, or polarized light image plane. This is the depth scaling factor for that point in the corresponding sensor coordinate system. This is the intrinsic parameter matrix corresponding to the sensor, containing focal length and principal point coordinate information. This is the rotation matrix from the stereo camera coordinate system to the corresponding sensor coordinate system. It is a translation vector. , , The three-dimensional spatial coordinates of the fish or environmental target point in the binocular vision coordinate system are used to calculate the pixel coordinates through the above mapping. and The pixel grayscale value or feature value at that location is assigned to the three-dimensional point. This allows thermal, spectral, and polarization information to be fused together based on the original three-dimensional geometric information through channel overlay.

[0024] Next, the static attribute data is processed. For discrete non-numerical data of fish species, one-hot encoding is used to convert it into a sparse vector. For continuous numerical data such as growth cycle days, activity score, and feed density and feed composition ratio, static attribute feature vectors are constructed. All the generated time-series water quality data, fused photoelectric spectrum data, and static attribute data are standardized to eliminate the influence of dimensions. The standardization uses the Z-score algorithm, and the calculation formula is as follows: ; In the formula, These are the standardized feature values. These are the original sensor values ​​or image pixel values. This is the mean of the feature over the historical dataset. Standard deviation To prevent the use of tiny constants with zero denominators, after processing, the normalized photoelectric spectrum tensor, water quality time series vector, and static attribute vector are encapsulated and combined into a standardized dataset with spatiotemporal labels.

[0025] By strictly aligning the temporal and spatial data and fusing multiple sources, the problem of inconsistent sampling frequencies and spatial perspective differences between different sensors was effectively solved, ensuring that the data input to the model corresponded accurately in a physical sense. This provided a high-quality data foundation for subsequent multi-channel neural networks to accurately extract the coupling features between the environment and organisms.

[0026] S2. Input standardized data into a multi-channel feature encoding neural network, and use parallel branches in the network to perform three-dimensional convolution operations and sequence encoding on photoelectric multi-mode data and water quality multi-mode data respectively. Combine the attribute data of fish body and bait to extract semantic features, and independently output high-dimensional visual space features, temporal evolution feature vectors and semantic embedding vectors. Furthermore, in S2, the convolution operation and sequence encoding of the visual image and the sensor time-series data are performed using parallel branches within the network, specifically including the following steps: A visual perception branch based on a three-dimensional convolutional neural network is constructed. The four-dimensional photoelectric fusion map in the standardized dataset is used as input. Multi-scale three-dimensional convolutional kernels are used to scan the spatial dimension and channel dimension of the image simultaneously to extract high-dimensional visual spatial features. A temporal analysis branch based on a temporal convolutional network is constructed. Water quality multi-model data in the standardized dataset is used as input. Historical data of oxygen content and temperature are sequence encoded through a gating mechanism, and the temporal evolution feature vector is output. An attribute mapping branch based on a multilayer perceptron is constructed. The static attribute feature vectors in the standardized dataset are input into the fully connected layer to perform nonlinear mapping and feature recombination on the physiological characteristics of fish species and the physical properties of bait, and to extract semantic embedding vectors. High-dimensional visual space features, temporal evolution feature vectors, and semantic embedding vectors are used as the encoding output of independent modalities.

[0027] Furthermore, extracting the semantic embedding vector specifically includes the following steps: Discrete fish species category labels are mapped to dense vectors using a pre-constructed entity embedding matrix. These dense vectors are then concatenated with normalized growth cycle values ​​and activity indices to construct the original biological feature set of the fish. The feed composition ratio data and feed density values ​​are input into a fully connected neural network to perform feature upsizing, and the physical property parameters are mapped to a high-dimensional feature space to construct the original physical feature set of the feed. The original biological feature set of the fish is input into the biological behavior reasoning subnetwork within the attribute mapping branch, and the biological behavior semantic vector is extracted through multi-layer nonlinear transformation; The original feature set of bait physics is input into the sedimentation dynamics simulation subnetwork within the attribute mapping branch. Semantic vectors of sedimentation characteristics are extracted, and the semantic vectors of biological habits and sedimentation characteristics are combined into the final semantic embedding vector.

[0028] Specifically, first, a multi-channel feature encoding neural network is initialized in a high-performance computing server. This network consists of three independent branches that work in parallel, corresponding to visual perception, temporal analysis and attribute mapping tasks, respectively. The standardized data generated in step S1 is decoupled and used as the input data source for each branch.

[0029] For the visual perception branch, a deep architecture based on a 3D residual convolutional neural network is constructed. The four-dimensional photoelectric fusion map containing depth, thermal, spectral, and polarization information constructed in step S1 is reshaped into a five-dimensional tensor input format, denoted as... ,in For batch size, The number of feature channels corresponds to the four dimensions of the optoelectronic fusion map. For the number of slices based on time or depth, and To capture the feeding surging patterns and spatial distribution changes of fish schools, a multi-scale 3D convolution kernel is used to simultaneously perform sliding scans in both the spatial and channel dimensions, representing the height and width of the image. The mathematical expression of the 3D convolution operation is as follows: ; In the formula, Indicates the first Layer Each feature map in coordinates Neuron activation values ​​at that location It is a ReLU nonlinear activation function. For the first Feature map index of the layer , , These represent the dimensions of the 3D convolution kernel in the length, width, and depth dimensions, respectively. For the first Layer The convolution kernel and the first Learnable weight parameters connected to each input feature map For bias terms, The output features of the previous layer are processed by multiple stacked 3D convolutional layers and 3D max pooling layers, and then flattened by a global average pooling layer to output a high-dimensional visual space feature vector.

[0030] For the time-series analysis branch, an architecture based on a temporal convolutional network (TCN) is constructed to organize the standardized multi-model water quality data, including historical sequences of oxygen content, temperature, and pH, into an input matrix. ,in The time window length, For water quality parameter features, causal dilated convolution is used to encode the sequence. An exponentially increasing dilation factor is used to expand the receptive field, and a gating mechanism is introduced to control the flow of historical information. The formula for the temporal convolution operation of a layer is as follows: ; In the formula, For the first Layer at time The output characteristics, As input to the previous layer, Indicates the expansion factor as dilated convolution operation, As a parameter controlling the sampling interval of the convolutional kernel, it determines the receptive field range of the network when processing long sequences. This indicates element-wise multiplication. Here is the weight matrix of the filter convolution kernel. Here is the weight matrix of the gated convolution kernel, and the tanh function is used to extract temporal features. The Sigmoid function is used as a gate signal to adjust the information throughput. After being processed by a multi-layer residual-connected dilated convolutional block, it outputs a temporal evolution feature vector, which represents the dynamic change trend of the water quality environment over time.

[0031] For the attribute mapping branch, a multilayer perceptron network is constructed to extract semantic features. First, the fish body attributes are processed, and a pre-constructed network of size is used. Entity embedding matrix ,in The total number of fish species is the preset number. To embed the vector dimension, index according to the input discrete fish species category label. Find the corresponding dense vector from the matrix. This dense vector is compared with the normalized growth cycle value. and activity index Channel splicing was performed to construct a set of original biological features of the fish. The feature set is then input into the biological behavior reasoning subnetwork, and biological behavior semantic vectors are extracted through fully connected layers and nonlinear transformations. Simultaneously, the feed component ratio data and feed density values ​​are combined into an input vector. The input is a subnetwork for settling dynamics simulation. A fully connected layer maps the physical property parameters to a high-dimensional space, extracting semantic vectors of settlement characteristics. The nonlinear mapping process of attribute features is as follows: ; In the formula, The corresponding output semantic vector, The corresponding input original feature set, , These are the weight matrices for different fully connected layers. , For bias vectors, Finally, the biological habit semantic vector is activated using the GELU activation function. Semantic vector of settlement characteristics The vectors are then concatenated and merged to form the final semantic embedding vector.

[0032] Finally, the high-dimensional visual space features, temporal evolution feature vectors, and semantic embedding vectors output from the above three branches are output as mutually independent modal coding results to the next stage. Through the parallel heterogeneous network architecture, unstructured video images, semi-structured time series, and structured attribute data are effectively mapped to the same high-dimensional semantic space, realizing the deep feature extraction and decoupling of multi-source heterogeneous information in complex aquaculture scenarios.

[0033] S3. Based on the distribution dispersion of high-dimensional visual space features, temporal evolution feature vectors and semantic embedding vectors, modal confidence weights are generated. Based on this, adaptive weighted fusion of each feature vector is performed, and a global state feature vector is generated using a multi-head attention mechanism. Furthermore, in S3, generating the global state feature vector specifically includes the following steps: High-dimensional visual space features, temporal evolution feature vectors, and semantic embedding vectors are input into their respective feature projection layers to map features of different dimensions to a unified latent feature space. The distribution dispersion of each modality feature in the latent feature space is calculated separately. The distribution dispersion is represented by the information entropy of the feature vector along the channel dimension, and the distribution dispersion is normalized to the real-time confidence weight of each modality through the activation function. The corresponding modal features are multiplied by real-time confidence weights to generate weighted high-dimensional visual space features, temporal evolution feature vectors, and semantic embedding vectors. The weighted modal features are concatenated into a multimodal feature sequence, which is then input into the multi-head self-attention mechanism module of the pre-trained large model. The cross-modal correlation matrix between visual, temporal, and attribute features is calculated, and the global state feature vector is output through weighted aggregation.

[0034] Specifically, firstly, three independent feature projection layers are constructed in the feature fusion module, each receiving the high-dimensional visual space features output from step S2. Temporal evolution feature vector and semantic embedding vectors Since these three feature vectors come from different network branches, their dimensions are respectively , and Furthermore, since they follow different statistical distributions, it is necessary to map them to a latent feature space of a unified dimension through a linear transformation. Let the unified dimension be 1. For any mode Projected latent feature vectors The calculation formula is as follows: ; In the formula, The learnable projection matrix for the corresponding modality has the following dimensions: Used to adjust feature dimensions. For bias vectors, The input is the original modal feature vector.

[0035] Subsequently, to evaluate the data quality and validity of each modality at the current time, the distribution dispersion of each latent feature vector along the channel dimension is calculated. First, the Softmax function is used to analyze the feature vectors along the channel dimension. Normalization yields the probability distribution form. , of which The probability value of each channel Calculated as Next, the distribution dispersion of this mode is calculated using the information entropy formula. This characterizes the information richness or uncertainty of the feature: ; Obtain the dispersion Then, it is passed through a fully connected node containing a Sigmoid activation function, mapping the entropy value to a range within... Real-time confidence weights between This process allows the network to automatically learn the non-linear relationship between entropy and feature reliability based on training feedback. After calculating the weights, the original latent features are adaptively weighted to obtain a weighted feature vector. : ; After weighting, the weighted feature vectors of the three modalities are concatenated along the sequence dimension to construct a multimodal feature sequence of length 3. The sequence is then input into the multi-head self-attention mechanism module of the pre-trained large model. Inside the module, the input sequence is first processed... Multiply by the query matrix respectively Key matrix Sum matrix Generate query vectors Key vector Sum value vector The dot product attention mechanism is used to calculate the cross-modal correlation matrix between visual, temporal, and attribute features, and the value vectors are weighted and aggregated accordingly. For the th... Each attention head, its output The calculation formula is as follows: ; In the formula, , , The first The query, key, and value matrix corresponding to each attention head. This is the transpose of the key matrix. The scaling factor is used to prevent the gradient from vanishing due to an excessively large dot product. The function operates on the sequence dimension, and the generated attention map reflects the interdependencies between features of different modalities. The outputs of all attention heads are concatenated and fused through a linear layer. Finally, the output sequence is subjected to global average pooling to output a global state feature vector containing rich cross-modal interaction information.

[0036] By introducing an adaptive weighting mechanism based on information entropy, the problem of partial mode failure caused by sensor malfunction or harsh environment in aquaculture is effectively solved. Furthermore, through a multi-head attention mechanism, the implicit correlation between visual behavior, environmental parameters and biological habits is deeply explored, generating a more robust and comprehensive global description vector of aquaculture status.

[0037] S4. Input the global state feature vector into the state prediction neural network, and output the feeding intensity index and environmental state index through the fully connected layer. Calculate the deviation of the feeding intensity index and environmental state index from their respective standard thresholds, correct the benchmark feeding curve, and generate dynamic correction parameters for the feeding strategy. Furthermore, in S4, calculating the deviation of the two indices relative to the standard threshold and correcting the baseline feeding curve specifically includes the following steps: The global state feature vector is input into the task-specific decoding layer of the state prediction neural network, and the fully connected operation is used to map and output the feeding intensity index and the environmental state index respectively. Read the preset standard feeding threshold and safe environment threshold, calculate the positive demand deviation between the predicted feeding intensity index and the standard feeding threshold, and the environmental pressure deviation between the environmental state index and the safe environment threshold; Based on the current number of days of aquaculture and the attributes of the fish species, the preset baseline feeding curve is invoked to query and obtain the theoretical basic feeding rate and theoretical basic total feeding amount at the current time point. By constructing dynamic correction coefficients using positive demand deviation and environmental pressure deviation, the theoretical basic feeding rate and theoretical basic total feeding amount are weighted and modulated to generate dynamic correction parameters for the feeding strategy that include the final feeding rate and the final feeding duration.

[0038] Furthermore, calculating the positive demand deviation between the predicted feeding intensity index and the standard feeding threshold, and the environmental stress deviation between the environmental state index and the safe environmental threshold, specifically includes the following steps: Based on the current fish growth cycle and sunshine duration, the corresponding standard feeding intensity benchmark value and water quality safety threshold value are retrieved from the preset database; The raw difference is obtained by subtracting the standard food intensity benchmark value from the food intensity index. Negative values ​​are filtered out using a linear rectification function, and the positive part is retained and multiplied by a preset food conversion coefficient to obtain the positive demand deviation. The difference between the environmental state index and the water quality safety threshold is calculated. When the environmental state index is lower than the safety threshold, the difference is mapped to a nonlinear environmental pressure deviation using an exponential decay function. The positive demand deviation and environmental pressure deviation are normalized, and the deviation parameters are output as the final correction calculation.

[0039] Specifically, first, a state prediction neural network is run in the central processing unit, and the global state feature vector output in step S3 is processed. The task-specific decoding layer, composed of multiple fully connected neural networks, is fed into the network. This layer maps high-dimensional abstract features to a specific physical state index space and outputs a feeding intensity index representing the current hunger level of the fish population through matrix operations. and environmental status index, which characterizes the current suitability of water quality. The calculation formula for the decoding layer is as follows: ; In the formula, This is the weight matrix of the decoding layer. For bias vectors, The sigmoid activation function is used to constrain the output exponent to... Within the standard range, A value close to 1 indicates an extremely strong appetite. A value close to 1 indicates excellent water quality.

[0040] After the prediction is completed, the system proceeds to the deviation calculation and threshold calibration stage. Based on the current fish growth cycle and real-time sunlight duration, the system retrieves the corresponding standard feeding intensity benchmark value from the preset expert knowledge database. and water quality safety threshold Subsequently, the deviation between the predicted index and the standard threshold is calculated. For the feeding dimension, the predicted feeding intensity index is used. Subtract the standard reference value The negative values ​​are filtered out using a linear rectified function (ReLU), retaining only the positive part representing the additional food intake requirement, and then multiplied by a preset food intake conversion factor. A positive demand deviation was obtained. For the environmental dimension, the environmental state index is calculated. Water quality safety threshold The difference between them, when When this occurs, it indicates that the water quality is in a non-ideal state. The difference is then mapped to a non-linear environmental pressure deviation using an exponential decay function. This is used to characterize the degree to which environmental degradation inhibits feeding; the deviation calculation formula is as follows: ; ; ; In the formula, The environmental pressure attenuation constant, As an environmental constraint factor, its value is in The greater the difference between them, the better. The exponential decay indicates that the smaller the allowable feeding ratio, the more pronounced the decay becomes in relation to the above calculations. and Perform normalization verification to ensure that the value is within the linear operation range allowed by the algorithm.

[0041] Finally, the final dynamic adjustment parameters for the feeding strategy are generated. Based on the current number of days of rearing and the fish species attributes, a preset baseline feeding curve is invoked, and the theoretical basic feeding rate at the current time point is retrieved. Total amount of feed based on theoretical basis Utilizing the aforementioned positive demand deviation Deviation from environmental pressure Constructing dynamic correction coefficients The theoretical baseline value is weighted and modulated. The logic for constructing the correction coefficient is as follows: positive demand is superimposed on the basic feeding amount, while being subject to multiplicative suppression by environmental pressure, resulting in the final feeding rate. With final feeding time The calculation formula is as follows: ; ; ; The system will calculate and The parameters are packaged into a dynamic correction parameter package for feeding strategy and prepared to be sent to the control layer. By combining neural network prediction and rule correction, closed-loop control of feeding decision is realized. It can respond to the real-time explosive feeding demand of fish and quickly cut off feeding when water quality deteriorates through an exponential decay mechanism, effectively avoiding pollution from uneaten feed and ensuring a dynamic balance between aquaculture benefits and environmental safety.

[0042] S5. Send the dynamic correction parameters of the feeding strategy to the control system, adjust the preset feeding rate and total amount, and control the equipment to perform feeding.

[0043] Furthermore, in S5, adjusting the preset feeding rate and total amount, and controlling the equipment to perform feeding specifically includes the following steps: The dynamic adjustment parameters of the feeding strategy are analyzed, and the final feeding rate and final feeding duration contained therein are extracted and used as the current execution-level feeding instructions. By calling the bait density attribute in the bait multi-model data, a mass-volume conversion model is constructed to convert the final feeding rate into the target rotation speed of the feeder motor; The target rotation speed and final feeding time are sent to the underlying control unit through the industrial control communication protocol, which drives the feeding actuator to start feeding at the updated rate. The actual cumulative feeding amount is monitored in real time by a metering device installed at the feeder outlet. When the monitored cumulative feeding amount reaches the target total feeding amount determined by the final feeding rate and the final feeding time, a stop command is triggered to interrupt the motor operation.

[0044] Specifically, the host computer of the control system first receives and parses the dynamic correction parameter package of the feeding strategy generated in step S4, and extracts the final feeding rate optimized by AI decision-making from it. With final feeding time The two are then multiplied to calculate the target total amount of feed for this feeding task. At this point, the system calls the pre-stored bait multi-model data and reads the bulk density attribute of the currently used bait. Based on the physical parameters of the feeder's mechanical structure, a conversion model between mass flow rate and motor speed is constructed. Considering the filling coefficient and slippage characteristics of the pelleted bait during the screw conveying process, the final feeding rate in terms of mass is converted into the target speed of the feeder's feeding motor using the following formula. : ; In the formula, This represents the target number of rotations per minute for the feeding motor. The final feeding rate output by the algorithm. This refers to the bulk density of the bait. This represents the theoretical geometric volume discharged per revolution of the feeder's screw conveyor. The conveying efficiency coefficient of the particulate medium is related to the particle size and surface friction coefficient of the feed. Through this model calculation, the abstract decision rate is transformed into kinematic commands that can be executed by the physical equipment.

[0045] After completing the speed conversion, the host computer uses the industrial-grade Modbus RTU communication protocol to transmit the calculated target speed. and final feeding time Encapsulated as a control frame, it is sent to the underlying programmable logic controller (PLC) or embedded microcontroller via RS485 bus. After receiving the instruction, the underlying controller uses a PID closed-loop control algorithm to adjust the output frequency of the frequency converter or the voltage of the DC motor, driving the feeding actuator to quickly reach and stabilize at the target speed, thereby starting the feeding operation at the updated and accurate rate.

[0046] During the feeding process, a dynamic weighing and metering device or flow sensor installed at the feeder outlet is activated to sample the current instantaneous feeding amount in real time at a millisecond frequency and perform integration and accumulation to obtain the actual cumulative feeding amount. The underlying controller continuously compares the actual cumulative feeding amount with the target total feeding amount. When it detects... When the time comes, a hardware interrupt signal is immediately triggered, the motor drive power is cut off and the holding brake is applied to forcibly stop the feeding action, and the final execution data of this task is fed back to the database to form a closed-loop log.

[0047] Through precise physical model mapping and industrial-grade closed-loop control, the virtual decisions at the algorithm level are seamlessly transformed into mechanical actions at the physical level, ensuring that the actual feeding amount is highly consistent with the optimal feeding strategy predicted by AI, thus achieving precise operation in the breeding process.

[0048] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for correcting precise feeding in aquaculture using multidimensional and multi-modal data, characterized in that, Includes the following steps: S1. Collect photoelectric multi-mode data and water quality multi-mode data in the aquaculture environment, and simultaneously input fish body multi-mode data and feed multi-mode data. After time stamp synchronization and spatial coordinate mapping, generate standardized data. S2. Input the standardized data into a multi-channel feature encoding neural network, and use the parallel branches in the network to perform three-dimensional convolution operations and sequence encoding on the photoelectric multi-mode data and water quality multi-mode data respectively. Combine the attribute data of fish body and bait to extract semantic features, and independently output high-dimensional visual space features, temporal evolution feature vectors and semantic embedding vectors. S3. Based on the distribution dispersion of the high-dimensional visual space features, temporal evolution feature vectors and semantic embedding vectors, generate modal confidence weights, and adaptively weight and fuse each feature vector accordingly, and use a multi-head attention mechanism to generate a global state feature vector. S4. Input the global state feature vector into the state prediction neural network, and output the feeding intensity index and the environmental state index through the fully connected layer. Calculate the deviation of the feeding intensity index and the environmental state index from their respective standard thresholds, correct the benchmark feeding curve, and generate dynamic correction parameters for the feeding strategy. S5. Send the dynamic correction parameters of the feeding strategy to the control system, adjust the preset feeding rate and total amount, and control the equipment to perform feeding.

2. The method for correcting precise feeding in aquaculture using multidimensional and multi-modal data according to claim 1, characterized in that, The optoelectronic multimode data includes depth point clouds acquired by a binocular camera, spectral features acquired by a multispectral sensor, and infrared thermal imaging data and polarized light data.

3. The method for correcting precise feeding in aquaculture using multidimensional and multi-modal data according to claim 1, characterized in that, In step S1, the process of generating standardized data through timestamp synchronization and spatial coordinate mapping includes the following steps: Establish a unified system clock reference, take the acquisition frequency of binocular vision data as the trigger axis, and use linear interpolation or nearest neighbor matching algorithm to upsample low-frequency water quality multimodal data and other photoelectric multimodal data to obtain time-aligned multimodal data; A three-dimensional spatial coordinate system is constructed based on binocular vision depth data. Using a pre-calibrated camera intrinsic parameter matrix and extrinsic parameter rotation and translation matrix between different sensors, the pixels of infrared thermal imaging images, multispectral images, and polarized light images are projected and mapped onto the three-dimensional spatial coordinate system to construct a four-dimensional optoelectronic fusion map containing depth information, thermal information, spectral information, and polarization information. Encode or embed features of fish species, growth cycle, and activity level in fish body multi-modal data, and feed composition and feed density in feed multi-modal data to generate static attribute feature vectors. The water quality multi-model data and the four-dimensional photoelectric fusion map are standardized, and the processed data is combined with the static attribute feature vector to form a standardized dataset.

4. The method for correcting precise feeding in aquaculture using multidimensional and multi-modal data according to claim 1, characterized in that, In step S2, the convolution operation and sequence encoding of the visual image and the sensor time-series data using parallel branches within the network specifically include the following steps: A visual perception branch based on a three-dimensional convolutional neural network is constructed. The four-dimensional photoelectric fusion map in the standardized dataset is used as input. Multi-scale three-dimensional convolutional kernels are used to scan the spatial dimension and channel dimension of the image simultaneously to extract high-dimensional visual spatial features. A temporal analysis branch based on a temporal convolutional network is constructed. Water quality multi-model data in the standardized dataset is used as input. Historical data on oxygen content and temperature are sequence encoded through a gating mechanism to output a temporal evolution feature vector. An attribute mapping branch based on a multilayer perceptron is constructed. The static attribute feature vectors in the standardized dataset are input into a fully connected layer to perform nonlinear mapping and feature recombination on the physiological characteristics of fish species and the physical properties of bait, and to extract semantic embedding vectors. The high-dimensional visual space features, temporal evolution feature vectors, and semantic embedding vectors are used as the encoded output of independent modalities.

5. The method for correcting precise feeding in aquaculture using multidimensional and multi-modal data according to claim 4, characterized in that, The extraction of semantic embedding vectors specifically includes the following steps: Discrete fish species category labels are mapped to dense vectors using a pre-constructed entity embedding matrix. These dense vectors are then concatenated with normalized growth cycle values ​​and activity indices to construct the original biological feature set of fish. The feed composition ratio data and feed density values ​​are input into a fully connected neural network to perform feature upsizing, and the physical property parameters are mapped to a high-dimensional feature space to construct the original physical feature set of the feed. The original biological feature set of the fish is input into the biological behavior reasoning subnetwork within the attribute mapping branch, and the biological behavior semantic vector is extracted through multi-layer nonlinear transformation; The original physical feature set of the bait is input into the sedimentation dynamics simulation subnetwork within the attribute mapping branch, sedimentation characteristic semantic vectors are extracted, and the biological habit semantic vectors are combined with the sedimentation characteristic semantic vectors to form the final semantic embedding vector.

6. The method for correcting precise feeding in aquaculture using multidimensional and multi-modal data according to claim 1, characterized in that, In step S3, generating the global state feature vector specifically includes the following steps: The high-dimensional visual space features, temporal evolution feature vectors, and semantic embedding vectors are respectively input into their respective feature projection layers, mapping features of different dimensions to a unified-dimensional latent feature space. The distribution dispersion of each modality feature in the latent feature space is calculated respectively. The distribution dispersion is characterized by the information entropy of the feature vector along the channel dimension, and the distribution dispersion is normalized to the real-time confidence weight of each modality through the activation function. The real-time confidence weights are used to perform dot product operations on the corresponding modal features to generate weighted high-dimensional visual spatial features, temporal evolution feature vectors, and semantic embedding vectors. The weighted modal features are concatenated into a multimodal feature sequence, which is then input into the multi-head self-attention mechanism module of the pre-trained large model. The cross-modal correlation matrix between visual, temporal, and attribute features is calculated, and the global state feature vector is output through weighted aggregation.

7. The method for correcting precise feeding in aquaculture using multidimensional and multi-modal data according to claim 1, characterized in that, In step S4, calculating the deviation of the two indices relative to the standard threshold and correcting the baseline feeding curve specifically includes the following steps: The global state feature vector is input into the task-specific decoding layer of the state prediction neural network, and the feeding intensity index and the environmental state index are mapped and output respectively through fully connected operations. Read the preset standard feeding threshold and safe environment threshold, calculate the positive demand deviation between the predicted feeding intensity index and the standard feeding threshold, and the environmental pressure deviation between the environmental state index and the safe environment threshold; Based on the current number of days of aquaculture and the attributes of the fish species, the preset baseline feeding curve is invoked to query and obtain the theoretical basic feeding rate and theoretical basic total feeding amount at the current time point. A dynamic correction coefficient is constructed using the positive demand deviation and environmental pressure deviation. The theoretical basic feeding rate and the theoretical basic total feeding amount are weighted and modulated to generate dynamic correction parameters for the feeding strategy that include the final feeding rate and the final feeding duration.

8. The method for correcting precise feeding in aquaculture using multidimensional and multi-modal data according to claim 7, characterized in that, The positive demand deviation between the calculated and predicted feeding intensity index and the standard feeding threshold, and the environmental pressure deviation between the environmental state index and the safe environmental threshold, specifically include the following steps: Based on the current fish growth cycle and sunshine duration, the corresponding standard feeding intensity benchmark value and water quality safety threshold value are retrieved from the preset database; Subtract the standard food intake intensity benchmark value from the food intake intensity index to obtain the original difference. Use a linear rectification function to filter out negative values, retain the positive part and multiply it by a preset food intake conversion coefficient to obtain the positive demand deviation. The difference between the environmental state index and the water quality safety threshold is calculated. When the environmental state index is lower than the safety threshold, the difference is mapped to a nonlinear environmental pressure deviation using an exponential decay function. The positive demand deviation and environmental pressure deviation are normalized to output the final deviation parameters used for correction calculations.

9. The method for correcting precise feeding in aquaculture using multidimensional and multi-modal data according to claim 1, characterized in that, In step S5, adjusting the preset feeding rate and total amount, and controlling the equipment to perform feeding specifically includes the following steps: The dynamic correction parameters of the feeding strategy are analyzed, and the final feeding rate and final feeding duration contained therein are extracted and used as the current execution-level feeding instruction. By calling the bait density attribute in the bait multi-model data, a mass-volume conversion model is constructed, and the final feeding rate is converted into the target rotation speed of the feeder motor. The target rotation speed and the final feeding time are sent to the underlying control unit through the industrial control communication protocol, driving the feeding actuator to start feeding at the updated rate. The actual cumulative feeding amount is monitored in real time by a metering device installed at the feeder outlet. When the monitored cumulative feeding amount reaches the target total feeding amount determined by the final feeding rate and the final feeding time, a stop command is triggered to interrupt the motor operation.

Images