A multi-modal data-driven visualization management method for aquaculture

Abstract

This invention discloses a multimodal data-driven visualization management method for aquaculture, belonging to the field of data visualization technology. It addresses the problem of error accumulation during aquaculture status assessment, making it difficult to accurately reflect the true correlation between fish behavior and the aquaculture environment. By collecting water turbidity data and constructing a visual modality suppression factor, the reliability of underwater visual recognition results is corrected. An apparent oxygen coupling index is generated by combining dissolved oxygen concentration. Based on the spatial coordinates of the fish population, cluster analysis of fish behavior differences is performed. The apparent oxygen coupling index and fish behavior differences are fused to generate an aquaculture status score. A feedback adjustment mechanism is constructed by combining historical aquaculture status scores to dynamically correct the visual modality suppression factor, obtaining a corrected aquaculture status score. This improves the accuracy of aquaculture status identification and the reliability of management decisions in complex aquatic environments, providing data support for visualized aquaculture management.

Description

Technical Field

[0001] This invention relates to the field of data visualization technology, and more specifically, to a multimodal data-driven visualization management method for aquaculture. Background Technology

[0002] With the development of large-scale and intensive aquaculture, aquaculture management is gradually shifting from manual inspection to digital and visual management. Existing aquaculture systems typically use underwater camera equipment to obtain information on fish activity and combine it with environmental monitoring data such as dissolved oxygen and temperature to assess the aquaculture status, thereby helping aquaculture personnel to understand the growth status, feeding status and environmental changes of the fish.

[0003] The existing technology has the following shortcomings: Currently, existing technologies for analyzing aquaculture status using underwater visual data and environmental monitoring data mostly employ static evaluation mechanisms. These mechanisms are insufficient to adaptively adjust the contribution of different modal data to the visual recognition quality fluctuations caused by changes in water transparency. This leads to the accumulation of errors during the aquaculture status assessment process, making it difficult to accurately reflect the true correlation between fish behavior and the aquaculture environment. Consequently, this affects the stability and reliability of visualization management results in complex aquaculture scenarios. Therefore, this paper proposes a multimodal data-driven visualization management method for aquaculture.

[0004] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a multimodal data-driven visualization management method for aquaculture. This method constructs a visual modality suppression factor using water turbidity data to correct the reliability of underwater visual recognition results. It also generates a visual-oxygen coupling index based on dissolved oxygen concentration and analyzes the differential characteristics of fish behavior through fish school clustering based on their spatial location. By fusing environmental conditions and behavioral characteristics, an aquaculture status score is generated. Furthermore, feedback adjustments are made based on historical aquaculture status to achieve dynamic retrospective correction of the visual modality, thereby obtaining a corrected aquaculture status score to address the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a multimodal data-driven visualization management method for aquaculture, comprising the following steps: Step S1: Retrieve the instantaneous turbidity sequence of the aquaculture area, calculate the visual modality suppression factor through the instantaneous turbidity sequence, and extract the original confidence level of visual recognition and the visual recognition anomaly category label from the underwater camera to evaluate the original features of the visual modality. Step S2: After inversely suppressing the original visual modality features according to the visual modality suppression factor, suppressed visual assessment features are generated. Dissolved oxygen concentration readings in the aquaculture area are detected. The visual oxygen coupling index is generated by combining the dissolved oxygen concentration readings and the suppressed visual assessment features. Step S3: Set the fish gathering threshold, collect the coordinates of individual targets in the underwater camera, perform clustering and calibration of the individual target coordinates based on the fish gathering threshold, analyze the differential characteristics of fish behavior based on the clustering and calibration results, and evaluate the current aquaculture status score of the aquaculture area in combination with the apparent oxygen coupling index. Step S4: Retrieve the historical aquaculture status score of the aquaculture area, combine it with the current aquaculture status score to generate an aquaculture feedback difference, determine whether to adjust the visual modality suppression factor based on the aquaculture feedback difference, and generate a corrected current aquaculture status score.

[0007] In a preferred embodiment, in step S1, the online turbidity sensor collects turbidity values ​​of the water body in the aquaculture area at a fixed sampling period, and the turbidity values ​​are arranged in time sequence to obtain the instantaneous turbidity sequence of the water body. The instantaneous turbidity sequence of the water body is standardized to obtain the standardized instantaneous turbidity sequence of the water body; The arithmetic mean of the standardized instantaneous turbidity sequence of the water body is taken as the baseline component of turbidity. Take the absolute value of the difference between the turbidity value and the turbidity baseline component in the standardized instantaneous turbidity sequence of the water body, sum the absolute values ​​of each difference and divide by the number of turbidity values ​​to obtain the turbidity disturbance component. By substituting the turbidity baseline component and the turbidity perturbation component into the exponential saturation mapping formula, the visual modality suppression factor is calculated.

[0008] In a preferred embodiment, in step S1, the pre-trained visual recognition model outputs two inference results for each frame of real-time image captured by the underwater camera: the original confidence level of visual recognition and the visual recognition anomaly category label. The original confidence level of visual recognition is the value of the largest component in the classification probability vector output by the pre-trained visual recognition model for a real-time image frame. The visual recognition anomaly category label is the discrete classification result output by the pre-trained visual recognition model for the behavior of fish in real-time image frames; By mapping incremental values ​​to the visual anomaly category labels, we obtain standardized visual anomaly category labels.

[0009] In a preferred embodiment, in step S2, the suppressed visual evaluation features are calculated from the original visual modality features and the visual modality suppression factor, using the following formula: Where F represents the original features of the visual modality. is the visual modality suppression factor, and S is the suppressed visual evaluation feature; The dissolved oxygen concentration in the aquaculture area is detected in real time using an online dissolved oxygen sensor, and the dissolved oxygen concentration reading is output. The dissolved oxygen concentration readings were standardized to obtain the standardized results corresponding to the dissolved oxygen concentration readings. The difference between 1 and the standardized dissolved oxygen concentration reading is taken as the dissolved oxygen deficit.

[0010] In a preferred embodiment, in step S2, the suppressed visual assessment features and dissolved oxygen deficit are used to calculate the visual-oxygen coupling index: Where S represents the suppressed visual assessment feature, and D represents the dissolved oxygen deficit. The oxygen coupling index is the visual-oxygen coupling index.

[0011] In a preferred embodiment, in step S3, a fish aggregation threshold is set. The fish aggregation threshold is a preset spatial distance parameter, which is derived from the average nearest neighbor distance statistics of fish swimming normally in historical aquaculture scenarios. The underwater camera outputs individual position coordinates within a continuous sampling period. An individual target refers to a basic analytical unit formed by identifying and separating a single fish in the aquaculture water through target detection and tracking algorithms in continuous video frames captured by the underwater camera. The coordinates of an individual target's position are horizontal plane coordinates and vertical depth coordinates. The spatial Euclidean distance between any two individual targets is calculated using these coordinates.

[0012] In a preferred embodiment, in step S3, when the spatial Euclidean distance between any two target individuals is less than or equal to the fish aggregation threshold, it is determined that the two target individuals belong to the same cluster set, thereby forming a fish cluster. For each fish swarm cluster within a time window, calculate the average of the target position coordinates of all individuals within that cluster to obtain the geometric center coordinates of the fish swarm cluster. The difference between the geometric center coordinates between adjacent time windows is calculated to obtain the cluster center offset. The offsets of all clusters are weighted and summed to obtain the differential features of fish swarm behavior. The fish behavior difference features are linearly fused with the apparent oxygen coupling index to obtain the current aquaculture status score.

[0013] In a preferred embodiment, in step S4, the historical aquaculture status score generated in the aquaculture area during a continuous historical monitoring period is retrieved. The historical aquaculture status score is derived from the statistical mean obtained by continuously calculating the multimodal fusion results of the aquaculture area within a preset time sliding window. Subtracting the historical breeding status score from the current breeding status score yields the breeding feedback difference. When the absolute value of the aquaculture feedback difference is greater than or equal to the preset feedback threshold, the visual modality inhibition factor correction mechanism is triggered. Conversely, the visual modality suppression factor correction mechanism will not be triggered.

[0014] In a preferred embodiment, in step S4, when the visual modality suppression factor correction mechanism is triggered, a callback correction is performed on the visual modality suppression factor to obtain the callback-corrected visual modality suppression factor. The current aquaculture status score is recalculated based on the visual modality suppression factor after the callback correction, resulting in the corrected current aquaculture status score: ; in, The revised score for the current aquaculture status. This is the visual modality suppression factor after callback correction. Rate the current breeding status. This is the preset visual impact adjustment coefficient.

[0015] The technical effects and advantages of this invention are as follows: This invention collects water turbidity data and constructs a visual modality suppression factor to correct the reliability of underwater visual recognition results. It generates a visual-oxygen coupling index based on dissolved oxygen concentration, and performs cluster analysis on the differential characteristics of fish behavior based on the spatial coordinates of the fish population. The visual-oxygen coupling index and the differential characteristics of fish behavior are then fused to generate a farming status score. A feedback adjustment mechanism is constructed based on historical farming status scores to dynamically adjust the visual modality suppression factor, resulting in a corrected farming status score. This enables a collaborative assessment of the farming environment and fish behavior, improving the accuracy of farming status recognition and the reliability of management decisions in complex aquatic environments, and providing data support for visualized management of aquaculture. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the implementation of a multimodal data-driven visualization management method for aquaculture according to the present invention.

[0017] Figure 2 This is a schematic diagram illustrating the steps of a multimodal data-driven visualization management method for aquaculture according to the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] This invention collects water turbidity data and constructs a visual modality suppression factor to correct the reliability of underwater visual recognition results. It generates a visual-oxygen coupling index based on dissolved oxygen concentration, and performs differential fish behavior characteristics based on fish spatial location coordinates. The visual-oxygen coupling index and fish behavior characteristics are then fused to generate a farming status score. A feedback adjustment mechanism is constructed based on historical farming status scores to dynamically correct the visual modality suppression factor, resulting in a corrected farming status score. This enables a collaborative assessment of the farming environment and fish behavior, improving the accuracy of farming status recognition and the reliability of management decisions in complex aquatic environments.

[0020] Example 1, such as Figures 1 to 2 As shown, a multimodal data-driven visualization management method for aquaculture has the following specific process: Step S1: Retrieve the instantaneous turbidity sequence of the aquaculture area, calculate the visual modality suppression factor through the instantaneous turbidity sequence, and extract the original confidence level of visual recognition and the visual recognition anomaly category label from the underwater camera to evaluate the original features of the visual modality. Step S2: After inversely suppressing the original visual modality features according to the visual modality suppression factor, suppressed visual assessment features are generated. Dissolved oxygen concentration readings in the aquaculture area are detected. The visual oxygen coupling index is generated by combining the dissolved oxygen concentration readings and the suppressed visual assessment features. Step S3: Set the fish gathering threshold, collect the coordinates of individual targets in the underwater camera, perform clustering and calibration of the individual target coordinates based on the fish gathering threshold, analyze the differential characteristics of fish behavior based on the clustering and calibration results, and evaluate the current aquaculture status score of the aquaculture area in combination with the apparent oxygen coupling index. Step S4: Retrieve the historical aquaculture status score of the aquaculture area, combine it with the current aquaculture status score to generate an aquaculture feedback difference, determine whether to adjust the visual modality suppression factor based on the aquaculture feedback difference, and generate a corrected current aquaculture status score.

[0021] The specific implementation is as follows: In step S1, an online turbidity sensor and an underwater camera are deployed in the aquaculture area. The online turbidity sensor is used to continuously measure the turbidity level of the water in the aquaculture area, and the underwater camera is used to collect real-time images of the underwater scene in the aquaculture area. The real-time image frames collected by the underwater camera are input frame by frame into the pre-trained visual recognition model for inference. It should be explained that an online turbidity sensor is a sensing device that uses the principle of optical transmission or scattering to measure the turbidity level of water in real time. In this example, it is used to continuously collect the turbidity values ​​of the water in the aquaculture area at a fixed sampling period. The pre-trained visual recognition model refers to an image classification model that completes parameter fixing by using a set of labeled underwater behavior images of aquaculture as training samples before the aquaculture area is put into operation. In this example, for each frame of real-time image from the underwater camera, two inference results are output simultaneously: the original confidence level of visual recognition and the visual recognition anomaly category label.

[0022] An online turbidity sensor collects turbidity values ​​of the water in an aquaculture area at a fixed sampling period. The turbidity values ​​are then arranged in time sequence to obtain an instantaneous turbidity sequence of the water.

[0023] The instantaneous turbidity sequence of water body is standardized, and the calculation formula is as follows: Where x is the turbidity value in the instantaneous turbidity sequence of the water body. This is the minimum detection limit for online turbidity sensors. This represents the maximum detection limit for the online turbidity sensor. This is the result after standardizing the turbidity value corresponding to x.

[0024] After standardizing each turbidity value in the instantaneous turbidity sequence of the water body, the standardized instantaneous turbidity sequence of the water body is obtained. The visual modal suppression factor is calculated from the standardized instantaneous turbidity sequence of water using the exponential saturation mapping rule. The specific steps are as follows: A1: Take the arithmetic mean of the standardized instantaneous turbidity sequence of the water body as the baseline component of turbidity; A2: Take the absolute value of the difference between the turbidity value and the turbidity baseline component in the standardized instantaneous turbidity sequence of the water body, sum the absolute values ​​of each difference and divide by the number of turbidity values ​​to obtain the turbidity disturbance component. A3: Substitute the turbidity baseline component and the turbidity perturbation component into the exponential saturation mapping formula to calculate the visual modality suppression factor: Where B is the baseline turbidity component and V is the turbidity disturbance component. It is a visual modality inhibitor; The turbidity baseline component reflects the overall level of turbidity in the aquaculture area within the analysis window, while the turbidity disturbance component reflects the degree of fluctuation of turbidity relative to its baseline. The larger the turbidity baseline component or the larger the turbidity disturbance component, the larger the visual modality suppression factor, and the stronger the ability to suppress the visual modality. When both the turbidity baseline component and the turbidity disturbance component approach 0, the visual modality suppression factor approaches 0 and has no suppressive effect on the visual modality.

[0025] The pre-trained visual recognition model outputs two inference results for each real-time image frame captured by the underwater camera: the original confidence score and the visual anomaly category label. The original confidence score is the value of the largest component in the classification probability vector output by the pre-trained visual recognition model for the real-time image frame, and its value lies within the interval [0,1]. The visual anomaly category label is the discrete classification result output by the pre-trained visual recognition model for the fish behavior in the real-time image frame, including four types: normal, clustered, surfacing, and wall-collision.

[0026] Normal type refers to fish swimming dispersedly and with regular behavior in the water; Aggregation type refers to fish clustering abnormally densely and shrinking their activity range; Floating type refers to fish rising to the vicinity of the water surface and frequently opening their mouths; Wall-banging type refers to fish exhibiting stress behavior of continuously banging against the walls of the aquaculture area. The visual recognition anomaly category labels are standardized as follows: normal type is mapped to 0 according to the severity of the anomaly, clustering type is mapped to a preset first level value, floating type is mapped to a preset second level value, and wall collision type is mapped to a preset third level value. The preset first level value, preset second level value, and preset third level value increase sequentially in the interval (0,1) according to the severity of the anomaly, thus obtaining the standardized visual recognition anomaly category labels.

[0027] It should be added that the preset first-level value, preset second-level value, and preset third-level value are set as follows: Select the set of underwater behavior images of aquaculture in the aquaculture area that have been marked as calibration samples, and calculate the mean mortality risk of the target individuals corresponding to the samples according to the aggregation type, surfacing type, and wall collision type. After scaling the mean mortality risk of the three types to the (0,1] interval, assign the preset first-level value, preset second-level value, and preset third-level value to them respectively from small to large, so as to ensure that the three-level values ​​correspond to the actual degree of harm of the corresponding type.

[0028] The original visual modality features are calculated using the short-board synthesis method, which combines the original confidence level of visual recognition with the standardized visual recognition anomaly category labels. The calculation formula is as follows: Where R is the original confidence level of visual recognition, L is the standardized visual recognition anomaly category label, and F is the original visual modality feature; The original visual modality features reflect the intensity of abnormal behavior signals detected by the visual recognition model in the current image frame. When both the original visual recognition confidence level and the standardized visual recognition anomaly category label are high, the original visual modality features achieve a high value. When either the original visual recognition confidence level or the standardized visual recognition anomaly category label is low, the original visual modality features are limited by that low value. When either the original visual recognition confidence level or the standardized visual recognition anomaly category label is low, the original visual modality features are both suppressed.

[0029] In step S2, the suppressed visual evaluation features are calculated using the linear inverse suppression method from the original visual modality features and the visual modality suppression factor. The calculation formula is as follows: Where F represents the original features of the visual modality. is the visual modality suppression factor, and S is the suppressed visual evaluation feature; The larger the visual modality suppression factor, the more severely the original visual modality features are compressed, and the lower the value of the suppressed visual evaluation features; when the visual modality suppression factor approaches 0, the suppressed visual evaluation features approach the original visual modality features themselves. Online dissolved oxygen sensors are deployed in aquaculture areas to monitor the dissolved oxygen concentration in the aquaculture water in real time. The dissolved oxygen concentration readings are output according to a fixed sampling period. The dissolved oxygen concentration readings reflect the dissolved oxygen concentration level of the aquaculture water at the current sampling time. The lower the value, the weaker the oxygen supply capacity of the water and the higher the risk of hypoxia stress in fish. The dissolved oxygen concentration readings were standardized to obtain the standardized results corresponding to the dissolved oxygen concentration readings. The larger the suppressed visual assessment feature, the stronger the remaining credible visual anomaly signal and the higher the risk; the larger the standardized dissolved oxygen concentration reading, the higher the dissolved oxygen level in the water and the lower the risk. The difference between 1 and the standardized dissolved oxygen concentration reading is taken as the dissolved oxygen deficit. Dissolved oxygen deficit reflects the degree of oxygen deficiency in the water body of an aquaculture area relative to the dissolved oxygen level required for the normal survival of fish. The higher the standardized dissolved oxygen reading, the smaller the dissolved oxygen deficit; the lower the standardized dissolved oxygen reading, the larger the dissolved oxygen deficit. The visual-oxygen coupling index is calculated using the second-order norm coupling method from suppressed visual assessment features and dissolved oxygen deficit: Where S represents the suppressed visual assessment feature, and D represents the dissolved oxygen deficit. The visual oxygen coupling index; The larger the apparent oxygen coupling index, the stronger the two risk signals of residual credible visual anomalies and insufficient dissolved oxygen in the water; the smaller the apparent oxygen coupling index, the lower the level of both risk signals. It should be explained that an online dissolved oxygen sensor is a sensing device that measures the dissolved oxygen concentration in water in real time based on the principle of electrochemical or optical fluorescence quenching. In this example, it is used to collect dissolved oxygen concentration readings of the aquaculture area water at fixed sampling cycles.

[0030] By combining the suppressive effect of water turbidity on underwater visual modalities with the degree of dissolved oxygen deficiency in water bodies into the apparent oxygen coupling index, an input is provided for subsequent assessment of the current aquaculture status score of aquaculture areas.

[0031] In step S3, a fish aggregation threshold is set, and the individual position coordinates output by the underwater camera within a continuous sampling period are collected. The individual position coordinates are then subjected to cluster calibration processing, and the fish behavior difference features are calculated based on the cluster calibration results. Finally, the current aquaculture status score is generated by combining the apparent oxygen coupling index.

[0032] The fish aggregation threshold is a preset spatial distance parameter, which is derived from the average nearest neighbor distance statistics of fish swimming normally in historical aquaculture scenarios. It is obtained by performing quantile analysis on the Euclidean distance distribution between fish under normal aquaculture conditions. For example, the 25th percentile of the distance distribution under normal conditions is taken as the fish aggregation threshold.

[0033] The coordinates of the individual target position are horizontal plane coordinates and vertical depth coordinates, which are derived from the spatial positioning results output by the underwater camera after multi-target recognition and three-dimensional reconstruction of the fish target. They are used to characterize the instantaneous spatial distribution state of a single fish in the aquaculture water, and the changes in their coordinates reflect the movement trajectory of the fish and changes in the group structure.

[0034] It should be noted that an underwater camera is an image acquisition device deployed inside or on the boundary structure of an aquaculture water body, used to continuously acquire continuous video frames or image sequences of the aquaculture area in an underwater environment; an individual target refers to the basic analysis unit formed by identifying and separating a single fish body in the aquaculture water body through target detection and tracking algorithms in the continuous video frames or image sequences acquired by the underwater camera, corresponding to the observable entity representation of a single fish body at the current moment.

[0035] Clustering is performed on the coordinates of individual target locations based on a fish swarm aggregation threshold. Specifically, the spatial Euclidean distance between any two target individuals is used as the clustering criterion. When the spatial Euclidean distance between any two target individuals is less than or equal to the fish swarm aggregation threshold, the corresponding two target individuals are determined to belong to the same cluster set, thus forming a fish swarm cluster.

[0036] Clustering labeling groups fish that are spatially close to each other into the same behavioral unit. The fewer the number of fish groups, the higher the concentration of the fish group; the more the number of fish groups, the higher the dispersion of the fish group.

[0037] Based on the clustering calibration results, the differential characteristics of fish school behavior are calculated. Specifically, for each fish school cluster within a time window, the average value of the target position coordinates of all individuals is calculated to obtain the geometric center coordinates of the fish school cluster. The geometric center coordinates are used to characterize the spatial center of gravity of the local group of fish, and their changes reflect the overall migration trend of the fish school.

[0038] Furthermore, the difference in geometric center coordinates between adjacent time windows is calculated to obtain the cluster center offset. The cluster center offset represents the temporal motion offset intensity of the fish school clusters. The larger the value, the more drastic the local group motion change of the fish school.

[0039] Based on a weighted summation of all cluster offsets, the differential features of fish swarm behavior are obtained: ; in, Differential features of fish school behavior For cluster weights, This is the index value for the fish school cluster. The total number of fish clusters. This represents the number of individual targets within a fish swarm cluster.

[0040] The differential characteristics of fish behavior represent the intensity of changes in the group structure of fish over time. The larger the value, the more significant the fluctuation in the overall movement state of the fish group and the higher the probability of abnormal behavior.

[0041] The fish behavior difference features are linearly fused with the apparent oxygen coupling index to obtain the current aquaculture status score: ; in, Rate the current breeding status. The oxygen coupling index is the visual-oxygen coupling index. Differential features of fish school behavior and The preset weighting coefficients are used, and they satisfy the following conditions: This is used to adjust the contribution ratio of visual-water quality coupling information and fish behavior information in the overall score.

[0042] The higher the current aquaculture status score, the higher the combined risk of abnormal water environment and abnormal fish behavior, and the greater the urgency for intervention and regulation in the corresponding aquaculture area.

[0043] It should be noted that the preset weighting coefficients are used to quantify the contribution ratio of the visual oxygen coupling index and the differential fish behavior features in the current aquaculture status score. The method of setting them is based on the multimodal correlation analysis results in the historical aquaculture dataset. Specifically, the weighting coefficients are obtained by normalizing the correlation coefficients between the visual water quality coupling features and the fish behavior features in the historical samples and the manually labeled aquaculture status results.

[0044] In step S4, the historical aquaculture status scores generated in the continuous historical monitoring period of the aquaculture area are retrieved, and the aquaculture feedback difference is calculated in combination with the current aquaculture status score. Based on the aquaculture feedback difference, the visual modality suppression factor is dynamically reverted and judged, and the corrected current aquaculture status score is output.

[0045] Among them, the historical aquaculture status score is derived from the statistical mean obtained by continuously calculating the multimodal fusion results of the aquaculture area within a preset time sliding window. Its calculation basis is consistent with the current aquaculture status score. It is used to characterize the comprehensive operational status level of the aquaculture area on a historical time scale and reflect the long-term stability benchmark of the aquaculture system. The larger the value, the higher the degree of aquaculture anomaly accumulation within the historical period.

[0046] Subtracting the historical farming status score from the current farming status score yields the farming feedback difference. This difference characterizes the degree of deviation of the current farming status from the historical baseline.

[0047] Among them, a difference in aquaculture feedback greater than 0 indicates that the current aquaculture status score is higher than the historical aquaculture status score, indicating that the degree of aquaculture abnormality in the current monitoring period has increased compared with the historical baseline; a difference in aquaculture feedback less than 0 indicates that the current aquaculture status score is lower than the historical aquaculture status score, indicating that the current aquaculture environment tends to improve compared with the historical period; the larger the absolute value of the difference in aquaculture feedback, the more significant the deviation between the current status and the historical baseline.

[0048] Compare the absolute value of the difference in aquaculture feedback with the preset feedback threshold: When the absolute value of the aquaculture feedback difference is greater than or equal to the preset feedback threshold, the visual modality inhibition factor correction mechanism is triggered. Conversely, the visual modality suppression factor correction mechanism will not be triggered.

[0049] It should be noted that the preset feedback threshold is determined by the standard deviation of the historical aquaculture status score sequence, and is used to characterize the system's response sensitivity to status shifts.

[0050] When the visual modality suppression factor correction mechanism is triggered, a callback correction is performed on the visual modality suppression factor: , ; in, This is the visual modality suppression factor after callback correction. This is a preset adjustment coefficient used to control the intensity of feedback correction. The results are the normalized values ​​of the aquaculture feedback. This represents the average difference in aquaculture feedback within the historical window. The standard deviation of the difference in aquaculture feedback within the historical window. This is to provide feedback on the difference in aquaculture practices.

[0051] It should be noted that the preset adjustment coefficient is used to control the influence of the aquaculture feedback difference on the correction amplitude of the visual modality suppression factor. The setting method is based on the optimization and determination of the system convergence speed and stability index during the historical feedback closed-loop adjustment process. After statistical analysis of the convergence error and oscillation amplitude of the visual modality suppression factor under different adjustment coefficient values, the optimal stable interval value is selected as the preset adjustment coefficient.

[0052] The callback-corrected visual modality suppression factor is used to readjust the weight ratio of visual modality in multimodal fusion. When the callback-corrected visual modality suppression factor decreases, it indicates a reduction in the degree of suppression of visual modality, thereby increasing the contribution ratio of visual information in the fusion result. When the callback-corrected visual modality suppression factor increases, it indicates an enhancement in the degree of suppression of visual modality, thereby reducing the influence weight of visual information in the fusion result.

[0053] Finally, the current aquaculture status score is recalculated based on the visual modality suppression factor after the callback correction, resulting in the corrected current aquaculture status score: ; in, The revised score for the current aquaculture status. This is the visual modality suppression factor after callback correction. Rate the current breeding status. This is a preset visual impact adjustment coefficient, used to control the degree of attenuation of the contribution of visual modalities in the final score.

[0054] The revised current aquaculture status score represents the final aquaculture status assessment result after feedback closed-loop adjustment. The larger the value, the higher the overall abnormal risk of the aquaculture area after considering the consistency of historical feedback and the dynamic correction of visual modality, and the higher the corresponding intervention priority.

[0055] It should be noted that the preset visual impact adjustment coefficient is used to control the attenuation ratio of the visual modality inhibition factor in the final current aquaculture status score correction process. The setting method is based on the mapping relationship between the statistical results of water turbidity distribution and the visual recognition error rate. The visual impact adjustment coefficient is obtained by performing regression analysis on the visual recognition deviation and fusion score offset under different turbidity levels.

[0056] Finally, it should be noted that in this paper, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0057] Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0058] In this document, the singular forms “a,” “an,” and “the” may also include the plural forms unless the context clearly indicates otherwise. It should also be understood that terms such as “comprising / including” or “having” specify the presence of the stated features, integrals, steps, operations, components, parts, or combinations thereof, but do not preclude the possibility of the presence or addition of one or more other features, integrals, steps, operations, components, parts, or combinations thereof. Meanwhile, the term “and / or” as used in this specification includes any and all combinations of the associated listed items.

[0059] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

[0060] The above description of the disclosed embodiments will enable those skilled in the art to make or use various modifications to these embodiments. It will be readily apparent to those skilled in the art that the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multimodal data-driven visualization management method for aquaculture, characterized in that: Includes the following steps: Step S1: Retrieve the instantaneous turbidity sequence of the aquaculture area, calculate the visual modality suppression factor through the instantaneous turbidity sequence, and extract the original confidence level of visual recognition and the visual recognition anomaly category label from the underwater camera to evaluate the original features of the visual modality. Step S2: After inversely suppressing the original visual modality features according to the visual modality suppression factor, suppressed visual assessment features are generated. Dissolved oxygen concentration readings in the aquaculture area are detected. The visual oxygen coupling index is generated by combining the dissolved oxygen concentration readings and the suppressed visual assessment features. Step S3: Set the fish gathering threshold, collect the coordinates of individual targets in the underwater camera, perform clustering and calibration of the individual target coordinates based on the fish gathering threshold, analyze the differential characteristics of fish behavior based on the clustering and calibration results, and evaluate the current aquaculture status score of the aquaculture area in combination with the apparent oxygen coupling index. Step S4: Retrieve the historical aquaculture status score of the aquaculture area, combine it with the current aquaculture status score to generate an aquaculture feedback difference, determine whether to adjust the visual modality suppression factor based on the aquaculture feedback difference, and generate a corrected current aquaculture status score.

2. The multimodal data-driven visualization management method for aquaculture according to claim 1, characterized in that: In step S1, the online turbidity sensor collects the turbidity values ​​of the water in the aquaculture area at a fixed sampling period, and the turbidity values ​​are arranged in time sequence to obtain the instantaneous turbidity sequence of the water body. The instantaneous turbidity sequence of the water body is standardized to obtain the standardized instantaneous turbidity sequence of the water body; The arithmetic mean of the standardized instantaneous turbidity sequence of the water body is taken as the baseline component of turbidity. Take the absolute value of the difference between the turbidity value and the turbidity baseline component in the standardized instantaneous turbidity sequence of the water body, sum the absolute values ​​of each difference and divide by the number of turbidity values ​​to obtain the turbidity disturbance component. By substituting the turbidity baseline component and the turbidity perturbation component into the exponential saturation mapping formula, the visual modality suppression factor is calculated.

3. The multimodal data-driven visualization management method for aquaculture according to claim 1, characterized in that: In step S1, the pre-trained visual recognition model outputs two inference results for each frame of real-time image captured by the underwater camera: the original confidence level of visual recognition and the visual recognition anomaly category label. The original confidence level of visual recognition is the value of the largest component in the classification probability vector output by the pre-trained visual recognition model for a real-time image frame. The visual recognition anomaly category label is the discrete classification result output by the pre-trained visual recognition model for the behavior of fish in real-time image frames; By mapping incremental values ​​to the visual anomaly category labels, we obtain standardized visual anomaly category labels.

4. The multimodal data-driven visualization management method for aquaculture according to claim 3, characterized in that: In step S2, the suppressed visual evaluation features are calculated from the original visual modality features and the visual modality suppression factor. The calculation formula is as follows: Where F represents the original features of the visual modality. is the visual modality suppression factor, and S is the suppressed visual evaluation feature; The dissolved oxygen concentration in the aquaculture area is detected in real time using an online dissolved oxygen sensor, and the dissolved oxygen concentration reading is output. The dissolved oxygen concentration readings were standardized to obtain the standardized results corresponding to the dissolved oxygen concentration readings. The difference between 1 and the standardized dissolved oxygen concentration reading is taken as the dissolved oxygen deficit.

5. The multimodal data-driven visualization management method for aquaculture according to claim 4, characterized in that: In step S2, the visual-oxygen coupling index is calculated based on the suppressed visual assessment features and dissolved oxygen deficit: Where S represents the suppressed visual assessment feature and D represents the dissolved oxygen deficit. The oxygen coupling index is the visual-oxygen coupling index.

6. The multimodal data-driven visualization management method for aquaculture according to claim 1, characterized in that: In step S3, a fish aggregation threshold is set. The fish aggregation threshold is a preset spatial distance parameter, which is derived from the average nearest neighbor distance statistics of fish swimming normally in historical aquaculture scenarios. The underwater camera outputs individual position coordinates within a continuous sampling period. An individual target refers to the basic analytical unit formed by identifying and separating a single fish in the aquaculture water through target detection and tracking algorithms in the continuous video frames captured by the underwater camera. The coordinates of an individual target's position are horizontal plane coordinates and vertical depth coordinates. The spatial Euclidean distance between any two individual targets is calculated using these coordinates.

7. The multimodal data-driven visualization management method for aquaculture according to claim 6, characterized in that: In step S3, when the spatial Euclidean distance between any two target individuals is less than or equal to the fish swarm aggregation threshold, it is determined that the two target individuals belong to the same cluster set, thereby forming a fish swarm cluster. For each fish swarm cluster within a time window, calculate the average of the target position coordinates of all individuals within that cluster to obtain the geometric center coordinates of the fish swarm cluster. The difference between the geometric center coordinates between adjacent time windows is calculated to obtain the cluster center offset. The offsets of all clusters are weighted and summed to obtain the differential features of fish swarm behavior. The fish behavior difference features are linearly fused with the apparent oxygen coupling index to obtain the current aquaculture status score.

8. The multimodal data-driven visualization management method for aquaculture according to claim 7, characterized in that: In step S4, retrieve the historical aquaculture status scores generated within the continuous historical monitoring period for the aquaculture area; The historical aquaculture status score is derived from the statistical mean obtained by continuously calculating the multimodal fusion results of the aquaculture area within a preset time sliding window. Subtracting the historical breeding status score from the current breeding status score yields the breeding feedback difference. When the absolute value of the aquaculture feedback difference is greater than or equal to the preset feedback threshold, the visual modality inhibition factor correction mechanism is triggered. Conversely, the visual modality suppression factor correction mechanism will not be triggered.

9. A multimodal data-driven visualization management method for aquaculture according to claim 8, characterized in that: In step S4, when the visual modality suppression factor correction mechanism is triggered, a callback correction is performed on the visual modality suppression factor to obtain the callback-corrected visual modality suppression factor. The current aquaculture status score is recalculated based on the visual modality suppression factor after the callback correction, resulting in the corrected current aquaculture status score: ； in, This is the revised score for the current aquaculture status. This is the visual modality suppression factor after callback correction. Rate the current breeding status. This is the preset visual impact adjustment coefficient.

Images