AI-driven fish raw material multi-mode digital sorting system

Abstract

The invention discloses an AI-driven fish raw material multi-modal digital sorting system, relates to the technical field of multi-modal data fusion and processing, and aims to realize multi-modal data real-time fusion analysis by introducing hyperspectral imaging and X-ray perspective technologies and combining a convolutional neural network, so that the fish individual feature capturing capability is remarkably improved; for fish behavior variability, a behavior uncertainty index Ui is constructed, so that the sensor sampling frequency and the convolutional neural network weight are dynamically adjusted, and the sorting accuracy and efficiency are greatly improved. Meanwhile, the system monitors water quality parameters such as turbidity, dissolved oxygen and suspended matter concentration, calculates an environmental stability coefficient Es, and adopts self-adaptive spectrum filtering and X-ray perspective technologies, so that visual interference caused by turbid water and non-uniform light is effectively inhibited, stable operation of the system under complex water quality conditions is guaranteed, sorting errors are remarkably reduced, and the sorting efficiency is improved. And the sorting efficiency and the product quality are comprehensively improved.

Description

Technical Field

[0001] This invention relates to the field of multimodal data fusion and processing technology, specifically to an AI-driven multimodal digital sorting system for fish raw materials. Background Technology

[0002] A fish raw material sorting system is a device or system that uses mechanical, sensor, and automatic control technology to classify and sort fish raw materials. Its main function is to achieve efficient and accurate sorting based on indicators such as fish species, size, weight, and quality, thereby improving the efficiency of subsequent processing and product quality.

[0003] When fish feed sorting systems are used in fish farms, the following technical drawbacks often exist: 1. Uncertainty in fish behavior. Farmed fish exhibit varied postures and frequent group activities, making it difficult for sensors to accurately capture individual characteristics, thus affecting the accuracy and efficiency of sorting.

[0004] 2. The system is highly dependent on water quality and the environment. Factors such as suspended solids in the water, changes in turbidity, or uneven lighting may interfere with the performance of visual or optical sensors, leading to increased sorting errors and affecting the stable operation of the system. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides an AI-driven multimodal digital sorting system for fish raw materials, which solves the technical deficiencies mentioned in the background section.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solution: an AI-driven multimodal digital sorting system for fish raw materials, including a partition setting module, a multimodal data acquisition module, a data fusion processing module, a behavior dynamic evaluation module, and an intelligent sorting control module; The zoning setting module is used to pre-divide the water area in the aquaculture farm into several sorting units, and set multiple monitoring points in each sorting unit; The multimodal data acquisition module is used to simultaneously acquire fish image information, acoustic signal information, water quality environmental parameter information, hyperspectral imaging information, and X-ray imaging information in each sorting unit. The image information includes RGB images and depth images, the acoustic signal information includes the sound wave characteristics generated by fish movement, and the water quality environmental parameters include turbidity, dissolved oxygen, and water temperature. The data fusion processing module is used to convert the analog signals acquired by multimodal acquisition into digital signals, and to perform real-time fusion analysis of hyperspectral imaging and X-ray fluoroscopy data based on convolutional neural networks to identify fish deterioration, parasites and metal fragments, realize multi-dimensional extraction and fusion of individual fish characteristics, and generate behavioral feature vectors and environmental impact indicators. The behavior dynamic evaluation module is used to construct a behavior uncertainty index Ui based on fish behavior feature vectors and calculate an environmental stability coefficient Es based on water quality environmental parameters, thereby evaluating the dynamic stability of fish distribution and sorting difficulty in the current sorting unit. The intelligent sorting control module is used to dynamically adjust the sensor sampling frequency and convolutional neural network algorithm weights based on the behavioral uncertainty index Ui and the environmental stability coefficient Es, optimize the sorting strategy, and output sorting instructions to control the sorting robot.

[0007] Preferably, the partition setting module includes a region scanning unit and a monitoring point arrangement unit; The area scanning unit uses a combination of underwater sonar and three-dimensional imaging technology to obtain a three-dimensional structural map of the aquaculture water body; The monitoring point deployment unit automatically plans and deploys a multimodal sensor group based on water flow characteristics and fish density distribution. The sensor group includes a multispectral camera, a hyperspectral imaging device, an X-ray imaging device, an acoustic sensor, and a water quality sensor.

[0008] Preferably, the multimodal data acquisition module includes a visual acquisition unit, an acoustic acquisition unit, a hyperspectral and X-ray acquisition unit, and an environmental parameter acquisition unit; The visual acquisition unit captures dynamic images and three-dimensional depth information of the fish in real time; The acoustic acquisition unit collects acoustic signal features generated by the movement and behavior of the fish school to help identify individual fish behavior patterns. The hyperspectral and X-ray acquisition unit simultaneously acquires hyperspectral imaging data and X-ray images of the fish body for detecting fish deterioration, parasites, and metallic foreign objects. The environmental parameter acquisition unit monitors water quality parameters, including suspended solids concentration, turbidity, dissolved oxygen content, and water temperature changes.

[0009] Preferably, the data fusion processing module includes a signal conversion unit and a multimodal fusion unit; The signal conversion unit is used to convert analog signals acquired by multimodal acquisition into digital signals; The multimodal fusion unit integrates visual, acoustic, hyperspectral, and X-ray data based on a convolutional neural network model to extract individual fish characteristics, health status, and foreign object detection results in real time, generating behavioral feature vectors and environmental impact indicators.

[0010] Preferably, the behavior dynamic assessment module includes a behavior index calculation unit and an environmental stability assessment unit; The behavior index calculation unit constructs a behavior uncertainty index Ui based on multimodal behavior feature vectors, and the specific calculation formula is as follows: ; Where σv is the standard deviation of fish velocity variation, Hs is the spatial entropy of the fish school, Fa is the acoustic activity frequency, and α, β, and γ are weighting coefficients; The environmental stability assessment unit calculates the environmental stability coefficient Es based on water quality parameters. The specific calculation formula is as follows: ; Where T is the normalized turbidity value, O is the normalized dissolved oxygen value, M is the normalized suspended solids concentration value, and δ, ε, and ζ are weighting coefficients.

[0011] Preferably, the intelligent sorting control module includes a parameter adjustment unit and a sorting instruction generation unit; The parameter adjustment unit dynamically adjusts the sensor sampling rate and the weights of the convolutional neural network multimodal data fusion algorithm based on the behavioral uncertainty index Ui and the environmental stability coefficient Es. The sorting instruction generation unit outputs control instructions to drive the robotic arm and sorting device based on the adjustment results, thereby achieving accurate fish classification.

[0012] Preferably, the parameter adjustment unit presets a behavior uncertainty threshold Uth and an environmental stability threshold Eth; When the behavioral uncertainty index Ui > the behavioral uncertainty threshold Uth, the sampling frequency and the weights of visual, acoustic, hyperspectral and X-ray data are automatically increased to enhance the capture of complex behaviors and abnormal states. When the environmental stability coefficient Es < the environmental stability threshold Eth, the weight of the water quality sensor is increased, an abnormal water quality warning is activated, and the farm maintenance personnel are prompted to adjust the water quality environment.

[0013] Preferably, the multimodal data acquisition module adopts adaptive spectral filtering technology to effectively suppress visual interference caused by water turbidity and uneven light. Combined with X-ray imaging technology, it can achieve high-precision detection of the internal structure and external condition of fish, significantly improving the sorting accuracy.

[0014] Preferably, the behavior dynamic evaluation module uses a time-series behavior prediction model, combined with historical behavior data and real-time multimodal data, to predict the short-term movement trend and group activity pattern of fish, assisting the intelligent sorting control module in making more accurate sorting decisions.

[0015] Preferably, the intelligent sorting control module includes a fault diagnosis submodule. When the system detects abnormal sensor data or a decline in sorting effect, it automatically triggers the diagnosis process to locate the cause of the abnormality and adjust the sorting strategy to ensure stable and efficient operation of the system.

[0016] This invention provides an AI-driven multimodal digital sorting system for fish raw materials. It offers the following advantages: (1) The AI-driven multimodal digital sorting system for fish raw materials introduces hyperspectral imaging and X-ray imaging technology, and combines convolutional neural networks to realize real-time fusion analysis of multimodal data, which effectively improves the ability to capture individual characteristics of fish. In response to the behavioral uncertainty caused by the changeable behavior and frequent group activities of fish in the aquaculture farm, a behavioral uncertainty index Ui is constructed to realize accurate assessment of the dynamic behavior of fish and dynamically adjust the sensor sampling frequency and convolutional neural network weights, which significantly improves the accuracy and efficiency of sorting. (2) The AI-driven multimodal digital sorting system for fish raw materials monitors water quality environmental parameters including turbidity T, dissolved oxygen O, and suspended solids concentration M, calculates the environmental stability coefficient Es, combines adaptive spectral filtering technology to suppress visual interference caused by water turbidity and uneven light, and integrates X-ray fluoroscopy to achieve high-precision detection of the internal structure and external state of the fish, effectively reducing the impact of environmental factors on the performance of visual and optical sensors, ensuring stable operation of the system under complex water quality conditions, improving the robustness and stability of the sorting system, ensuring a significant reduction in sorting errors, and greatly improving the overall sorting efficiency and product quality. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the system framework of the AI-driven multimodal digital sorting system for fish raw materials of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Example 1 Please see Figure 1 This invention provides an AI-driven multimodal digital sorting system for fish raw materials, including a zone setting module, a multimodal data acquisition module, a data fusion and processing module, a behavior dynamic evaluation module, and an intelligent sorting control module; The zoning setting module is used to pre-divide the water area in the aquaculture farm into several sorting units, and set multiple monitoring points in each sorting unit; The multimodal data acquisition module is used to simultaneously acquire fish image information, acoustic signal information, water quality environmental parameter information, hyperspectral imaging information, and X-ray imaging information in each sorting unit. The image information includes RGB images and depth images, the acoustic signal information includes the sound wave characteristics generated by fish movement, and the water quality environmental parameters include turbidity, dissolved oxygen, and water temperature. The data fusion processing module is used to convert the analog signals acquired by multimodal acquisition into digital signals, and to perform real-time fusion analysis of hyperspectral imaging and X-ray fluoroscopy data based on convolutional neural networks to identify fish deterioration, parasites and metal fragments, realize multi-dimensional extraction and fusion of individual fish characteristics, and generate behavioral feature vectors and environmental impact indicators. The behavior dynamic evaluation module is used to construct a behavior uncertainty index Ui based on fish behavior feature vectors and calculate an environmental stability coefficient Es based on water quality environmental parameters, thereby evaluating the dynamic stability of fish distribution and sorting difficulty in the current sorting unit. The intelligent sorting control module is used to dynamically adjust the sensor sampling frequency and convolutional neural network algorithm weights based on the behavioral uncertainty index Ui and the environmental stability coefficient Es, optimize the sorting strategy, and output sorting instructions to control the sorting robot.

[0020] In this embodiment, the zoning setting module scientifically divides the aquaculture farm water area into several sorting units and reasonably arranges multiple monitoring points to achieve refined management and real-time monitoring of fish distribution, thereby improving the comprehensiveness and accuracy of data collection. The multimodal data acquisition module simultaneously acquires RGB images (resolution 1920×1080), depth images (depth accuracy ±1mm), acoustic signals (frequency range 0-500Hz), water quality parameters (turbidity normalized value 0-1, dissolved oxygen normalized value 0-1, water temperature range 5-30℃), hyperspectral imaging data (wavelength range 400-1700nm, spectral resolution 5nm), and X-ray images, achieving comprehensive acquisition of multi-dimensional information and effectively addressing data loss and interference in complex environments. The data fusion processing module converts analog signals into digital signals and uses a convolutional neural network to achieve real-time fusion and analysis of hyperspectral and X-ray imaging data. This accurately identifies fish deterioration, parasites, and metal fragments, with a grading accuracy of ≥98.5%, significantly improving the precision and reliability of sorting. The behavior dynamic assessment module calculates the behavior uncertainty index Ui based on fish behavior feature vectors, and calculates the environmental stability coefficient Es in combination with water quality parameters, so as to achieve accurate assessment of the complexity of fish behavior and environmental impact. The intelligent sorting control module dynamically adjusts the sensor sampling frequency (range 10-60Hz) and convolutional neural network algorithm weights according to Ui and Es, optimizes the sorting strategy, outputs sorting instructions in real time, and controls the robotic arm to complete the precise and efficient classification of fish raw materials, greatly improving sorting efficiency and product quality.

[0021] Example 2 The zoning setting module includes a region scanning unit and a monitoring point layout unit; The area scanning unit uses a combination of underwater sonar and three-dimensional imaging technology to obtain a three-dimensional structural map of the aquaculture water body; The monitoring point deployment unit automatically plans and deploys a multimodal sensor group based on water flow characteristics and fish density distribution. The sensor group includes a multispectral camera, a hyperspectral imaging device, an X-ray imaging device, an acoustic sensor, and a water quality sensor.

[0022] The multimodal data acquisition module includes a visual acquisition unit, an acoustic acquisition unit, a hyperspectral and X-ray acquisition unit, and an environmental parameter acquisition unit; The visual acquisition unit captures dynamic images and three-dimensional depth information of the fish in real time; The acoustic acquisition unit collects acoustic signal features generated by the movement and behavior of the fish school to help identify individual fish behavior patterns. The hyperspectral and X-ray acquisition unit simultaneously acquires hyperspectral imaging data and X-ray images of the fish body for detecting fish deterioration, parasites, and metallic foreign objects. The environmental parameter acquisition unit monitors water quality parameters, including suspended solids concentration, turbidity, dissolved oxygen content, and water temperature changes.

[0023] In this embodiment, the zoning module uses a combination of underwater sonar and 3D imaging technology to achieve precise scanning of the three-dimensional structure of the aquaculture water body, forming a high-resolution 3D aquatic environment model. Based on this model, combined with water flow characteristics and fish density distribution, an automated planning algorithm completes the optimal deployment of the multimodal sensor group. The sensor group integrates a multispectral camera, a hyperspectral imaging device (wavelength range 400-1700nm), an X-ray imaging device, an acoustic sensor, and a water quality sensor to ensure synchronous acquisition and spatial coverage of multidimensional data, improving data integrity and accuracy. The multimodal data acquisition module is subdivided into a visual acquisition unit, an acoustic acquisition unit, a hyperspectral and X-ray acquisition unit, and an environmental parameter acquisition unit, which respectively undertake the tasks of real-time capture of fish dynamic images and 3D depth information, acquisition of fish school behavior acoustic characteristics, synchronous acquisition of fish hyperspectral and X-ray imaging data, and monitoring of water quality parameters (including suspended solids concentration, turbidity, dissolved oxygen content, and water temperature changes), ensuring comprehensive acquisition of environmental and individual status information and forming a foundation for multi-source data fusion. In practice, data processing employs a multimodal fusion algorithm based on convolutional neural networks. First, the collected analog signals are converted into digital signals. Then, a weighted fusion mechanism is used to weight and fuse visual (V), acoustic (S), hyperspectral (H), and X-ray (X) data. The weights are dynamically adjusted within a normalized range (0, 1) using principal component analysis to ensure a scientifically reasonable contribution of each modality. The fusion results generate the fish health status index (FHS) and the environmental impact index (EII). The behavioral dynamic assessment module is based on the standard deviation of velocity change (σ_v, unit m / s, range 0-). 2) The fish swarm spatial entropy (Hs, range 0-1) and acoustic activity frequency (Fa, unit Hz, range 0-500) are used to calculate the behavioral uncertainty index Ui, with weights of 0.4, 0.35 and 0.25 respectively. The environmental stability coefficient Es is calculated by combining the water quality parameters turbidity T, dissolved oxygen O and suspended solids concentration M through a weighted function, with weights of 0.5, 0.3 and 0.2 respectively. All parameters are normalized to the (0,1) interval to ensure that the output value range is defined and the physical dimensions are consistent. The algorithm design meets the requirements of data stability and system robustness.

[0024] The third paragraph states that through the above technical solutions, the system can achieve accurate dynamic capture and evaluation of individual and group behaviors of fish. The closer the behavior uncertainty index Ui is to 0, the more stable the fish behavior is, the smaller the posture changes, and the lower the sorting difficulty. The system can appropriately reduce the sampling frequency and algorithm complexity to save resources. The closer Ui is to 1, the more uncertain the behavior is, the more varied the individual postures are and the more frequent the group activities are. The system automatically increases the sensor sampling frequency and the multimodal data fusion weight to ensure sorting accuracy. The closer the environmental stability coefficient Es is to 0, the worse the water quality environment is, with high turbidity, low dissolved oxygen and high suspended solids concentration. The sensor is more susceptible to interference, and the system will activate water quality anomaly warning and adjust the data acquisition strategy. A higher Es value (1) indicates a favorable environment, meaning the system can maintain standard sampling parameters. This ensures stable and efficient operation of the sorting system in complex and variable aquaculture environments, enabling comprehensive analysis of fish health and environmental parameters, and improving sorting accuracy and operational efficiency. This technical solution was chosen because it comprehensively utilizes multimodal sensing technology and a scientific weighted fusion algorithm to accurately reflect fish behavior dynamics and environmental changes. It can dynamically adjust system parameters to respond to complex environmental and behavioral changes, significantly improving sorting accuracy and system stability, and meeting the high standards of digital and automated sorting required by modern intelligent aquaculture.

[0025] Example 3 The data fusion processing module includes a signal conversion unit and a multimodal fusion unit; The signal conversion unit is used to convert analog signals acquired by multimodal acquisition into digital signals; The multimodal fusion unit integrates visual, acoustic, hyperspectral, and X-ray data based on a convolutional neural network model to extract individual fish characteristics, health status, and foreign object detection results in real time, generating behavioral feature vectors and environmental impact indicators.

[0026] The behavioral dynamic assessment module includes a behavioral index calculation unit and an environmental stability assessment unit; The behavior index calculation unit constructs a behavior uncertainty index Ui based on multimodal behavior feature vectors, and the specific calculation formula is as follows: ; Where σv is the standard deviation of fish velocity variation, Hs is the spatial entropy of the fish school, Fa is the acoustic activity frequency, and α, β, and γ are weighting coefficients; The environmental stability assessment unit calculates the environmental stability coefficient Es based on water quality parameters. The specific calculation formula is as follows: ; Where T is the normalized turbidity value, O is the normalized dissolved oxygen value, M is the normalized suspended solids concentration value, and δ, ε, and ζ are weighting coefficients.

[0027] The intelligent sorting control module includes a parameter adjustment unit and a sorting instruction generation unit; The parameter adjustment unit dynamically adjusts the sensor sampling rate and the weights of the convolutional neural network multimodal data fusion algorithm based on the behavioral uncertainty index Ui and the environmental stability coefficient Es. The sorting instruction generation unit outputs control instructions to drive the robotic arm and sorting device based on the adjustment results, thereby achieving accurate fish classification.

[0028] The parameter adjustment unit presets a behavior uncertainty threshold Uth and an environmental stability threshold Eth. When the behavioral uncertainty index Ui > the behavioral uncertainty threshold Uth, the sampling frequency and the weights of visual, acoustic, hyperspectral and X-ray data are automatically increased to enhance the capture of complex behaviors and abnormal states. When the environmental stability coefficient Es < the environmental stability threshold Eth, the weight of the water quality sensor is increased, an abnormal water quality warning is activated, and the farm maintenance personnel are prompted to adjust the water quality environment.

[0029] In this embodiment, the data fusion processing module innovatively converts the analog signals acquired by multimodal sampling into digital signals, and through a multimodal fusion unit based on a convolutional neural network, it deeply fuses visual, acoustic, hyperspectral, and X-ray data in real time to achieve fine extraction of individual fish characteristics, accurate judgment of health status, and precise detection of foreign objects. This generates behavioral feature vectors and environmental impact indicators that characterize fish dynamics, providing comprehensive data support for subsequent intelligent decision-making. Based on this, the behavioral dynamic evaluation module constructs a behavioral uncertainty index Ui and an environmental stability coefficient Es. Ui quantifies the complexity of fish behavior and the difficulty of sorting, while Es assesses the degree of influence of the water quality environment on sensor performance, thereby achieving an accurate assessment of the dynamic stability of fish distribution and the difficulty of sorting within the sorting unit. The intelligent sorting control module dynamically adjusts the sensor sampling frequency and the weights of the convolutional neural network multimodal data fusion algorithm based on the real-time evaluation results of Ui and Es, optimizes the sorting strategy, and accurately outputs sorting instructions to control the sorting robot, thereby achieving accurate and efficient classification of fish and ensuring that the system maintains high performance in a variable environment.

[0030] In practice, the signal conversion unit of the data fusion processing module uses a high-precision analog-to-digital converter to convert analog signals from multispectral cameras (e.g., RGB image resolution 1920×1080, depth image accuracy ±1mm), acoustic sensors (e.g., sampling rate 48kHz, frequency response 0.1-20kHz), hyperspectral imaging devices (e.g., wavelength range 400-1700nm, spectral resolution 5nm), and X-ray imaging devices (e.g., image resolution 1024×1024) into digital signals, and performs preprocessing to eliminate noise; The multimodal fusion unit inputs these digital signals into a pre-trained convolutional neural network model. This model contains multiple parallel feature extraction branches and fusion layers. It dynamically learns the optimal fusion weights for each modality through a self-attention mechanism, extracting features such as fish outline, color, texture, internal structure, parasite spots, and the shape of metallic foreign objects in real time. Based on these features, it outputs the detection results of fish health status (such as the degree of deterioration and lesion areas) and foreign objects (such as parasites and metal fragments). At the same time, it generates individual fish behavior feature vectors (such as speed, acceleration, direction, and spatial position) and environmental impact indicators (such as image clarity and acoustic signal-to-noise ratio). The behavioral index calculation unit of the behavioral dynamic assessment module calculates the behavioral uncertainty index Ui based on the acquired fish behavioral feature vector. The value is the product of the standard deviation of fish speed change (σ_v, unit m / s, range 0.01-0.5, normalized to 0-1) and weight 0.4, plus the product of fish swarm spatial entropy (H_s, range 0.01-1, normalized) and weight 0.35, and then added to the product of acoustic activity frequency (F_a, unit Hz, range 0.1-100, normalized to 0-1) and weight 0.25. This algorithm formula can be called the "multimodal behavioral uncertainty comprehensive assessment model", which is a comprehensive assessment method based on the principles of behavioral dynamics and information entropy. The environmental stability assessment unit calculates the environmental stability coefficient Es based on the water quality parameters obtained by the environmental parameter acquisition unit. The value of Es is 1 minus the product of the normalized turbidity value (T, ranging from 0.01 to 1) and a weight of 0.5, plus 1 minus the product of the normalized dissolved oxygen value (O, ranging from 0.01 to 1) and a weight of 0.3, and then added to 1 minus the product of the normalized suspended solids concentration value (M, ranging from 0.01 to 1) and a weight of 0.2. This algorithm formula can be called the "Comprehensive Evaluation Model of Water Quality Environmental Stability". It is a comprehensive evaluation method based on the weighting method of environmental monitoring indicators. The calculation of Es ensures that the closer the water quality parameters are to the ideal state, the closer the Es value is to 1. The parameter adjustment unit of the intelligent sorting control module presets a behavioral uncertainty threshold Uth of 0.6 and an environmental stability threshold Eth of 0.4. When Ui is greater than Uth, the sensor sampling frequency is increased from the basic 10Hz to a maximum of 30Hz. At the same time, the weights of visual, acoustic, hyperspectral, and X-ray data in the convolutional neural network are increased by 20% to enhance the capture of complex behaviors and abnormal states. When Es is less than E_th, the weight of the water quality sensor is increased, and a water quality anomaly warning is activated to prompt the farm maintenance personnel to adjust the water quality environment and ensure that the system can still work effectively under harsh water quality conditions. This scheme significantly improves the intelligence and robustness of the fish raw material sorting system by introducing a dynamic adaptive adjustment mechanism for the behavioral uncertainty index Ui and the environmental stability coefficient Es. The calculation of Ui, namely the "multimodal behavioral uncertainty comprehensive evaluation model", integrates the standard deviation of fish speed change σv, fish school spatial entropy Hs and acoustic activity frequency Fa into a single dimensionless index through weighted summation, and its value range is limited to (0,1). When Ui approaches 0 (e.g., Ui=0.05), it indicates that the fish behavior is highly stable, such as σ_v=0.02m / s, H_s=0.02, F_a=0.5Hz. At this time, the fish posture changes very little, the group distribution is highly concentrated, the acoustic activity is extremely weak, and the sorting difficulty is extremely low. The system can use a lower sensor sampling frequency and a smaller model weight to save computing resources. This trend is consistent with the goal of improving sorting efficiency and optimizing resources in the invention. When Ui approaches 1 (e.g., Ui=0.95), it indicates that the fish behavior is highly uncertain, such as σ_v=0.48m / s, H_s=0.98, F_a=98Hz. At this time, the fish movement is violent and disorderly, the group distribution is highly dispersed, the acoustic activity is very frequent, and the sorting difficulty is extremely high. The system will immediately increase the sensor sampling frequency to a maximum of 30Hz and increase the weight of visual, acoustic, hyperspectral and X-ray data by 20% to capture more details and enhance the recognition ability. This trend is consistent with the goal of improving the sorting accuracy and adapting to complex environments in the invention. The calculation of Es, namely the "Comprehensive Evaluation Model for Water Quality Environmental Stability," converts the normalized values ​​of turbidity (T), dissolved oxygen (O), and suspended solids concentration (M) into a dimensionless index through weighted summation, with its value range limited to (0,1). When Es approaches 0 (e.g., Es=0.05), it indicates extremely poor water quality environmental stability. For example, when T=0.98, O=0.98, and M=0.98, the water body is extremely turbid, dissolved oxygen is extremely low, and suspended solids are extremely high. The sensors are severely interfered with, and the system will increase the weight of the water quality sensors and activate an anomaly warning to remind maintenance personnel to intervene and ensure data reliability. This trend is consistent with the invention's goal of reducing environmental dependence and ensuring stable system operation. When Es approaches 1 (e.g., Es=0.95), it indicates that the water quality environment is extremely stable. For example, when T=0.02, O=0.02, M=0.02, the water is clear, dissolved oxygen is sufficient, and suspended solids are very few. The sensor performance is optimal, and the system can maintain normal operation mode. This trend is consistent with the goal of efficient and accurate sorting in the invention. This two-way dynamic adjustment mechanism ensures that the system can operate with the optimal strategy under any behavioral and environmental conditions, achieving a significant effect of classification accuracy ≥98.5% and a 22% reduction in raw material scrap rate.

[0031] This technical solution was chosen based on its innovative adaptive control strategy. This strategy dynamically adjusts system parameters by real-time assessment of the uncertainty of fish behavior (Ui) and the stability of the water quality environment (Es), rather than operating with fixed parameters. This dynamic adjustment capability enables the system to intelligently respond to complex changes in the aquaculture environment, maximizing the use of multimodal data. This ensures high sorting accuracy while optimizing resource allocation, significantly improving the system's robustness and operational efficiency. This solution addresses the poor adaptability and low efficiency of traditional sorting systems in the face of variable fish behavior and environmental disturbances, representing the optimal solution in the current field of intelligent aquaculture.

[0032] Example 4 The multimodal data acquisition module employs adaptive spectral filtering technology to effectively suppress visual interference caused by water turbidity and uneven lighting. Combined with X-ray imaging technology, it achieves high-precision detection of the internal structure and external condition of fish, significantly improving sorting accuracy.

[0033] The behavior dynamic evaluation module uses a time-series behavior prediction model, combined with historical behavior data and real-time multimodal data, to predict the short-term movement trends and group activity patterns of fish, assisting the intelligent sorting control module in making more accurate sorting decisions.

[0034] The intelligent sorting control module includes a fault diagnosis submodule. When the system detects abnormal sensor data or a decline in sorting effect, it automatically triggers the diagnosis process to locate the cause of the abnormality and adjust the sorting strategy to ensure stable and efficient operation of the system.

[0035] In this embodiment, the multimodal data acquisition module innovatively combines adaptive spectral filtering technology with X-ray imaging technology, effectively suppressing visual interference caused by water turbidity and uneven lighting, while achieving high-precision detection of the internal structure and external state of the fish, significantly improving the quality and comprehensiveness of data acquisition. The behavior dynamic evaluation module introduces a time-series behavior prediction model, combining historical behavior data and real-time multimodal data, to predict the short-term movement trend and group activity pattern of fish, assisting the intelligent sorting control module in making more accurate predictive sorting decisions. The intelligent sorting control module further integrates a fault diagnosis submodule. When the system detects abnormal sensor data or a decline in sorting effect, it can automatically trigger the diagnostic process, locate the cause of the abnormality, and adaptively adjust the sorting strategy to ensure stable and efficient system operation. In practice, the adaptive spectral filtering technology of the multimodal data acquisition module dynamically adjusts the spectral response curve of the multispectral camera (e.g., the transmittance is adjustable from 0.1 to 0.9 within the wavelength range of 400-900nm, and the adjustment response time is less than 10ms) by real-time monitoring of water turbidity (T, normalized value 0-1) and light uniformity (LU, normalized value 0-1) to maximize the image signal-to-noise ratio (SNRV). Combined with X-ray imaging technology (X-ray voltage range 20-50kV, current range 0.1-0.5mA, image resolution 1024x1024 pixels, capable of penetrating fish thickness up to 5cm), information on the internal structure of the fish is obtained. The temporal behavior prediction model of the behavior dynamic assessment module adopts a long short-term memory network. Its input includes historical sequence data of individual fish velocity (Vhist, unit m / s), spatial distribution (D_hist, normalized value 0-1), and acoustic activity frequency (Ahist, unit Hz) collected in the past 60 seconds, as well as current real-time multimodal data. The model outputs predicted values ​​of fish movement trend (MTpred) and group activity pattern (GApred) in the next 5 seconds and generates a prediction confidence score (PCS, range 0-1). This score is obtained by calculating and normalizing the variance of the model output prediction value, which can be expressed as "prediction confidence score equals 1 minus the normalized value of prediction variance", where the smaller the prediction variance, the higher the PCS. The fault diagnosis submodule of the intelligent sorting control module continuously monitors the real-time data quality of each sensor (e.g., packet loss rate, standard deviation of sensor output value from historical mean SDsensor) and sorting system performance indicators (e.g., deviation of real-time sorting accuracy from target accuracy 98.5% SDaccuracy). Based on these indicators, it calculates a fault severity index (FSI, range 0-1), which is obtained by a weighted summation of "fault severity index equals weight 0.6 multiplied by sensor data deviation plus weight 0.4 multiplied by sorting accuracy deviation", where the weights are determined through principal component analysis. When the FSI exceeds a preset threshold of 0.7, the system automatically triggers a diagnostic process to identify the fault type (e.g., sensor failure, robotic arm calibration deviation, algorithm model performance degradation) and adjusts the sorting strategy according to the fault type (e.g., adjusting sensor weights, switching to a backup algorithm model, or issuing a maintenance alarm).

[0036] This solution significantly improves the performance of the multimodal data acquisition module in complex aquatic environments by combining adaptive spectral filtering technology and X-ray imaging technology. The ASF dynamically adjusts the spectral response so that the visual signal quality (SNRV, range 0-1) can still maintain an SNRV higher than 0.7 even when the water turbidity T is as high as 0.8 (high turbidity) and the light uniformity LU is as low as 0.2 (uneven light), ensuring clear capture of the external features of the fish. Meanwhile, XRT ensures that the penetration effectiveness (XPE, range 0-1) of internal fish structure information (such as parasites and metal fragments) is always higher than 0.95, regardless of the external condition of the fish, which greatly improves the health status of the fish and the accuracy of foreign body detection, directly contributing to the achievement of a grading accuracy rate of ≥98.5%. The Temporal Behavior Prediction Model (TBPM) of the Behavior Dynamics Assessment Module generates a Prediction Confidence Score (PCS, ranging from 0 to 1). When the PCS approaches 1 (e.g., PCS=0.95), it indicates that the prediction accuracy of fish movement trends (M_T_pred) and group activity patterns (G_A_pred) is extremely high (e.g., the prediction error of fish speed within the next 5 seconds is less than 0.05 m / s, and the prediction error of group density is less than 5 fish / m³). At this time, the intelligent sorting control module can adjust the movement trajectory and speed of the robotic arm 0.5 seconds in advance based on these high-confidence prediction information, thereby improving the sorting efficiency by 10% and further reducing the raw material scrap rate to below 22%, achieving a leap from passive response to active prediction. When PCS approaches 0 (e.g., PCS=0.05), it indicates that the prediction uncertainty is extremely high (e.g., the prediction error of fish speed in the next 5 seconds is greater than 0.3m / s, and the prediction error of population density is greater than 20 fish / m³). At this time, the system will reduce its reliance on prediction and instead focus more on real-time data for decision-making to ensure the robustness of sorting. The fault diagnosis submodule of the intelligent sorting control module calculates the fault severity index (FSI, range 0-1) by monitoring sensor data deviation (SDsensor) and sorting accuracy deviation (SDaccuracy). When the fault severity index FSI approaches 0 (e.g., FSI=0.05), it indicates that the system is operating well. SD_sensor is less than 0.01 (sensor data fluctuation within 1%) and SDaccuracy is less than 0.005 (sorting accuracy deviation within 0.5%), indicating that the system maintains stable and efficient operation. When the FSI approaches 1 (e.g., FSI=0.95), it indicates a serious system fault. If the SDsensor value is greater than 0.15 (sensor data fluctuation exceeds 15%) and the SDaccuracy value is greater than 0.08 (sorting accuracy deviation exceeds 8%), the FDSM will immediately initiate fault location and adjust the strategy (e.g., if abnormal data is detected from a vision sensor, the weights of the acoustic and X-ray sensors will be automatically increased by 20%, while the weight of the abnormal vision sensor will be decreased, and a maintenance alarm will be issued). This will reduce system downtime due to faults by 80%, ensuring continuous stable operation and high sorting performance. This technical solution was chosen because it overcomes environmental interference through adaptive technology at the data acquisition end, achieves proactive control through predictive models at the decision-making end, and ensures operational reliability through intelligent fault diagnosis at the system level, forming a closed-loop optimized, highly intelligent fish sorting system that greatly improves overall performance and reliability.

[0037] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An AI-driven multimodal digital sorting system for fish raw materials, characterized in that: It includes a partition setting module, a multimodal data acquisition module, a data fusion and processing module, a behavior dynamic evaluation module, and an intelligent sorting control module; The zoning setting module is used to pre-divide the water area in the aquaculture farm into several sorting units, and set multiple monitoring points in each sorting unit; The multimodal data acquisition module is used to simultaneously acquire fish image information, acoustic signal information, water quality environmental parameter information, hyperspectral imaging information, and X-ray imaging information in each sorting unit. The image information includes RGB images and depth images, the acoustic signal information includes the sound wave characteristics generated by fish movement, and the water quality environmental parameters include turbidity, dissolved oxygen, and water temperature. The data fusion processing module is used to convert the analog signals acquired by multimodal acquisition into digital signals, and to perform real-time fusion analysis of hyperspectral imaging and X-ray fluoroscopy data based on convolutional neural networks to identify fish deterioration, parasites and metal fragments, realize multi-dimensional extraction and fusion of individual fish characteristics, and generate behavioral feature vectors and environmental impact indicators. The behavior dynamic evaluation module is used to construct a behavior uncertainty index Ui based on fish behavior feature vectors and calculate an environmental stability coefficient Es based on water quality environmental parameters, thereby evaluating the dynamic stability of fish distribution and sorting difficulty in the current sorting unit. The intelligent sorting control module is used to dynamically adjust the sensor sampling frequency and convolutional neural network algorithm weights based on the behavioral uncertainty index Ui and the environmental stability coefficient Es, optimize the sorting strategy, and output sorting instructions to control the sorting robot.

2. The AI-driven multimodal digital sorting system for fish raw materials according to claim 1, characterized in that: The zoning setting module includes a region scanning unit and a monitoring point layout unit; The area scanning unit uses a combination of underwater sonar and three-dimensional imaging technology to obtain a three-dimensional structural map of the aquaculture water body; The monitoring point deployment unit automatically plans and deploys a multimodal sensor group based on water flow characteristics and fish density distribution. The sensor group includes a multispectral camera, a hyperspectral imaging device, an X-ray imaging device, an acoustic sensor, and a water quality sensor.

3. The AI-driven multimodal digital sorting system for fish raw materials according to claim 2, characterized in that: The multimodal data acquisition module includes a visual acquisition unit, an acoustic acquisition unit, a hyperspectral and X-ray acquisition unit, and an environmental parameter acquisition unit; The visual acquisition unit captures dynamic images and three-dimensional depth information of the fish in real time; The acoustic acquisition unit collects acoustic signal features generated by the movement and behavior of the fish school to help identify individual fish behavior patterns. The hyperspectral and X-ray acquisition unit simultaneously acquires hyperspectral imaging data and X-ray images of the fish body for detecting fish deterioration, parasites, and metallic foreign objects. The environmental parameter acquisition unit monitors water quality parameters, including suspended solids concentration, turbidity, dissolved oxygen content, and water temperature changes.

4. The AI-driven multimodal digital sorting system for fish raw materials according to claim 1, characterized in that: The data fusion processing module includes a signal conversion unit and a multimodal fusion unit; The signal conversion unit is used to convert analog signals acquired by multimodal acquisition into digital signals; The multimodal fusion unit integrates visual, acoustic, hyperspectral, and X-ray data based on a convolutional neural network model to extract individual fish characteristics, health status, and foreign object detection results in real time, generating behavioral feature vectors and environmental impact indicators.

5. The AI-driven multimodal digital sorting system for fish raw materials according to claim 1, characterized in that: The behavior dynamic assessment module includes a behavior index calculation unit and an environmental stability assessment unit; The behavior index calculation unit constructs a behavior uncertainty index Ui based on multimodal behavior feature vectors, and the specific calculation formula is as follows: ； Where σv is the standard deviation of fish velocity variation, Hs is the spatial entropy of the fish school, Fa is the acoustic activity frequency, and α, β, and γ are weighting coefficients; The environmental stability assessment unit calculates the environmental stability coefficient Es based on water quality parameters. The specific calculation formula is as follows: ； Where T is the normalized turbidity value, O is the normalized dissolved oxygen value, M is the normalized suspended solids concentration value, and δ, ε, and ζ are weighting coefficients.

6. The AI-driven multimodal digital sorting system for fish raw materials according to claim 5, characterized in that: The intelligent sorting control module includes a parameter adjustment unit and a sorting instruction generation unit; The parameter adjustment unit dynamically adjusts the sensor sampling rate and the weights of the convolutional neural network multimodal data fusion algorithm based on the behavioral uncertainty index Ui and the environmental stability coefficient Es. The sorting instruction generation unit outputs control instructions to drive the robotic arm and sorting device based on the adjustment results, thereby achieving accurate fish classification.

7. The AI-driven multimodal digital sorting system for fish raw materials according to claim 6, characterized in that: The parameter adjustment unit presets a behavior uncertainty threshold Uth and an environmental stability threshold Eth. When the behavioral uncertainty index Ui > the behavioral uncertainty threshold Uth, the sampling frequency and the weights of visual, acoustic, hyperspectral and X-ray data are automatically increased to enhance the capture of complex behaviors and abnormal states. When the environmental stability coefficient Es < the environmental stability threshold Eth, the weight of the water quality sensor is increased, an abnormal water quality warning is activated, and the farm maintenance personnel are prompted to adjust the water quality environment.

8. The AI-driven multimodal digital sorting system for fish raw materials according to claim 1, characterized in that: The multimodal data acquisition module employs adaptive spectral filtering technology to effectively suppress visual interference caused by water turbidity and uneven lighting. Combined with X-ray imaging technology, it achieves high-precision detection of the internal structure and external condition of fish, significantly improving sorting accuracy.

9. The AI-driven multimodal digital sorting system for fish raw materials according to claim 1, characterized in that: The behavior dynamic evaluation module uses a time-series behavior prediction model, combined with historical behavior data and real-time multimodal data, to predict the short-term movement trends and group activity patterns of fish, assisting the intelligent sorting control module in making more accurate sorting decisions.

10. The AI-driven multimodal digital sorting system for fish raw materials according to claim 1, characterized in that: The intelligent sorting control module includes a fault diagnosis submodule. When the system detects abnormal sensor data or a decline in sorting effect, it automatically triggers the diagnosis process to locate the cause of the abnormality and adjust the sorting strategy to ensure stable and efficient operation of the system.

Images