Intelligent detection and grading system for freshwater fish based on machine vision

Abstract

The present application relates to the freshwater fish culture technical field, especially be a kind of freshwater fish intelligent detection and grading system based on machine vision, including image acquisition unit, image processing unit, image analysis unit, grading decision unit and control and output unit, image acquisition unit acquires multi-view image sequence under controlled illumination environment, after target area is extracted by image processing unit, image analysis unit completes three-dimensional profile reconstruction form parameter calculation activity stress evaluation and body surface defect detection;Grading decision unit realizes double-dimension grading of specification and quality according to multidimensional characteristics, and control and output unit drives actuating mechanism to complete sorting and generate report.The present application realizes the multi-dimension automatic detection and accurate grading of the size, activity, health condition of freshwater fish, realizes the whole process automation of freshwater fish grading, improves grading accuracy and efficiency, reduces labor cost, and is suitable for large-scale freshwater fish culture processing scene.

Description

Technical Field

[0001] This invention relates to the field of freshwater fish farming technology, specifically to a machine vision-based intelligent detection and grading system for freshwater fish. Background Technology

[0002] Freshwater fish farming is a crucial component of the aquaculture economy. Before being marketed, fish products typically require grading based on size and quality to meet diverse market demands and enhance economic benefits. Traditional grading relies heavily on manual labor, with workers judging fish size, vitality, and overall condition by sight and touch. This method is inefficient, labor-intensive, and susceptible to subjective biases, leading to inconsistent grading standards and high long-term operating costs. While some mechanical weight or length sorting devices have emerged in recent years, enabling preliminary automated measurement of size indicators, they still cannot quantify key quality indicators such as fish vitality, stress levels, and surface damage, let alone achieve comprehensive intelligent judgment of both size and quality.

[0003] With the development of machine vision and artificial intelligence technologies, existing research has attempted to apply image analysis to fish detection. For example, using single-view images to estimate fish length or using threshold segmentation to identify obvious lesions on the body surface. However, these methods often have significant limitations. A single view cannot acquire three-dimensional morphological information of the fish, leading to large errors in size measurement and volume estimation. The lack of analysis of fish movement sequences makes it difficult to scientifically assess their vitality and stress response. Traditional image processing methods are not robust enough for detecting minor surface damage, abnormal spots, or parasites, and their accuracy is limited in complex backgrounds. More importantly, most existing research remains at the stage of visual detection and analysis, failing to form an efficient closed loop with automated actuators and failing to build a complete system integrating multi-view imaging, multi-dimensional feature extraction, intelligent decision-making, and precise sorting.

[0004] Therefore, a machine vision-based intelligent detection and grading system for freshwater fish is proposed to address the above problems. Summary of the Invention

[0005] The purpose of this invention is to provide a machine vision-based intelligent detection and grading system for freshwater fish to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A machine vision-based intelligent detection and grading system for freshwater fish includes:

[0008] The image acquisition unit is used to continuously acquire multi-view image sequences of a single freshwater fish under controlled lighting conditions in the target water body.

[0009] The image processing unit is connected in communication with the image acquisition unit. It is used to receive multi-view image sequences and perform image preprocessing, background segmentation and target extraction operations to obtain the target image region of the freshwater fish to be tested.

[0010] The image analysis unit, communicatively connected to the image processing unit, is used to perform multi-dimensional feature analysis on the target image region and generate multi-dimensional feature analysis results, including at least:

[0011] The morphological parameter analysis module is used to reconstruct the three-dimensional contour model of the freshwater fish under test based on multi-view images, and calculate its body length, body height, body width and estimated weight accordingly.

[0012] The motion state assessment module is used to analyze the displacement, attitude angle changes, and activity intensity of the target image region in a continuous image sequence to assess its vitality and stress state.

[0013] The fish appearance detection module is used to identify damaged areas, abnormal spots, and parasite attachment characteristics on the surface of freshwater fish under test based on high-resolution images and a trained convolutional neural network model.

[0014] The hierarchical decision-making unit communicates with the image analysis unit to receive multi-dimensional feature analysis results and make comprehensive decisions based on a preset hierarchical rule base. The hierarchical rule base integrates at least weight-based specification hierarchical rules and vitality and appearance feature-based quality hierarchical rules to generate hierarchical decision results.

[0015] The control and output unit communicates with the hierarchical decision unit and is used to generate control commands based on the hierarchical decision results to drive the actuator to sort the freshwater fish to be tested into containers of the corresponding level and output a hierarchical report.

[0016] As can be seen from the technical solution provided by the present invention above, the beneficial effects of the machine vision-based intelligent detection and grading system for freshwater fish provided by the present invention are:

[0017] Improving the accuracy of freshwater fish detection and grading: The system relies on multi-view image acquisition technology and three-dimensional contour reconstruction algorithm to accurately calculate morphological parameters such as body length, body height, body width and estimated weight; combined with convolutional neural network model to identify surface defects, and with motion state analysis to quantify vitality and stress level, multi-dimensional data supports grading decisions, eliminates subjective errors of manual grading, and significantly improves the accuracy and fairness of grading results.

[0018] The system automates the detection and grading process: from image acquisition, image processing, image analysis, grading decision-making to sorting output, the entire process requires no manual intervention. It can continuously process freshwater fish to be tested in batches, greatly shortening the grading time for a single fish, improving the overall detection and grading efficiency, and adapting to the production needs of large-scale freshwater fish farming and processing.

[0019] Enhance the system's versatility and adaptability: The system can dynamically adjust the light intensity and spectral band according to the different species and surface reflectivity of freshwater fish; the grading rule base supports flexible configuration of specification grading thresholds and quality scoring standards, which can adapt to the grading needs of various freshwater fish without large-scale hardware modifications, thus enhancing the system's applicability.

[0020] Achieve traceability and refined management of graded data: The system can automatically integrate the multi-dimensional feature analysis results, grading decision results, and sorting information of the freshwater fish to be tested, and generate a detailed record of the grading of a single fish and a batch grading report; these data can be used for subsequent quality traceability, aquaculture optimization, and market traceability, thereby improving the refined management level of the freshwater fish industry chain;

[0021] Reduce the overall cost of freshwater fish farming and processing: Automated processes reduce the labor costs of manual grading, and accurate grading results can prevent high-quality freshwater fish from being misjudged and downgraded, thereby increasing the market value of the product; the results of vitality and stress status assessment can guide subsequent transportation and temporary holding stages, reduce the loss rate of freshwater fish in the circulation process, and further reduce overall costs. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the structure of a machine vision-based intelligent detection and grading system for freshwater fish according to the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0024] To better understand the above technical solutions, the following will provide a detailed description of the technical solutions in conjunction with the accompanying drawings and specific embodiments.

[0025] like Figure 1 As shown, this embodiment of the invention provides a machine vision-based intelligent detection and grading system for freshwater fish, comprising:

[0026] The image acquisition unit is used to continuously acquire multi-view image sequences of a single freshwater fish under controlled lighting conditions in the target water body.

[0027] The image processing unit is connected in communication with the image acquisition unit. It is used to receive multi-view image sequences and perform image preprocessing, background segmentation and target extraction operations to obtain the target image region of the freshwater fish to be tested.

[0028] The image analysis unit, communicatively connected to the image processing unit, is used to perform multi-dimensional feature analysis on the target image region and generate multi-dimensional feature analysis results, including at least:

[0029] The morphological parameter analysis module is used to reconstruct the three-dimensional contour model of the freshwater fish under test based on multi-view images, and calculate its body length, body height, body width and estimated weight accordingly.

[0030] The motion state assessment module is used to analyze the displacement, attitude angle changes, and activity intensity of the target image region in a continuous image sequence to assess its vitality and stress state.

[0031] The fish appearance detection module is used to identify damaged areas, abnormal spots, and parasite attachment characteristics on the surface of freshwater fish under test based on high-resolution images and a trained convolutional neural network model.

[0032] The hierarchical decision-making unit communicates with the image analysis unit to receive multi-dimensional feature analysis results and make comprehensive decisions based on a preset hierarchical rule base. The hierarchical rule base integrates at least weight-based specification hierarchical rules and vitality and appearance feature-based quality hierarchical rules to generate hierarchical decision results.

[0033] The control and output unit communicates with the hierarchical decision unit and is used to generate control commands based on the hierarchical decision results to drive the actuator to sort the freshwater fish to be tested into containers of the corresponding level and output a hierarchical report.

[0034] In this embodiment, the image acquisition unit is the core sensing front end of the machine vision-based freshwater fish intelligent detection and grading system. It acquires high-quality multi-view image sequences of the freshwater fish under test by constructing a controlled lighting environment and a multi-view synchronous imaging mechanism, providing an accurate and reliable data source for subsequent image processing and analysis.

[0035] The image acquisition unit is primarily responsible for providing standardized freshwater fish image data to the system. It constructs a uniform and adjustable lighting environment through a closed imaging enclosure, simultaneously acquiring top-down and side-view images of the freshwater fish using industrial cameras arranged from multiple perspectives. A transparent support channel guides the fish through the acquisition area, maintaining a stable posture. Finally, the continuously acquired multi-view image sequence is structured and transmitted to the image processing unit, ensuring the consistency and spatiotemporal correlation of the image data throughout the process. The image acquisition unit includes:

[0036] Enclosed imaging enclosure:

[0037] Lighting environment construction: A diffuse reflection coating is set on the inner wall of the closed imaging box. The coating can make the incident light diffusely reflect uniformly, eliminating the problems of direct light and shadow inside the box; an adjustable brightness multispectral LED light source array is configured inside the box. This array can provide light in different spectral bands to meet the surface imaging needs of different kinds of freshwater fish.

[0038] Dynamic adjustment of illumination parameters: Based on the species and surface reflectance characteristics of the freshwater fish under test, the illumination intensity and spectral band of the multispectral LED light source array are dynamically adjusted. The adjustment formula is as follows: (in, The adjusted light intensity, As a reference light intensity, Light correction factor for freshwater fish species. (Light correction coefficient corresponding to the reflectivity of freshwater fish body surface); By adjusting the parameters, a controlled lighting environment with optimized background contrast is formed in the imaging chamber, improving the distinction between the fish body and the background;

[0039] At least two industrial cameras:

[0040] Industrial camera setup and triggering: At least two industrial cameras are arranged at a fixed angle above and to the side of the imaging box. The installation angle of the cameras is calibrated to fully cover the top and side views of the freshwater fish to be tested. When the freshwater fish to be tested enters the preset acquisition area, the system controls multiple industrial cameras to start shooting synchronously or sequentially based on the trigger signal of the position sensor, ensuring that fish images from different perspectives are acquired at the same time.

[0041] Transparent carrier channel guidance: A transparent carrier channel is set inside the imaging box. The channel material has high light transmittance and will not obstruct the image acquisition. The size of the channel is designed according to the body shape of common freshwater fish, which can guide the freshwater fish to be tested to pass through the image acquisition area in a basically fixed posture, avoiding large changes in the posture of the fish during the acquisition process and ensuring the compatibility of multi-view images.

[0042] Image sequence processing and transmission submodule:

[0043] Multi-view image sequence acquisition: Based on the acquisition of multi-view image pairs at a single time point, the industrial camera is continuously triggered to repeatedly acquire images at a preset time interval to obtain multiple sets of multi-view image pairs of the freshwater fish under test passing through the acquisition area. These image pairs are arranged in chronological order to form a multi-view image sequence under continuous time series.

[0044] Structured processing and transmission: The acquired multi-view image sequence is timestamped to ensure that images from different views at the same time point have a unified time identifier; after timestamping, the image data is encapsulated to form a structured image data stream containing the image data acquisition time and viewpoint information, and then the data stream is transmitted to the image processing unit.

[0045] Furthermore, the controlled lighting environment is constructed based on the theory of diffuse reflection and spectral matching. The diffuse reflection coating can convert the direct light from the LED light source into uniform ambient light, reducing the interference of reflection and shadow on the fish surface. The multispectral LED light source array can match the optimal spectral band and light intensity according to the surface color and reflectivity characteristics of different freshwater fish, so that the fish features are clearly presented in the image, laying the foundation for subsequent target extraction and feature analysis.

[0046] Multi-view synchronous triggering imaging is based on position sensing and timing control theory; the position sensor monitors the position of the freshwater fish under test in the transparent carrier channel in real time, and sends a trigger signal when the fish reaches the preset acquisition area; after receiving the signal, the timing controller synchronously drives multiple industrial cameras to take pictures, ensuring that the images from different perspectives are completely aligned in the time dimension, and ensuring the spatial consistency of subsequent multi-view image fusion.

[0047] The structured image data stream generation is based on data standardization and temporal coding theory. The system adds a unique timestamp and viewpoint code to each acquired image. The timestamp is used to identify the time of image acquisition, and the viewpoint code is used to distinguish between top-view and side-view images. By encapsulating the image data, the scattered image information is integrated into a data stream with a unified format, which facilitates efficient parsing and processing by the image processing unit.

[0048] The workflow of the image acquisition unit is as follows:

[0049] Initialization phase:

[0050] After the image acquisition unit is started, it first completes the self-test of the hardware equipment, including the sealing test of the closed imaging box, the light emission status test of the multispectral LED light source array, the shooting function test of the industrial camera, and the unobstructedness test of the transparent carrier channel, to ensure that all hardware equipment is in normal working condition.

[0051] Load the system's preset freshwater fish species illumination parameter configuration information and camera trigger timing parameters, establish a communication connection with the image processing unit, and prepare to receive the detection task instructions for the freshwater fish to be tested;

[0052] Lighting parameter configuration stage:

[0053] Receive information on the species of freshwater fish to be tested from the system, and retrieve the corresponding light intensity reference value and spectral band parameters for that species;

[0054] Based on the pre-stored data of the reflectivity of freshwater fish bodies, the illumination correction coefficient is calculated and substituted into the adjustment formula to determine the final operating parameters of the multispectral LED light source array. The light source array is then started and adjusted to the target state to complete the construction of the illumination environment.

[0055] Image acquisition stage:

[0056] The freshwater fish to be tested enters the transparent carrying channel, and the position sensor monitors the position of the fish in real time. When the fish reaches the preset image acquisition area, the sensor sends a trigger signal to the control system.

[0057] After receiving the signal, the control system synchronously triggers industrial cameras placed at different angles to acquire top-down and side-view images at that time point, forming a multi-view image pair at a single time point.

[0058] The camera is repeatedly triggered to capture images at preset time intervals until the freshwater fish under test completely passes through the image acquisition area, thereby acquiring multiple sets of multi-view image pairs covering the entire movement of the fish.

[0059] Image sequence processing stage:

[0060] All acquired multi-view image pairs are timestamped to ensure that images from different perspectives at the same time point have the same time identifier, thus guaranteeing the temporal consistency of the image sequence.

[0061] Add viewpoint encoding information to each image, bind the timestamp and viewpoint encoding to the image data, complete the structured encapsulation process, and generate a structured image data stream;

[0062] Data transmission phase:

[0063] The encapsulated structured image data stream is transmitted to the image processing unit through a preset communication link. During the transmission process, the data transmission status is monitored in real time to ensure that the data is complete and without loss.

[0064] End phase:

[0065] Once all images of a batch of freshwater fish to be tested have been acquired, the system shuts down the multispectral LED light source array and industrial camera, cleans residual impurities in the transparent carrier channel, and waits for the next batch of testing task instructions. If there are no new tasks for a long time, it automatically enters a low-power standby state.

[0066] In this embodiment, the image processing unit is the core data processing hub of the machine vision-based freshwater fish intelligent detection and grading system. It performs a series of precise processing on the multi-view image sequences transmitted by the image acquisition unit to extract the target image region of the freshwater fish to be tested from the complex background, providing a high-quality data source for the multi-dimensional feature extraction of the subsequent image analysis unit.

[0067] The image processing unit is primarily responsible for the full-process optimization of the received multi-view image sequence. It first eliminates image noise and uneven illumination through preprocessing, then separates the fish from the background through preliminary segmentation. Next, it constructs spatial projection data of the fish using multi-view fusion and 3D projection. Then, it refines and removes interference information using motion consistency, and finally accurately extracts the target image region through 3D reconstruction and backprojection. The entire process transforms the original image into a clean target region, ensuring the accuracy of subsequent morphological parameter analysis, motion state assessment, and fish appearance detection. The image processing unit includes:

[0068] Image preprocessing module:

[0069] Noise filtering: Noise filtering is performed on each original image in the multi-view image sequence. An adaptive filtering algorithm is used to suppress random noise and salt-and-pepper noise in the image, while preserving key details such as fish body edges and textures, thereby improving image clarity.

[0070] Illumination equalization: To address the issue of localized illumination differences that may still exist under controlled lighting conditions, illumination equalization is performed on the filtered image. The equalization formula is as follows: (in To output the image in coordinates Pixel value at that location, For the input image in coordinates Pixel value at that location, The minimum pixel value of the input image. (where the maximum pixel value is the input image); this formula maps the pixel values ​​of the image to the full grayscale range, eliminates overly bright or dark areas, and generates preprocessed image data corresponding to each viewpoint;

[0071] Preliminary segmentation module:

[0072] Color space conversion: Convert the preprocessed image from RGB color space to HSV color space, and construct the color feature threshold of the fish body area by utilizing the differences between the fish body and the background in three dimensions: hue saturation and brightness.

[0073] Texture feature analysis: Extract texture feature parameters of the image, including indicators such as contrast entropy and correlation of the gray-level co-occurrence matrix, and construct a multi-feature fusion segmentation criterion by combining color feature thresholds;

[0074] Background and foreground segmentation: Based on the constructed segmentation criteria, preliminary segmentation of background and foreground is performed on each preprocessed image data. Pixels that meet the feature conditions are identified as candidate fish regions, and preliminary segmentation results containing candidate freshwater fish regions to be tested are obtained.

[0075] Multi-view fusion and 3D projection module:

[0076] Feature point detection and matching: Feature point detection is performed on the preliminary segmentation results from different viewpoints at the same time point to extract key feature points such as corners and inflection points of the fish body edge; a feature point matching algorithm is used to calculate the matching similarity of feature points from different viewpoints, and the matching similarity formula is as follows: (in For feature points and feature points Matching similarity, For feature points The Each feature component For feature points The Each feature component For feature points The mean of the characteristic components, For feature points The mean of the characteristic components, (The total number of feature components); Select feature point pairs with similarity higher than a set threshold to complete cross-view feature point matching;

[0077] Spatial alignment and 3D projection: Based on the pre-calibrated intrinsic and extrinsic parameters of the industrial camera, the preliminary segmentation results from different perspectives are aligned in spatial coordinates, and the aligned 2D segmentation results are projected into a unified virtual 3D coordinate system to generate 3D spatial projection data that characterizes the spatial occupancy of the freshwater fish under test.

[0078] Motion Consistency Refinement Module:

[0079] Inter-frame difference calculation: Perform inter-frame difference operation on three-dimensional spatial projection data under continuous time series, calculate the difference value of projection data between adjacent time frames, and obtain inter-frame difference image;

[0080] Motion trajectory clustering: Cluster analysis is performed on motion regions in inter-frame difference images to construct motion trajectories for different regions; the motion trajectories are compared with the expected motion patterns of the freshwater fish under test to eliminate false motion regions caused by static background noise and instantaneous interference;

[0081] Refined point cloud extraction: Preserve the area consistent with the movement of the freshwater fish under test, and generate a refined 3D target point cloud to provide accurate 3D data support for subsequent target area extraction;

[0082] Target region extraction module:

[0083] 3D convex hull calculation: Perform 3D convex hull calculation on the refined 3D target point cloud to construct the minimum convex hull structure that can wrap the entire fish body and fully represent the spatial contour of the fish body;

[0084] Surface reconstruction: Surface meshing reconstruction is performed based on a three-dimensional convex hull structure to generate a smooth and continuous three-dimensional surface model of the fish body;

[0085] Back projection and region delineation: The reconstructed 3D surface model is back projected onto the original image coordinate system of each viewpoint. Based on the coordinate information of the back projection, the target image region of the freshwater fish to be tested is accurately delineated and extracted on the images of each viewpoint.

[0086] Furthermore, the image preprocessing technology is based on adaptive filtering and gray-level mapping theory; the adaptive filtering algorithm can dynamically adjust the filtering parameters according to the texture features of local areas of the image, and retain the edge details of the fish body while removing noise; the illumination equalization technology expands the pixel value distribution of the image to the full gray-level range through gray-level linear mapping, eliminates the segmentation error caused by uneven illumination, and lays a good foundation for subsequent segmentation operations.

[0087] Multi-view feature matching and 3D projection technology is based on camera calibration and stereo vision theory. By pre-calibrating the camera's intrinsic and extrinsic parameters, a mapping relationship between two-dimensional image coordinates and three-dimensional world coordinates can be established. The feature point matching algorithm calculates the similarity of feature points from different viewpoints to achieve accurate matching of cross-viewpoint feature points. Then, through spatial projection, the two-dimensional segmentation results are transformed into three-dimensional spatial data to construct the spatial outline of the fish.

[0088] Motion consistency analysis technology is based on inter-frame difference and cluster analysis theory. Inter-frame difference operation can quickly capture motion regions in image sequences, while cluster analysis can divide motion regions into different trajectory clusters. By comparing the consistency between trajectory clusters and the expected motion pattern of the fish, background noise and transient interference can be effectively eliminated, the real motion region of the fish can be preserved, and the purity of the target point cloud can be improved.

[0089] 3D convex hull reconstruction and back projection technology are based on computational geometry and coordinate transformation theory. 3D convex hull calculation can generate the minimum convex set that encloses the fish body, which can fully represent the spatial morphology of the fish body. Surface mesh reconstruction can transform discrete point cloud data into a continuous 3D surface model. Back projection technology maps the 3D surface model back to the 2D image coordinate system through inverse coordinate transformation, thereby accurately defining the target area of ​​the fish body on the original image.

[0090] The workflow of the image processing unit is as follows:

[0091] Initialization phase:

[0092] After the image processing unit starts up, it completes the self-test of the hardware device, including the connectivity detection of the data receiving interface, the stability detection of the algorithm running environment, and the loading detection of the parameter configuration file, to ensure that the unit is in normal working condition.

[0093] Load the preset configuration information such as filtering parameters, color space conversion parameters, feature point matching threshold, and 3D projection parameters, establish a communication connection with the image acquisition unit and the image analysis unit, and prepare to receive multi-view image sequences;

[0094] Image reception and preprocessing stage:

[0095] Receive the structured image data stream transmitted by the image acquisition unit, parse the image data timestamps and viewpoint information in the data stream, and classify and organize the multi-view image sequence according to time order and viewpoint.

[0096] For each original image, noise filtering and illumination equalization are performed sequentially to eliminate image noise and uneven illumination, generating preprocessed image data corresponding to each viewpoint;

[0097] Preliminary segmentation stage:

[0098] The preprocessed image is converted from the RGB color space to the HSV color space, and the color and texture feature parameters of the image are extracted.

[0099] Based on the preset multi-feature fusion segmentation criteria, the preprocessed image is initially segmented into background and foreground to obtain preliminary segmentation results containing candidate regions of freshwater fish to be tested.

[0100] Multi-view fusion and 3D projection stage:

[0101] Feature point detection is performed on the preliminary segmentation results from different perspectives at the same time point, the feature point matching similarity is calculated, and cross-view feature point matching is completed.

[0102] Based on the camera calibration parameters, the matched segmentation results are spatially aligned and projected onto a unified virtual 3D coordinate system to generate 3D spatial projection data.

[0103] Motion consistency refinement stage:

[0104] Inter-frame difference operations are performed on continuous time series three-dimensional spatial projection data to obtain inter-frame difference images;

[0105] Cluster analysis is performed on the motion regions in the inter-frame difference images to construct motion trajectory clusters, noise regions that do not conform to the movement patterns of the fish are removed, and the three-dimensional target point cloud is extracted and refined.

[0106] Target region extraction and output stage:

[0107] Perform 3D convex hull calculation and surface meshing reconstruction on the refined 3D target point cloud to generate a 3D surface model of the fish body;

[0108] The 3D surface model is back-projected onto the coordinate system of the original image from each viewpoint to accurately define and extract the target image region;

[0109] The extracted target image region is transmitted to the image analysis unit, and key data during the processing is saved, awaiting the input of the next batch of image data.

[0110] In this embodiment, the image analysis unit is the core feature mining hub of the machine vision-based freshwater fish intelligent detection and grading system. It performs multi-dimensional feature analysis on the target image region output by the image processing unit, extracts key information from three levels: morphological parameters, motion state, and body surface features, and provides accurate and comprehensive quantitative analysis basis for the grading decision-making unit.

[0111] The image analysis unit is primarily responsible for deep feature mining of the target image region. It reconstructs the three-dimensional contour of the fish using the morphological parameter analysis module, calculates body length, height, width, and estimated weight, quantifies the fish's displacement, posture, and activity intensity through the motion state assessment module to determine its vitality and stress state, and identifies abnormal spots and parasite attachments on the body surface using the fish appearance detection module. Finally, it integrates the analysis results from these three dimensions to form a standardized multi-dimensional feature analysis dataset, which is then transmitted to the hierarchical decision-making unit. The morphological parameter analysis module is used to reconstruct the three-dimensional contour model of the freshwater fish under test based on multi-view images. The image analysis unit includes:

[0112] Morphological parameter analysis module:

[0113] Cross-view feature point matching: Receive and parse target image regions from various viewpoints transmitted by the image processing unit, perform feature point detection on target image regions from different viewpoints at the same time point, and extract key features such as the corner points and inflection points of the fish body edges; filter matching feature point pairs through feature point matching algorithms to generate a cross-view matching feature point pair set.

[0114] 3D point cloud generation: Based on the set of matching feature point pairs and the pre-calibrated intrinsic and extrinsic parameters of the industrial camera, the 3D spatial coordinates of the matching feature point pairs are calculated by triangulation to generate a sparse 3D point cloud that represents the key geometric features of the fish surface; using the sparse 3D point cloud as a geometric constraint, stereo matching and depth calculation are performed on the target image regions from various perspectives to generate a dense 3D point cloud that covers the complete outline of the fish.

[0115] 3D contour mesh reconstruction: The surface of the dense 3D point cloud is reconstructed by meshing to build a 3D contour mesh model of the freshwater fish under test, which fully represents the spatial morphological characteristics of the fish.

[0116] Morphological parameter calculation: Perform principal axis analysis on the three-dimensional contour mesh model to determine the principal axis from head to tail; calculate the maximum projected length of the three-dimensional contour mesh model along the principal axis as the body length; on the cross section perpendicular to the principal axis, calculate the maximum vertical dimension as the current cross section height, calculate the maximum horizontal dimension as the current cross section width, and use the maximum values ​​of body height and body width in all representative cross sections as the overall height and overall width of the fish body;

[0117] Estimated weight conversion: The volume of the internal space enclosed by the surface of the 3D contour mesh model is calculated as the estimated volume. The estimated weight is then calculated based on the pre-stored fish species density conversion relationship. The conversion formula is as follows: (in, To estimate the weight, The body density corresponding to the freshwater fish species to be tested. (For estimated volume);

[0118] Motion status assessment module:

[0119] The target centroid displacement tracking submodule calculates the centroid coordinates of the fish in the two-dimensional image coordinate system at each time point based on the target image region from each viewpoint in a continuous time series. According to the pre-calibrated camera parameters, the two-dimensional centroid coordinates of each viewpoint are fused and converted into three-dimensional centroid coordinates in a unified world coordinate system. Based on the time series of the three-dimensional centroid coordinates, the displacement vector and instantaneous velocity between adjacent time points are calculated to generate the fish's motion trajectory and velocity change curve.

[0120] The 3D attitude angle extraction submodule receives the 3D space centroid coordinate sequence and the refined 3D target point cloud, performs principal component analysis on the refined 3D target point cloud at each time point, and determines the direction of the fish's 3D space principal axis at that time point; based on the angle change of the 3D space principal axis direction relative to the world coordinate system reference axis, it calculates the fish's attitude angle sequence in the pitch, yaw and roll directions.

[0121] The activity intensity quantification submodule simultaneously receives instantaneous motion velocity sequences and attitude angle sequences. It performs integration on the instantaneous motion velocity sequence within a time window to obtain the linear motion cumulative intensity; it performs differentiation on the attitude angle sequence to obtain the angular velocity sequence, and integrates the angular velocity sequence within a time window to obtain the rotational motion cumulative intensity; finally, it weights and fuses the linear and rotational motion cumulative intensities to calculate the comprehensive activity intensity index. The calculation formula is as follows: (in To form a comprehensive activity intensity index, The cumulative intensity weighting coefficient for linear motion. For linear motion cumulative intensity, The cumulative intensity weighting coefficient for rotational motion. (cumulative intensity of rotational motion);

[0122] The vitality and stress state assessment submodule receives the movement trajectory velocity change curve, attitude angle sequence, and comprehensive activity intensity index; extracts the regularity of the movement trajectory, the fluctuation frequency and amplitude of the velocity change curve, the abrupt change characteristics of the attitude angle sequence, and the time-domain distribution pattern of the comprehensive activity intensity index; it then matches and compares the extracted features with a preset vitality benchmark model and stress feature library, outputting the fish's vitality level score and stress state quantitative indicators, specifically including:

[0123] Motion trajectory regularity extraction: The three-dimensional motion trajectory data of the freshwater fish to be tested is received, and the deviation between the trajectory and a standard smooth trajectory is calculated. The smaller the deviation, the more regular the trajectory. The regularity is calculated using the deviation accumulation method, with the following formula:

[0124] (in, For the regularity of the motion trajectory, The actual coordinates of the trajectory. The coordinates of the standard trajectory are the points corresponding to the coordinates. The total number of trajectory coordinate points. (Total length of the standard trajectory);

[0125] Speed ​​variation curve feature extraction: Receive speed variation curve data, count the number of speed peaks per unit time to obtain the speed fluctuation frequency, calculate the difference between the maximum and minimum speed values ​​to obtain the speed fluctuation amplitude; weight and fuse the fluctuation frequency and amplitude according to a preset ratio to generate a comprehensive speed fluctuation value.

[0126] Attitude angle sequence mutation feature extraction: Receive attitude angle sequence data in pitch, yaw and roll directions, calculate the rate of change of attitude angle at each time point; when the rate of change exceeds a preset threshold, it is determined as an attitude angle mutation, and the total number of mutations within the time window is used as the attitude angle mutation feature value.

[0127] Extraction of the temporal distribution of comprehensive activity intensity: Receive time series data of the comprehensive activity intensity index and calculate the variance of the intensity index within the time window; the smaller the variance, the more uniform the distribution of comprehensive activity intensity; the larger the variance, the more drastic the fluctuation of the intensity distribution, thus characterizing the temporal distribution pattern of comprehensive activity intensity.

[0128] Vitality Benchmark Model Matching: The extracted motion trajectory regularity, velocity fluctuation comprehensive value, attitude angle abrupt change feature value, and comprehensive activity intensity time-domain distribution value are input into the vitality benchmark model; the model calculates the vitality matching degree through a multi-feature weighted fusion algorithm, and the fusion formula is as follows: (in, Rate your vitality level. This is the weighting coefficient for the regularity of the motion trajectory. For the regularity of the motion trajectory, The weighting coefficient for the comprehensive value of speed fluctuation. This is the comprehensive value of speed fluctuation. The weighting coefficients for the eigenvalues ​​of abrupt changes in attitude angles. These are the characteristic values ​​of abrupt changes in attitude angle. The weighting coefficient is the time-domain distribution value of the comprehensive activity intensity. (This represents the time-domain distribution value of comprehensive activity intensity); the weighting coefficients are preset based on the movement characteristics of different freshwater fish species and can be dynamically adjusted.

[0129] Comparative analysis of stress feature database: The abrupt change in posture angle feature values ​​are matched with the threshold range in the stress feature database. Simultaneously, the degree of stress response in the tested freshwater fish is analyzed by combining the fluctuation of the temporal distribution of comprehensive activity intensity. The formula for calculating the quantitative index of stress state is as follows: (in As a quantitative indicator of stress state, This is a stress intensity correction factor. The number of attitude angle abrupt changes. (This refers to the length of the time window); the larger the index value, the stronger the stress response of the freshwater fish.

[0130] Vitality Level Determination: The calculated vitality level score It matches the fish with a preset vitality level threshold range; different scoring ranges correspond to different vitality levels, and the higher the score, the better the vitality of the freshwater fish.

[0131] Stress state quantification output: The calculated stress state quantification index Standardization is performed to output specific quantitative values; at the same time, stress levels are classified according to the magnitude of the indicator values, providing a clear reference basis for stress levels for quality grading.

[0132] Results integration and transmission: The vitality level score, vitality level and stress state quantitative indicators, and stress state level are integrated into a motion state assessment result dataset and transmitted to the hierarchical decision-making unit for subsequent quality grading determination.

[0133] Fish appearance detection module:

[0134] High-resolution image input submodule: Receives the target image region output by the image processing unit, performs standardized size adjustment and enhancement processing on the target image region to improve the image clarity and contrast, and generates the image block to be detected;

[0135] Multi-scale feature extraction submodule: Input the image patch to be detected into the trained convolutional neural network model, and extract local detail features and global context features under different receptive fields through multiple parallel convolutional paths in the model, generating multiple sets of depth feature maps with different scales.

[0136] Feature Fusion and Enhancement Submodule: Performs cross-scale feature fusion and attention weighting on multiple sets of depth feature maps with different scales, highlights feature responses related to fish surface defects, suppresses interference from irrelevant background features, and generates fused enhanced feature maps.

[0137] Defect classification and localization submodule: Based on enhanced feature maps, the classification branch of the convolutional neural network model outputs the probability prediction of the presence of abnormal spots or parasite attachment on the fish body surface; at the same time, the localization branch of the model outputs the bounding box coordinates or pixel-level segmentation mask of the abnormal spots or parasite attachment features in the damaged area in the image block to be detected.

[0138] The results analysis and output submodule determines whether there are specific types of appearance defects on the fish body surface based on probability prediction and bounding box coordinates or pixel-level segmentation masks; it quantifies and calculates the total area ratio and distribution information of defects, and generates a fish appearance inspection report.

[0139] Furthermore, the 3D contour reconstruction and parameter calculation technology is based on stereo vision and computational geometry theory. The feature points of the 2D image are converted into 3D spatial coordinates through triangulation to generate a sparse point cloud. Then, stereo matching is performed using the sparse point cloud as a constraint to complete the dense point cloud data of the fish body surface. A 3D contour model is constructed through surface mesh reconstruction. Combined with principal axis analysis and cross-sectional measurement, the precise quantification of the fish body morphology parameters is achieved. The weight conversion is based on the linear relationship between volume and density, and the prediction is completed using the body density parameters corresponding to the fish species.

[0140] The motion state quantitative assessment technology is based on target tracking and kinematic analysis theory. It achieves accurate tracking of the three-dimensional motion trajectory of the fish through spatiotemporal transformation of the centroid coordinates. It determines the direction of the fish's main axis based on principal component analysis and calculates the posture angle sequence to characterize the spatial posture changes of the fish. It quantifies the cumulative intensity of linear and rotational motion through integral calculation and then obtains a comprehensive activity intensity index through weighted fusion. Finally, it combines the matching analysis of motion characteristics and benchmark models to achieve an objective assessment of vitality and stress status.

[0141] The convolutional neural network appearance defect detection technology is based on deep learning and computer vision theory. It uses a multi-path convolutional structure to extract multi-scale features, taking into account both local details and global contextual information. It strengthens the response weight of defect features through attention-weighted fusion, thereby improving the sensitivity of defect recognition. It utilizes the parallel output of classification and localization branches to simultaneously determine the defect type and locate the defect. Combined with quantitative calculation, it obtains indicators such as the area ratio and quantity of defects, thus completing a comprehensive evaluation of appearance quality.

[0142] The workflow of the image analysis unit is as follows:

[0143] Initialization phase:

[0144] After the image analysis unit starts, it completes the loading and self-check of the algorithm model, including integrity checks of the 3D reconstruction algorithm, motion evaluation algorithm, and convolutional neural network model, to ensure the stability of the algorithm's operating environment.

[0145] Load the preset fish species morphological parameter benchmark values, vitality assessment benchmark model and stress feature library, establish communication connection with the image processing unit and hierarchical decision unit, and prepare to receive target image area data;

[0146] Morphological parameter analysis stage:

[0147] The system receives the target image region transmitted by the image processing unit, performs feature point detection and matching on images from different perspectives, and generates a cross-perspective feature point pair set.

[0148] A sparse 3D point cloud is generated by triangulation, and a dense 3D point cloud is further constructed and the surface mesh reconstruction is completed to obtain a 3D contour mesh model.

[0149] Calculate morphological parameters such as body length, height and width, convert and estimate weight based on the relationship between volume and body density, and generate a morphological parameter analysis dataset.

[0150] Exercise status assessment phase:

[0151] The centroid coordinates of the target image region in a continuous time series are calculated and 3D transformed to generate motion trajectory and velocity change curves.

[0152] Extract the three-dimensional posture angle sequence of the fish body, calculate the cumulative intensity of linear motion and the cumulative intensity of rotational motion, and obtain the comprehensive activity intensity index by weighting;

[0153] By combining the matching analysis of motion characteristics and benchmark models, the activity level score and stress state quantitative index are output to generate a motion state assessment dataset.

[0154] Fish body appearance inspection stage:

[0155] The target image region is standardized to generate image patches to be detected, which are then input into a convolutional neural network model.

[0156] Multi-scale deep features are extracted and fused for enhancement. Defect probability prediction and location information are output through classification and localization branches.

[0157] The area proportion, quantity, and distribution of defects are analyzed to generate a fish appearance inspection report and appearance feature dataset;

[0158] Results integration and output stage:

[0159] By integrating the analysis data from three dimensions—morphological parameters, motion state, and apparent features—standardized multi-dimensional feature analysis results are generated.

[0160] The standardized feature dataset is transmitted to the hierarchical decision-making unit, while key data from the analysis process is saved, awaiting the input of the next batch of data.

[0161] In this embodiment, the grading decision unit is the core decision center of the machine vision-based freshwater fish intelligent detection and grading system. It integrates the multi-dimensional feature data output by the image analysis unit, performs dual judgment on specifications and quality according to the preset grading rule library, and finally generates accurate grading decision results, providing the core basis for the sorting operation of the control and output unit.

[0162] The hierarchical decision-making unit is primarily responsible for receiving, parsing, and comprehensively deciding on the multi-dimensional feature analysis results transmitted by the image analysis unit. It first standardizes the heterogeneous data of morphological parameters, motion states, and appearance detection. Then, based on a pre-defined hierarchical rule base, it completes specification hierarchies and quality hierarchies respectively. Finally, it generates the final hierarchical decision-making result through hierarchical mapping relationships, simultaneously outputting control commands and hierarchical report data. This ensures the objectivity and accuracy of the hierarchical results throughout the process. The hierarchical decision-making unit includes:

[0163] Hierarchical data receiving and parsing module:

[0164] Multi-dimensional data reception: Receives multi-dimensional feature analysis results from the image analysis unit, including morphological parameter datasets, motion state evaluation datasets, and fish appearance detection datasets of the freshwater fish under test.

[0165] Data format standardization: The received heterogeneous data is format unified and parsed. Data such as body length, body height, body width, estimated weight, vitality level score, stress state quantification index in morphological parameters, and defect area proportion and quantity distribution information in appearance detection are converted into a standardized format that meets the requirements of the hierarchical rule base, generating a standardized feature dataset containing three types of datasets.

[0166] Specification grading determination submodule:

[0167] Weight data extraction: The estimated weight data of the freshwater fish to be tested is extracted from the standardized feature dataset. This data is calculated by the morphological parameter analysis module based on the three-dimensional contour model and body density.

[0168] Threshold range matching: The estimated weight data is input into the preset weight-based specification classification rules. The rules have multiple weight threshold ranges stored in the pre-stored rules, and each range corresponds to a specification level. By matching the estimated weight with each threshold range one by one, the specification level of the freshwater fish to be tested is determined.

[0169] Quality grading determination submodule:

[0170] Sub-item data extraction: The motion state assessment dataset and the fish appearance detection dataset are extracted in parallel from the standardized feature dataset and used for the calculation of the vitality quality sub-item score and the appearance quality sub-item score, respectively.

[0171] Vitality quality sub-score calculation: Input the exercise status assessment dataset into the preset vitality assessment sub-rules, and calculate the vitality quality sub-scores based on the vitality level score and stress state quantitative indicators according to the scoring standards in the rules.

[0172] Appearance quality sub-score calculation: Input the fish appearance detection dataset into the preset appearance evaluation sub-rules, and calculate the appearance quality sub-score based on the total area ratio and distribution information of defects according to the deduction standards in the rules.

[0173] Comprehensive Quality Score Fusion: Based on a preset fusion algorithm, the vitality quality sub-scores and appearance quality sub-scores are weighted and fused to calculate the comprehensive quality score. The calculation formula is as follows: (in For overall quality scoring, The weighting coefficients for the vitality quality sub-item score. Scoring for the vitality and quality sub-item. The weighting coefficients for the appearance quality sub-item scores. (Scoring of appearance quality sub-items);

[0174] Quality grade determination: The comprehensive quality score is matched with the preset quality score range. Each score range corresponds to a quality grade, and the quality grade of the freshwater fish to be tested is determined accordingly.

[0175] Integrated decision-making and instruction generation module:

[0176] Grade mapping combination: Receive the specification grade output by the specification grading determination submodule and the quality grade output by the quality grading determination submodule, and combine the specification grade and quality grade according to the preset grade mapping relationship to generate the final grading decision result of the freshwater fish to be tested.

[0177] Control command generation: Based on the final hierarchical decision results, control commands containing target level identifiers are generated. These commands are used to guide the control and output units to drive the actuators to perform sorting operations.

[0178] Grading report data generation: Integrate standardized feature datasets and grading decision results to generate grading report data that includes the size, quality, and various characteristic indicators of the freshwater fish to be tested, providing data support for subsequent report synthesis;

[0179] Furthermore, the multi-dimensional data standardization technology is based on the theory of data heterogeneity unification. It establishes a unified data description standard for three types of data from different sources and in different formats: morphological parameters, motion states, and appearance detection. Through format conversion and index normalization, it eliminates format and dimensional differences between heterogeneous data, enabling various types of data to adapt to the matching requirements of the hierarchical rule base and ensuring the smooth implementation of hierarchical decision-making.

[0180] The hierarchical rule base matching technology is based on rule engine theory. The rule base integrates two core rule categories: specification hierarchical rules and quality hierarchical rules. The specification hierarchical rules use weight threshold ranges as the core matching conditions, while the quality hierarchical rules use sub-item scoring standards and comprehensive scoring ranges as the core matching conditions. The system automatically determines the level by comparing standardized data with the conditions in the rule base one by one, thereby improving the efficiency and consistency of hierarchical decision-making.

[0181] The weighted fusion decision-making technology is based on the multi-index comprehensive evaluation theory. It sets different weight coefficients for the two core quality dimensions of vitality quality and apparent quality according to their impact on the overall quality of freshwater fish. The weighted fusion algorithm integrates the two sub-item scores into a comprehensive quality score, realizing a comprehensive evaluation of the quality of freshwater fish. The weight coefficients can be flexibly adjusted according to the actual application scenario and fish species characteristics to enhance the adaptability of the system.

[0182] The workflow of a hierarchical decision-making unit is as follows:

[0183] Initialization phase:

[0184] After the hierarchical decision-making unit is started, it completes the loading and self-check of the hierarchical rule base, including the integrity check of the weight threshold range of specification grading, the scoring criteria of quality grading and the level mapping relationship, to ensure that the rule base is free of missing and conflicting information.

[0185] Load the preset weighted fusion algorithm parameters, establish communication connections with the image analysis unit and the control and output unit, and prepare to receive multi-dimensional feature analysis results;

[0186] Data reception and parsing stage:

[0187] Receive the multi-dimensional feature analysis results transmitted by the image analysis unit, perform data integrity verification, and remove invalid or abnormal data;

[0188] The data that passes the verification are standardized to generate a standardized feature dataset containing morphological parameters, motion state, and appearance detection data.

[0189] Specification grading determination stage:

[0190] Extract estimated weight data from the standardized feature dataset;

[0191] The estimated weight data is matched with the weight threshold range in the size grading rules to determine the size grade of the freshwater fish to be tested.

[0192] Quality grading and determination stage:

[0193] Extract the motion state assessment dataset and the fish appearance detection dataset from the standardized feature dataset;

[0194] Calculate the scores for the vitality quality sub-item and the appearance quality sub-item separately;

[0195] The comprehensive quality score is calculated based on a weighted fusion algorithm. The comprehensive quality score is then matched with the quality score range to determine the quality grade of the freshwater fish to be tested.

[0196] Comprehensive decision-making stage:

[0197] The system receives the determination results of specification level and quality level, combines them according to the level mapping relationship, and generates the final classification decision result.

[0198] Based on the final hierarchical decision results, control instructions and hierarchical report data are generated.

[0199] Result output stage:

[0200] Control commands and hierarchical report data are transmitted to the control and output unit to guide subsequent sorting operations and report synthesis;

[0201] Save the standardized feature dataset and hierarchical decision results of the current freshwater fish to be tested, and wait for the next batch of data input.

[0202] In this embodiment, the control and output unit is the execution and feedback hub of the machine vision-based freshwater fish intelligent detection and grading system. It receives instructions from the grading decision unit, generates precise control signals to drive the actuator to complete the sorting of freshwater fish, and simultaneously realizes the synthesis output of action feedback confirmation and grading report, ensuring the closed-loop controllability and data integrity of the entire grading process.

[0203] The control and output unit is primarily responsible for converting the grading decision results into executable sorting actions and completing the data recording and report output throughout the entire process. It first parses the grading decision instructions to extract the target grade and identity information, then calculates the precise trigger time and action parameters of the actuator through spatiotemporal registration. Subsequently, it generates control instructions to drive the actuator to complete the sorting. Next, it confirms the effectiveness of the sorting actions through sensor feedback, and finally integrates all detection and sorting data to generate a grading report. This ensures the accuracy of freshwater fish sorting and the completeness of the grading data throughout the entire process. The control and output unit includes:

[0204] Hierarchical instruction receiving and parsing module:

[0205] Hierarchical decision instruction reception: Receives hierarchical decision results transmitted from the hierarchical decision unit. The data includes the target level identifier and unique identification number of the hierarchical conclusion of the freshwater fish to be tested.

[0206] Command content parsing: The received hierarchical decision results are structured and parsed to extract the target level identifiers directly related to the sorting operation. At the same time, the freshwater fish identification numbers corresponding to the identifiers are associated with them, providing core data support for subsequent spatiotemporal registration and sorting control.

[0207] Target spatiotemporal registration module:

[0208] Real-time location and speed query: Based on the freshwater fish identification number obtained from the analysis, the real-time location information and movement speed data of the freshwater fish under test in the transparent carrying channel are queried. This data is continuously collected and uploaded by the position sensor of the image acquisition unit.

[0209] Precise trigger parameter calculation: Based on the physical distance between the actuator and the image acquisition area, combined with the real-time movement speed of the freshwater fish, the precise trigger time point driving the actuator's action is calculated. The calculation formula is as follows: (in To accurately trigger the time point, The physical distance between the actuator and the image acquisition area. (The real-time movement speed of the freshwater fish to be measured); Simultaneously determine key parameters such as the range of motion of the actuator to ensure that the actuator moves accurately when the freshwater fish reaches the designated position;

[0210] Sorting control signal generation module:

[0211] Control command parameter integration: Integrate parameters such as precise trigger time point, action type, and action amplitude output by the target spatiotemporal registration module, and construct a standardized set of control command parameters according to the drive protocol requirements of the actuator;

[0212] Command sequence generation: Based on the parameter set, a sorting control command sequence containing the target level, action type, action amplitude, and trigger time is generated. The command sequence adopts a time-series encoding format to ensure that the actuator can accurately execute various operations in chronological order.

[0213] Actuator drive module:

[0214] Control command transmission: The sorting control command sequence is sent to the corresponding actuator in real time. The actuator includes different types such as pneumatic nozzle array servo robotic arms or guide flaps, which can be flexibly configured according to the actual grading scenario.

[0215] Action-driven: After the command sequence is triggered, the actuator is driven to perform the corresponding action at a precise time point; the pneumatic nozzle array performs the pushing action, the servo robotic arm performs the grasping action, and the guide flap performs the guiding action, so as to accurately sort the freshwater fish to be tested into the target container of the corresponding level.

[0216] Action feedback and confirmation module:

[0217] Sensor signal reception: Receives signals transmitted by feedback sensors installed at the end of the actuator. The sensors monitor the actuator's operating status in real time, including whether the action has started, whether the action amplitude meets the standard, and whether freshwater fish have entered the target container.

[0218] Sorting validity confirmation: Analyze the signals from the feedback sensors to determine whether the sorting action has been executed correctly; if the signal shows that the action meets the preset standard and the freshwater fish has entered the target container, a sorting completion confirmation signal for the freshwater fish to be tested is generated; if the signal shows that the action is abnormal, a sorting failure warning signal is generated and a system alarm is triggered.

[0219] Tiered report synthesis and output module:

[0220] Single fish data integration: Receive the grading decision results and sorting completion confirmation signal, integrate the morphological parameters, motion state, appearance detection and other multi-dimensional feature analysis results of the freshwater fish to be tested, as well as the grading decision results and sorting completion information, and generate a grading detail record for a single fish;

[0221] Batch data statistics: Accumulate and statistically analyze the grading details of all freshwater fish to be tested within the preset batch. The statistics include core data such as the quantity distribution, quality overview indicators, and sorting success rate of freshwater fish of each grade.

[0222] Hierarchical report generation and output: Based on statistical data, batch hierarchical reports are synthesized. The report format is adapted to the transmission requirements of display terminals and data storage servers. The reports are synchronously output to designated terminals or servers to complete data archiving and sharing.

[0223] Furthermore, the spatiotemporal registration technology, based on kinematic analysis and position tracking theory, associates the real-time position and movement speed data of freshwater fish with their identification numbers. Combined with the physical distance between the actuator and the data collection area, it calculates the precise action trigger time. This technology achieves spatiotemporal synchronization between hierarchical decision-making instructions and the physical position of freshwater fish, ensuring that the actions of the actuator and the movement trajectory of the freshwater fish are perfectly matched, thus avoiding sorting timing deviations.

[0224] The precision drive technology for actuators is based on electromechanical control and timing instruction theory. It formulates differentiated drive instruction sequences for different types of actuators based on their motion characteristics. The instruction sequence includes parameters such as trigger time, action type, and amplitude, and can be adapted to actuators with different drive methods such as pneumatic, hydraulic, or electric, ensuring that all types of actuators can accurately complete sorting actions.

[0225] Closed-loop feedback control technology is based on sensor detection and action confirmation theory. By setting feedback sensors at the end of the actuator, a closed-loop control link of "command sending, action execution status feedback" is constructed. The sensors collect action status data in real time and transmit it back. The system judges the validity of the sorting action based on the feedback data, realizes dynamic monitoring and anomaly correction of the sorting process, and improves the reliability of the sorting process.

[0226] Multi-source data integration technology, based on data fusion and standardized coding theory, normalizes the characteristic analysis data, grading decision data, and sorting action data of freshwater fish, and completes the classification, statistics, and visualization of the data according to the preset report template. This technology achieves seamless connection of data throughout the entire process of detection, grading, and sorting, and the generated report can intuitively reflect the grading quality and sorting efficiency of a batch of freshwater fish.

[0227] The workflow of the control and output unit is as follows:

[0228] Initialization phase:

[0229] After the control and output unit is started, it completes the self-test of the actuator, including the air pressure detection of the pneumatic nozzle array, the joint flexibility detection of the servo robotic arm, and the opening and closing detection of the guide flap, to ensure that the actuator is in normal working condition.

[0230] Load the preset spatiotemporal registration parameters, execute the mechanism driving protocol and hierarchical report template, establish a communication connection with the hierarchical decision-making unit execution mechanism and data storage server, and prepare to receive hierarchical decision-making instructions;

[0231] Hierarchical instruction receiving and parsing stage:

[0232] Receive the hierarchical decision results transmitted by the hierarchical decision unit, perform integrity verification on the data, and remove invalid or erroneous data;

[0233] The target level identifier and freshwater fish identification number in the decision results are parsed, and the parsed data is temporarily stored in the local cache for later use.

[0234] Target spatiotemporal registration stage:

[0235] Based on the freshwater fish's identification number, the fish's real-time location information and movement speed data are retrieved from the image acquisition unit;

[0236] Substituting the trigger time calculation formula and combining it with the physical distance between the actuator and the acquisition area, the precise trigger time point and action parameters are calculated.

[0237] Sorting control signal generation stage:

[0238] Integrate parameters such as trigger time, action type, and action amplitude, and generate a standardized sorting control instruction sequence according to the actuator drive protocol;

[0239] The instruction sequence is time-coded to ensure that the instructions can be received and executed by the execution mechanism in a preset time order;

[0240] Actuator-driven phase:

[0241] The sorting control command sequence is sent to the corresponding actuator, and the trigger time point is waited for to arrive.

[0242] Once the trigger time is reached, the executor will perform push grabbing or diversion actions according to the instruction sequence to sort the freshwater fish into the target container;

[0243] Action feedback and confirmation phase:

[0244] Receive feedback signals from the end sensors of the actuator and analyze them to determine whether the sorting action has been effectively completed;

[0245] If the sorting action is valid, a sorting completion confirmation signal is generated; if the action is abnormal, a warning signal is generated and a system alarm is triggered.

[0246] The stages of tiered report synthesis and output:

[0247] Integrate the multi-dimensional feature analysis results, grading decision results, and sorting completion information of a single fish to generate a detailed record of the grading of a single fish;

[0248] The grading data of all freshwater fish in a batch is statistically analyzed, a batch grading report is generated, and the data is output to a display terminal or data storage server.

[0249] End phase:

[0250] Once a batch of freshwater fish has been sorted, the drive power of the actuator is turned off, temporary data in the local cache is cleared, the batch grading report is saved to a local backup, and the system awaits the next batch grading task instruction.

[0251] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A machine vision-based intelligent detection and grading system for freshwater fish, characterized in that: include: The image acquisition unit is used to continuously acquire multi-view image sequences of a single freshwater fish under controlled lighting conditions. The image acquisition unit includes: The enclosed imaging chamber has a diffuse reflection coating on its inner wall and is equipped with an adjustable brightness multispectral LED light source array to provide a uniform and spectrally adjustable lighting environment. At least two industrial cameras are arranged at a fixed angle above and to the side of the imaging box to acquire top and side views of the freshwater fish under test, either synchronously or sequentially. A transparent carrying channel, located inside the imaging chamber, is used to guide the freshwater fish under test through the image acquisition area in a basically fixed posture. The image acquisition unit acquires multi-view image sequences including: By using a multispectral LED light source array with adjustable brightness, the light intensity and spectral band are dynamically adjusted based on the species and surface reflectivity of the freshwater fish under test, so as to form a uniform and background-contrast-optimized controlled lighting environment in a closed imaging chamber. When the freshwater fish to be tested in the transparent carrying channel arrives at the preset image acquisition area, at least two industrial cameras are synchronously triggered based on the position sensor signal to acquire the top view and side view images of the freshwater fish to be tested, respectively, and generate a multi-view image pair at a single time point. Based on the acquisition of multi-view image pairs at a single time point, the industrial camera is continuously triggered to repeatedly acquire multi-view image pairs at a single time point according to a preset time interval, thereby obtaining multiple sets of multi-view image pairs of the freshwater fish under test passing through the image acquisition area throughout the entire process, forming a multi-view image sequence in a continuous time series. The multi-view image sequence is time-stamp aligned and data encapsulated to form a structured image data stream, which is then transmitted to the image processing unit. The image processing unit is connected in communication with the image acquisition unit. It is used to receive multi-view image sequences and perform image preprocessing, background segmentation and target extraction operations to obtain the target image region of the freshwater fish to be tested. The image analysis unit, communicatively connected to the image processing unit, is used to perform multi-dimensional feature analysis on the target image region and generate multi-dimensional feature analysis results, including at least: The morphological parameter analysis module is used to reconstruct the three-dimensional contour model of the freshwater fish under test based on multi-view images, and calculate its body length, body height, body width and estimated weight accordingly. The motion state assessment module is used to analyze the displacement, attitude angle changes, and activity intensity of the target image region in a continuous image sequence to assess its vitality and stress state. The fish appearance detection module is used to identify damaged areas, abnormal spots, and parasite attachment characteristics on the surface of freshwater fish under test based on high-resolution images and a trained convolutional neural network model. The hierarchical decision-making unit communicates with the image analysis unit to receive multi-dimensional feature analysis results and make comprehensive decisions based on a preset hierarchical rule base. The hierarchical rule base integrates at least weight-based specification hierarchical rules and vitality and appearance feature-based quality hierarchical rules to generate hierarchical decision results. The control and output unit communicates with the hierarchical decision unit and is used to generate control commands based on the hierarchical decision results to drive the actuator to sort the freshwater fish to be tested into containers of the corresponding level and output a hierarchical report.

2. The machine vision-based intelligent detection and grading system for freshwater fish according to claim 1, characterized in that: The image processing unit is used to receive multi-view image sequences and perform image preprocessing, background segmentation, and target extraction operations to obtain the target image region of the freshwater fish to be tested, including: The image preprocessing module is used to perform noise filtering and illumination equalization on each original image in the received multi-view image sequence to generate preprocessed image data corresponding to each view. The preliminary segmentation module, connected to the image preprocessing module, is used to perform preliminary segmentation of the background and foreground of each preprocessed image data based on color space conversion and texture feature analysis, to obtain preliminary segmentation results containing candidate regions of freshwater fish to be tested. The multi-view fusion and 3D projection module is connected to the preliminary segmentation module. It is used to match feature points and align space of the preliminary segmentation results from different perspectives at the same time point, and project them into a unified virtual 3D coordinate system to generate 3D spatial projection data that characterizes the spatial occupancy of the freshwater fish under test. The motion consistency refinement module is connected to the multi-view fusion and 3D projection module. It is used to analyze 3D spatial projection data under continuous time series. Through inter-frame difference and motion trajectory clustering, it removes static background noise and instantaneous interference, and extracts a refined 3D target point cloud that is consistent with the motion of the freshwater fish under test. The target region extraction module is connected to the motion consistency refinement module. It is used to perform 3D convex hull calculation and surface reconstruction on the refined 3D target point cloud, and back-project it onto the original image coordinate system of each viewpoint, so as to accurately define and extract the target image region of the freshwater fish to be tested on the image of each viewpoint.

3. The machine vision-based intelligent detection and grading system for freshwater fish according to claim 1, characterized in that: The morphological parameter analysis module is used to reconstruct a three-dimensional contour model of the freshwater fish under test based on multi-view images, and to calculate its body length, body height, body width, and estimated weight accordingly, including: The system receives and parses the target image regions from various perspectives transmitted by the image processing unit, performs feature point detection and matching on the target image regions from different perspectives at the same time point, and generates a set of matching feature point pairs across perspectives. Based on the set of matching feature point pairs and the pre-calibrated internal and external parameters of each industrial camera, the three-dimensional spatial coordinates of the matching feature point pairs are calculated by triangulation, generating a sparse three-dimensional point cloud that characterizes the key geometric features of the surface of the freshwater fish to be measured. Using sparse 3D point cloud as geometric constraint, stereo matching and depth calculation are performed on the target image region from each viewpoint to generate a dense 3D point cloud covering the complete outline of the surface of the freshwater fish to be tested. The surface of the dense 3D point cloud is then reconstructed by surface meshing to obtain the 3D outline mesh model of the freshwater fish to be tested. Principal axis analysis is performed on the three-dimensional contour mesh model to determine the principal axis from the head to the tail of the freshwater fish to be tested. The maximum projected length of the three-dimensional contour mesh model along the principal axis is calculated as the body length. On the cross section perpendicular to the principal axis, the maximum dimension of the three-dimensional contour mesh model in the vertical direction is calculated as the body height of the current cross section. The maximum dimension of the three-dimensional contour mesh model in the horizontal direction is calculated as the body width of the current cross section. The maximum value of body height and body width in all representative cross sections is taken as the overall height and overall width of the freshwater fish to be tested. The volume of the internal space enclosed by the surface of the three-dimensional contour mesh model is calculated as the estimated volume. Based on the pre-stored conversion relationship of body density corresponding to the freshwater fish species to be tested, the estimated volume is converted into the estimated weight.

4. The machine vision-based intelligent detection and grading system for freshwater fish according to claim 1, characterized in that: The motion state assessment module is used to analyze the displacement, attitude angle changes, and activity intensity of the target image region in a continuous image sequence to assess its vitality and stress state, including: The target centroid displacement tracking submodule is used to calculate the centroid coordinates of the freshwater fish under test in the two-dimensional image coordinate system at each time point based on the target image region from each viewpoint in a continuous time series. It also fuses the two-dimensional centroid coordinates of each viewpoint into three-dimensional centroid coordinates in a unified world coordinate system according to the pre-calibrated camera parameters. Based on the time series of the three-dimensional centroid coordinates, it calculates the displacement vector and instantaneous velocity between adjacent time points to generate the motion trajectory and velocity change curve of the freshwater fish under test. The 3D attitude angle extraction submodule is connected to the target centroid displacement tracking submodule. It is used to receive the 3D spatial centroid coordinate sequence and the refined 3D target point cloud provided by the image processing unit. Principal component analysis is performed on the refined 3D target point cloud at each time point to determine the direction of the 3D spatial principal axis of the freshwater fish under test at that time point. Based on the angle change of the 3D spatial principal axis direction relative to the world coordinate system reference axis, the attitude angle sequence of the freshwater fish under test in the pitch, yaw and roll directions is calculated. The activity intensity quantification submodule, connected to the 3D attitude angle extraction submodule, is used to simultaneously receive instantaneous motion velocity sequences and attitude angle sequences; it performs integration operations on the instantaneous motion velocity sequences within a time window to obtain the linear motion cumulative intensity; it performs differentiation operations on the attitude angle sequences to obtain the angular velocity sequences, and performs integration operations on the angular velocity sequences within a time window to obtain the rotational motion cumulative intensity; it then performs weighted fusion of the linear motion cumulative intensity and the rotational motion cumulative intensity to calculate the comprehensive activity intensity index. The vitality and stress state assessment submodule, connected to the activity intensity quantification submodule, receives motion trajectory, velocity change curve, posture angle sequence, and comprehensive activity intensity index; extracts the regularity of the motion trajectory, the fluctuation frequency and amplitude of the velocity change curve, the abrupt change characteristics of the posture angle sequence, and the temporal distribution pattern of the comprehensive activity intensity index; matches and compares the extracted features with the preset vitality benchmark model and stress feature library, and finally outputs the vitality level score and stress state quantification index of the freshwater fish under test.

5. The machine vision-based intelligent detection and grading system for freshwater fish according to claim 1, characterized in that: The fish surface detection module is used to identify damaged areas, abnormal spots, and parasite attachment characteristics on the surface of freshwater fish under test based on high-resolution images and a trained convolutional neural network model, including: The high-resolution image input submodule is used to receive the target image region of the freshwater fish to be tested output by the image processing unit, and to perform normalized size adjustment and enhancement processing on the target image region to generate the image block to be detected. The multi-scale feature extraction submodule is connected to the high-resolution image input submodule. It is used to input the image patch to be detected into the trained convolutional neural network model. Through multiple parallel convolutional paths in the convolutional neural network model, the local detail features and global context features of the image patch to be detected under different receptive fields are extracted respectively, generating multiple sets of depth feature maps with different scales. The feature fusion and enhancement submodule is connected to the multi-scale feature extraction submodule. It is used to perform cross-scale feature fusion and attention weighting on multiple sets of depth feature maps with different scales, highlight the feature responses related to fish appearance defects, and generate the fused enhanced feature map. The defect classification and localization submodule is connected to the feature fusion and enhancement submodule. It is used to output the probability prediction of damage, abnormal spots or parasite attachment on the surface of the freshwater fish under test based on the enhanced feature map through the classification branch in the convolutional neural network model. At the same time, it outputs the bounding box coordinates or pixel-level segmentation mask of the damaged area, abnormal spots or parasite attachment features in the image block to be detected through the localization branch in the convolutional neural network model. The results parsing and output submodule is connected to the defect classification and localization submodule. It is used to determine whether there are specific types of appearance defects on the surface of the freshwater fish under test based on probability prediction and bounding box coordinates or pixel-level segmentation masks, and to quantify the total area ratio, number and distribution information of defects, and generate a fish appearance inspection report.

6. The intelligent detection and grading system for freshwater fish based on machine vision according to claim 1, characterized in that: The hierarchical decision-making unit is used to receive multi-dimensional feature analysis results and make comprehensive decisions based on a preset hierarchical rule base to generate hierarchical decision results, including: The hierarchical data receiving and parsing module is used to receive the multi-dimensional feature analysis results from the image analysis unit, and to parse and standardize the data format of the multi-dimensional feature analysis results to generate a standardized feature dataset containing the morphological parameter dataset of the freshwater fish to be tested, the motion state evaluation dataset, and the fish appearance detection dataset. The specification grading determination submodule is connected to the grading data receiving and parsing module. It is used to extract the estimated weight data of the freshwater fish to be tested from the standardized feature dataset; input the estimated weight data into the preset weight-based specification grading rules; and determine the specification level of the freshwater fish to be tested by matching the estimated weight with multiple pre-stored weight threshold ranges. The quality grading determination submodule, connected to the grading data receiving and parsing module, is used to extract the motion state assessment dataset and the fish appearance detection dataset in parallel from the standardized feature dataset. The motion state assessment dataset is input into a preset vitality assessment sub-rule, and a vitality quality sub-score is calculated based on the vitality level score and stress state quantification index. The fish appearance detection dataset is input into a preset appearance assessment sub-rule, and an appearance quality sub-score is calculated based on the total area ratio, number, and distribution information of defects. The vitality quality sub-score and appearance quality sub-score are weighted and fused according to a preset fusion algorithm to generate a comprehensive quality score, and the quality grade of the freshwater fish under test is determined based on the score range in which the comprehensive quality score falls. The integrated decision-making and instruction generation module is connected to the specification grading judgment submodule and the quality grading judgment submodule, respectively. It is used to receive the judgment results of specification level and quality level; according to the preset level mapping relationship, it combines the specification level and quality level to map the final grading decision result; and based on the final grading decision result, it generates control instructions and grading report data containing the target level identifier.

7. The intelligent detection and grading system for freshwater fish based on machine vision according to claim 1, characterized in that: The control and output unit is used to generate control commands based on the grading decision results to drive the actuator to sort the freshwater fish to be tested into containers of the corresponding grades, and output a grading report, including: The hierarchical instruction receiving and parsing module is used to receive the hierarchical decision results from the hierarchical decision-making unit, parse the hierarchical decision results, and extract the target level identifier and corresponding identity number of the freshwater fish to be tested. The target spatiotemporal registration module is connected to the hierarchical instruction receiving and parsing module. It is used to associate and query the real-time position information and movement speed of the freshwater fish under test in the transparent carrier channel of the image acquisition unit according to the identification number. Based on the real-time position information, movement speed and physical distance between the actuator and the image acquisition area, the precise triggering time point and action parameters for driving the actuator are calculated. The sorting control signal generation module is connected to the target spatiotemporal registration module. It is used to generate a sorting control instruction sequence containing target level, action type and action amplitude based on the precise trigger time point and action parameters. The actuator drive module is connected to the sorting control signal generation module and is used to send the sorting control command sequence to the corresponding actuator. The actuator includes at least a pneumatic nozzle array, a servo robotic arm or a guide flap, and is used to perform pushing, grabbing or guiding actions at precise triggering time points according to the sorting control command sequence, so as to sort the freshwater fish to be tested into the target container of the corresponding level. The action feedback and confirmation module is connected to the actuator drive module and is used to receive the sensor feedback signal set at the end of the actuator to confirm whether the sorting action has been executed correctly; after confirming that the sorting action has been completed, it generates a sorting completion confirmation signal for the freshwater fish to be tested. The grading report synthesis and output module is connected to the grading instruction receiving and parsing module and the action feedback and confirmation module, respectively. It is used to receive grading decision results and sorting completion confirmation signals; integrate the multi-dimensional feature analysis results of the current freshwater fish to be tested, the grading decision results, and the sorting completion information to generate a grading detail record for a single fish; accumulate and statistically analyze the grading detail records of all freshwater fish to be tested in a preset batch, synthesize a batch grading report including grade distribution statistics, quality overview, and sorting success rate, and output the batch grading report to the display terminal or data storage server.

Images