A method and system for generating a fishing multi-source fusion dataset based on migration reconstruction

Abstract

This invention relates to the field of data generation technology, and discloses a method and system for generating a multi-source fusion dataset of fish catches based on transfer reconstruction. The method includes: dividing acquired fish catch observation data into source domain data and target domain data; performing edge enhancement on first underwater image data and second underwater image data respectively to obtain first enhanced underwater image data and second enhanced underwater image data; performing fish identification on the first enhanced underwater image data and second enhanced underwater image data based on the reflective properties of fish scales respectively to generate first fish feature vectors and second fish feature vectors; learning the correlation pattern between the first fish feature vector and the first fish behavior data, and transferring the correlation pattern to the target domain data to complete the second fish feature vector; integrating the spatiotemporally aligned data and the source domain data to obtain a multi-source fusion dataset of fish catches. This invention can provide reliable data support for fisheries applications.

Description

Technical Field

[0001] This invention relates to the field of data generation technology, and in particular to a method and system for generating multi-source fusion datasets of fish catches based on migration reconstruction. Background Technology

[0002] Fish catch data is the core basis for fisheries resource assessment, fishery condition forecasting, and scientific management. However, its collection process is affected by the deployment of observation equipment and the complexity of the underwater environment, resulting in uneven spatial and temporal distribution. Some long-term monitored fishing areas have accumulated a large amount of fish catch data, while newly established monitored fishing areas and remote offshore fishing areas have significant data gaps, making it difficult to form a complete dataset.

[0003] In existing technologies, only the statistics and storage of catch characteristics are generally performed for fishing areas with abundant data, while traditional data completion methods such as linear interpolation and mean filling are used for fishing areas with sparse data. These methods cannot use the patterns of fishing areas with abundant data to fill the feature gaps in fishing areas with sparse data, resulting in insufficient completeness and reliability of the generated catch dataset, which cannot meet the needs of high-precision fishery resource analysis. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a method and system for generating multi-source fusion datasets of fish catches based on migration reconstruction.

[0005] In a first aspect, embodiments of the present invention provide a method for generating a multi-source fusion dataset of fish catches based on migration reconstruction, including:

[0006] The acquired fish catch observation data is divided into source domain data and target domain data. The source domain data includes first underwater image data and first fish behavior data corresponding to fishing areas with abundant observation data. The target domain data includes second underwater image data and second fish behavior data corresponding to fishing areas with sparse observation data.

[0007] Edge enhancement is performed on the first underwater image data and the second underwater image data based on the characteristics of the ambient incident light to obtain the first enhanced underwater image data and the second enhanced underwater image data.

[0008] Based on the reflective properties of fish scales, fish identification is performed on the first enhanced underwater image data and the second enhanced underwater image data respectively, generating a first fish feature vector corresponding to the source domain data and a second fish feature vector corresponding to the target domain data;

[0009] Using the first fish feature vector as a training sample, the association pattern between the first fish feature vector and the first fish behavior data is learned, and the association pattern is transferred to the target domain data to complete the second fish feature vector;

[0010] The completed second fish feature vector and the second fish behavior data are spatiotemporally aligned, and the spatiotemporally aligned data is integrated with the source domain data to obtain a multi-source fusion dataset of fish catch.

[0011] Preferably, the step of performing edge enhancement on the first underwater image data and the second underwater image data based on the characteristics of ambient incident light to obtain first enhanced underwater image data and second enhanced underwater image data includes:

[0012] The environmental incident light characteristic parameters are extracted from the source domain data and the target domain data, respectively.

[0013] Based on the environmental incident light characteristic parameters, the first underwater image data and the second underwater image data are divided into several illumination areas respectively;

[0014] Edge enhancement is performed on each of the illuminated areas in the first underwater image data and the second underwater image data, respectively.

[0015] Each illuminated region after edge enhancement is fused to obtain first enhanced underwater image data corresponding to the first underwater image data and second enhanced underwater image data corresponding to the second underwater image data.

[0016] Preferably, the step of dividing the first underwater image data and the second underwater image data into several illumination regions based on the environmental incident light characteristic parameters includes:

[0017] Extract the light intensity gradient distribution from the environmental incident light characteristic parameters, and divide the first underwater image data and the second underwater image data into a strong light saturation area, a weak light blur area, and a normal lighting area based on the light intensity gradient distribution.

[0018] Preferably, the step of performing edge enhancement on each of the illuminated areas in the first underwater image data and the second underwater image data includes:

[0019] Based on adaptive threshold compression, the light intensity of the strong light saturation region in the first underwater image data and the second underwater image data is suppressed respectively;

[0020] Top-hat transformation is performed on the low-light blurred areas in the first underwater image data and the second underwater image data, respectively;

[0021] Edge detection operators are used to enhance the edge contours of the normally illuminated areas in the first underwater image data and the second underwater image data, respectively.

[0022] Preferably, the step of performing fish identification on the first enhanced underwater image data and the second enhanced underwater image data based on the reflective properties of fish scales, and generating a first fish feature vector corresponding to the source domain data and a second fish feature vector corresponding to the target domain data, includes:

[0023] All reflective areas with a gray level higher than the background are extracted from the first and second enhanced underwater image data respectively using adaptive threshold segmentation.

[0024] Frequency domain decomposition is performed on each of the reflective regions to obtain the high-frequency component energy ratio of the corresponding reflective region, and all candidate fish scale reflective regions that meet the first fish scale reflective characteristics are screened based on the high-frequency component energy ratio, wherein the first fish scale reflective characteristics include the high-frequency component energy ratio exceeding a first threshold.

[0025] Analyze the reflective point arrangement pattern of each candidate fish scale reflective region, and select all target fish scale reflective regions that meet the second fish scale reflective characteristics based on the reflective point arrangement pattern, wherein the second fish scale reflective characteristics include a reflective point density exceeding a second threshold.

[0026] For each of the target fish scale reflective regions, feature matching is performed to generate a first fish feature vector corresponding to the source domain data and a second fish feature vector corresponding to the target domain data.

[0027] Preferably, the step of performing frequency domain decomposition on each of the reflective regions to obtain the energy ratio of the high-frequency components corresponding to the reflective regions, and screening out all candidate fish scale reflective regions that meet the first fish scale reflective characteristics based on the energy ratio of the high-frequency components, includes:

[0028] Multi-scale wavelet transform is used to perform frequency domain decomposition on each of the reflective regions to obtain the energy ratio of the high-frequency components of the corresponding reflective regions, and all candidate fish scale reflective regions that meet the first fish scale reflective characteristics are screened based on the energy ratio of the high-frequency components.

[0029] Preferably, the step of analyzing the arrangement pattern of reflective points in each of the candidate fish scale reflective regions, and selecting all target fish scale reflective regions that meet the second fish scale reflective characteristics based on the arrangement pattern of reflective points, includes:

[0030] Based on the spatial distribution entropy analysis, the reflective point arrangement pattern of each candidate fish scale reflective region is analyzed, and all target fish scale reflective regions that meet the second fish scale reflective characteristics are selected according to the reflective point arrangement pattern.

[0031] Preferably, the step of using the first fish feature vector as a training sample, learning the correlation pattern between the first fish feature vector and the first fish behavior data, and transferring the correlation pattern to the target domain data to complete the second fish feature vector includes:

[0032] A transfer learning model incorporating an attention mechanism is constructed, with the first fish feature vector as the input and the first fish behavior data as the output. The nonlinear mapping relationship between the first fish feature vector and the first fish behavior data is learned through the attention mechanism.

[0033] The second fish feature vector is input into the optimized transfer learning model for feature adaptation and missing information completion, generating a complete fish feature vector that matches the fishing conditions corresponding to the target domain data. The feature adaptation is configured to adapt the nonlinear mapping relationship to the second fish feature vector.

[0034] Preferably, a domain adaptation loss function is introduced to optimize the transfer learning model, wherein the domain adaptation loss function is configured to minimize the feature distribution distance between the first fish feature vector and the second fish feature vector.

[0035] Secondly, embodiments of the present invention provide a fish catch multi-source fusion dataset generation system based on migration reconstruction, comprising:

[0036] The data partitioning module is used to divide the acquired fish catch observation data into source domain data and target domain data. The source domain data includes first underwater image data and first fish behavior data corresponding to fishing areas with abundant observation data, and the target domain data includes second underwater image data and second fish behavior data corresponding to fishing areas with sparse observation data.

[0037] An edge enhancement module is used to perform edge enhancement on the first underwater image data and the second underwater image data based on the characteristics of ambient incident light, respectively, to obtain first enhanced underwater image data and second enhanced underwater image data.

[0038] The fish identification module is used to identify fish in the first enhanced underwater image data and the second enhanced underwater image data based on the reflective properties of fish scales, and to generate a first fish feature vector corresponding to the source domain data and a second fish feature vector corresponding to the target domain data.

[0039] The transfer completion module is used to learn the correlation pattern between the first fish feature vector and the first fish behavior data using the first fish feature vector as a training sample, and transfer the correlation pattern to the target domain data to complete the second fish feature vector;

[0040] The alignment and fusion module is used to perform spatiotemporal alignment of the completed second fish feature vector and the second fish behavior data, and to integrate the spatiotemporally aligned data with the source domain data to obtain a multi-source fusion dataset of fish catch.

[0041] Compared with existing technologies, the beneficial effects of the fish catch multi-source fusion dataset generation method and system based on migration reconstruction proposed in this invention are as follows:

[0042] (1) To address the pain point of poor underwater image quality, edge enhancement is performed based on the characteristics of ambient incident light to accurately eliminate interference such as uneven lighting, overexposure, or low light blur. Combined with the reflective characteristics of fish scales, targeted fish identification is achieved, effectively eliminating non-target reflective interference such as bubbles and aquatic plants, and improving the accuracy of fish identification.

[0043] (2) By learning the correlation between fish characteristics and behavioral data through source domain data, and by using transfer learning to adapt the pattern to the target domain, the gaps in fish characteristics in sparse fishing areas can be accurately filled, effectively improving the reliability of data in sparse areas and solving the problem of data distortion caused by uneven distribution of data between fishing areas.

[0044] (3) By aligning the source domain data with the target domain data after completion, the complete source domain data and the target domain data are integrated into a unified fish catch dataset, which can support high-precision fishery resource assessment, dynamic prediction of fishery conditions and scientific management decision-making, and provide data basis for digital management of fisheries. Attached Figure Description

[0045] Figure 1 This is a flowchart illustrating a method for generating a multi-source fusion dataset of fish catch based on migration reconstruction according to an embodiment of the present invention.

[0046] Figure 2 This is a schematic diagram of the edge enhancement process according to an embodiment of the present invention;

[0047] Figure 3 This is a schematic diagram of the process of fish identification based on the reflective properties of fish scales in an embodiment of the present invention;

[0048] Figure 4 This is a schematic diagram of fish identification results according to an embodiment of the present invention;

[0049] Figure 5 This is a flowchart illustrating the process of learning and transferring association rules in an embodiment of the present invention;

[0050] Figure 6 This is a schematic diagram of the structure of a fish catch multi-source fusion dataset generation system based on migration reconstruction according to an embodiment of the present invention;

[0051] Figure label:

[0052] 01. Data partitioning module; 02. Edge enhancement module; 03. Fish identification module; 04. Migration completion module; 05. Alignment and fusion module. Detailed Implementation

[0053] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0054] In the description of this invention, it should be understood that the terms "first" and "second," etc., are used to distinguish different objects, rather than to describe a specific order.

[0055] In the description of this invention, it should be noted that, unless otherwise defined, all technical and scientific terms used in this invention have the same meaning as commonly understood by those skilled in the art. The terminology used in this specification is for the purpose of describing specific embodiments only and is not intended to limit the invention. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0056] like Figure 1 The diagram shown is a flowchart illustrating a method for generating a multi-source fusion dataset of fish catch based on migration reconstruction, according to an embodiment of the present invention. (Refer to...) Figure 1 This invention provides a method for generating a multi-source fusion dataset of fish catch based on migration reconstruction, comprising the following steps:

[0057] S1. Divide the acquired fish catch observation data into source domain data and target domain data;

[0058] The source domain data includes the first underwater image data and the first fish behavior data corresponding to the fishing area with abundant observation data, while the target domain data includes the second underwater image data and the second fish behavior data corresponding to the fishing area with sparse observation data.

[0059] Specifically, a multi-source observation system consisting of underwater high-definition cameras, acoustic detection equipment, and fish behavior tracking sensors was used to collect fish catch observation data within a pre-defined fishing area. Specifically, for fishing areas with abundant observation data, such as those with long-term deployed monitoring equipment, high data collection frequency, and continuous recording, the first underwater image data and the first fish behavior data (such as migration trajectories and foraging time records) were collected. For fishing areas with sparse observation data, such as newly established monitoring stations and remote offshore areas with long data collection intervals and insufficient sample sizes, the second underwater image data and the second fish behavior data were collected.

[0060] Furthermore, Gaussian filtering was applied to all underwater image data to remove noise interference caused by suspended particles in the water, and the image resolution was standardized to 1920×1080 pixels to achieve format standardization. Outliers caused by abnormal fluctuations in the sensor were removed from all fish behavior data using the 3σ principle, and the timestamps of the behavior data were uniformly corrected to UTC time to ensure temporal consistency.

[0061] Finally, based on the collection frequency, data volume, and completeness indicators of the fishery observation data, the preprocessed fish catch observation data were divided into source domain data (the first underwater image data and the first fish behavior data corresponding to the fishery areas with abundant observation data) and target domain data (the second underwater image data and the second fish behavior data corresponding to the fishery areas with sparse observation data).

[0062] S2. Based on the characteristics of ambient incident light, edge enhancement is performed on the first underwater image data and the second underwater image data respectively to obtain the first enhanced underwater image data and the second enhanced underwater image data.

[0063] The acquired first and second underwater image data are still subject to interference from incident light unique to the underwater environment, such as light intensity attenuation at different depths, directional shifts caused by water surface refraction, and spectral distortion caused by water absorption. This can easily lead to problems such as blurred fish edges and loss of detail, directly affecting the accuracy of subsequent fish identification. Therefore, it is necessary to perform targeted edge enhancement processing on the two types of underwater image data based on the characteristics of environmental incident light to achieve accurate enhancement of fish edges and improvement of image quality.

[0064] like Figure 2 As shown, this is a flowchart illustrating step S2. (Refer to...) Figure 2 Step S2 includes:

[0065] S201. Extract the environmental incident light characteristic parameters from the source domain data and the target domain data respectively;

[0066] For the first underwater image data and the accompanying environmental monitoring records (including data collected by light sensors) in the source domain data, the corresponding environmental incident light characteristic parameters are extracted by multispectral analysis, including the light intensity gradient distribution (calculated by using the image gray value gradient operator to calculate the rate of change of light intensity in different areas) and the spectral attenuation coefficient (calculated based on the relationship between the propagation distance and energy attenuation of light of different wavelengths in water).

[0067] For the second underwater image data in the target domain data, if there is light sensor recording, it is extracted with reference to the source domain method. If it only contains image data, the light intensity benchmark value is estimated by the dark channel prior algorithm. The spectral attenuation characteristics are inferred by combining the image color shift. Finally, the environmental incident light characteristic parameters corresponding to the source domain and the target domain are obtained respectively.

[0068] S202. Based on the characteristic parameters of the ambient incident light, the first underwater image data and the second underwater image data are divided into several illumination areas respectively;

[0069] The light intensity gradient distribution in the environmental incident light characteristic parameters is extracted, and the first underwater image data and the second underwater image data are divided into a strong light saturation area, a weak light blur area, and a normal lighting area based on the light intensity gradient distribution.

[0070] Specifically, the first underwater image data and the second underwater image data are converted to grayscale, the grayscale value distribution of a single image is statistically analyzed, and the grayscale mean μ and standard deviation σ are calculated, which are used as the basis for dividing the illumination intensity.

[0071] In this embodiment, areas with grayscale values ​​> μ+1.5σ and light intensity gradient values ​​< 5 are identified as strong light saturation areas. In these areas, the fish's edge details are obscured due to excessive light, and the light distribution tends to be uniform with weak gradient changes. Areas with grayscale values ​​< μ-1.2σ and light intensity gradient values ​​< 5 are identified as weak light blurring areas. In these areas, the overall grayscale value is low due to insufficient light, resulting in low contrast between the fish and the background, and the gradient signal of the detailed texture is obscured by noise. Areas with remaining grayscale values ​​in the range of μ-1.2σ to μ+1.5σ and light intensity gradient values ​​≥ 5 are identified as normal lighting areas. In these areas, the lighting is balanced and the gradient signal is clear, which can completely preserve the fish's edge and texture features.

[0072] It should be noted that, in response to the differences in water transparency between different fishing areas in the source and target domains, the calculation range of grayscale mean and standard deviation can be locally adjusted. For example, the threshold range for weak light areas can be appropriately expanded for turbid water to ensure that the division of illumination areas is adapted to the actual underwater environment.

[0073] S203. Perform edge enhancement on each illuminated area under the first underwater image data and the second underwater image data respectively;

[0074] Specifically, step S203 includes:

[0075] 1) Based on adaptive threshold compression, the light intensity in the strong light saturation region under the first underwater image data and the second underwater image data is suppressed respectively;

[0076] The strong light saturation areas in the first and second underwater image data are first divided into regions. The gray-level distribution characteristics of each strong light saturation area are extracted through regional gray-level statistical analysis to determine the upper limit of gray-level saturation in the region (i.e., the critical value at which the gray-level value no longer changes with the increase of light intensity). This is used as the basis for calculating the adaptive threshold. Then, the threshold size is dynamically adjusted by combining the gray-level average value of the normal illumination area around the region to ensure that the threshold can cover the overexposed gray-level range in the region and form a smooth transition with the gray-level of the surrounding area.

[0077] Gradient compression is applied to grayscale values ​​exceeding the adaptive threshold within the region, with the compression ratio dynamically adjusted based on the difference between the grayscale value and the threshold. In other words, the larger the difference, the higher the compression ratio, quickly reducing brightness interference from excessively strong light; conversely, the smaller the difference, the lower the compression ratio, preserving edge details close to the normal grayscale range. Simultaneously, the grayscale change trend of potential fish edges within the region is monitored during compression to avoid blurring of edge textures due to over-compression, ultimately reducing the overall brightness of the strongly lit saturated area to a range compatible with the grayscale of the normally lit area.

[0078] Specifically, the pixel value distribution in the strong light saturation area is statistically analyzed using grayscale histograms to locate the interval where the pixel value no longer increases with light intensity. The lower limit of this interval is taken as the initial grayscale saturation upper limit. The spatial distance weight between the strong light saturation area and the surrounding normal lighting area is calculated, and the initial threshold is weighted and corrected by combining the grayscale mean of the normal area to generate a dynamic threshold matrix for each pixel within the area.

[0079] Furthermore, for pixel values ​​exceeding the dynamic threshold, a piecewise nonlinear compression mapping is employed. Specifically, pixels significantly above the threshold (difference > 50% of the threshold) are rapidly reduced in brightness using an exponential compression function, while pixels slightly above the threshold (difference ≤ 50% of the threshold) are fine-tuned using a linear compression function. Simultaneously, potential fish-edge pixels within the pre-detection region are pre-detected using the Canny operator, and the compression ratio for these pixels is further reduced to preserve edge texture. After compression, the grayscale variance between the saturated high-light region and the adjacent normal region is calculated. If the variance exceeds a set threshold, the dynamic threshold matrix is ​​iteratively optimized until the grayscale transition between the two regions is continuous, ultimately achieving coordinated preservation of edge details while suppressing strong light.

[0080] 2) Perform top-hat transformation on the weak light blurred areas in the first underwater image data and the second underwater image data respectively;

[0081] Top-hat transformation, a morphological operation that suppresses low-grayscale uniform backgrounds and highlights bright details smaller than structuring elements in an image, can specifically address the problem of bright features being masked in low-light blurred areas due to overall low grayscale and insufficient contrast between the fish body and the background.

[0082] Specifically, based on the typical size of the fish in the dim light blur area (combined with the body length data of common fish in the fishing area), a disk-shaped structural element with a suitable radius is selected. This structural element can adapt to the contour features of fish in different postures, avoiding edge distortion caused by the mismatch between the shape of the structural element and the fish body.

[0083] Furthermore, a morphological opening operation is performed on the original image of the blurred low-light area. This involves first eroding the original image using a structuring element (to eliminate tiny bright noise points), then dilating the erosion result (to restore the grayscale continuity of the background area), yielding the opening result. Finally, the opening result is subtracted from the original image to obtain the image with enhanced bright details after top-hat transformation. Specifically, this process can be characterized by the following formula:

[0084]

[0085] in, This indicates the result of the top-hat transformation. This represents the original image of the blurred area in low light. Represents a structural element. Represents the original image Regarding structural elements The opening operation results. Through the above operations, the bright edges of the fish body and the reflective features of the fish scales that are obscured by the background in low-light environments can be effectively extracted.

[0086] 3) Edge detection operators are used to enhance the edge contours of the normally illuminated areas under the first and second underwater image data respectively.

[0087] Specifically, the Canny edge detection operator is used to enhance the edge contours of the normally illuminated areas that have been divided in the first and second underwater image data.

[0088] The edge contour enhancement process is explained below using edge detection operators:

[0089] a) Gaussian filtering preprocessing is applied to images in normally lit areas. An adaptively selected Gaussian kernel smooths the image, suppressing high-frequency noise caused by tiny water particles. The Gaussian kernel size is dynamically adjusted based on the average width of the fish.

[0090] b) Calculate the gradient magnitude and direction of the image (use the Sobel operator to obtain the horizontal and vertical gradients respectively, and obtain the edge intensity and direction by gradient synthesis) to locate potential edge pixels;

[0091] c) By performing non-maximum suppression, local maximum pixels are retained in the gradient direction, and non-edge pixels are removed to refine the edge width. A double thresholding method is used (a high threshold is used to filter strong edges, and a low threshold is used to identify weak edges) to retain only weak edges connected to strong edges and eliminate false edges formed by isolated noise points.

[0092] d) Combining the geometric continuity of the fish's outline in the normal lighting area, such as the consistency of the edge direction along the fish's axis, morphological connection operations are performed on the detected edges to fill the edge gaps, ultimately resulting in enhanced edges with clear outlines, continuity, and less noise interference.

[0093] S204. Fuse each illuminated region after edge enhancement to obtain the first enhanced underwater image data corresponding to the first underwater image data and the second enhanced underwater image data corresponding to the second underwater image data.

[0094] Spatial boundary calibration is performed on the strong light saturation area, weak light blur area and normal lighting area after edge enhancement. The pixel range of each area and the boundary line of adjacent areas are determined by the image coordinate system to ensure that the position of each area corresponds accurately in the original image coordinate system.

[0095] A weighted fusion strategy based on boundary transition is adopted to process the boundary zone of the region. A transition pixel band with a preset pixel width is set at the boundary of adjacent regions. The gray value of the pixels in the transition band is calculated according to the distance weighting principle. That is, the closer the pixel is to a certain enhancement region, the higher the weight of its gray value is given by the enhancement result of that region. This achieves a smooth transition of gray value and texture between adjacent regions and avoids obvious splicing marks. For pixels inside non-boundary regions, the processing results after edge enhancement of each region are directly retained to ensure that the details of the fish body edge are not lost.

[0096] After fusion, the grayscale continuity of the overall image is checked, and the grayscale difference between adjacent pixels is calculated. If the difference exceeds the preset threshold, the transition zone weighting coefficient is iteratively optimized until the overall image is visually consistent and there are no obvious regional discontinuities. Finally, the first enhanced underwater image data after the fusion of the first underwater image data and the second enhanced underwater image data after the fusion of the second underwater image data are output respectively.

[0097] S3. Based on the reflective properties of fish scales, fish identification is performed on the first enhanced underwater image data and the second enhanced underwater image data respectively, generating the first fish feature vector corresponding to the source domain data and the second fish feature vector corresponding to the target domain data;

[0098] It should be noted that although the first and second enhanced underwater image data have effectively enhanced the edges of the fish and suppressed light interference, it is still necessary to accurately distinguish the fish target from non-target objects such as bubbles and aquatic plants in the image. Fish scales, as a unique structure of fish, have reflective properties that are significantly different from other objects, including but not limited to high-frequency energy concentration and regular arrangement of reflective points. Therefore, targeted fish identification can be achieved based on these characteristics.

[0099] like Figure 3 As shown, this is a flowchart illustrating step S3. (Refer to...) Figure 3 Step S3 includes:

[0100] S301. Extract all reflective areas with a gray level higher than the background from the first enhanced underwater image data and the second enhanced underwater image data respectively by adaptive threshold segmentation;

[0101] Specifically, the first and second enhanced underwater image data are first preprocessed with Gaussian smoothing to eliminate high-frequency noise that may be introduced during the enhancement process and to avoid noise being misjudged as reflective points. Then, a sliding window (the window size is dynamically adjusted based on the average cross-sectional area of ​​the fish) is used to divide the image into several local sub-blocks. The mean and standard deviation of the background gray level of each sub-block are calculated as the basis for the calculation of the adaptive threshold. The background gray level is obtained by excluding the top 5% of pixels with the highest gray level in the sub-block to ensure that it is not affected by potential reflective points.

[0102] Furthermore, a dynamic threshold is set for each sub-block. ( This represents the average grayscale value of the background. Indicates the standard deviation of background grayscale. The adaptive coefficient is adjusted according to the overall brightness of the sub-block within the range of [1.2, 1.8] (with larger values ​​for bright areas and smaller values ​​for dark areas). Pixels with gray values ​​higher than the dynamic threshold within the sub-block are marked as reflective candidate pixels. Finally, connected component analysis is performed on all reflective candidate pixels, and adjacent candidate pixels are merged into continuous regions using the 8-neighborhood method to obtain all reflective regions with gray values ​​higher than the background in the first and second enhanced underwater image data.

[0103] S302. Perform frequency domain decomposition on each reflective region to obtain the energy ratio of the high-frequency component of the corresponding reflective region, and screen out all candidate fish scale reflective regions that meet the first fish scale reflective characteristics based on the energy ratio of the high-frequency component.

[0104] The first fish scale reflectivity characteristic includes a high-frequency component energy ratio exceeding a first threshold. Specifically, multi-scale wavelet transform is used to perform frequency domain decomposition on each reflectivity region to obtain the high-frequency component energy ratio of the corresponding reflectivity region, and all candidate fish scale reflectivity regions that meet the first fish scale reflectivity characteristic are selected based on the high-frequency component energy ratio.

[0105] For each reflective region, size standardization is first performed, followed by frequency domain decomposition using multi-scale wavelet transform. Specifically, a wavelet basis adapted to the details of fish scale texture is selected, and the reflective region signal is decomposed into low-frequency approximate components (reflecting the overall brightness trend of the region) and high-frequency detail components (reflecting subtle changes in scale edges and reflective points) through multi-level decomposition. For the multi-scale high-frequency characteristics of fish scale reflection caused by the scale layering structure, high-frequency sub-bands (including high-frequency variations in horizontal, vertical, and diagonal directions) are extracted from each decomposition layer. The ratio of the sum of the energy of all high-frequency sub-bands to the total energy of the reflective region is calculated to obtain the energy proportion of the high-frequency components. Further, the energy proportion of the high-frequency components is compared with a preset first threshold, and reflective regions with a proportion exceeding the first threshold are selected as candidate fish scale reflective regions that meet the first fish scale reflective characteristics. The first threshold is determined based on the high-frequency energy distribution characteristics of known fish scale reflective samples.

[0106] In this embodiment, the reflective area is uniformly cropped to 128×128 pixels, and a 3-layer multi-scale decomposition is performed using the db4 wavelet basis. Specifically, the first layer of high-frequency sub-bands reflects subtle reflective fluctuations at the 1-2 pixel scale, the second layer reflects scale edge changes at the 2-4 pixel scale, and the third layer reflects local reflective cluster features at the 4-8 pixel scale. Six high-frequency sub-bands are extracted from the 3-layer decomposition (each layer contains three directions: horizontal, vertical, and diagonal, and redundant sub-bands with an energy percentage <5%), and the ratio of their total energy to the total energy of the region is calculated, i.e., the high-frequency component energy percentage. Further, based on the statistical analysis of 200 sets of fish scale reflective samples (high-frequency percentage concentrated between 62% and 80%) and 150 sets of non-fish scale reflective samples (bubbles, aquatic plants, etc., with high-frequency percentage concentrated between 30% and 55%), the first threshold is set to 60%, and finally, reflective areas with a high-frequency component energy percentage >60% are selected as candidate fish scale reflective areas. As shown in Table 1, it reflects the energy ratio of high-frequency components of different reflective types and the determination results of the first fish scale reflective characteristics.

[0107] Table 1. Results of determining the energy percentage of high-frequency components and fish scale characteristics for different reflective types.

[0108]

[0109] S303. Analyze the arrangement pattern of reflective points in each candidate fish scale reflective area, and select all target fish scale reflective areas that meet the second fish scale reflective characteristics based on the arrangement pattern of reflective points.

[0110] The second fish scale reflectivity characteristic includes a reflectivity density exceeding a second threshold. Specifically, the reflectivity pattern of each candidate fish scale reflectivity region is analyzed based on spatial distribution entropy, and all target fish scale reflectivity regions that meet the second fish scale reflectivity characteristic are selected based on the reflectivity pattern.

[0111] For each candidate fish scale reflective region, independent reflective points are first extracted through binarization. Then, morphological denoising is used to remove isolated points with an area smaller than a single fish scale reflective unit, ensuring the purity of the reflective point data. Next, spatial distribution entropy is introduced to analyze the arrangement pattern of reflective points. The candidate fish scale reflective regions are divided into uniform grids, and the number of reflective points in each grid is counted. The spatial distribution entropy is calculated using the Shannon entropy formula; the lower the entropy value, the more ordered the reflective points are. It should be noted that because fish scales are arranged in overlapping / interlaced patterns, the distribution of reflective points is regular, and the entropy value is significantly lower than that of disordered reflective features such as bubbles and aquatic plants.

[0112] Simultaneously, the reflective point density (the number of reflective points per unit area) is calculated and compared with a preset second threshold. Spatial distribution entropy is used to assist in the judgment (excluding dense but disordered non-fish scale reflective areas). Finally, regions with reflective point density exceeding the second threshold and spatial distribution entropy conforming to the fish scale arrangement pattern are selected as target fish scale reflective regions that meet the second fish scale reflective characteristics. The second threshold is determined based on the statistical characteristics of the density of known fish scale samples.

[0113] In this embodiment, the candidate fish scale reflective areas are first uniformly cropped to 64×64 pixels. Otsu adaptive binarization is used to segment the reflective points, and the opening operation of 3×3 structuring elements is used to remove isolated noise points with an area of ​​less than 3 pixels. The region is divided into 64 uniform grids of 8×8 pixels, and the spatial distribution entropy is calculated using the Shannon entropy formula. The total number of reflective points is counted and converted into reflective point density.

[0114] Furthermore, based on statistical analysis of 200 sets of fish scale reflective samples and 150 sets of non-fish scale reflective samples, this embodiment sets the second threshold to 8 samples / mm. 2 Furthermore, the spatial distribution entropy is ≤1.5. Finally, regions that simultaneously meet both conditions are selected as target fish scale reflective regions. As shown in Table 2, it reflects the spatial distribution entropy, reflective point density, and second fish scale reflective characteristics determination results for different reflective types.

[0115] Table 2. Reflective characteristics of different reflective types and determination results of the reflective properties of the second fish scale.

[0116]

[0117] The unit for reflectivity density is units / mm. 2 It is understandable that bubble reflection does not meet the second fish scale reflection characteristic because its entropy value exceeds the standard and its density is insufficient, while aquatic plant reflection does not meet the second fish scale reflection characteristic because both its entropy value and density exceed the standard.

[0118] By analyzing the characteristic parameters of different reflective types, fish can be effectively identified. For example... Figure 4 The diagram shown illustrates the fish identification results of an embodiment of the present invention. (Refer to...) Figure 4 It clearly shows the underwater distribution of fish, bubbles, and aquatic plants.

[0119] S304. Perform feature matching on each target fish scale reflective area to generate the first fish feature vector corresponding to the source domain data and the second fish feature vector corresponding to the target domain data.

[0120] The system utilizes a pre-built database of known fish samples from the source domain. This database contains fish scale reflective templates corresponding to different fish species, and each template is labeled with characteristic benchmarks such as reflective intensity range, high-frequency component frequency distribution pattern, and reflective point arrangement entropy range. For each target fish scale reflective region, the mean reflective intensity, peak high-frequency component frequency, and reflective point arrangement entropy are extracted to construct a region-specific feature parameter matrix. This matrix is ​​then compared with the feature matrices of each fish template in the database to calculate similarity. The template with the highest similarity exceeding a preset matching threshold is selected to determine the fish species corresponding to the target region. The similarity calculation can use cosine similarity or Euclidean distance, and is not limited here.

[0121] The number of reflective areas of target fish scales corresponding to each fish species in the same image is counted. The fish species label, number of areas, average reflective intensity, peak frequency of high-frequency components, and entropy value of reflective point arrangement are correlated and integrated to finally generate the first fish feature vector of source domain data (covering the species, quantity and feature parameters of all fish in the source domain image) and the second fish feature vector of target domain data (containing the species, quantity and feature parameters of all fish in the target domain image).

[0122] S4. Using the feature vector of the first fish as a training sample, learn the correlation pattern between the feature vector of the first fish and the behavioral data of the first fish, and transfer the correlation pattern to the target domain data to complete the feature vector of the second fish.

[0123] Because target domain data often lacks fish behavior data, it's impossible to directly establish a correlation between the second fish feature vector and fish behavior. In contrast, source domain data already possesses complete first fish feature vectors and corresponding first fish behavior data. Therefore, transfer learning is needed to transfer the feature-behavior correlation patterns from the source domain to the target domain, thereby completing the second fish feature vector.

[0124] like Figure 5 As shown, this is a flowchart illustrating step S4. (Refer to...) Figure 5 Step S4 includes:

[0125] S401. Construct a transfer learning model that includes an attention mechanism, using the feature vector of the first fish as the input of the transfer learning model and the behavioral data of the first fish as the output of the transfer learning model, and learn the nonlinear mapping relationship between the feature vector of the first fish and the behavioral data of the first fish through the attention mechanism.

[0126] Specifically, a transfer learning model incorporating an attention mechanism is constructed, which consists of an input layer, a feature encoding layer, an attention interaction layer, a feature transformation layer, and an output layer.

[0127] The following is a detailed explanation of each layer of the transfer learning model:

[0128] 1) Input layer;

[0129] The input layer receives the first fish feature vector (containing multi-dimensional information such as fish species label, quantity, and reflective feature parameters). The embedding layer converts the discrete species label into a computable vector form, and at the same time, it standardizes the continuous feature parameters (such as the mean reflective intensity, high frequency peak, etc.) to unify the feature dimensions to adapt to subsequent network calculations.

[0130] 2) Feature coding layer;

[0131] The feature encoding layer employs a 3-layer fully connected network, using the ReLU activation function to perform a non-linear transformation on the input features, extracting a more abstract higher-order feature representation. Then, a multi-head self-attention mechanism module is connected. This module calculates the correlation weights of each dimension within the feature vector, dynamically assigning higher attention weights to feature dimensions that are strongly correlated with the behavioral data, suppressing interference from irrelevant or weakly correlated features, and thus focusing on key features.

[0132] 3) Feature transformation layer;

[0133] The feature transformation layer further compresses and fuses the attention-weighted feature vectors through a two-layer fully connected network, while introducing a Dropout layer to prevent overfitting. Finally, the output layer uses a linear activation function to output a prediction vector that matches the dimension of the first fish behavior data.

[0134] Furthermore, during the model training phase, a domain adaptation loss function is introduced to optimize the transfer learning model. This domain adaptation loss function is configured to minimize the feature distribution distance between the feature vectors of the first fish and the second fish.

[0135] The domain adaptation loss function is introduced to address the feature distribution shift between the source and target domains caused by differences in the underwater environment. If the model is trained solely on source domain data, the learned nonlinear mapping may fail to directly adapt to the target domain due to the inconsistency in feature distributions between the two domains, thus affecting the accuracy of subsequent completion of the second fish feature vector. Therefore, by incorporating the minimization of the feature distribution distance between the first fish feature vector (source domain) and the second fish feature vector (target domain) into the model optimization objective through the domain adaptation loss function, alignment of the feature spaces of the two domains can be achieved, improving the model's transferability.

[0136] The domain adaptation loss function uses the maximum mean difference as a measure of distribution distance because it can effectively measure the distribution difference between two sample sets in a high-dimensional feature space, without requiring prior assumptions about the data distribution, thus adapting to the multidimensionality of fish feature vectors. In the calculation process, the source domain feature subset (corresponding to the encoding result of the first fish feature vector) and the target domain feature subset (corresponding to the preliminary encoding result of the second fish feature vector, at which point the fish behavior data is not yet complete) are extracted from the feature vector output by the feature transformation layer. Then, the mean difference between the two subsets in the regeneration kernel Hilbert space is calculated using the maximum mean difference. The smaller this difference value, the closer the feature distributions of the two domains are.

[0137] In constructing the overall model loss function, the domain adaptation loss and the original behavior prediction loss (using mean squared error (MSE) to measure the deviation between the model's output behavior prediction vector and the true value of the first fish behavior data) are weighted and fused to form the total loss function. The behavior prediction loss weight (0.6-0.8 in this embodiment) ensures that the model prioritizes learning the core mapping relationship between features and behavior; the domain adaptation loss weight (0.2-0.4 in this embodiment) balances the distribution alignment requirement with task priority, and the domain adaptation loss weight can be dynamically adjusted with each training round. A larger value is used in the early stages of training to accelerate distribution alignment, and the value is gradually reduced in later stages to focus on prediction accuracy.

[0138] By driving the model iterative optimization through the total loss function, the attention mechanism can not only more accurately capture the nonlinear relationship between the first fish feature vector and the behavioral data, but also force the feature distributions of the source domain and the target domain to converge to the same space, ultimately obtaining a transfer learning model that balances accurate behavior mapping capability and cross-domain adaptation capability.

[0139] S402. Input the second fish feature vector into the optimized transfer learning model to perform feature adaptation and missing information completion, and generate a complete fish feature vector that is adapted to the fishing conditions corresponding to the target domain data.

[0140] In this context, feature adaptation is configured to adapt the nonlinear mapping relationship to the feature vector of the second fish species.

[0141] Specifically, for the input second fish feature vector, the discrete fish species labels in the feature vector are converted into a fixed-dimensional computable vector through the embedding layer. At the same time, the continuous reflective feature parameters are standardized to ensure that their feature dimensions and data format are completely consistent with the first fish feature vector during model training, thus meeting the model input requirements.

[0142] Further, in the feature adaptation stage, the optimized transfer learning model calls the parameters of the feature encoding layer and the attention interaction layer, adjusted by the domain adaptation loss function. On one hand, based on the previously learned source domain nonlinear mapping relationship (the correlation between the features and behavior of the first fish), the model extracts high-order features from the feature vector of the second fish through the feature encoding layer, automatically fine-tuning the encoding weights to adapt to the distribution characteristics of the target domain features. On the other hand, the attention mechanism continues its ability to focus on the feature-behavior correlation dimension, dynamically assigning high attention weights to dimensions in the second fish feature vector that are strongly correlated with behavioral data, suppressing weakly correlated features in the target domain caused by environmental interference, ensuring that the feature adaptation process accurately anchors the core correlation dimension, and achieving effective transfer of the nonlinear mapping relationship to the feature vector of the second fish.

[0143] After feature adaptation is complete, the model enters the missing information completion stage. Specifically, the feature transformation layer compresses and fuses the adapted high-order features, and combines them with the nonlinear mapping rules learned by the model to predict and calculate the missing fish behavior data in the second fish feature vector. At the same time, the model calls on the target domain environmental features learned in the domain adaptation stage to make reasonable corrections to the predicted behavior data, avoiding behavior parameters that exceed the physiological range of the fish, and ensuring that the completed information matches the actual fishing conditions in the target domain.

[0144] Finally, the complete feature vector is checked for completeness to confirm that the feature vector contains fish species labels, target area quantity, reflective feature parameters, and completed behavioral data, and that each parameter is logically consistent. For example, the behavioral data does not conflict with fish species or target domain environmental features. Finally, a complete fish feature vector that matches the fishing conditions corresponding to the target domain data is output.

[0145] S5. The completed second fish feature vector and second fish behavior data are spatiotemporally aligned, and the spatiotemporally aligned data and source domain data are integrated to obtain a multi-source fusion dataset of fish catch.

[0146] It should be noted that the completed second fish feature vector and the second fish behavior data have a spatiotemporal reference discrepancy. The former's timestamp corresponds to the image acquisition time and its spatial coordinates are camera pixel coordinates, while the latter's timestamp may come from the sensor sampling time and its spatial coordinates are geographic latitude and longitude. Therefore, it is necessary to eliminate the discrepancy through spatiotemporal alignment first.

[0147] Specifically, in the time alignment stage, the unified system time for the acquisition of target domain data is used as the benchmark. The image acquisition timestamp of the second fish feature vector and the sensor sampling timestamp of the second fish behavior data are extracted. Linear interpolation is used to complete the data whose timestamp interval exceeds the acquisition period. The average value of the overlapping timestamp data is taken and fused to ensure that the two types of data correspond one-to-one in the time dimension.

[0148] Furthermore, in the spatial alignment stage, a coordinate mapping relationship is established based on the calibration parameters of the target domain acquisition device. The pixel coordinates of the second fish feature vector are converted into actual geographic coordinates through camera intrinsic parameters (focal length, pixel size) and camera extrinsic parameters (installation position, pitch angle). At the same time, the sensor coordinates of the second fish behavior data are mapped to the same geographic coordinate system through a coordinate system transformation matrix (combining the relative positions of the sensor and the camera). This ensures that the feature data and behavior data of the same individual fish correspond to the same spatial location, avoiding data misalignment due to differences in device position.

[0149] After spatiotemporal alignment is completed, the data integration stage begins. Specifically, the data aligned to the target domain and the data from the source domain are first normalized to unify fish species codes, feature parameter units (such as grayscale values ​​of reflectivity intensity and frequency units of behavioral data), and spatiotemporal formats, eliminating format differences. Then, through data conflict detection, conflicting data are selected based on the principle of prioritizing the target domain (target domain data is more relevant to the current fishing conditions), and conflict-free historical data from the source domain is spliced ​​with real-time data from the target domain. Finally, a multi-source fusion dataset of fish catches is formed, containing historical feature-behavioral correlation data from the source domain and spatiotemporally aligned data from the target domain.

[0150] This invention discloses a method for generating a multi-source fusion dataset of fish catch based on transfer learning. Addressing the pain point of poor underwater image quality, it enhances edges based on the characteristics of incident light in the environment, accurately eliminating interference such as uneven lighting, overexposure, or low-light blurring. Combined with the reflective characteristics of fish scales, it enables targeted fish identification, effectively eliminating non-target reflective interference such as bubbles and aquatic plants, thus improving the accuracy of fish identification. By learning the correlation between fish features and behavioral data from source domain data and applying this correlation to the target domain through transfer learning, it accurately fills in the gaps in fish features in sparse fishing areas, effectively improving the reliability of data in sparse regions and solving the problem of incompleteness caused by uneven data distribution between fishing areas. Through spatiotemporal alignment and data integration, it merges the complete source domain data with the completed target domain data into a unified fish catch dataset, supporting high-precision fishery resource assessment, dynamic prediction of fishing conditions, and scientific management decisions, providing data support for digital management of fisheries.

[0151] like Figure 6 The diagram shown is a structural schematic of a fish catch multi-source fusion dataset generation system based on migration reconstruction, according to an embodiment of the present invention. (Refer to...) Figure 6An embodiment of the present invention provides a fish catch multi-source fusion dataset generation system based on migration reconstruction, comprising:

[0152] The data partitioning module 01 is used to divide the acquired fish catch observation data into source domain data and target domain data. The source domain data includes the first underwater image data and the first fish behavior data corresponding to the fishing area with abundant observation data, and the target domain data includes the second underwater image data and the second fish behavior data corresponding to the fishing area with sparse observation data.

[0153] Edge enhancement module 02 is used to perform edge enhancement on the first underwater image data and the second underwater image data based on the characteristics of ambient incident light, to obtain the first enhanced underwater image data and the second enhanced underwater image data.

[0154] Specifically, edge enhancement is performed on the first underwater image data and the second underwater image data based on the characteristics of the ambient incident light, respectively, to obtain the first enhanced underwater image data and the second enhanced underwater image data, including:

[0155] Extract the environmental incident light characteristic parameters from the source domain data and the target domain data respectively;

[0156] Based on the environmental incident light characteristic parameters, the first underwater image data and the second underwater image data are divided into several illumination areas respectively;

[0157] Edge enhancement is performed on each illuminated region in the first and second underwater image data respectively;

[0158] Each illuminated region after edge enhancement is fused to obtain first enhanced underwater image data corresponding to the first underwater image data and second enhanced underwater image data corresponding to the second underwater image data.

[0159] Furthermore, based on the characteristics of the ambient incident light, the first underwater image data and the second underwater image data are divided into several illumination regions, including:

[0160] The light intensity gradient distribution in the environmental incident light characteristic parameters is extracted, and the first underwater image data and the second underwater image data are divided into a strong light saturation area, a weak light blur area, and a normal lighting area based on the light intensity gradient distribution.

[0161] Furthermore, edge enhancement is performed on each illuminated area in the first underwater image data and the second underwater image data, including:

[0162] Adaptive threshold compression is used to suppress the light intensity in the strong light saturation region under the first underwater image data and the second underwater image data, respectively;

[0163] Top-hat transform is performed on the low-light blurred areas in the first and second underwater image data respectively;

[0164] Edge detection operators are used to enhance the edge contours of normally illuminated areas under the first and second underwater image data, respectively.

[0165] Fish recognition module 03 is used to perform fish recognition on the first enhanced underwater image data and the second enhanced underwater image data based on the reflective properties of fish scales, and generate a first fish feature vector corresponding to the source domain data and a second fish feature vector corresponding to the target domain data.

[0166] Specifically, based on the reflective properties of fish scales, fish identification is performed on the first enhanced underwater image data and the second enhanced underwater image data, respectively, generating a first fish feature vector corresponding to the source domain data and a second fish feature vector corresponding to the target domain data, including:

[0167] All reflective areas with a gray level higher than the background are extracted from the first and second enhanced underwater image data respectively using adaptive threshold segmentation.

[0168] Frequency domain decomposition is performed on each reflective region to obtain the high-frequency component energy ratio of the corresponding reflective region, and all candidate fish scale reflective regions that meet the first fish scale reflective characteristics are screened based on the high-frequency component energy ratio. The first fish scale reflective characteristics include the high-frequency component energy ratio exceeding the first threshold.

[0169] Analyze the arrangement pattern of reflective points in each candidate fish scale reflective area, and select all target fish scale reflective areas that meet the second fish scale reflective characteristics based on the reflective point arrangement pattern. The second fish scale reflective characteristics include a reflective point density exceeding a second threshold.

[0170] For each target fish scale reflective area, feature matching is performed to generate the first fish feature vector corresponding to the source domain data and the second fish feature vector corresponding to the target domain data.

[0171] Furthermore, frequency domain decomposition is performed on each reflective region to obtain the energy ratio of the high-frequency components of the corresponding reflective region. Based on the energy ratio of the high-frequency components, all candidate fish scale reflective regions that meet the first fish scale reflective characteristics are screened out, including:

[0172] Multi-scale wavelet transform is used to decompose each reflective region in the frequency domain to obtain the energy ratio of the high-frequency components of the corresponding reflective region. Based on the energy ratio of the high-frequency components, all candidate fish scale reflective regions that meet the first fish scale reflective characteristics are selected.

[0173] Furthermore, the arrangement pattern of reflective points in each candidate fish scale reflective region was analyzed, and all target fish scale reflective regions that meet the second fish scale reflective characteristics were selected based on the arrangement pattern of reflective points, including:

[0174] Based on the spatial distribution entropy analysis, the reflective point arrangement pattern of each candidate fish scale reflective area is analyzed, and all target fish scale reflective areas that meet the second fish scale reflective characteristics are selected according to the reflective point arrangement pattern.

[0175] The transfer completion module 04 is used to learn the correlation between the first fish feature vector and the first fish behavior data using the first fish feature vector as a training sample, and then transfer the correlation to the target domain data to complete the second fish feature vector.

[0176] Specifically, using the feature vector of the first fish species as training samples, the correlation between the feature vector and the behavioral data of the first fish species is learned, and this correlation is transferred to the target domain data to complete the feature vector of the second fish species, including:

[0177] A transfer learning model incorporating an attention mechanism is constructed. The first fish feature vector is used as the input of the transfer learning model, and the first fish behavior data is used as the output of the transfer learning model. The nonlinear mapping relationship between the first fish feature vector and the first fish behavior data is learned through the attention mechanism.

[0178] The second fish feature vector is input into the optimized transfer learning model for feature adaptation and missing information completion, generating a complete fish feature vector that matches the fishing conditions corresponding to the target domain data. The feature adaptation is configured to adapt the nonlinear mapping relationship to the second fish feature vector.

[0179] In this study, a domain adaptation loss function is introduced to optimize the transfer learning model. The domain adaptation loss function is configured to minimize the feature distribution distance between the feature vectors of the first fish and the feature vectors of the second fish.

[0180] Alignment and fusion module 05 is used to perform spatiotemporal alignment of the completed second fish feature vector and the second fish behavior data, and to integrate the spatiotemporally aligned data with the source domain data to obtain a multi-source fusion dataset of fish catch.

[0181] This invention discloses a fish catch multi-source fusion dataset generation system based on transfer learning and reconstruction. Addressing the pain point of poor underwater image quality, it enhances edges based on environmental incident light characteristics, accurately eliminating interference such as uneven lighting, overexposure, or low-light blurring. Combined with the reflective properties of fish scales, it enables targeted fish identification, effectively eliminating non-target reflective interference such as bubbles and aquatic plants, thus improving fish identification accuracy. By learning the correlation between fish features and behavioral data from source domain data and applying this correlation to the target domain through transfer learning, it accurately fills in the gaps in fish features in sparse fishing areas, effectively improving the reliability of data in sparse regions and solving the problem of data distortion caused by uneven data distribution between fishing areas. Through spatiotemporal alignment and data integration, it merges the complete source domain data with the supplemented target domain data into a unified fish catch dataset, supporting high-precision fishery resource assessment, dynamic prediction of fishing conditions, and scientific management decisions, providing data support for digital management of fisheries.

[0182] It should be noted that each module in the aforementioned fish catch multi-source fusion dataset generation system based on migration reconstruction can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module. For specific limitations regarding the fish catch multi-source fusion dataset generation system based on migration reconstruction, please refer to the limitations of the fish catch multi-source fusion dataset generation method based on migration reconstruction mentioned above; both have the same function and role, and will not be repeated here.

[0183] In summary, this invention provides a method and system for generating a multi-source fusion dataset of fish catch based on transfer learning. Addressing the pain point of poor underwater image quality, it enhances edges based on the characteristics of incident light in the environment, accurately eliminating interference such as uneven lighting, overexposure, or low-light blurring. Combined with the reflective characteristics of fish scales, it enables targeted fish identification, effectively eliminating non-target reflective interference such as bubbles and aquatic plants, thus improving the accuracy of fish identification. By learning the correlation between fish features and behavioral data from source domain data and applying this correlation to the target domain through transfer learning, it accurately fills in the gaps in fish features in sparse fishing areas, effectively improving the reliability of data in sparse regions and solving the problem of incompleteness caused by uneven data distribution between fishing areas. Through spatiotemporal alignment and data integration, it merges the complete source domain data with the completed target domain data into a unified fish catch dataset, supporting high-precision fishery resource assessment, dynamic prediction of fishing conditions, and scientific management decisions, providing data support for digital management of fisheries.

[0184] The various embodiments in this specification are described in a progressive manner. For directly identical or similar parts of the embodiments, refer to each other. Each embodiment focuses on its differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. It should be noted that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0185] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and substitutions can be made without departing from the technical principles of the present invention, and these improvements and substitutions should also be considered within the scope of protection of the present invention.

Claims

1. A method for generating multi-source fusion datasets of fish catch based on transfer learning and reconstruction, characterized in that, include: The acquired fish catch observation data is divided into source domain data and target domain data. The source domain data includes first underwater image data and first fish behavior data corresponding to fishing areas with abundant observation data. The target domain data includes second underwater image data and second fish behavior data corresponding to fishing areas with sparse observation data. Edge enhancement is performed on the first underwater image data and the second underwater image data based on the characteristics of the ambient incident light to obtain the first enhanced underwater image data and the second enhanced underwater image data. Based on the reflective properties of fish scales, fish identification is performed on the first enhanced underwater image data and the second enhanced underwater image data respectively, generating a first fish feature vector corresponding to the source domain data and a second fish feature vector corresponding to the target domain data; Using the first fish feature vector as a training sample, the association pattern between the first fish feature vector and the first fish behavior data is learned, and the association pattern is transferred to the target domain data to complete the second fish feature vector; The completed second fish feature vector and the second fish behavior data are spatiotemporally aligned, and the spatiotemporally aligned data is integrated with the source domain data to obtain a multi-source fusion dataset of fish catch. The step of using the first fish feature vector as a training sample, learning the correlation pattern between the first fish feature vector and the first fish behavior data, and transferring the correlation pattern to the target domain data to complete the second fish feature vector includes: A transfer learning model incorporating an attention mechanism is constructed, with the first fish feature vector as the input and the first fish behavior data as the output. The nonlinear mapping relationship between the first fish feature vector and the first fish behavior data is learned through the attention mechanism. The second fish feature vector is input into the optimized transfer learning model for feature adaptation and missing information completion, generating a complete fish feature vector that matches the fishing conditions corresponding to the target domain data. The feature adaptation is configured to adapt the nonlinear mapping relationship to the second fish feature vector.

2. The method for generating multi-source fusion datasets of fish catch based on migration reconstruction according to claim 1, characterized in that, The method of performing edge enhancement on the first underwater image data and the second underwater image data based on the characteristics of ambient incident light to obtain first enhanced underwater image data and second enhanced underwater image data includes: The environmental incident light characteristic parameters are extracted from the source domain data and the target domain data, respectively. Based on the environmental incident light characteristic parameters, the first underwater image data and the second underwater image data are divided into several illumination areas respectively; Edge enhancement is performed on each of the illuminated areas in the first underwater image data and the second underwater image data, respectively. Each illuminated region after edge enhancement is fused to obtain first enhanced underwater image data corresponding to the first underwater image data and second enhanced underwater image data corresponding to the second underwater image data.

3. The method for generating multi-source fusion datasets of fish catch based on migration reconstruction according to claim 2, characterized in that, The first underwater image data and the second underwater image data are divided into several illumination regions based on the environmental incident light characteristic parameters, including: Extract the light intensity gradient distribution from the environmental incident light characteristic parameters, and divide the first underwater image data and the second underwater image data into a strong light saturation area, a weak light blur area, and a normal lighting area based on the light intensity gradient distribution.

4. The method for generating multi-source fusion datasets of fish catch based on migration reconstruction according to claim 3, characterized in that, The step of performing edge enhancement on each of the illuminated regions in the first underwater image data and the second underwater image data includes: Based on adaptive threshold compression, the light intensity of the strong light saturation region in the first underwater image data and the second underwater image data is suppressed respectively; Top-hat transformation is performed on the low-light blurred areas in the first underwater image data and the second underwater image data, respectively; Edge detection operators are used to enhance the edge contours of the normally illuminated areas in the first underwater image data and the second underwater image data, respectively.

5. The method for generating multi-source fusion datasets of fish catch based on migration reconstruction according to claim 1, characterized in that, The step of performing fish identification on the first enhanced underwater image data and the second enhanced underwater image data based on the reflective properties of fish scales, and generating a first fish feature vector corresponding to the source domain data and a second fish feature vector corresponding to the target domain data, includes: All reflective areas with a gray level higher than the background are extracted from the first and second enhanced underwater image data respectively using adaptive threshold segmentation. Frequency domain decomposition is performed on each of the reflective regions to obtain the high-frequency component energy ratio of the corresponding reflective region, and all candidate fish scale reflective regions that meet the first fish scale reflective characteristics are screened based on the high-frequency component energy ratio, wherein the first fish scale reflective characteristics include the high-frequency component energy ratio exceeding a first threshold. Analyze the reflective point arrangement pattern of each candidate fish scale reflective region, and select all target fish scale reflective regions that meet the second fish scale reflective characteristics based on the reflective point arrangement pattern, wherein the second fish scale reflective characteristics include a reflective point density exceeding a second threshold. For each of the target fish scale reflective regions, feature matching is performed to generate a first fish feature vector corresponding to the source domain data and a second fish feature vector corresponding to the target domain data.

6. The method for generating multi-source fusion datasets of fish catch based on migration reconstruction according to claim 5, characterized in that, The step involves performing frequency domain decomposition on each reflective region to obtain the energy ratio of the high-frequency components corresponding to that reflective region, and then filtering out all candidate fish scale reflective regions that meet the first fish scale reflective characteristics based on the energy ratio of the high-frequency components, including: Multi-scale wavelet transform is used to perform frequency domain decomposition on each of the reflective regions to obtain the energy ratio of the high-frequency components of the corresponding reflective regions, and all candidate fish scale reflective regions that meet the first fish scale reflective characteristics are screened based on the energy ratio of the high-frequency components.

7. The method for generating multi-source fusion datasets of fish catch based on migration reconstruction according to claim 5, characterized in that, The analysis of the reflective point arrangement pattern of each candidate fish scale reflective region, and the selection of all target fish scale reflective regions that meet the second fish scale reflective characteristics based on the reflective point arrangement pattern, includes: Based on the spatial distribution entropy analysis, the reflective point arrangement pattern of each candidate fish scale reflective region is analyzed, and all target fish scale reflective regions that meet the second fish scale reflective characteristics are selected according to the reflective point arrangement pattern.

8. The method for generating multi-source fusion datasets of fish catch based on migration reconstruction according to claim 1, characterized in that, The transfer learning model is optimized by introducing a domain adaptation loss function, wherein the domain adaptation loss function is configured to minimize the feature distribution distance between the first fish feature vector and the second fish feature vector.

9. A system for generating multi-source fusion datasets of fish catches based on transfer reconstruction, characterized in that, include: The data partitioning module is used to divide the acquired fish catch observation data into source domain data and target domain data. The source domain data includes first underwater image data and first fish behavior data corresponding to fishing areas with abundant observation data, and the target domain data includes second underwater image data and second fish behavior data corresponding to fishing areas with sparse observation data. An edge enhancement module is used to perform edge enhancement on the first underwater image data and the second underwater image data based on the characteristics of ambient incident light, respectively, to obtain first enhanced underwater image data and second enhanced underwater image data. The fish identification module is used to identify fish in the first enhanced underwater image data and the second enhanced underwater image data based on the reflective properties of fish scales, and to generate a first fish feature vector corresponding to the source domain data and a second fish feature vector corresponding to the target domain data. The transfer completion module is used to learn the correlation pattern between the first fish feature vector and the first fish behavior data using the first fish feature vector as a training sample, and transfer the correlation pattern to the target domain data to complete the second fish feature vector; The alignment and fusion module is used to perform spatiotemporal alignment of the completed second fish feature vector and the second fish behavior data, and to integrate the spatiotemporally aligned data with the source domain data to obtain a multi-source fusion dataset of fish catch; The step of using the first fish feature vector as a training sample, learning the correlation pattern between the first fish feature vector and the first fish behavior data, and transferring the correlation pattern to the target domain data to complete the second fish feature vector includes: A transfer learning model incorporating an attention mechanism is constructed, with the first fish feature vector as the input and the first fish behavior data as the output. The nonlinear mapping relationship between the first fish feature vector and the first fish behavior data is learned through the attention mechanism. The second fish feature vector is input into the optimized transfer learning model for feature adaptation and missing information completion, generating a complete fish feature vector that matches the fishing conditions corresponding to the target domain data. The feature adaptation is configured to adapt the nonlinear mapping relationship to the second fish feature vector.

Images