AI-based camera-based system for acquiring catch information on fishing boats using light-covered nets.

Abstract

This invention relates to the field of fishing vessel monitoring technology, specifically to a catch information acquisition system for fishing vessels using light-covered nets based on AI cameras. The system includes: a reflective point extraction module, a trajectory fitting module, an occlusion classification module, an edge redrawing module, and a density analysis module. In this invention, reflective points on the surface of underwater fish are extracted and high-brightness areas are selected to construct a set of gloss response points. The movement trajectory of the reflective point group is analyzed using a multi-frame image sequence. A random sampling consensus algorithm is used to fit the path features, identifying the continuity of the trajectory and changes in the movement pattern. An occlusion state classification is performed on the path features based on a support vector machine model, determining whether it is extended occlusion, cross occlusion, or boundary detachment. Based on the classification results, the edge contours of the reflective points are aligned with direction vectors and redrawn using path matching to restore the contours lost due to occlusion. By analyzing the brightness change trajectory of the fin region in the redrawn edge path, multi-granular data support is provided for catch statistics.

Description

Technical Field

[0001] This invention relates to the field of fishing vessel monitoring technology, and in particular to a system for acquiring catch information of fishing vessels using light-covered nets based on AI cameras. Background Technology

[0002] The field of fishing vessel monitoring technology encompasses the technical applications of collecting, recording, and managing the operational status, environmental information, and operational data of fishing vessels during operations. The core of this technology includes the monitoring and analysis of information such as vessel location, speed, operating time, operating area, and the type and quantity of catch, typically achieved through integrated collaboration of communication equipment, sensing devices, and information platforms.

[0003] Among them, the catch information acquisition system for light-net fishing vessels refers to a system that automatically collects data on the type, weight, and time of catches during the light-net operation by integrating video image recognition equipment, weighing sensors, and time recording devices.

[0004] Existing technologies rely on video image recognition equipment and weighing sensors to directly collect data. In dynamic underwater environments, these technologies are susceptible to interference from factors such as light reflection and mutual occlusion caused by densely swimming fish, leading to decreased image recognition accuracy. For example, when fish change their swimming posture or overlap, traditional image recognition struggles to accurately segment individual outlines, causing species identification errors. Simultaneously, weighing sensors are prone to data drift in the swaying environment of fishing vessels, making it impossible to distinguish the weight proportions of different fish species in a single catch. Time-recording devices only record the fishing period, lacking the ability to dynamically monitor the spatial distribution and density of the catch. Furthermore, existing systems underutilize the reflective features of fish surfaces and fail to establish a correlation analysis between the movement trajectories of reflective points and fish behavior patterns, making it difficult to address image blurring caused by turbid water or rapid swimming in complex underwater scenarios. These problems result in shortcomings in practical applications, including limited statistical dimensions of catch information, weak adaptability to dynamic scenes, and low accuracy in identifying dense fish schools, affecting the comprehensiveness of fisheries resource assessment. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies and propose a system for acquiring catch information of fishing vessels using light-covered nets based on AI cameras.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: A system for acquiring catch information on a fishing vessel using a light-covered net based on an AI camera includes:

[0007] The reflective point extraction module uses a camera to capture image frame sequences of fishing boats operating under the light cover net underwater, and performs reflective point filtering on the bright areas of the fish surface in the image frame sequence to obtain a set of gloss response points on the fish surface;

[0008] The trajectory fitting module estimates the motion trajectory of the fish body reflective point group in the image frame sequence of the fish body surface gloss response point set, and constructs a fish body path fitting feature sequence;

[0009] The occlusion classification module uses a support vector machine model to determine the occlusion status of the fish body path fitted feature sequence image and forms a fish body occlusion type classification result.

[0010] The edge redrawing module refers to the fish body occlusion type classification result to redraw the edge contour of the fish body surface gloss response point set and outputs the redrawn fish body edge path set.

[0011] The density analysis module obtains the brightness change trajectory group of the corresponding fin region in the bright area of ​​the fish surface based on the redrawn fish body edge path set, performs time series analysis on the gray value of the corresponding position in the image frame sequence, and outputs the local density information of the fish species.

[0012] As a further aspect of the present invention, the set of gloss response points on the fish surface includes the number of reflective points, spatial distribution characteristics, and sorting position information; the fish path fitting feature sequence includes trajectory continuity status, motion pattern change type, and image frame correlation; the fish occlusion type classification result specifically includes extended occlusion, cross occlusion, and boundary detachment; the redrawn fish edge path set includes edge contour morphology, path coverage area, and position fitting result; and the local density information of the fish species includes the number of fin response trajectory groups, gray value change trend, and frame sequence time characteristics.

[0013] As a further aspect of the present invention, the reflective point extraction module includes:

[0014] The image frame acquisition submodule acquires image frame sequences of fishing boats operating under the light net through a camera, extracts grayscale channel information of the fish surface in each image, and obtains image frame sequence grayscale dataset;

[0015] The bright area recognition submodule filters areas whose brightness exceeds twice the standard deviation of the image mean based on the brightness of each pixel in the grayscale dataset of the image frame sequence, locates the bright pixel group on the surface of the fish in the image, and obtains a set of candidate bright area points.

[0016] The reflective point screening submodule sorts the candidate points in the bright area in descending order of brightness, extracts the top five pixels with the highest brightness values ​​to construct a point set, and obtains the gloss response point set on the fish surface.

[0017] As a further aspect of the present invention, the trajectory fitting module includes:

[0018] The trajectory path estimation submodule extracts the coordinate change sequence of reflective points at the same position in the image frame sequence based on the set of gloss response points on the fish surface, and uses the RANSAC random sampling consensus algorithm to fit the spatial trajectory of gloss points on the fish surface in multiple frames of images to obtain the set of fitted paths for gloss points on the fish.

[0019] The abnormal segment identification submodule identifies trajectory segments whose fitting errors exceed the consistency threshold for multiple consecutive frames based on the inter-frame fitting residual of each trajectory in the fish body gloss point fitting path set, marks the corresponding segments as path interruption areas, and obtains the trajectory interruption segment identification group.

[0020] The feature sequence construction submodule performs normalization calculations on the change amplitude of the motion direction of the reflective points before and after the interruption point of the trajectory interruption segment identifier group, the inter-frame displacement gradient value, and the response point distribution density function value, and integrates them to form a fish body path fitting feature sequence.

[0021] As a further aspect of the present invention, the occlusion classification module includes:

[0022] The input vector construction submodule fits the feature sequence of the fish path, obtains the change of orientation angle between adjacent frames in the trajectory of the gloss points on the fish surface, the average displacement of each trajectory in the frame sequence, and the degree of curvature change of the trajectory line segment, and combines the three parameters to construct the input dataset and generate a path feature input vector group.

[0023] The occlusion state recognition submodule inputs the trajectory feature vector set into the support vector machine model, uses the training boundary to perform sample classification processing on the input data, identifies the occlusion state of the gloss point trajectory in the image frame sequence, and obtains the fish body occlusion state recognition result.

[0024] The classification result generation submodule assigns three classification labels to the identified areas in the image based on the fish occlusion status recognition results: single fish extension, fish intersection, or boundary detachment, and outputs the fish occlusion type classification results.

[0025] As a further aspect of the present invention, the edge redrawing module includes:

[0026] The occlusion area filtering module calls the fish body occlusion type classification result, filters the image area with occlusion type of fish body intersection, and extracts all points in the fish body surface gloss response point set that are located in the area to obtain the response point group of the intersection occlusion area.

[0027] The contour path matching submodule obtains an image contour trajectory template for the response point group in the cross-occlusion area, calculates the cosine value of the direction vector between the template path and the current response point group, selects the path with a direction cosine angle difference less than the specified range of the template as the reference path, performs direction vector alignment and length parameter scaling, and generates a candidate contour path set.

[0028] The edge coverage drawing submodule calculates the ratio of the overlap area between each path and the current response point group in spatial coordinates based on the candidate contour path set, filters the paths with an overlap ratio greater than a specified reference value, performs contour redrawing of the target path in the image edge region, and outputs the redrawn fish body edge path set.

[0029] As a further aspect of the present invention, the density analysis module includes:

[0030] The fin region extraction submodule identifies points with fin position features among the gloss points on the fish surface based on the redrawn fish body edge path set, extracts the pixel coordinates of the corresponding points in the image frame sequence between consecutive frames, and obtains the brightness trajectory point group of the fin region.

[0031] The brightness trajectory analysis submodule performs time-series statistics on the grayscale changes of each point in the brightness trajectory point group of the fin region in the frame sequence, calculates three indicators for each trajectory between consecutive frames: the grayscale fluctuation frequency, the overlap ratio of the amplitude peak, and the consistency value of the brightness change direction, and generates a set of brightness change feature indicators.

[0032] The density information generation submodule performs weighted processing on the three indicators in the brightness change feature index set, merges and evaluates the weighted results, and outputs the local density information of the fish species.

[0033] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0034] In this invention, reflective points on the surface of underwater fish are extracted and high-brightness areas are selected to construct a set of gloss response points. The movement trajectory of the reflective point group is analyzed using a multi-frame image sequence. A random sampling consensus algorithm is used to fit the path features, identifying the continuity of the trajectory and changes in movement patterns. An occlusion state classification is performed on the path features based on a support vector machine model, determining whether it is extended occlusion, cross occlusion, or boundary detachment. Based on the classification results, the edge contours of the reflective points are aligned with direction vectors and redrawn using path matching to restore the contours lost due to occlusion. By analyzing the brightness change trajectory of the fin region in the redrawn edge path, time-series statistics are performed on the grayscale fluctuation frequency, peak amplitude overlap ratio, and consistency of brightness change direction. Combined with weighted evaluation, local fish density information is generated. This process enhances the accuracy of fish contour recognition in dynamic underwater environments, reduces interference from uneven lighting and motion blur, improves target segmentation stability in cross-occlusion scenarios, and achieves accurate differentiation of densely distributed multi-fish scenes. By integrating trajectory fitting and occlusion compensation mechanisms, this method overcomes the dependence of traditional methods on static image analysis, enhances adaptability to complex motion scenes, and deepens the fish species identification dimension by combining gray-scale temporal features of fin regions, providing multi-granular data support for catch statistics. Attached Figure Description

[0035] Figure 1 This is a system flowchart of the present invention;

[0036] Figure 2 This is a flowchart of the reflective point extraction module of the present invention;

[0037] Figure 3 This is a flowchart of the trajectory fitting module of the present invention;

[0038] Figure 4 This is a flowchart of the occlusion classification module of the present invention;

[0039] Figure 5 This is a flowchart of the edge redrawing module of the present invention;

[0040] Figure 6 This is a flowchart of the density analysis module of the present invention. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0042] In the description of this invention, it should be understood that the terms "length," "width," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships, are based on the orientation or positional relationships shown in the accompanying drawings and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, in the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0043] Please see Figure 1 The catch information acquisition system for fishing vessels using AI cameras and light-covered nets includes:

[0044] The reflective point extraction module uses a camera to capture image frame sequences of fishing boats operating under the light cover net underwater, and performs reflective point filtering on the bright areas of the fish surface in the image frame sequence to obtain a set of gloss response points on the fish surface;

[0045] The trajectory fitting module estimates the motion trajectory of the fish body reflective point group in the image frame sequence of the fish body surface gloss response point set, and constructs the fish body path fitting feature sequence;

[0046] The occlusion classification module uses a support vector machine model to determine the occlusion status of the fish path fitted feature sequence image and generate a fish occlusion type classification result.

[0047] The edge redrawing module redraws the edge contour of the fish body surface gloss response point set based on the fish body occlusion type classification results, and outputs the redrawn fish body edge path set.

[0048] The density analysis module obtains the brightness change trajectory group of the corresponding fin region in the bright area of ​​the fish body surface based on the set of redrawn fish body edge paths, performs temporal analysis on the gray value of the corresponding position in the image frame sequence, and outputs the local density information of the fish species.

[0049] The set of gloss response points on the fish surface includes the number of reflective points, spatial distribution characteristics, and sorting position information. The fish path fitting feature sequence includes trajectory continuity, motion pattern change type, and image frame correlation. The fish occlusion type classification results are specifically extended occlusion, cross occlusion, and boundary detachment. The redrawn fish edge path set includes edge contour morphology, path coverage area, and position fitting results. The local density information of the fish species includes the number of fin response trajectory groups, gray value change trend, and frame sequence time characteristics.

[0050] Please see Figure 2 The reflective point extraction module includes:

[0051] The image frame acquisition submodule acquires image frame sequences of fishing boats operating under the light net through a camera, extracts grayscale channel information of the fish surface in each image, and obtains image frame sequence grayscale dataset;

[0052] The system captures image frames of fishing boats operating under the light-covered nets underwater using a camera. In actual operation scenarios, a waterproof high-definition camera is fixedly installed on the bottom or side of the boat, with its shooting angle directly facing the area affected by the light. A high-speed image acquisition card achieves an image acquisition frequency of 25 frames per second. After 30 seconds of continuous recording, a sequence of 750 images is obtained. For each image, the green channel of its RGB three channels is extracted as the basis for grayscale channel information. The average value is calculated pixel by pixel to convert it into a grayscale image. If the RGB value of a certain frame is [(40, 50, 55), (48, 58, 60), (52, 65, 70)], then the corresponding grayscale value after conversion is [48, 55, 62]. After all frames have been converted to grayscale, they form an image frame grayscale dataset. This dataset is stored in a three-dimensional array format. The first dimension is the frame number (1 to 750), the second dimension is the image height (e.g., 480), and the third dimension is the image width (e.g., 640). This completes the process of image frame acquisition and grayscale dataset construction.

[0053] The bright area recognition submodule filters areas whose brightness exceeds twice the standard deviation of the image mean based on the brightness of each pixel in the grayscale dataset of the image frame sequence, locates the bright pixel group on the surface of the fish in the image, and obtains a set of candidate bright areas.

[0054] The image frame sequence grayscale dataset is filtered based on the brightness of each pixel. In practice, this involves first statistically analyzing the pixel values ​​of each frame in the entire grayscale dataset frame by frame. For example, in frame 235, the grayscale matrix is ​​480×640, and the total brightness after pixel summation is 7,526,400. The total number of pixels in this frame is 307,200, so the average brightness is 24.5. The grayscale value set is then processed using the standard deviation formula. Assuming the standard deviation of this frame is 7.2, then double the standard deviation... With a standard deviation of 14.4, the brightness screening threshold is 24.5 + 14.4 = 38.9. Pixels with a brightness greater than or equal to 39 in the frame are retained and recorded as bright pixels according to their coordinates. After processing all frames in this way, the high-brightness points in each frame form a set of bright candidate points for the frame sequence. Considering system memory optimization, only a maximum of 1000 high-brightness pixels are retained in each frame. If the number of bright spots in a frame exceeds the threshold, they are sorted in descending order of grayscale value, and the first 1000 points are retained to finally construct the overall candidate point set.

[0055] The reflective point filtering submodule sorts the candidate points in the bright area in descending order of brightness, extracts the top five pixels with the highest brightness values ​​to construct a point set, and obtains the gloss response point set on the fish surface.

[0056] The reflective point filtering submodule sorts the candidate points in the bright area in descending order of brightness. In the specific implementation, for the bright point set of each frame, the gray value of each pixel is first compared. A fast sorting algorithm is used to sort the gray values ​​from high to low. After sorting, the coordinates and gray values ​​of the top five pixels are recorded as response points. For example, if the gray value sequence of candidate points in frame 121 is [55, 52, 49, 49, 48, 47, 45], then the gray values ​​of the top five points are [55, 52, 49, 49, 48], and their corresponding coordinates are [(120, 310), (98, 305), (102, 295), (88, 288), (135, 270)]. After obtaining the response point set of this frame, the next frame is processed. Finally, the response points of each frame in 750 frames are merged to form the gloss response point set of the fish surface.

[0057] Please see Figure 3 The trajectory fitting module includes:

[0058] The trajectory path estimation submodule is based on the set of gloss response points on the fish surface. It extracts the coordinate change sequence of the reflective point at the same position in the image frame sequence, and uses the RANSAC random sampling consensus algorithm to fit the spatial trajectory of the gloss points on the fish surface in multiple frames to obtain the set of fitted paths for the gloss points.

[0059] Based on the set of gloss response points on the fish surface, in practice, five gloss response points extracted from each frame are used as the basis. The coordinates of each response point are extracted sequentially across 750 frames. For example, if the coordinates of a gloss point in frame 100 are (320, 240), and it moves to (323, 242) in frame 101, then the coordinates of that point change to (3, 2). Each gloss response point forms a sequence trajectory in the time dimension. By comparing the continuity and consistency of the spatial coordinates of the gloss points in each frame, the change sequence is recorded for path fitting. During the fitting process, the RANSAC random sampling consensus method is used to iteratively sample the trajectory point set. Two sets of coordinate points are randomly selected from the frame sequence each time as the fitting starting point. A hypothetical path is constructed based on a three-dimensional straight line model, and the remaining points are substituted into the model to calculate the fitting error. The error is defined as the shortest Euclidean distance between the gloss point and the hypothetical path. To enhance the robustness of the algorithm to low-quality frames, a confidence weight w for the fitting points is introduced. i The following improved formula for weighted average of fitted residuals is used to evaluate path quality:

[0060]

[0061] The parameters are defined as follows: ε: weighted residual mean, reflecting the overall error of path fitting, in pixels; i: frame number of the response point in the path, ranging from 1 to n; n: number of gloss response points in a single trajectory, such as n = 30 if each path covers 30 frames; P i : The actual pixel coordinates of the fish's gloss response points in the i-th frame image, in the form of a two-dimensional vector (x i ,y i (), in pixels; The coordinates of the point mapped by the current fitted path at frame number i are in the form of a two-dimensional vector. The unit is pixels; w i The confidence weight of the response point in the i-th frame is calculated from the normalized grayscale value using the formula w. i =G i / 255, where G i The value is a grayscale value, ranging from 0 to 255.

[0062] For example, assuming the Euclidean distances between the actual and fitted points of a certain 5-frame response point in a 30-frame path segment are [3.2, 2.1, 4.5, 3.8, 2.6], corresponding to grayscale values ​​of [210, 185, 190, 220, 200], then the normalized weights are [0.824, 0.725, 0.745, 0.863, 0.784]. The squared distances multiplied by the weights are [8.467, 3.196, 15.059, 12.458, 5.317], the weight sum is 3.941, the total weighted squared residual sum is 44.497, and the final weighted residual mean is: This value indicates that the residual of this path segment is relatively high. If the judgment threshold is set to 10 pixels, this segment is not a well-fitted path. The RANSAC algorithm will exclude this path combination based on the error result and resample. It will filter out path segments with residuals less than the set threshold from all trajectory segments and output the path segments. The output includes the path start and end frame numbers, the start coordinates, the path direction vector, and the residual value. Finally, the trajectory fitting path set of the fish body gloss points is obtained.

[0063] The abnormal segment identification submodule identifies trajectory segments whose fitting errors exceed the consistency threshold for multiple consecutive frames based on the inter-frame fitting residual of each trajectory in the fish body gloss point fitting path set, marks the corresponding segments as path interruption areas, and obtains the trajectory interruption segment identification group.

[0064] The analysis is based on the inter-frame fitting residuals of each trajectory in the fitted path set of fish body gloss points. In practice, the residual variation amplitude between adjacent frames is calculated for each fitted path, and the residual time series is extracted on a continuous frame basis. For example, for a certain trajectory, the residual sequence in its corresponding 30-frame path segment is: [2.5, 3.1, 3.4, 3.6, 4.0, 10.2, 11.8, 12.5, 11.2, 4.8, 3.9]. It can be observed that there is a residual jump from frame 6 to frame 9. To determine whether it constitutes an abnormal segment, the mean residual of continuous frames within this time period needs to be calculated and compared with a consistency threshold. The threshold setting method refers to the mean difference between the maximum value and the median of the residuals in the effective path. If the maximum residual value in the effective trajectory is 6.5 and the median is 3.5, then... The consistency threshold is set to θ = 3.5 + (6.5 - 3.5) / 2 = 5.0. The mean of the residual sequence in this segment is (10.2 + 11.8 + 12.5 + 11.2) / 4 = 11.425, which is significantly higher than the consistency threshold. Therefore, frames 6 to 9 are marked as abnormal path segments, and the start and end frame numbers are added to the trajectory interruption segment identifier group. If concentrated abnormal positions appear in multiple trajectory paths, such as frames 225 to 228 being multiple path abnormal overlapping segments, they are further treated as key segments for subsequent feature analysis. Each segment identifier group is recorded in the form of frame segment number, such as [(225, 228), (315, 319)]. The system supports real-time updating of the identifier group and its one-to-one correspondence with the original trajectory path set, which facilitates the subsequent module to identify response point features.

[0065] The feature sequence construction submodule performs normalization calculations on the change in motion direction of reflective points before and after the interruption point of the trajectory interruption segment identifier group, the inter-frame displacement gradient value, and the response point distribution density function value, and integrates them to form a fish path fitting feature sequence.

[0066] The motion direction change amplitude, inter-frame displacement gradient value, and response point distribution density function value of the reflective points before and after the interruption point of the trajectory interruption segment marker group are normalized and calculated. In the specific execution process, the coordinate sequence of the reflective points of the three frames before and after each abnormal segment is first extracted, and their unit time direction vector difference is calculated. For example, if the three points in frames 222 to 224 before the interruption are [(320, 240), (323, 244), (327, 248)], and the point in frame 229 after the interruption is [(320, 240), (323, 244), (327, 248)], then the coordinate sequence of the reflective points in frames 229 after the interruption is calculated. Frame 231 is [(332, 255), (336, 259), (339, 262)]. The forward direction unit vector is [(3, 4), (4, 4)], which, after normalization, becomes [(0.6, 0.8), (0.707, 0.707)]. The subsequent direction vector is [(4, 4), (3, 3)], which, after normalization, becomes [(0.707, 0.707), (0.707, 0.707)]. The direction difference is calculated using the vector angle remainder. The difference coefficient sequence is obtained using the chord formula; next, the inter-frame displacement gradient is calculated, and a gradient sequence is constructed based on the Euclidean distance difference between adjacent frame points, for example, [5.0, 5.4, 6.1] is transformed into [0.4, 0.7]; then, the number of gloss points per unit pixel area in each frame is extracted, and the 480×640 image is divided into a 30×40 grid, with each grid area being 16×16 pixels. The point density in each grid is calculated, and a density function value sequence is constructed. The difference between the maximum density and the mean density is normalized to a ratio range of [0, 1]; finally, the three types of indicators are unified and normalized to the same scale before the sequence is integrated. The integration method is to take three feature values ​​from each frame as three-dimensional vectors and assemble them to form a feature vector sequence, which is output in array form, such as: [(0.62, 0.35, 0.58), (0.68, 0.38, 0.63), ...], where each triplet represents the direction difference, displacement gradient, and density ratio respectively, constituting the final fish path fitting feature sequence.

[0067] Please see Figure 4 The occlusion classification module includes:

[0068] The input vector construction submodule is based on the fish path fitting feature sequence to obtain the direction angle change between adjacent frames in the trajectory of the gloss points on the fish surface, the average displacement of each trajectory in the frame sequence, and the curvature change of the trajectory line segment. The three parameters are combined to construct the input dataset and generate the path feature input vector group.

[0069] Based on the fish path fitting feature sequence, the direction vector change, average displacement, and curvature change of the trajectory line segment in consecutive frames are extracted during the specific execution. These are used as three types of feature parameters for combined input. The direction angle change is calculated using the angle between unit vectors between two frames. For example, if the movement direction of the reflective point on the fish surface in frames 300 and 301 is (3, 4) and (5, 4), then the corresponding unit vectors are (0.6, 0.8) and (0.780, 0.625). The angle is calculated using cosine similarity, with the cosine value being cosθ = 0.6·0.780 + 0.8·0.625 = 0.468 + 0.500 = 0.968. Therefore, the direction angle change is calculated as arccos(0.968) ≈ 14.5°. The average displacement of each trajectory in the 30-frame sequence is obtained by summing the Euclidean distances between adjacent frame points and taking the average. For example... A total of 29 displacements are represented by [4.0, 4.2, 4.3, 4.5, ...], with an average value of 4.3 pixels. The curvature change is obtained by constructing an angle change sequence from the broken line segments formed every three frames. For example, two vectors [(3,3), (4,3)] are formed by (320, 240), (323, 243), and (327, 246), with unit vectors of (0.707, 0.707) and (0.8, 0.6), and the included angle is arccos(0.707·0.8+0.707·0.6)=arccos(0.989)≈8.1°. This process generates the curvature change sequence between points in the entire trajectory. The three dimensions are summarized according to the frame order to form the input feature matrix. Each row of vectors has the structure of [direction angle change, average displacement, curvature change], for example: [14.5, 4.3, 8.1]. The path feature input vector is established accordingly.

[0070] The occlusion state recognition submodule inputs the trajectory feature vector set into the support vector machine model, uses the training boundary to perform sample classification processing on the input data, identifies the occlusion state of the gloss point trajectory in the image frame sequence, and obtains the occlusion state recognition result of the fish body.

[0071] The trajectory feature vector set is input into the support vector machine model. In actual recognition, the pre-trained support vector set and boundary function are used to distinguish the input sample points. The predicted label of the input feature point is calculated based on its distance from the segmentation hyperplane, and it is determined whether it belongs to the fish body extension, crossing, or boundary detachment state. The classification function is represented by the SVM decision function as follows:

[0072]

[0073] The parameters are defined as follows: x: the current input path feature vector, which is a three-dimensional real vector, and the components represent the change in orientation angle (unit: degrees), the average inter-frame displacement (unit: pixels), and the degree of curvature change (unit: degrees); The i1th support vector is the important sample point retained in the training set after model training for constructing the classification boundary. Its structure is the same as x, and it is represented as a three-dimensional real vector, i1∈{1,2,...,m}. The true classification label corresponding to the i1th support vector takes a value of +1 or -1, which is used to characterize the occlusion state type of the sample. For example, +1 represents "single fish body extension", and -1 represents "fish body crossing" or "boundary detachment". The Lagrange multiplier of the i1th support vector is a parameter obtained through optimization during training. It represents the weight contribution of the support vector in the model and is usually a positive real number. Kernel function, used to calculate the relationship between input vector x and support vectors. The nonlinear similarity between them is represented here using radial basis functions (RBF), which are calculated as follows: Where γ is the kernel width parameter. represents the Euclidean distance; b: the bias constant term obtained from model training, used to adjust the discrimination boundary to better fit the actual distribution of training samples; m: the total number of support vectors, which is the size of the feature vector subset selected during training, generally much smaller than the total number of original samples; sign(): the sign function, used to judge the positive or negative of the overall summation value, returning +1 or -1 as the final predicted classification label.

[0074] For example, if the current input vector is x = [14.5, 4.3, 8.1], the three support vectors in the training model are: x1 = [13.0, 4.1, 7.5], y1 = +1, α1 = 0.9; x2 = [17.2, 4.5, 9.0], y2 = -1, α2 = 0.6; x3 = [10.1, 3.8, 6.5], y3 = +1, α3 = 0.7.

[0075] When calculating the kernel function value for each support vector, the kernel function parameter γ is set to 0.5, and the squared Euclidean distance is: ||x-x1|| 2 =(14.5-13.0) 2 +(4.3-4.1) 2 +(8.1-7.5) 2 =2.25 + 0.04 + 0.36 = 2.65, therefore K1 = exp(-0.5·2.65) = exp(-1.325) ≈ 0.266; ||x-x2|| 2 =(14.5-17.2) 2 +(4.3-4.5) 2 +(8.1-9.0) 2=7.29 + 0.04 + 0.81 = 8.14, therefore K2 = exp(-0.5·8.14) = exp(-4.07) ≈ 0.017; ||x-x3|| 2 =(14.5-10.1) 2 +(4.3-3.8) 2 +(8.1-6.5) 2 =19.36+0.25+2.56=22.17, so K3=exp(-0.5·22.17)=exp(-11.085)≈0.000015.

[0076] Substituting the support vector weights and classification labels, we get:

[0077] f(x)=sign(0.9·1·0.266+0.6·(-1)·0.017+0.7·1·0.000015-0.3)=sign(0.2394-0.0102+0.0000105-0.3)=sign(-0.0708)=-1.

[0078] The final classification result is 1, corresponding to the "fish body crossing" state. The identification label will be fed back to the subsequent classification module for image region labeling.

[0079] The classification result generation submodule assigns three classification labels to the identified regions in the image based on the fish occlusion status recognition results: single fish body extension, fish body intersection, or boundary detachment, and outputs the fish occlusion type classification results.

[0080] Based on the fish occlusion status recognition results, the recognized areas are marked in the image frame sequence and assigned corresponding classification label values ​​such as "single fish body extension", "fish body intersection", or "boundary detachment". During the execution process, the occlusion status sequence is first read, and the position of each trajectory result is mapped frame by frame. An occlusion mark layer is established in the original image coordinate system, and the corresponding trajectory point area is filled with the mark. For example, if trajectory A is identified as "boundary detachment" in frames 340 to 370, a separation mark is added to the corresponding trajectory point position in that frame segment. The mark data is stored in the form of numbers or masks, such as [frame number, trajectory number, occlusion type] = [340, A1, 3]. All classification outputs are output in JSON structure or frame index file format.

[0081] Please see Figure 5 The edge redrawing module includes:

[0082] The occlusion area filtering module calls the fish occlusion type classification results, filters the image region with the occlusion type of fish body intersection, and extracts all points in the fish body surface gloss response point set that are located in this region to obtain the response point group of the intersection occlusion region.

[0083] The system retrieves the fish occlusion type classification results, reads each frame in the image frame sequence, establishes a mapping relationship between frame number and path number, compares the occlusion classification label of each path, and extracts the path segment number marked as "fish body intersection". An occlusion frame sequence table is generated internally. After filtering, the system performs region-based extraction, marking a rectangular region in the grayscale matrix of each frame image based on the path coordinate range. For example, in frame 480, there is an intersection occlusion path A3, and its response point coordinate distribution range is [(310, 260), (312, 261), (315, 261 ... 63), (318, 265)], then construct the extraction window with the upper left point (310, 260) and the lower right point (318, 265). The corresponding area has a pixel width of 9 and a height of 6. The coordinates of all response points in the extraction window form the intra-frame cross-occlusion point set. Perform the above extraction operation on each frame in sequence. During the extraction process, remove the duplicate coordinate points and record them with unified numbers. After the point set extraction is completed for all cross-occlusion image frames under all paths, it is summarized into a three-dimensional array structure. The first dimension is the path number, the second dimension is the frame sequence number, and the third dimension is the two-dimensional coordinate value of the response point, forming the response point group of the cross-occlusion area.

[0084] The contour path matching submodule obtains the image contour trajectory template for the response point group in the cross-occlusion area, calculates the cosine value of the direction vector between the template path and the current response point group, selects the path with a direction cosine angle difference less than the specified range of the template as the reference path, performs direction vector alignment and length parameter scaling, and generates a candidate contour path set.

[0085] For processing response point groups in cross-occlusion areas, the standard image contour trajectory template database is first invoked. Contour templates selected from the database must meet the following requirements: the number of point sets must be no less than 80% of the response point group, the number of direction vectors must be no less than 20, and the template path format must be an ordered coordinate chain. These are connected sequentially to form a direction vector sequence. For each response point path, a direction vector is generated based on the coordinate difference between its adjacent frames. For example, the point sequence [(310, 260), (313, 263), (316, 266)] corresponds to the direction vector [(3, 3), (3, 3)]. After normalization, a unit vector sequence is obtained. The cosine value is then calculated by pairing the template direction sequence with the current path direction vector group. For the direction vector (3, 3) and the template direction vector (4, 2), the cosine value is calculated as follows:

[0086]

[0087] In all matching direction pairs, if the cosine angle difference (calculated by the inverse cosine function) is less than the set range, for example, set to 20°, that is, the corresponding cosine value is greater than 0.94, then it is considered a qualified match, and the template path is recorded as a reference path. Then, the path is scaled by the length parameter, with the current path length divided by the template path length as the scaling factor. For example, if the response path length is 27 pixels and the template path length is 30 pixels, then the scaling factor is 0.9. After adjusting all coordinates of the template path proportionally according to the scaling factor, the starting point coordinates are aligned to the starting point of the response path, thus completing the direction alignment and scale normalization of the matching path. All successfully matched paths are stored in the candidate contour path set.

[0088] The edge coverage drawing submodule calculates the ratio of the overlap area between each path and the current response point group in spatial coordinates based on the candidate contour path set, filters the paths with an overlap ratio greater than a specified reference value, performs contour redrawing of the target path in the image edge region, and outputs the redrawn fish body edge path set.

[0089] Image mapping is performed based on the candidate contour path set. First, each candidate path is converted into a set of pixels in the image coordinate system. The ratio of the pixel overlap area between the candidate path and the current cross-occlusion response point group is calculated. The ratio is calculated based on the ratio of the intersection area to the total area of ​​the response point set. For example, if the total number of response point pixels is 45 and the overlapping area of ​​the candidate path mapping area is 38 pixels, then the overlap area ratio is 38 / 45≈0.844. It is then determined whether this value is greater than a specified benchmark value. The benchmark value is set based on the minimum effective coverage rate. For example, if it is set to 0.80, then the path meets the overlap condition and is retained as an edge path. Otherwise, if the ratio is less than the benchmark value, the path is discarded and not included in the drawing. After filtering, the remaining paths are drawn using image overlay processing. The path coordinates are mapped onto the edge layer according to the image frame order. The line segment width is set to 2 pixels and the color is set to neutral gray. Finally, the drawing layer coordinate file or image format file is output to form a set of fish body edge redrawing paths.

[0090] Please see Figure 6 The density analysis module includes:

[0091] The fin region extraction submodule is based on the redrawn fish body edge path set, identifies points with fin position features among the gloss points on the fish body surface, extracts the pixel coordinates of such points in the image frame sequence between consecutive frames, and obtains the brightness trajectory point group of the fin region.

[0092] Based on the set of redrawn fish body edge paths, the coordinates of reflective points around the redrawn paths are analyzed frame by frame in the image frame sequence. Combining the dense distribution characteristics, geometric positioning characteristics, and inter-frame motion stability of reflective points, it is determined whether they have fin position features. The specific judgment criteria are as follows: if there are bright spots with brightness values ​​greater than 1.5 times the image average in a 16×16 pixel area around a certain point in 5 consecutive frames, and the coordinate changes of these points do not exceed 3 pixels, then the point is marked as a suspected fin feature point. For example, the bright spot located at (328, 244) in frame 422 has a maximum coordinate offset of 2 pixels in frames 423 to 426, with corresponding gray values ​​of [210, 215, 220, 218, 212] and an average image gray value of 100. Then, the point is determined to meet the fin feature condition. All points that meet the condition are numbered according to the frame sequence to construct a trajectory chain. The pixel coordinates of each point are extracted from the trajectory chain and integrated into multiple sets of trajectory paths to form a fin region brightness trajectory point group.

[0093] The brightness trajectory analysis submodule performs temporal statistics on the grayscale changes of each point in the brightness trajectory point group of the fin region in the frame sequence, calculates three indicators for each trajectory between consecutive frames: the grayscale fluctuation frequency, the overlap ratio of the amplitude peak, and the consistency value of the brightness change direction, and generates a set of brightness change feature indicators.

[0094] Statistical analysis was performed on the grayscale changes of each point in the brightness trajectory point group of the fin region in the frame sequence. The grayscale sequence of consecutive frames was extracted, and three indicators were calculated. First, the grayscale fluctuation frequency was calculated using the following formula: Where: h: is the length of the luminance trajectory, in frames; i2: frame sequence number subscript, traversing the interval from 1 to h-1; The grayscale value of the i2th frame; T: grayscale transition threshold, set to 15; δ(): judgment function, returns 1 if true, returns 0 if false. When the grayscale sequence is [210, 215, 190, 195, 220, 185], the inter-frame difference is [5, 25, 5, 25, 35], of which 3 are greater than the threshold T = 15, h = 6, calculated as follows: Secondly, the peak overlap ratio is calculated using the following formula: Where: N co : The number of other tracks with the same grayscale peak frame number at the peak frame position of this track; N total The total number of tracks currently participating in the overlap statistics; if the current track has a maximum grayscale value of 220 in the 5th frame, and there are 3 other tracks that also have their maximum grayscale values ​​in the 5th frame, and the total number of tracks participating in the comparison is 5, then: Finally, the formula for calculating the consistency value of brightness change direction is: Where: L maxh-1: The length of the longest segment with consecutive identical grayscale differences; h-1: The total number of difference segments; The grayscale difference is [5, -25, 5, 25, -35], the symbol sequence is [+1, -1, +1, +1, -1], the longest consistent segment is [+1, +1], and its length is 2, then: In summary, the three indicators for the current point under a trajectory length of 6 frames are: frequency Freq = 0.6, overlap ratio PeakRatio = 0.6, and consistency Cons = 0.4, which constitute the complete brightness change feature vector of this point [0.6, 0.6, 0.4].

[0095] The density information generation submodule performs weighted processing on three indicators in the brightness change feature index set, merges and evaluates the weighted results, and outputs local density information of fish species.

[0096] Weighted processing was performed on the brightness change feature index set. Before setting the weights, the discriminative contribution of different indicators in dense structure recognition needed to be evaluated. The evaluation method was to calculate the Pearson correlation coefficient between each of the three indicators and the artificial density annotation level in the pre-sampled sample set. In 300 trajectory samples, the correlation coefficient between grayscale change frequency and artificial density level was 0.78, the peak overlap ratio was 0.65, and the brightness change consistency was 0.52. Using the normalized result of the correlation coefficient as the basis for weighting, the three coefficients were normalized to a sum of 1, resulting in the following ratio: To simplify calculations and maintain discriminative power, the adjusted weights were set to 0.4, 0.35, and 0.25. During weighted calculation, the three indicator vectors [Freq, PeakRatio, Cons] for each point were multiplied by their respective weights and then summed to obtain the local density value for that point. Taking a trajectory point feature index of [0.6, 0.6, 0.4] as an example, the calculation is as follows: Density point =0.6×0.4+0.6×0.35+0.4×0.25=0.55. Next, all trajectory points in each frame of the image are merged and summarized, and the image region is divided into a grid (e.g., 32×32 pixels per grid). In each grid, the density value of all points located within that grid is counted, and the average value is used to obtain the grid density, which is expressed as: Where: MeanDensity: the average density value of this grid area, used to characterize the local concentration of fish activity in this area; k: the number of brightness trajectory points in this grid; The weighted density value of the i3th trajectory point, in dimensionless real number, ranges from [0, 1]; i3: trajectory point index number, ranging from 1 to k; if there are 6 trajectory points in a grid, and their weighted density values ​​are [0.45, 0.50, 0.62, 0.55, 0.48, 0.57], then:

[0097]

[0098] Finally, all grid density values ​​are classified into intervals, with the classification criteria set as: high density (MeanDensity≥0.60), medium density (0.40≤MeanDensity<0.60), and low density (MeanDensity<0.40). According to the above results, 0.528 belongs to the medium density interval, and is recorded as this grid density level in a two-dimensional coordinate matrix. The system outputs all grid coordinates and corresponding levels as the final local density information dataset of fish species.

[0099] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. An AI camera-based light cover net fishing boat catch information acquisition system, characterized by, The system comprises: The light reflection point extraction module acquires a sequence of image frames under the water surface of the fishing net of the fishing vessel through a camera, performs light reflection point screening on the bright area of the fish body surface in the sequence of image frames, and obtains a set of fish body surface gloss response points; The trajectory fitting module estimates the motion trajectory of the fish body light reflection point group in the set of fish body surface gloss response point image frames, and constructs a fish body path fitting feature sequence; The occlusion classification module uses a support vector machine model to judge the occlusion state of the fish body path fitting feature sequence image, and forms a fish body occlusion type classification result; The edge redrawing module refers to the fish body occlusion type classification result to redraw the edge contour of the set of fish body surface gloss response points, and outputs a set of redrawing fish body edge paths; The density analysis module obtains the brightness change trajectory group of the corresponding fin strip area in the bright area of the fish body surface based on the set of redrawing fish body edge paths, performs time series analysis on the gray value of the corresponding position in the image frame, and outputs fish local density information; The set of fish body surface gloss response points includes the number of light reflection points, spatial distribution characteristics, and sorting position information, the fish body path fitting feature sequence includes trajectory continuity state, motion mode change type, and image frame correlation, the fish body occlusion type classification result is specifically an extension occlusion, a cross occlusion, and a boundary separation, the set of redrawing fish body edge paths includes edge contour shape, path coverage area, and position fitting result, and the fish local density information includes fin strip response trajectory group number, gray value change trend, and frame sequence time characteristics; The trajectory fitting module comprises: The trajectory path estimation submodule extracts the coordinate change sequence of the light reflection points at the same position in the image frame sequence based on the set of fish body surface gloss response points, uses the RANSAC random sample consensus algorithm to fit the spatial trajectory of the fish body surface gloss points in multiple image frames, and obtains a set of fish body gloss point fitting paths; The abnormal segment identification submodule identifies the trajectory segment whose fitting error exceeds the consistency threshold in a plurality of continuous frames according to the inter-frame fitting residual of each trajectory in the set of fish body gloss point fitting paths, marks the corresponding segment as a path interruption area, and obtains a set of trajectory interruption segment identifiers; The feature sequence construction submodule performs normalization calculation on the motion direction change amplitude, inter-frame displacement gradient value, and response point distribution density function value of the light reflection points before and after the breakpoints in the set of trajectory interruption segment identifiers, and integrates to form a fish body path fitting feature sequence; The edge redrawing module comprises: The occlusion area screening submodule calls the fish body occlusion type classification result, screens the image area of the fish body cross occlusion type, extracts all point sets in the set of fish body surface gloss response points located in the area, and obtains a cross occlusion area response point group; The contour path matching submodule obtains an image contour trajectory template for the cross occlusion area response point group, calculates the direction vector cosine value of the template path and the current response point group, selects the path with a direction cosine angle difference less than the specified range of the template as a reference path, performs direction vector alignment and length parameter scaling, and generates a candidate contour path set; The edge coverage drawing sub-module calculates an overlapping area ratio of each path with the current response point group in spatial coordinates based on the candidate contour path set, screens paths with an overlapping ratio greater than a specified reference value, performs contour redrawing of the target path in the image edge region, and outputs a redrawing fish body edge path set. 2.The AI camera-based light cover net fishing boat's catch information acquisition system according to claim 1, wherein The light reflection point extraction module comprises: The image frame acquisition sub-module acquires a sequence of image frames under the water surface of the fishing boat operating with the light shield net through a camera, extracts the gray channel information of the fish body surface in each image, and obtains a gray data set of the sequence of image frames; The bright area identification sub-module screens an area with a brightness greater than twice the standard deviation of the image mean value according to the brightness of each pixel in the gray data set of the sequence of image frames, locates a group of bright pixel points on the fish body surface in the image, and obtains a candidate point set of bright areas; The light reflection point screening sub-module arranges the candidate point set of bright areas in descending order of brightness, extracts the top five pixel points in brightness value to construct a point set, and obtains a set of fish body surface gloss response points. 3.The AI camera-based light cover net fishing boat's catch information acquisition system according to claim 1, characterized in that, The occlusion classification module comprises: The input vector construction sub-module obtains the direction angle change between adjacent frames, the average displacement amount of each trajectory in the frame sequence, and the curvature change degree of the trajectory segment in the fish body path fitting feature sequence, combines the three parameters to construct an input data set, and generates a path feature input vector group; The occlusion state recognition sub-module inputs the trajectory feature vector set into a support vector machine model, performs sample classification processing on the input data using a trained boundary, recognizes the occlusion state of the gloss point trajectory in the sequence of image frames, and obtains a fish body occlusion state recognition result; The classification result generation sub-module assigns three classification labels of single fish body extension, fish body intersection, or boundary separation to the recognized area in the image according to the fish body occlusion state recognition result, and outputs a fish body occlusion type classification result. 4.The AI camera-based light cover net fishing boat's catch information acquisition system according to claim 1, wherein, The density analysis module comprises: The fin strip region extraction sub-module identifies points with fin strip position characteristics among the fish body surface gloss points based on the redrawing fish body edge path set, extracts the pixel coordinates of the corresponding points in the sequence of image frames between consecutive frames, and obtains a fin strip region brightness trajectory point group; The brightness trajectory analysis sub-module performs time series statistics on the gray scale change of each point in the fin strip region brightness trajectory point group in the frame sequence, calculates three indexes of the gray scale fluctuation frequency, the peak value of the amplitude, and the consistency value of the brightness change direction of each trajectory between consecutive frames, and generates a brightness change feature index set; The density information generation sub-module performs weighted processing on the three indexes in the brightness change feature index set, merges and evaluates the weighted results, and outputs local density information of the fish species.

Images