AI camera-based fishing information acquisition system of lamplight cover net fishing boat

Abstract

The invention relates to the technical field of fishing boat monitoring, in particular to an AI camera-based fishing information acquisition system for a lamplight cover net fishing boat, which comprises a reflective point extraction module, a track fitting module, a shielding classification module, an edge redrawing module and a density analysis module. The method comprises the following steps: extracting reflective points on the surface of an underwater fish body, screening a high-brightness area, constructing a gloss response point set, analyzing a motion track of a reflective point group in combination with a multi-frame image sequence, fitting path features by adopting a random sampling consistency algorithm, and identifying a track continuity state and a motion mode change. And performing shielding state classification on the path features based on a support vector machine model, judging extension shielding, cross shielding or boundary separation types, performing direction vector alignment and path matching redrawing on the edge contours of the reflective points according to a classification result, and recovering contour missing caused by shielding. And multi-granularity data support is provided for fish catch statistics by analyzing a brightness change track of a fin ray region in a redrawn edge path.

Description

Technical Field

[0001] The present invention relates to the technical field of fishing vessel monitoring, and in particular to a fishing catch information acquisition system for a light-covered net fishing vessel based on an AI camera. Background Art

[0002] Fishing vessel monitoring encompasses the collection, recording, and management of operational status, environmental information, and operational data from fishing vessels during operations. Core aspects of this technology include monitoring and analyzing vessel location, speed, operating time, operating area, and catch type and quantity, typically through integrated communication equipment, sensors, and information platforms.

[0003] Among them, the fishing catch information acquisition system of the light-net fishing vessel refers to a system that automatically collects the type, weight and time data of the fish caught by the fishing vessel during the light-net operation by integrating video image recognition equipment, weighing sensors and time recording devices.

[0004] Existing technologies rely on direct data collection from video image recognition equipment and weighing sensors. These systems are susceptible to interference in dynamic underwater environments, such as light reflections and occlusion caused by densely swimming fish, resulting in reduced image recognition accuracy. For example, when fish bodies change their swimming posture or overlap, traditional image recognition struggles to accurately segment individual outlines, leading to species identification errors. Furthermore, weighing sensors are subject to data drift in the swaying environment of fishing vessels, making it impossible to distinguish the weight contribution of different fish species in a single catch. Time recording devices only record the duration of the catch and lack the ability to dynamically monitor the spatial distribution and density of the catch. Furthermore, existing systems underutilize the reflective characteristics of fish surfaces and fail to correlate the motion trajectories of reflective points with fish behavior patterns. This makes it difficult to address image blurring caused by turbid water or rapid swimming in complex underwater scenarios. These issues result in existing technologies suffering from limitations in practical applications, such as a single statistical dimension for catch information, poor adaptability to dynamic scenarios, and low accuracy in identifying dense fish schools, impacting the comprehensiveness of fishery resource assessments. Summary of the Invention

[0005] The purpose of the present invention is to solve the shortcomings of the prior art and propose a fishing information acquisition system for a light-covered net fishing vessel based on an AI camera.

[0006] In order to achieve the above-mentioned purpose, the present invention adopts the following technical solution: A fishing catch information acquisition system for a light-covered net fishing boat based on an AI camera comprises:

[0007] The reflection point extraction module uses a camera to collect a sequence of image frames of a light-covered fishing boat operating under the water surface, and performs reflection point screening on the bright areas on the surface of the fish in the image frame sequence to obtain a set of gloss response points on the surface of the fish;

[0008] The trajectory fitting module estimates the motion trajectory of the fish body reflective point group in the image frame sequence of the gloss response point set on the fish body surface, and constructs a fish body path fitting feature sequence;

[0009] The occlusion classification module uses a support vector machine model to judge the occlusion state of the fish body path fitting feature sequence image to form a fish body occlusion type classification result;

[0010] The edge redrawing module redraws the edge contour of the gloss response point set on the surface of the fish body according to the fish body occlusion type classification result, and outputs a redrawn fish body edge path set;

[0011] The density analysis module obtains the brightness change trajectory group of the corresponding fin area in the bright area on the surface of the fish body based on the redrawn fish edge path set, performs time series analysis on the grayscale values ​​of the corresponding positions in the image frame sequence, and outputs the local density information of the fish species.

[0012] As a further solution of the present invention, the set of gloss response points on the fish surface includes the number of reflective points, spatial distribution characteristics, and sorting position information; the fish path fitting feature sequence includes trajectory continuity status, motion mode change type, and image frame association relationship; the fish occlusion type classification results are specifically extended occlusion, cross occlusion, and boundary detachment; the redrawn fish edge path set includes edge contour morphology, path coverage area, and position fitting results; the local density information of the fish species includes the number of fin response trajectory groups, grayscale value change trend, and frame sequence time characteristics.

[0013] As a further solution of the present invention, the reflection point extraction module includes:

[0014] The image frame acquisition submodule uses a camera to collect image frame sequences of the light-covered net fishing boat operating under the water surface, extracts the grayscale channel information of the fish body surface in each frame of the image, and obtains the image frame sequence grayscale data set;

[0015] The bright light area recognition submodule selects areas whose brightness exceeds two standard deviations of the image mean based on the brightness of each pixel in the grayscale dataset of the image frame sequence, locates the bright light pixel point group on the surface of the fish body in the image, and obtains a set of candidate points of the bright light area;

[0016] The reflective point screening submodule arranges the candidate point set in descending order of brightness from the bright area, extracts the top five pixel points with the highest brightness values ​​to construct a point set, and obtains a gloss response point set on the fish body surface.

[0017] As a further solution of the present invention, the trajectory fitting module includes:

[0018] The trajectory path estimation submodule extracts the coordinate change sequence of the reflective point at the same position in the image frame sequence based on the set of gloss response points on the fish body surface, and uses the RANSAC random sampling consensus algorithm to perform path fitting on the spatial trajectory of the gloss points on the fish body surface in multiple frames of images to obtain the set of fitted paths of the gloss points on the fish body;

[0019] The abnormal segment identification submodule identifies the trajectory segments whose fitting errors of consecutive frames exceed the consistency threshold based on the inter-frame fitting residual of each trajectory in the fish body gloss point fitting path set, marks the corresponding segments as path interruption areas, and obtains the trajectory interruption segment identification group;

[0020] The feature sequence construction submodule normalizes and calculates the movement direction change amplitude, inter-frame displacement gradient value and response point distribution density function value of the reflective points before and after the interruption point of the trajectory interruption segment identification group, and integrates them to form a fish body path fitting feature sequence.

[0021] As a further solution of the present invention, the occlusion classification module includes:

[0022] The input vector construction submodule obtains the angular change between adjacent frames in the glossy point trajectory of the fish body surface, the average displacement of each trajectory in the frame sequence, and the curvature change of the trajectory segment based on the fish body path fitting feature sequence, and combines these three parameters to construct an input data set to generate a path feature input vector group;

[0023] The occlusion state recognition submodule inputs the trajectory feature vector set into the support vector machine model, performs sample classification processing on the input data using the training boundary, identifies the occlusion state to which the glossy point trajectory in the image frame sequence belongs, and obtains the fish body occlusion state recognition result;

[0024] The classification result generation submodule assigns three classification labels of single fish body extension, fish body intersection or boundary separation to the identified area in the image based on the fish body occlusion state recognition result, and outputs the fish body occlusion type classification result.

[0025] As a further solution of the present invention, the edge redrawing module includes:

[0026] The occlusion area screening submodule calls the fish body occlusion type classification result, screens the image area where the occlusion type is fish body intersection, and correspondingly extracts all point sets located in the area from the fish body surface gloss response point set to obtain the intersection occlusion area response point group;

[0027] The contour path matching submodule obtains the image contour trajectory template for the response point group in the cross-occlusion area, calculates the direction vector cosine value of the template path and the current response point group, selects the path whose direction cosine angle difference is less than the template specified range as the reference path, performs direction vector alignment and length parameter scaling, and generates a candidate contour path set;

[0028] The edge coverage drawing submodule calculates the overlapping area ratio of each path and the current response point group in the spatial coordinates based on the candidate contour path set, screens the paths whose overlapping ratio is greater than the specified reference value, performs contour redrawing of the target path in the edge area of ​​the image, and outputs the redrawn fish body edge path set.

[0029] As a further solution of the present invention, the density analysis module includes:

[0030] The fin ray region extraction submodule identifies points with fin ray position characteristics among the glossy points on the fish body surface based on the redrawn fish body edge path set, extracts the pixel coordinates of the corresponding points between consecutive frames in the image frame sequence, and obtains the fin ray region brightness trajectory point group;

[0031] The brightness trajectory analysis submodule performs time series statistics on the grayscale changes of each point in the brightness trajectory point group of the fin area in the frame sequence, calculates three indicators of each trajectory between consecutive frames: grayscale fluctuation frequency, amplitude peak overlap ratio, and brightness change direction consistency value, and generates a brightness change feature indicator set;

[0032] The density information generation submodule performs weighted processing on the three indicators in the brightness change characteristic index set, merges and evaluates the weighted results, and outputs the local density information of the fish species.

[0033] Compared with the prior art, the advantages and positive effects of the present invention are:

[0034] In the present invention, by extracting reflective points on the surface of underwater fish and screening high-brightness areas, a set of gloss response points is constructed, and the motion trajectory of the reflective point group is analyzed in combination with a multi-frame image sequence. The random sampling consistency algorithm is used to fit the path features, and the trajectory continuity state and motion mode changes are identified. Based on the support vector machine model, the path features are classified into occlusion states, and the extended occlusion, cross occlusion or boundary separation type is determined. According to the classification results, the direction vector alignment and path matching of the edge contour of the reflective point are redrawn to restore the contour loss caused by occlusion. By analyzing the brightness change trajectory of the fin area in the redrawn edge path, the grayscale fluctuation frequency, the amplitude peak overlap ratio and the brightness change direction consistency are time-series statistics, and the local density information of the fish species is generated in combination with weighted evaluation. The above process enhances the accuracy of fish contour recognition in a dynamic underwater environment, reduces the interference of uneven lighting and motion blur, improves the stability of target segmentation in cross-occlusion scenarios, and realizes accurate distinction of scenes with dense distribution of multiple fish bodies. By integrating trajectory fitting and occlusion compensation mechanisms, we overcome the traditional method's reliance on static image analysis and enhance its adaptability to complex motion scenes. At the same time, we combine the grayscale temporal characteristics of the fin area to deepen the dimension of fish species identification and provide multi-granularity data support for catch statistics. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 is a system flow chart of the present invention;

[0036] Figure 2 This is a flow chart of the reflective point extraction module of the present invention;

[0037] Figure 3 This is a flow chart of the trajectory fitting module of the present invention;

[0038] Figure 4 This is a flow chart of the occlusion classification module of the present invention;

[0039] Figure 5 is a flow chart of the edge redrawing module of the present invention;

[0040] Figure 6 This is a flow chart of the density analysis module of the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0042] In the description of the present invention, it should be understood that the terms "length," "width," "up," "down," "front," "back," "left," "right," "vertical," "horizontal," "top," "bottom," "inside," "outside," and the like, indicating positions or relationships, are based on the positions or relationships shown in the accompanying drawings and are intended only to facilitate the description of the present invention and simplify the description. They do not indicate or imply that the devices or elements referred to must have a specific orientation, be constructed, or operate in a specific orientation. Therefore, they should not be construed as limiting the present invention. Furthermore, in the description of the present invention, "plurality" means two or more, unless otherwise expressly and specifically defined.

[0043] See also Figure 1 The fishing information acquisition system for light-covered net fishing vessels based on AI cameras includes:

[0044] The reflection point extraction module uses a camera to collect a sequence of image frames of a light-covered fishing boat operating under the water surface, and performs reflection point screening on the bright areas on the surface of the fish in the image frame sequence to obtain a set of gloss response points on the surface of the fish;

[0045] The trajectory fitting module estimates the motion trajectory of the fish's reflective point group in the image frame sequence of the fish's surface gloss response point set, and constructs the fish's path fitting feature sequence;

[0046] The occlusion classification module uses the support vector machine model to judge the occlusion status of the fish path fitting feature sequence image and form the fish occlusion type classification result;

[0047] The edge redrawing module redraws the edge contour of the fish body surface gloss response point set based on the fish body occlusion type classification result, and outputs the redrawn fish body edge path set;

[0048] The density analysis module obtains the brightness change trajectory group of the corresponding fin area in the bright area of ​​the fish body surface based on the redrawn fish edge path set, performs time series analysis on the grayscale values ​​of the corresponding positions in the image frame sequence, and outputs the local density information of the fish species;

[0049] The set of gloss response points on the fish surface includes the number of reflective points, spatial distribution characteristics, and sorting position information. The fish path fitting feature sequence includes the trajectory continuity status, motion mode change type, and image frame association relationship. The fish occlusion type classification results are specifically extended occlusion, cross occlusion, and boundary detachment. The redrawn fish edge path set includes edge contour morphology, path coverage area, and position fitting results. The local density information of fish species includes the number of fin response trajectory groups, grayscale value change trend, and frame sequence time characteristics.

[0050] See also Figure 2 , the reflection point extraction module includes:

[0051] The image frame acquisition submodule uses a camera to collect image frame sequences of the light-covered net fishing boat operating under the water surface, extracts the grayscale channel information of the fish body surface in each frame of the image, and obtains the image frame sequence grayscale data set;

[0052] A camera is used to capture a sequence of underwater image frames of a fishing boat operating under a light-covered net. In the actual operation scenario, a waterproof high-definition camera is fixed on the bottom or side of the hull so that its shooting angle faces the area under the water where the light is applied. A high-speed image acquisition card is used to achieve an image acquisition frequency of 25 frames per second. After 30 seconds of continuous recording, a sequence of 750 frames of images is obtained. For each frame of image, the green channel of its RGB three channels is extracted as the basis for the grayscale channel information. The image is converted into a grayscale image by calculating the average value pixel by pixel. If the RGB value of a frame is [(40, 50, 55), (48, 58, 60), (52, 65, 70)], the corresponding grayscale value after conversion is [48, 55, 62]. After all frames are grayscale converted, an image frame grayscale dataset is formed. The dataset is stored in a three-dimensional array format, with the first dimension being the frame number (1 to 750), the second dimension being the image height (e.g., 480), and the third dimension being the image width (e.g., 640). This completes the process of image frame acquisition and grayscale dataset construction.

[0053] The bright light area recognition submodule selects the areas whose brightness exceeds twice the standard deviation of the image mean based on the brightness of each pixel in the grayscale dataset of the image frame sequence, locates the bright light pixel point group on the surface of the fish in the image, and obtains the set of candidate points of the bright light area;

[0054] According to the brightness of each pixel in the grayscale data set of the image frame sequence, in the specific implementation, all pixel values ​​of each frame of the image in the entire grayscale data set must be counted frame by frame. For example, the grayscale matrix in the 235th frame is 480×640, and the total brightness value after pixel summation is 7,526,400. The total number of pixels in this frame is 307,200, so the average brightness of the image is 24.5. The standard deviation formula is used to process the grayscale value set. Assuming that the standard deviation of this frame is 7.2, then twice the standard deviation is 24.5. The standard deviation is 14.4, and the brightness screening threshold is 24.5 + 14.4 = 38.9. Pixels with brightness greater than or equal to 39 in the frame are retained and recorded as bright pixels according to their coordinates. After all frames are processed in this way, the highlight points in each frame form a frame sequence bright light candidate point set. Considering system memory optimization, only a maximum of 1000 highlight pixels are retained in each frame. If the number of bright spots in a frame exceeds the threshold, they are sorted in descending order according to grayscale value, and the top 1000 points are retained to finally construct the overall candidate point set.

[0055] The reflective point screening submodule sorts the candidate points in the bright area in descending order of brightness, extracts the top five pixels in brightness value to construct a point set, and obtains the gloss response point set on the fish surface;

[0056] The reflective point screening submodule arranges the candidate points in the bright area in descending order according to brightness. In the specific implementation, for the bright point set of each frame, the grayscale value of each pixel is first compared, and the grayscale value is arranged from high to low using the quick sorting algorithm. After sorting, the coordinates of the top five pixels and their grayscale values ​​are recorded as response points. For example, the grayscale value sequence of the candidate points in the 121st frame is [55, 52, 49, 49, 48, 47, 45]. The grayscale values ​​of the top five points screened out are [55, 52, 49, 49, 48], and their corresponding coordinates are [(120, 310), (98, 305), (102, 295), (88, 288), (135, 270)]. After obtaining the response point set of this frame, the next frame is processed, and finally the response points of each frame in 750 frames are merged to form the gloss response point set of the fish body surface.

[0057] See also Figure 3 , the trajectory fitting module includes:

[0058] The trajectory path estimation submodule extracts the coordinate change sequence of the reflective point at the same position in the image frame sequence based on the set of gloss response points on the fish body surface. It uses the RANSAC random sampling consensus algorithm to fit the spatial trajectory of the gloss points on the fish body surface in multiple frames of images to obtain the set of fitted paths of the gloss points on the fish body.

[0059] Based on the set of gloss response points on the fish body surface, in actual operation, the five gloss response points extracted from each frame image are used as the basis, and the coordinate position of each response point is extracted in 750 frames of image in chronological order. For example, if the coordinates of a gloss point in the 100th frame are (320, 240) and move to (323, 242) in the 101st frame, the coordinate change of the point is (3, 2). Each gloss response point forms a sequence trajectory in the time dimension. By comparing the continuity and consistency of the spatial coordinates of the gloss point in each frame, the change sequence is recorded for path fitting processing. In the fitting process, the RANSAC random sampling consistency method is used to perform iterative sampling on the trajectory point set. Each time, two groups of coordinate points in the frame sequence are randomly selected as the fitting starting point. A hypothetical path is constructed based on the three-dimensional straight line model, and the remaining points are brought into the model to calculate the fitting error. The error is defined as the shortest Euclidean distance between the gloss point and the hypothetical path. In order to enhance the robustness of the algorithm to low-quality frames, the fitting point confidence weight w is introduced i , the path quality is evaluated using the following improved fitting residual mean weighted formula:

[0060]

[0061] The parameters are defined as follows: ε: weighted residual mean, reflecting the overall path fitting error, in pixels; i: the frame number of the response point in the path, ranging from 1 to n; n: the number of gloss response points contained in a single trajectory, if each path covers 30 frames, then n = 30; P i : The actual pixel coordinates of the fish body gloss response point in the i-th frame image, in the form of a two-dimensional vector (x i ,y i ), the unit is pixel; The coordinates of the points mapped by the current fitting path on the corresponding frame number i, in the form of a two-dimensional vector The unit is pixel; w i : The confidence weight of the response point in the i-th frame, calculated by normalizing the grayscale value, and the calculation method is w i =G i / 255, where G i Grayscale value, ranging from 0 to 255.

[0062] For example, assuming that the Euclidean distances between the actual and fitted points of a 5-frame response point in a 30-frame path segment are: [3.2, 2.1, 4.5, 3.8, 2.6], and the corresponding grayscale values ​​are [210, 185, 190, 220, 200], then the normalized weights are: [0.824, 0.725, 0.745, 0.863, 0.784], and the squared distances multiplied by the weights are: [8.467, 3.196, 15.059, 12.458, 5.317], the weight sum is 3.941, the total weighted square residual sum is 44.497, and the final weighted residual mean is: This value indicates that the residual of this path segment is relatively high. If the judgment threshold is set to 10 pixels, this segment does not belong to the well-fitted path. The RANSAC algorithm will exclude this path combination based on the error result and resample it. The path segments with residuals less than the set threshold will be screened and output from all trajectory segments. The output content includes the path start and end frame numbers, starting point coordinates, path direction vector, and residual value, and finally the trajectory fitting path set of the fish body gloss point is obtained.

[0063] The abnormal segment identification submodule identifies the trajectory segments whose fitting errors exceed the consistency threshold based on the inter-frame fitting residual of each trajectory in the fish body gloss point fitting path set, marks the corresponding segments as path interruption areas, and obtains the trajectory interruption segment identification group;

[0064] According to the inter-frame fitting residual of each trajectory in the set of fish body gloss point fitting paths, the inter-frame fitting residual variation of each fitting path is calculated. In actual operation, the residual time series is extracted in units of continuous frames. For example, for a certain trajectory, the residual sequence in the 30-frame path segment corresponding to it is: [2.5, 3.1, 3.4, 3.6, 4.0, 10.2, 11.8, 12.5, 11.2, 4.8, 3.9]. It can be observed that there is a residual jump from the 6th frame to the 9th frame. To determine whether it constitutes an abnormal segment, the mean of the residuals of the continuous frames in this time period needs to be calculated and compared with the consistency threshold. The threshold setting method refers to the mean difference between the maximum value and the median of the residuals in the valid path. If the maximum value of the residual in the valid trajectory is 6.5 and the median is 3.5, The consistency threshold is set to θ = 3.5 + (6.5 - 3.5) / 2 = 5.0, and the mean of the residual sequence in this segment is (10.2 + 11.8 + 12.5 + 11.2) / 4 = 11.425, which is significantly higher than the consistency threshold. Therefore, the 6th to 9th frames are marked as abnormal path segments, and the start and end frame numbers are added to the trajectory interruption segment identification group. If concentrated abnormal positions appear in multiple trajectory paths, for example, frames 225 to 228 are multiple path abnormal overlap segments, they are further processed as key segments for subsequent feature analysis. Each segment identification group is recorded in the form of frame segment number, such as [(225, 228), (315, 319)]. The system supports real-time update of the identification group and one-to-one correspondence with the original trajectory path set, which facilitates the subsequent module to perform response point feature recognition.

[0065] The feature sequence construction submodule normalizes the change amplitude of the motion direction of the reflective points before and after the interruption point in the trajectory interruption segment identification group, the inter-frame displacement gradient value, and the response point distribution density function value, and integrates them to form the fish body path fitting feature sequence;

[0066] The normalized calculation is performed for the change amplitude of the motion direction of the reflective points before and after the interruption point of the trajectory interruption segment identification group, the inter-frame displacement gradient value and the response point distribution density function value. In the specific execution process, the coordinate sequence of the reflective points of each abnormal segment before and after the three frames is first extracted, and the direction vector difference per unit time is calculated. For example, if the three points from the 222nd frame to the 224th frame before the interruption are [(320, 240), (323, 244), (327, 248)], the coordinate sequence of the reflective points of the 229th frame after the interruption is [(320, 240), (323, 244), (327, 248)], the coordinate sequence of the reflective points of the three frames before and after the abnormal segment ... To the 231st frame is [(332, 255), (336, 259), (339, 262)], then the forward direction unit vector is [(3, 4), (4, 4)], after normalization it is [(0.6, 0.8), (0.707, 0.707)], the back section direction vector is [(4, 4), (3, 3)], after normalization it is [(0.707, 0.707), (0.707, 0.707)], the direction difference is calculated by the vector angle co The chord formula is used to obtain a sequence of difference coefficients. Next, the inter-frame displacement gradient is calculated, and a gradient sequence is constructed based on the Euclidean distance difference between adjacent frame points, for example, [5.0, 5.4, 6.1] is transformed into [0.4, 0.7]. The number of gloss points per unit pixel area in each frame is then extracted. The 480×640 image is divided into 30×40 grids, each with an area of ​​16×16 pixels. The point density in each grid is calculated, and a density function value sequence is constructed. The value is normalized to a ratio range of [0, 1] by the difference between the maximum density and the mean density. Finally, the three indicators are normalized to the same scale and then sequenced. The integration method is to take three eigenvalues ​​of each frame as a three-dimensional vector and assemble them to form a feature vector sequence. The resulting vector sequence is output as an array, such as [(0.62, 0.35, 0.58), (0.68, 0.38, 0.63), …], where each triplet represents the direction difference, displacement gradient, and density ratio, respectively, forming the final fish path fitting feature sequence.

[0067] See also Figure 4 ,The occlusion classification module includes:

[0068] The input vector construction submodule is based on the fish path fitting feature sequence. It obtains the angular change between adjacent frames in the trajectory of glossy points on the fish body surface, the average displacement of each trajectory in the frame sequence, and the curvature change of the trajectory segment. It combines these three parameters to construct the input data set and generate a path feature input vector group.

[0069] Based on the fish path fitting feature sequence, the direction vector change, average displacement and curvature change degree of the trajectory segment of the continuous frames in the sequence are extracted in the specific execution, and are respectively input as three types of feature parameters for combined input. The direction angle change is calculated by the unit vector angle between the two frames. For example, the movement directions of the reflective points on the surface of the fish body in the 300th frame and the 301st frame are (3, 4) and (5, 4), then the corresponding unit vectors are (0.6, 0.8) and (0.780, 0.625). The angle calculation adopts the cosine similarity method, and the cosine value is cosθ=0.6·0.780+0.8·0.625=0.468+0.500=0.968, and then the degree of direction angle change is obtained as arccos(0.968)≈14.5°. The average displacement of each trajectory in the 30-frame sequence is averaged by summing the Euclidean distance sequence of adjacent frame points, for example There are 29 displacements in total, [4.0, 4.2, 4.3, 4.5, …], with an average of 4.3 pixels. The curvature change is calculated by constructing an angle change sequence from the broken line segments formed every three frames. For example, the two vectors [(3, 3), (4, 3)] are formed from (320, 240), (323, 243), and (327, 246). The unit vectors are (0.707, 0.707) and (0.8, 0.6), respectively, with an angle of arccos(0.707·0.8+0.707·0.6)=arccos(0.989)≈8.1°. This process generates a curvature change sequence between points in the entire trajectory. The three dimensions are summarized in frame order to form the input feature matrix. Each row of the vector has the structure [direction angle change, average displacement, curvature change], for example: [14.5, 4.3, 8.1]. This is how the path feature input vector is constructed.

[0070] The occlusion state recognition submodule inputs the trajectory feature vector set into the support vector machine model, uses the training boundary to perform sample classification processing on the input data, identifies the occlusion state to which the glossy point trajectory in the image frame sequence belongs, and obtains the fish body occlusion state recognition result;

[0071] The trajectory feature vector set is input into the support vector machine model. In actual recognition, the pre-trained support vector set and boundary function are used to discriminate the input sample points. The input feature points are predicted based on their distance from the segmentation hyperplane to determine whether they belong to the fish body extension, intersection or boundary separation state. The classification function uses the SVM decision function to express it as follows:

[0072]

[0073] The parameters are defined as follows: x: the path feature vector of the current input, which is a three-dimensional real vector whose components represent the direction angle change (unit: degree), the average inter-frame displacement (unit: pixel), and the curvature change (unit: degree) in sequence; The i1th support vector, i.e., the important sample points retained in the training set for classification boundary construction after model training, has the same structure as x and is represented as a three-dimensional real vector, i1∈{1,2,...,m}; The true classification label corresponding to the i1th support vector is +1 or -1, which is used to represent the occlusion state type of the sample. For example, +1 indicates "single fish body extension" and -1 indicates "fish body crossing" or "boundary detachment"; The Lagrange multiplier of the i1th support vector is a parameter obtained through optimization during the training process. It represents the weight contribution of the support vector in the model and is usually a positive real number. Kernel function, used to calculate the input vector x and support vector The nonlinear similarity between them is calculated by using radial basis function (RBF). Where γ is the kernel function width parameter, represents the Euclidean distance; b: the bias constant term obtained from model training, which is used to adjust the discrimination boundary to make it more consistent with the actual distribution of training samples; m: the total number of support vectors, which is the size of the feature vector subset screened during the training process and is generally much smaller than the total number of original samples; sign(): the sign function, which is used to judge the positive or negative of the overall sum value and return +1 or -1 as the final predicted classification label.

[0074] For example, the current input vector is x = [14.5, 4.3, 8.1], and the three support vectors in the training model are: x1 = [13.0, 4.1, 7.5], y1 = +1, α1 = 0.9; x2 = [17.2, 4.5, 9.0], y2 = -1, α2 = 0.6; x3 = [10.1, 3.8, 6.5], y3 = +1, α3 = 0.7.

[0075] When calculating the kernel function value of each support vector, the kernel function parameter γ is taken as 0.5, and the square of the Euclidean distance is: ||x-x1|| 2 =(14.5-13.0) 2 +(4.3-4.1) 2 +(8.1-7.5) 2 =2.25+0.04+0.36=2.65, so K1=exp(-0.5·2.65)=exp(-1.325)≈0.266; ||x-x2|| 2 =(14.5-17.2) 2 +(4.3-4.5) 2 +(8.1-9.0) 2=7.29+0.04+0.81=8.14, so K2=exp(-0.5·8.14)=exp(-4.07)≈0.017; ||x-x3|| 2 =(14.5-10.1) 2 +(4.3-3.8) 2 +(8.1-6.5) 2 =19.36+0.25+2.56=22.17, so K3=exp(-0.5·22.17)=exp(-11.085)≈0.000015.

[0076] After substituting the support vector weights and classification labels, we get:

[0077] f(x)=sign(0.9·1·0.266+0.6·(-1)·0.017+0.7·1·0.000015-0.3)=sign(0.2394-0.0102+0.0000105-0.3)=sign(-0.0708)=-1.

[0078] The final classification result is 1, corresponding to the “fish body cross” state, and the identification label will be fed back to the subsequent classification module for image area labeling.

[0079] The classification result generation submodule assigns three classification labels, namely, single fish extension, fish intersection, or boundary separation, to the identified area in the image based on the fish occlusion status recognition result, and outputs the fish occlusion type classification result;

[0080] Based on the fish occlusion status recognition results, the identified areas are marked in the image frame sequence and assigned corresponding classification labels such as "single fish extension," "fish crossing," or "boundary separation." During execution, the occlusion status sequence is first read, and the frame-by-frame position mapping is performed for each trajectory result. An occlusion marker layer is created in the original image coordinate system, and the corresponding trajectory point area is filled with markers. For example, if trajectory A is identified as "boundary separation" between frames 340 and 370, a separation marker is added to the corresponding trajectory point position in this frame segment. The marker data is stored as a number or mask, such as [frame number, trajectory number, occlusion type] = [340, A1, 3]. All classification outputs are output as a JSON structure or frame index file.

[0081] See also Figure 5 , the edge redrawing module includes:

[0082] The occlusion area screening submodule calls the fish body occlusion type classification result, filters the image area where the occlusion type is fish body intersection, and correspondingly extracts all the point sets in the fish body surface gloss response point set located in the area to obtain the cross occlusion area response point group;

[0083] The fish occlusion type classification result is called, and each frame result is read one by one in the image frame sequence. A mapping relationship is established between the frame number and the path number, and the occlusion classification label of each path is compared. The path segment number with the occlusion type marked as "fish body crossing" is extracted, and an occlusion frame sequence table is generated within the system. After the screening is completed, the region corresponding extraction operation is entered. The rectangular area is calibrated according to the path coordinate range in the grayscale matrix of each frame image. For example, there is a cross occlusion path A3 in the 480th frame, and the coordinate distribution range of its response point is [(310, 260), (312, 261), (315, 2 63), (318, 265)], then the extraction window is constructed with the upper left point (310, 260) and the lower right point (318, 265), the corresponding area pixel width is 9, and the height is 6. The coordinates of all response points in the extraction window constitute the intra-frame cross-occlusion point set, and the above extraction operation is performed on each frame in turn. During the extraction process, duplicate coordinate points are removed and recorded with a unified number. After completing the point set extraction for the cross-occlusion image frames under all paths, they are summarized into a three-dimensional array structure, the first dimension is the path number, the second dimension is the frame sequence number, and the third dimension is the two-dimensional coordinate value of the response point, forming a cross-occlusion area response point group.

[0084] The contour path matching submodule obtains the image contour trajectory template for the response point group in the cross-occlusion area, calculates the direction vector cosine value between the template path and the current response point group, selects the path whose direction cosine angle difference is less than the template specified range as the reference path, performs direction vector alignment and length parameter scaling, and generates a candidate contour path set;

[0085] For processing the response point group in the cross-occlusion area, first call the standard image contour trajectory template database. The contour template screened in the database must meet the requirement that the number of point sets is not less than 80% of the response point group and the number of direction vectors is not less than 20. The template path format is an ordered coordinate chain, which is connected in sequence to form a direction vector sequence. For each response point path, the direction vector is generated according to the difference between its adjacent frame coordinates. For example, the point sequence [(310, 260), (313, 263), (316, 266)] corresponds to the direction vector [(3, 3), (3, 3)]. After normalization, the unit vector sequence is obtained. The cosine value of the template direction sequence and the current path direction vector group is calculated by one-to-one pairing. For the direction vector (3, 3) and the template direction vector (4, 2), the cosine value is calculated as follows:

[0086]

[0087] Among all matching direction pairs, if the cosine angle difference (calculated by the inverse cosine function) is less than the set range, for example, set to 20°, that is, the corresponding cosine value is greater than 0.94, then it is considered to be a qualified match, and the template path is recorded as a reference path. Then the length parameter of the path is scaled, and the current path length is divided by the template path length as the scaling factor. For example, if the response path length is 27 pixels and the template path length is 30 pixels, the scaling factor is 0.9. After all coordinates of the template path are adjusted proportionally according to the scaling factor, the coordinates of the starting point are aligned to the starting point of the response path, that is, the direction alignment and scale normalization of the matching path are completed, and all successfully matched paths are stored in the candidate contour path set.

[0088] The edge coverage drawing submodule calculates the overlapping area ratio of each path and the current response point group in spatial coordinates based on the candidate contour path set, selects paths with an overlapping ratio greater than a specified reference value, redraws the contour of the target path in the edge area of ​​the image, and outputs the redrawn fish body edge path set;

[0089] Image mapping operation is performed based on the candidate contour path set. First, each candidate path is converted into a pixel point set in the image coordinate system, and the pixel overlap area ratio with the current cross-occlusion response point group is calculated. The ratio of the intersection area to the total area of ​​the response point set is used as the basis for ratio calculation. For example, if the total number of response point pixels is 45 and the number of pixels of the candidate path mapping area overlapping with it is 38, then the overlap area ratio is 38 / 45≈0.844. It is judged whether this value is greater than the specified benchmark value. The benchmark value setting method refers to the minimum recognition effective coverage rate as the basis. For example, if it is set to 0.80, the path meets the overlap condition and is retained as the edge path; otherwise, if the ratio is less than the benchmark value, the path is eliminated from drawing. After screening, the remaining paths are drawn. The drawing method adopts image overlay processing. The path coordinates are mapped on the edge layer according to the image frame sequence. The line segment width is set to 2 pixels and the color is set to neutral gray. Finally, the drawing layer coordinate file or image format file is output to form the fish body edge redrawing path set.

[0090] See also Figure 6 , the density analysis module includes:

[0091] The fin ray region extraction submodule identifies points with fin ray position characteristics among the glossy points on the fish body surface based on the redrawn fish body edge path set, extracts the pixel coordinates of such points between consecutive frames in the image frame sequence, and obtains the fin ray region brightness trajectory point group;

[0092] Based on the set of redrawn fish body edge paths, the coordinates of the reflective points around the redrawn paths are analyzed frame by frame in the image frame sequence. The dense distribution characteristics, geometric positioning characteristics and inter-frame motion stability of the reflective points are combined to determine whether they have fin ray position characteristics. The specific judgment standard is: if there are highlight points with brightness values ​​greater than 1.5 times the image mean in 16×16 pixel areas around a point in 5 consecutive frames, and the coordinate change of these points does not exceed 3 pixels, then the point is marked as a suspected fin ray feature point. For example, the glossy point at (328, 244) in frame 422 has a maximum coordinate offset of 2 pixels in frames 423 to 426, and the corresponding grayscale values ​​are [210, 215, 220, 218, 212]. The average grayscale of the image is 100, so the point is determined to meet the fin ray feature conditions. All points that meet the conditions are constructed into a trajectory chain according to the frame sequence number. The pixel coordinates of each point in the trajectory chain are extracted and integrated into multiple trajectory paths to form a fin ray area brightness trajectory point group.

[0093] The brightness trajectory analysis submodule performs time series statistics on the grayscale changes of each point in the fin area brightness trajectory point group in the frame sequence. It calculates three indicators for each trajectory between consecutive frames: the grayscale fluctuation frequency, the overlap ratio of the amplitude peak value, and the consistency value of the brightness change direction, and generates a set of brightness change feature indicators.

[0094] Perform statistics on the grayscale changes of each point in the fin area brightness trajectory point group in the frame sequence, extract the continuous frame grayscale sequence, and calculate the three indicators. First, calculate the grayscale fluctuation frequency using the following formula: Where: h: is the length of the brightness trajectory, in frames; i2: the frame number subscript, the traversal interval is 1 to h-1; Grayscale value of the i2th frame; T: grayscale jump threshold, set to 15; δ(): judgment function, returns 1 if it is true, and returns 0 if it is false. When the grayscale sequence is [210, 215, 190, 195, 220, 185], the inter-frame difference is [5, 25, 5, 25, 35], of which 3 are greater than the threshold T = 15, h = 6, and the calculation is: Secondly, calculate the amplitude peak coincidence ratio, the formula is: Where: N co : The number of other tracks with the same grayscale peak frame number at the peak frame position of the track; N total : The total number of tracks currently participating in the coincidence statistics; if the current track has a maximum grayscale value of 220 in the 5th frame, and there are 3 other tracks that also have their respective maximum grayscales in the 5th frame, the total number of tracks participating in the comparison is 5, then: Finally, the calculation formula for the brightness change direction consistency value is: Where: L max: The longest segment length of continuous grayscale differences of the same symbol; h-1: The total number of difference segments; The grayscale difference is [5, -25, 5, 25, -35], the symbol sequence is [+1, -1, +1, +1, -1], the longest consistent segment is [+1, +1], and its length is 2, then: In summary, the three indicators of the current point under the trajectory length of 6 frames are: frequency Freq = 0.6, overlap ratio PeakRatio = 0.6, and consistency Cons = 0.4, which constitute the complete brightness change feature indicator vector of the point [0.6, 0.6, 0.4].

[0095] The density information generation submodule performs weighted processing on the three indicators in the brightness change characteristic index set, merges and evaluates the weighted results, and outputs the local density information of fish species;

[0096] A weighted processing is performed on the brightness change feature index set. Before setting the weight, it is necessary to evaluate the contribution of different indicators to the identification of dense structures. The evaluation method is to calculate the Pearson correlation coefficient of the three indicators separately with the manual density annotation level in the pre-sampling sample set. In the 300 groups of trajectory samples, the correlation coefficient between the grayscale change frequency and the manual density level is 0.78, the peak coincidence ratio is 0.65, and the brightness change consistency is 0.52. According to the normalized result of the correlation coefficient as the weight basis, the three coefficients are normalized to a sum of 1, and the obtained ratio is To simplify the calculation and maintain the discrimination, the weights are adjusted to 0.4, 0.35, and 0.25 respectively. When performing weighted calculation, the three index vectors [Freq, PeakRatio, Cons] of each point are multiplied by their respective weights and then summed to obtain the local density value of the point. Taking the trajectory point feature index of [0.6, 0.6, 0.4] as an example, the calculation is as follows: Density point =0.6×0.4+0.6×0.35+0.4×0.25=0.55. Next, all trajectory points in each frame are merged and summarized, and the image area is divided into a grid (for example, 32×32 pixels per grid). In each grid, the density values ​​of all points within the grid are counted, and the average value is calculated to obtain the grid density, which is expressed as: Where: MeanDensity: the average density value of the grid area, used to characterize the local concentration of fish activity in the area; k: the number of brightness track points in the grid; The weighted density value of the i3th trajectory point is a dimensionless real number in the range [0, 1]. i3 is the trajectory point index number, ranging from 1 to k. If there are 6 trajectory points in a grid, and their weighted density values ​​are [0.45, 0.50, 0.62, 0.55, 0.48, 0.57], then:

[0097]

[0098] Finally, all grid density values ​​were classified into intervals, and the classification criteria were set as: high density (MeanDensity ≥ 0.60), medium density (0.40 ≤ MeanDensity < 0.60), and low density (MeanDensity < 0.40). According to the above results, 0.528 belongs to the medium density interval, and the grid density level is recorded in the two-dimensional coordinate matrix. The system outputs all grid coordinates and corresponding levels as the final fish species local density information dataset.

[0099] The above are merely preferred embodiments of the present invention and do not limit the present invention in any other form. Any technician familiar with the profession may use the technical content disclosed above to change or modify it into an equivalent embodiment with equivalent changes and apply it to other fields. However, any simple modification, equivalent change and modification made to the above embodiment based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall still fall within the scope of protection of the technical solution of the present invention.

Claims

1. The fishing information acquisition system for light-covered net fishing boats based on AI cameras is characterized by: The system comprises: The reflection point extraction module uses a camera to collect a sequence of image frames of a light-covered fishing boat operating under the water surface, and performs reflection point screening on the bright areas on the surface of the fish in the image frame sequence to obtain a set of gloss response points on the surface of the fish; The trajectory fitting module estimates the motion trajectory of the fish body reflective point group in the image frame sequence of the gloss response point set on the fish body surface, and constructs a fish body path fitting feature sequence; The occlusion classification module uses a support vector machine model to judge the occlusion state of the fish body path fitting feature sequence image to form a fish body occlusion type classification result; The edge redrawing module redraws the edge contour of the gloss response point set on the surface of the fish body according to the fish body occlusion type classification result, and outputs a redrawn fish body edge path set; The density analysis module obtains the brightness change trajectory group of the corresponding fin area in the bright area on the surface of the fish body based on the redrawn fish edge path set, performs time series analysis on the grayscale values ​​of the corresponding positions in the image frame sequence, and outputs the local density information of the fish species.

2. The fishing catch information acquisition system for a light-covered net fishing vessel based on an AI camera according to claim 1 is characterized in that: The set of gloss response points on the fish surface includes the number of reflective points, spatial distribution characteristics, and sorting position information; the fish path fitting feature sequence includes trajectory continuity status, motion mode change type, and image frame association relationship; the fish occlusion type classification results are specifically extended occlusion, cross occlusion, and boundary detachment; the redrawn fish edge path set includes edge contour morphology, path coverage area, and position fitting results; the local density information of the fish species includes the number of fin response trajectory groups, grayscale value change trend, and frame sequence time characteristics.

3. The fishing catch information acquisition system for a light-covered net fishing vessel based on an AI camera according to claim 1, characterized in that: The reflective point extraction module includes: The image frame acquisition submodule uses a camera to collect image frame sequences of the light-covered net fishing boat operating under the water surface, extracts the grayscale channel information of the fish body surface in each frame of the image, and obtains the image frame sequence grayscale data set; The bright light area recognition submodule selects areas whose brightness exceeds two standard deviations of the image mean based on the brightness of each pixel in the grayscale dataset of the image frame sequence, locates the bright light pixel point group on the surface of the fish body in the image, and obtains a set of candidate points of the bright light area; The reflective point screening submodule arranges the candidate point set in descending order of brightness from the bright area, extracts the top five pixel points with the highest brightness values ​​to construct a point set, and obtains a gloss response point set on the fish body surface.

4. The fishing catch information acquisition system for a light-covered net fishing vessel based on an AI camera according to claim 3 is characterized in that: The trajectory fitting module includes: The trajectory path estimation submodule extracts the coordinate change sequence of the reflective point at the same position in the image frame sequence based on the set of gloss response points on the fish body surface, and uses the RANSAC random sampling consensus algorithm to perform path fitting on the spatial trajectory of the gloss points on the fish body surface in multiple frames of images to obtain the set of fitted paths of the gloss points on the fish body; The abnormal segment identification submodule identifies the trajectory segments whose fitting errors of consecutive frames exceed the consistency threshold based on the inter-frame fitting residual of each trajectory in the fish body gloss point fitting path set, marks the corresponding segments as path interruption areas, and obtains the trajectory interruption segment identification group; The feature sequence construction submodule normalizes and calculates the movement direction change amplitude, inter-frame displacement gradient value and response point distribution density function value of the reflective points before and after the interruption point of the trajectory interruption segment identification group, and integrates them to form a fish body path fitting feature sequence.

5. The fishing catch information acquisition system for a light-covered net fishing vessel based on an AI camera according to claim 4 is characterized in that: The occlusion classification module includes: The input vector construction submodule obtains the angular change between adjacent frames in the glossy point trajectory of the fish body surface, the average displacement of each trajectory in the frame sequence, and the curvature change of the trajectory segment based on the fish body path fitting feature sequence, and combines these three parameters to construct an input data set to generate a path feature input vector group; The occlusion state recognition submodule inputs the trajectory feature vector set into the support vector machine model, performs sample classification processing on the input data using the training boundary, identifies the occlusion state to which the glossy point trajectory in the image frame sequence belongs, and obtains the fish body occlusion state recognition result; The classification result generation submodule assigns three classification labels of single fish body extension, fish body intersection or boundary separation to the identified area in the image based on the fish body occlusion state recognition result, and outputs the fish body occlusion type classification result.

6. The fishing catch information acquisition system for a light-covered net fishing vessel based on an AI camera according to claim 5, characterized in that: The edge redrawing module includes: The occlusion area screening submodule calls the fish body occlusion type classification result, screens the image area where the occlusion type is fish body intersection, and correspondingly extracts all point sets located in the area from the fish body surface gloss response point set to obtain the intersection occlusion area response point group; The contour path matching submodule obtains the image contour trajectory template for the response point group in the cross-occlusion area, calculates the direction vector cosine value of the template path and the current response point group, selects the path whose direction cosine angle difference is less than the template specified range as the reference path, performs direction vector alignment and length parameter scaling, and generates a candidate contour path set; The edge coverage drawing submodule calculates the overlapping area ratio of each path and the current response point group in the spatial coordinates based on the candidate contour path set, screens the paths whose overlapping ratio is greater than the specified reference value, performs contour redrawing of the target path in the edge area of ​​the image, and outputs the redrawn fish body edge path set.

7. The fishing catch information acquisition system for a light-covered net fishing vessel based on an AI camera according to claim 6, characterized in that: The density analysis module includes: The fin ray region extraction submodule identifies points with fin ray position characteristics among the glossy points on the fish body surface based on the redrawn fish body edge path set, extracts the pixel coordinates of the corresponding points between consecutive frames in the image frame sequence, and obtains the fin ray region brightness trajectory point group; The brightness trajectory analysis submodule performs time series statistics on the grayscale changes of each point in the brightness trajectory point group of the fin area in the frame sequence, calculates three indicators of each trajectory between consecutive frames: grayscale fluctuation frequency, amplitude peak overlap ratio, and brightness change direction consistency value, and generates a brightness change feature indicator set; The density information generation submodule performs weighted processing on the three indicators in the brightness change characteristic index set, merges and evaluates the weighted results, and outputs the local density information of the fish species.

Images