Fish species recognition method and system based on contour key points and attention mechanism

Abstract

The application discloses a fish species identification method and system based on contour key points and an attention mechanism, and relates to the field of fish species identification.The method comprises the following steps: acquiring a target image of a target fish body; inputting the target image into a precise contour key point model to obtain contour points and contour key points of the target image; using a sparse sampling method to calculate distance feature data of the target fish body according to the contour points of the target image and to calculate angle feature data of the target fish body according to the contour key points of the target image; fusing the distance feature data of the target fish body and the angle feature data of the target fish body to obtain target fusion features; and inputting the target fusion features into a fish species identification model to obtain the species of the target fish body.The application can improve the accuracy of fish species identification.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of fish species identification, and in particular to a fish species identification method and system based on contour key points and attention mechanisms. BACKGROUND

[0002] In order to analyze and study fish diversity, the most critical issue is to identify the species of fish. In response to this demand, a large number of scientific and technical personnel have done a lot of research work and have achieved certain results. However, for the image or video data to be identified, the image quality will be affected due to the underwater environment, and there are problems such as noise, distortion, severe target occlusion, and multiple poses of fish. Therefore, the fish species identification algorithm based on images has great difficulty.

[0003] The traditional fish species identification scheme relies on manual extraction of target features, which has a certain effect, but is severely affected by image quality. At the same time, the algorithm is only applicable to a specific scene, lacks universality, and is not easy to improve, with poor stability.

[0004] In recent years, with the development of artificial intelligence, especially the landing application of deep learning in image target identification, fish identification algorithms based on deep learning provide a new solution to the problem of fish identification. However, if open source data, algorithms and programs are used directly without considering the specific problems of the actual scene, the actual effect cannot meet the demand. The specific problems and difficulties include: algorithm universality problem, algorithm applicability problem, target occlusion problem, target multi-scale change problem, target deformation problem, fine-grained target classification problem, and low detection rate problem.

[0005] In order to solve the above problems, at present, the commonly used deep learning species identification algorithm scheme is to unify the target positioning problem and the fish classification task into one task. When designing the deep learning network, the one-stage method is adopted, and the task is designed as an end-to-end processing method, which improves the algorithm efficiency, but at the same time also has certain problems. Since the target positioning uses a target detection method, this method will have more background areas when positioning the target, which will interfere with the target identification and affect the identification accuracy. In addition, if a target segmentation algorithm is used, it can reduce the background interference to a certain extent, but usually the segmentation is not accurate, which leads to inaccurate contour extraction. Moreover, since the positioning and classification are designed as one task, the weights of the two tasks cannot be accurately designed, which may result in lower identification accuracy of one of the tasks. Therefore, the accuracy of fish species identification still needs to be improved. SUMMARY

[0006] Based on this, embodiments of the present invention provide a fish species identification method and system based on contour key points and attention mechanisms to improve the accuracy of fish species identification.

[0007] To achieve the above objectives, the present invention provides the following solution:

[0008] Fish species identification methods based on contour keypoints and attention mechanisms include:

[0009] Obtain the target image of the target fish body;

[0010] The target image is input into the precise contour key point model to obtain the contour points and contour key points of the target image;

[0011] Using a sparse sampling method, distance feature data of the target fish body is calculated based on the contour points of the target image, and angle feature data of the target fish body is calculated based on the contour key points of the target image;

[0012] The distance feature data and the angle feature data of the target fish are fused to obtain the target fusion feature;

[0013] The target fusion features are input into the fish species identification model to obtain the species of the target fish.

[0014] The method for determining the precise contour key point model is as follows:

[0015] Obtain training images of labeled fish species;

[0016] Extract the contour region of the fish in the training image to obtain the contour points of the training image;

[0017] The points among the contour points that meet the set conditions are determined as contour key points; the set conditions include the contour point curvature being greater than a set curvature value, the distance between the contour point and the centroid being greater than the distance between the adjacent contour point and the centroid, or the distance between the contour point and the centroid being less than the distance between the adjacent contour point and the centroid.

[0018] The training image, the contour points of the training image, and the contour key points of the training image are input into a deep learning model, and the model is trained with the goal of minimizing the loss function to obtain the accurate contour key point model.

[0019] The method for determining the fish species identification model is as follows:

[0020] Using a sparse sampling method, the distance feature data of the training fish is calculated based on the contour points of the training image, and the angle feature data of the training fish is calculated based on the contour key points of the training image.

[0021] The distance feature data and angle feature data of the training fish are fused to obtain the training fusion feature;

[0022] The training fusion features and corresponding label data are input into an attention-based deep learning model for training to obtain the fish species identification model.

[0023] Optionally, the method of employing sparse sampling to calculate the distance feature data of the target fish body based on the contour points of the target image, and to calculate the angle feature data of the target fish body based on the contour key points of the target image, specifically includes:

[0024] Calculate the first centroid based on the contour points of the target image;

[0025] The distance between each of the contour points calculated in a clockwise direction and the first centroid is determined as length feature data;

[0026] Determine the maximum and minimum values ​​in the length distance data to obtain the maximum length feature and the minimum length feature;

[0027] Based on the maximum length feature and the minimum length feature, the distance value of the length feature is normalized to obtain normalized length data;

[0028] The normalized length data is sampled according to the first set sampling interval to obtain the distance feature data of the target fish.

[0029] Calculate the second centroid based on the contour key points of the target image;

[0030] Calculate the angle between each of the contour key points and the second centroid to obtain angle data;

[0031] The angle data is sampled according to the second set sampling interval to obtain the angle feature data of the target fish.

[0032] Optionally, the formula for calculating the loss function is:

[0033] loss = α*loss1 + β*loss2;

[0034] Wherein, loss represents the loss function; loss1 represents the loss function for ordinary points, which are points other than the contour key points among the contour points; loss2 represents the loss function for contour key points; α represents the weight of the loss function for ordinary points; β represents the weight of the loss function for contour key points; β > α.

[0035] Optionally, acquiring training images of labeled fish species specifically includes:

[0036] Obtain initial images of the labeled training fish species;

[0037] The initial image is subjected to denoising and enhancement processing to obtain an enhanced image;

[0038] The enhanced image is then subjected to data augmentation processing to obtain an augmented image;

[0039] The augmented image is then cleaned to obtain the training image.

[0040] This invention also provides a fish species identification system based on contour key points and attention mechanisms, comprising:

[0041] The target image acquisition module is used to acquire target images of the target fish body.

[0042] The target contour point extraction module is used to input the target image into the accurate contour key point model to obtain the contour points and contour key points of the target image;

[0043] The target feature extraction module is used to calculate the distance feature data of the target fish body based on the contour points of the target image using a sparse sampling method, and to calculate the angle feature data of the target fish body based on the contour key points of the target image.

[0044] The target feature fusion module is used to fuse the distance feature data and the angle feature data of the target fish body to obtain the target fused feature;

[0045] The species identification module is used to input the target fusion features into the fish species identification model to obtain the species of the target fish.

[0046] A contour key point model determination module is used to determine the precise contour key point model; the contour key point model determination module specifically includes:

[0047] The training image acquisition unit is used to acquire training images of fish species that have been labeled.

[0048] The training contour point extraction unit is used to extract the contour region of the fish in the training image to obtain the contour points of the training image.

[0049] A key point extraction unit is used to identify points among the contour points that meet set conditions as contour key points; the set conditions include the curvature of the contour point being greater than a set curvature value, the distance between the contour point and the centroid being greater than the distance between adjacent contour points and the centroid, or the distance between the contour point and the centroid being less than the distance between adjacent contour points and the centroid.

[0050] The first model training unit is used to input the training image, the contour points of the training image, and the contour key points of the training image into the deep learning model, and train it with the goal of minimizing the loss function to obtain the accurate contour key point model.

[0051] A recognition model determination module is used to determine the fish species recognition model; the recognition model determination module specifically includes:

[0052] The training feature extraction unit is used to calculate the distance feature data of the training fish body based on the contour points of the training image and to calculate the angle feature data of the training fish body based on the contour key points of the training image using a sparse sampling method.

[0053] The training feature fusion unit is used to fuse the distance feature data and the angle feature data of the training fish to obtain training fusion features.

[0054] The second model training unit is used to input the training fusion features and corresponding label data into the deep learning model based on the attention mechanism for training, so as to obtain the fish species identification model.

[0055] Optionally, the target feature extraction module specifically includes:

[0056] The first centroid calculation unit is used to calculate the first centroid based on the contour points of the target image;

[0057] The length data calculation unit is used to determine the distance between each of the contour points calculated in a clockwise direction and the first centroid as length feature data;

[0058] The extreme value feature determination unit is used to determine the maximum and minimum values ​​in the length distance data to obtain the maximum length feature and the minimum length feature;

[0059] A normalization unit is used to normalize the distance value of the length feature based on the maximum length feature and the minimum length feature to obtain normalized length data.

[0060] The first sampling unit is used to sample the normalized length data according to a first set sampling interval to obtain the distance feature data of the target fish.

[0061] The second centroid calculation unit is used to calculate the second centroid based on the contour key points of the target image;

[0062] An angle data calculation unit is used to calculate the angle between each of the contour key points and the second centroid to obtain angle data;

[0063] The second sampling unit is used to sample the angle data according to the second set sampling interval to obtain the angle feature data of the target fish.

[0064] Optionally, the formula for calculating the loss function in the first model training unit is:

[0065] loss = α*loss1 + β*loss2;

[0066] Wherein, loss represents the loss function; loss1 represents the loss function for ordinary points, which are points other than the contour key points among the contour points; loss2 represents the loss function for contour key points; α represents the weight of the loss function for ordinary points; β represents the weight of the loss function for contour key points; β > α.

[0067] Optionally, the training image acquisition unit specifically includes:

[0068] The initial image acquisition subunit is used to acquire the initial images of the labeled training fish.

[0069] The image enhancement subunit is used to perform denoising and enhancement processing on the initial image to obtain an enhanced image;

[0070] The data augmentation subunit is used to perform data augmentation processing on the enhanced image to obtain an augmented image;

[0071] The data cleaning subunit is used to perform data cleaning processing on the augmented image to obtain the training image.

[0072] Compared with the prior art, the beneficial effects of the present invention are:

[0073] This invention proposes a fish species identification method and system based on contour keypoints and an attention mechanism. First, the target image of the target fish is input into a precise contour keypoint model to obtain the contour points and contour keypoints of the target image. Using a sparse sampling method, distance feature data of the target fish is calculated based on the contour points, and angular feature data is calculated based on the contour keypoints. The distance feature data and angular feature data of the target fish are then fused to obtain a target fusion feature. Finally, the target fusion feature is input into a fish species identification model to determine the species of the target fish. This invention employs a two-step identification approach. First, a precise contour keypoint model is used to extract contour points, ensuring accurate positioning. Then, features are obtained based on the contour points, and a fish species identification model constructed using an attention mechanism-based deep learning model is used for accurate classification. This invention combines contour keypoints with an attention mechanism, which is fundamentally different from traditional deep learning target recognition schemes and improves the accuracy of fish species identification. Attached Figure Description

[0074] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0075] Figure 1 A flowchart of a fish species identification method based on contour key points and attention mechanism provided in an embodiment of the present invention;

[0076] Figure 2 A flowchart illustrating the method for determining a precise contour key point model provided in an embodiment of the present invention;

[0077] Figure 3 A flowchart illustrating the method for determining a fish species identification model provided in an embodiment of the present invention;

[0078] Figure 4 This is a structural diagram of a fish species identification system based on contour key points and attention mechanism provided in an embodiment of the present invention. Detailed Implementation

[0079] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0080] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0081] Based on the problems of existing solutions, this invention considers designing algorithms for accurate extraction of target contours, accurate localization of key points, and fine-grained classification of targets. On the basis of traditional segmentation algorithms, a two-step recognition scheme is designed. First, accurate localization is ensured, and then accurate classification is performed on the target region. This scheme is fundamentally different from traditional deep learning target recognition schemes and solves the accuracy problem of fish type recognition from the overall solution.

[0082] Figure 1 A flowchart illustrating a fish species identification method based on contour key points and an attention mechanism provided in an embodiment of the present invention. See also... Figure 1 The method includes:

[0083] Step 101: Obtain the target image of the target fish.

[0084] Step 102: Input the target image into the precise contour key point model to obtain the contour points and contour key points of the target image.

[0085] Step 103: Using a sparse sampling method, calculate the distance feature data of the target fish body based on the contour points of the target image, and calculate the angle feature data of the target fish body based on the contour key points of the target image.

[0086] Step 104: Fuse the distance feature data and the angle feature data of the target fish body to obtain the target fused feature.

[0087] Step 105: Input the target fusion features into the fish species identification model to obtain the species of the target fish.

[0088] See Figure 2 The method for determining the precise contour key point model is as follows:

[0089] Step 201: Obtain training images of the labeled training fish species.

[0090] Step 202: Extract the outline region of the fish in the training image to obtain the outline points of the training image.

[0091] Step 203: Determine the points among the contour points that meet the set conditions as contour key points.

[0092] The set conditions include the curvature of the contour point being greater than a set curvature value, the distance between the contour point and the center of mass being greater than the distance between adjacent contour points and the center of mass, or the distance between the contour point and the center of mass being less than the distance between adjacent contour points and the center of mass. For example, the fish mouth, dorsal fin, anal fin, pectoral fin, pelvic fin, and caudal fin satisfy the above set conditions; these are key contour points, and these key contour points constitute the key contour of the fish body.

[0093] Step 204: Input the training image, the contour points of the training image, and the contour key points of the training image into the deep learning model, and train it with the goal of minimizing the loss function to obtain the accurate contour key point model.

[0094] See Figure 3 The method for determining the fish species identification model is as follows:

[0095] Step 301: Using a sparse sampling method, calculate the distance feature data of the training fish body based on the contour points of the training image, and calculate the angle feature data of the training fish body based on the contour key points of the training image.

[0096] Step 302: Fuse the distance feature data and angle feature data of the training fish to obtain the training fusion feature.

[0097] Step 303: Input the training fusion features and corresponding label data into the deep learning model based on the attention mechanism for training to obtain the fish species identification model.

[0098] Step 103 specifically includes:

[0099] 1) Calculate the first centroid (C) based on the contour points of the target image. x1 C y1 ).

[0100]

[0101]

[0102] C x1 C represents the coordinates of the first centroid in the x-direction; y1 Indicates the coordinates of the first centroid in the y-direction; x i This represents the coordinates of the i-th contour point in the x-direction, with a corresponding pixel value of p. ai ;y i This represents the y-coordinate of the i-th contour point, with a corresponding pixel value of p. bi n represents the total number of contour points.

[0103] 2) The distance between each contour point calculated in a clockwise direction and the first centroid is determined as length feature data. Specifically:

[0104]

[0105] D i This represents the distance between the i-th contour point and the first centroid.

[0106] 3) Determine the maximum and minimum values ​​in the length distance data to obtain the maximum length feature D. max and minimum length feature D min Specifically:

[0107] D max =max(D i );

[0108] D min =min(D i );

[0109] 4) Normalize the distance values ​​of the length features based on the maximum and minimum length features to obtain normalized length data. Specifically:

[0110]

[0111] D ni This represents the normalized distance corresponding to the i-th contour point in the normalized length data.

[0112] 5) Sample the normalized length data according to the first preset sampling interval to obtain the distance feature data of the target fish. Specifically:

[0113] Sampling extracts 20 length feature data points, i.e., from n D... ni The distance feature data obtained by sampling 20 data values ​​is [D] n0 D n1 ,......D n20 ].

[0114] 6) Calculate the second centroid (C) based on the contour key points of the target image. x2 C y2 ), C x2 C represents the coordinates of the second centroid in the x-direction; y2 This represents the coordinates of the second centroid in the y-direction; the calculation process is similar to step 1), and will not be repeated here.

[0115] 7) Calculate the angle between each of the contour key points and the second centroid to obtain angle data. Specifically:

[0116] A j =atan2(x j -C x2 ,y j -C y2 )*180 / 3.1415926;

[0117] A j The angle between the j-th contour key point and the second centroid; x j This represents the x-coordinate of the j-th contour keypoint; y j This represents the coordinates of the j-th contour keypoint in the y-direction.

[0118] 8) Sample the angle data according to the second set sampling interval to obtain the angle feature data of the target fish body.

[0119] Based on the shape features of the fish's fins, tail, and head, the number of key points is between 15 and 25. To ensure data consistency, this embodiment extracts 20 angular feature data points; that is, from j key points... j Sampling 20 data values ​​yields angle feature data in the form [A0, A1, ..., A...]. 20 ].

[0120] Step 104 specifically includes:

[0121] Distance and angle feature data are fused to form unified feature data, which is then used for subsequent classification and recognition. The target fusion feature is represented as follows:

[0122] [D n0 D n1 ,......D n20 A0, A1......A 20 ].

[0123] Step 201 specifically includes:

[0124] 1) Obtain the initial images of the labeled training fish.

[0125] 2) Perform denoising and enhancement processing on the initial image to obtain an enhanced image. Specifically:

[0126] The initial image is an underwater image. Underwater images typically suffer from numerous quality issues such as color cast, blurriness, uneven lighting, and low contrast, significantly impacting fish identification. Therefore, the initial image is first subjected to image denoising and enhancement processing. Image enhancement algorithms include traditional white balance methods, dark channel prior algorithms, and deep learning modeling algorithms. In practice, due to the unavailability of a large amount of original, undisturbed underwater photographs as Ground Truth data, this embodiment employs traditional algorithms for image denoising and enhancement processing. The specific method for denoising and enhancement processing is as follows: First, the initial image is mapped from the RGB color space to the HSV color space. Then, white balance processing, histogram transformation, and dark channel prior processing are performed to enhance the initial image, resulting in an enhanced image.

[0127] 3) Perform data augmentation processing on the enhanced image to obtain an augmented image. Specifically:

[0128] Considering the limited number of training images, which significantly impacts deep learning training, data augmentation is performed on the enhanced images. The specific method for data augmentation involves performing image flipping, color transformation, and blurring on the enhanced images, thereby expanding the training samples.

[0129] 4) Perform data cleaning processing on the augmented image to obtain the training image.

[0130] For partially annotated augmented images, due to certain annotation errors, such as incorrect target categories, incorrect bounding boxes, and incorrect contour points, data cleaning is still required. The specific methods for data cleaning are: quickly identifying erroneous data in the augmented images by generating sample mini-images; simultaneously, clustering the augmented images to remove outliers, resulting in training images that ensure the effectiveness of the training.

[0131] In step 204, the formula for calculating the loss function is as follows:

[0132] loss = α*loss1 + β*loss2;

[0133] Wherein, loss represents the loss function; loss1 represents the loss function for ordinary points, which are points other than the contour key points among the contour points; loss2 represents the loss function for contour key points; α represents the weight of the loss function for ordinary points; β represents the weight of the loss function for contour key points; β > α.

[0134]

[0135] Where l represents the total number of ordinary points; p l This represents the position of the l-th ordinary point extracted by the model; g l This indicates the position of the l-th ordinary point.

[0136]

[0137] Where m represents the total number of contour key points; p j This indicates the position of the j-th contour keypoint extracted by the model; g j This indicates the position of the j-th contour key point; m+r=n, where n represents the total number of contour points.

[0138] In practical applications, a more specific implementation of the fish species identification method based on contour key points and attention mechanisms is as follows:

[0139] 1. Data augmentation and preprocessing

[0140] The initial images are subjected to data augmentation, data expansion, and data cleaning to obtain training images.

[0141] 2. Data labeling

[0142] Traditional target recognition algorithms extract the bounding rectangle of the target, and therefore label only the bounding rectangle region. However, when labeling fish targets in this way, the unique shapes of fish such as fins and tails mean the bounding rectangle includes a large amount of background, significantly interfering with fish identification. Therefore, we employ a target contour extraction method for target recognition.

[0143] The outline of a target object is essentially composed of a series of pixels, which form an ordered set of points, known as the outline points. The outline region of a fish is its entire body, without any other interfering areas.

[0144] Further refinement of the contour points is required. All pixels of the aforementioned contour points have the same weight, without distinguishing the importance of different contour points. This may result in the deep learning target training failing to extract some key contour points, which is not conducive to fish identification.

[0145] Based on the characteristics of fish targets, certain points in the outline are defined as key points. These key points are characterized by either a large curvature, a greater distance from the center of mass than adjacent points, or a smaller distance from the center of mass than adjacent points. According to this definition, the mouth, dorsal fin, anal fin, pectoral fin, pelvic fin, and caudal fin are key points, and these key points constitute the key outline of the fish's body.

[0146] In the data annotation process, the contour point data is first annotated. The original fish image data is input, and the contour points of the fish body are annotated using a polygon annotation tool to form n-dimensional data, with the data format [p1, p2, ..., pn]. Next, the contour key point data is annotated to form m-dimensional data, with the data format [k1, k2, ..., km]. Generally, m is less than n.

[0147] 3. Training of a precise contour key point model

[0148] 1) Algorithm combining contour segmentation extraction and keypoint regression

[0149] Based on the above data, a network and loss function combining contour segmentation and keypoint regression were designed for training. Since keypoints play a crucial role in the contour, different weights were assigned to them, increasing their weight in regression and thus giving them a higher weight factor, while contour extraction had a lower weight factor. The formula for calculating the loss function is not detailed here.

[0150] During the iterative training of the deep learning model, training stops when the result of the loss function is less than a set threshold or essentially no longer changes, thus obtaining the final accurate contour keypoint model. The input to the training phase is the training image and the corresponding labeled contour points and contour keypoints, and the output is the accurate contour keypoint model.

[0151] 4. Fine-grained classification and recognition algorithm based on sparse sampling fusion features

[0152] The model trained through the above steps, when input into image data of a real-world scene, can extract the target region from the image; this is the inference process of the precise contour key point model. Since the target region is an irregular polygonal area, classifying it using polygonal image data would be difficult due to the large dimensionality of the image data, and would not fully utilize the information of the polygons. Therefore, a fine-grained classification and recognition algorithm based on sparse sampling and fusion features is designed to determine the fish species identification model.

[0153] Sparse sampling refers to reducing the effective data dimensionality of a polygonal target by designing corresponding algorithms. Since the data dimension calculated by this algorithm is much smaller than that of image data, it is defined as sparse sampling.

[0154] The calculation steps for sparse sampling are similar to steps 1)-8) in step 103, and will not be repeated here. After sparse sampling, the distance feature data and angle feature data of the training fish are obtained. The two are then fused to obtain the training fused feature.

[0155] Through the above steps, the obtained training feature fusion features have characteristics such as rotation invariance, low dimensionality, and easy differentiation. This data can effectively reflect the shape of the target, and at the same time, the data dimensionality is low, providing effective feature data for fine-grained and accurate classification and recognition.

[0156] Each set of training fusion features corresponds to a type of fish. Based on the correspondence between the training fusion features and the labels, a deep learning model based on the attention mechanism is established. After training, the mapping relationship between features and fish types is determined, thus obtaining the fish species identification model.

[0157] 5. Recognition Network with Lightweight Attention Mechanism

[0158] Traditional classification and recognition networks require first extracting image features and then iteratively training based on the labels of the image data. This results in problems such as long training time, high resource consumption, and large data requirements. Since the above steps have already proposed more effective feature fusion, there is no need to extract features again. Based on this, the fish species recognition model designed is a lightweight recognition network. The main improvement of this network compared with traditional classification and recognition networks is that it reduces the feature extraction module, thereby unifying the network training task to classification and recognition, achieving the goal of focusing only on classification and recognition, and realizing the recognition of a lightweight attention mechanism.

[0159] This invention also provides a fish species identification system based on contour key points and attention mechanisms. Figure 4 This is a structural diagram of a fish species identification system based on contour key points and attention mechanism provided in an embodiment of the present invention.

[0160] See Figure 4 The system includes:

[0161] The target image acquisition module 401 is used to acquire the target image of the target fish body.

[0162] The target contour point extraction module 402 is used to input the target image into the precise contour key point model to obtain the contour points and contour key points of the target image.

[0163] The target feature extraction module 403 is used to calculate the distance feature data of the target fish body based on the contour points of the target image using a sparse sampling method, and to calculate the angle feature data of the target fish body based on the contour key points of the target image.

[0164] The target feature fusion module 404 is used to fuse the distance feature data and the angle feature data of the target fish body to obtain the target fusion feature.

[0165] The species identification module 405 is used to input the target fusion features into the fish species identification model to obtain the species of the target fish.

[0166] In one example, the system further includes:

[0167] A contour key point model determination module is used to determine the precise contour key point model. The contour key point model determination module specifically includes:

[0168] The training image acquisition unit is used to acquire training images of fish whose species have been labeled.

[0169] The training contour point extraction unit is used to extract the contour region of the fish in the training image to obtain the contour points of the training image.

[0170] A key point extraction unit is used to identify points among the contour points that meet set conditions as contour key points; the set conditions include the curvature of the contour point being greater than a set curvature value, the distance between the contour point and the centroid being greater than the distance between adjacent contour points and the centroid, or the distance between the contour point and the centroid being less than the distance between adjacent contour points and the centroid.

[0171] The first model training unit is used to input the training image, the contour points of the training image, and the contour key points of the training image into the deep learning model, and train it with the goal of minimizing the loss function to obtain the accurate contour key point model.

[0172] In one example, the system further includes: a recognition model determination module, used to determine the fish species recognition model. The recognition model determination module specifically includes:

[0173] The training feature extraction unit is used to calculate the distance feature data of the training fish body based on the contour points of the training image using a sparse sampling method, and to calculate the angle feature data of the training fish body based on the contour key points of the training image.

[0174] The training feature fusion unit is used to fuse the distance feature data and the angle feature data of the training fish to obtain training fusion features.

[0175] The second model training unit is used to input the training fusion features and corresponding label data into the deep learning model based on the attention mechanism for training, so as to obtain the fish species identification model.

[0176] In one example, the target feature extraction module 403 specifically includes:

[0177] The first centroid calculation unit is used to calculate the first centroid based on the contour points of the target image.

[0178] The length data calculation unit is used to determine the distance between each of the contour points calculated in a clockwise direction and the first centroid as length feature data.

[0179] The extreme value feature determination unit is used to determine the maximum and minimum values ​​in the length distance data to obtain the maximum length feature and the minimum length feature.

[0180] The normalization unit is used to normalize the distance value of the length feature based on the maximum length feature and the minimum length feature to obtain normalized length data.

[0181] The first sampling unit is used to sample the normalized length data according to a first set sampling interval to obtain the distance feature data of the target fish.

[0182] The second centroid calculation unit is used to calculate the second centroid based on the contour key points of the target image.

[0183] An angle data calculation unit is used to calculate the angle between each of the contour key points and the second centroid to obtain angle data.

[0184] The second sampling unit is used to sample the angle data according to the second set sampling interval to obtain the angle feature data of the target fish.

[0185] In one example, the formula for calculating the loss function in the first model training unit is:

[0186] loss = α*loss1 + β*loss2;

[0187] Wherein, loss represents the loss function; loss1 represents the loss function for ordinary points, which are points other than the contour key points among the contour points; loss2 represents the loss function for contour key points; α represents the weight of the loss function for ordinary points; β represents the weight of the loss function for contour key points; β > α.

[0188] In one example, the training image acquisition unit specifically includes:

[0189] The initial image acquisition subunit is used to acquire initial images of the labeled training fish.

[0190] The image enhancement subunit is used to perform denoising and enhancement processing on the initial image to obtain an enhanced image.

[0191] The data augmentation subunit is used to perform data augmentation processing on the enhanced image to obtain an augmented image.

[0192] The data cleaning subunit is used to perform data cleaning processing on the augmented image to obtain the training image.

[0193] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0194] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A fish species recognition method based on contour key points and attention mechanism, characterized in that, The method comprises the following steps: obtaining a target image of a target fish body; inputting the target image into a precise contour key point model to obtain contour points and contour key points of the target image; calculating distance feature data of the target fish body according to the contour points of the target image and calculating angle feature data of the target fish body according to the contour key points of the target image by using a sparse sampling method; fusing the distance feature data of the target fish body and the angle feature data of the target fish body to obtain target fusion features; inputting the target fusion features into a fish species identification model to obtain the species of the target fish body; wherein the determination method of the precise contour key point model is as follows: obtaining training images of training fish bodies with labeled species; extracting the contour region of fish in the training images to obtain contour points of the training images; determining points in the contour points that meet a set condition as contour key points; the set condition includes that the curvature of a contour point is greater than a set curvature value, the distance between the contour point and the centroid is greater than the distance between the adjacent contour point and the centroid, or the distance between the contour point and the centroid is less than the distance between the adjacent contour point and the centroid; inputting the training images, the contour points of the training images and the contour key points of the training images into a deep learning model, and training the deep learning model with the objective of minimizing the loss function to obtain the precise contour key point model; wherein the determination method of the fish species identification model is as follows: calculating distance feature data of the training fish body according to the contour points of the training images and calculating angle feature data of the training fish body according to the contour key points of the training images by using a sparse sampling method; fusing the distance feature data of the training fish body and the angle feature data of the training fish body to obtain training fusion features; inputting the training fusion features and corresponding label data into a deep learning model based on an attention mechanism to train the deep learning model to obtain the fish species identification model.

2. The fish species recognition method based on contour key points and attention mechanism according to claim 1, characterized in that, The method for calculating the distance feature data of the target fish body according to the contour points of the target image and the angle feature data of the target fish body according to the contour key points of the target image by using a sparse sampling method comprises the following steps: calculating a first centroid according to the contour points of the target image; determining the distance between each contour point calculated in a clockwise direction and the first centroid as length feature data; determining the maximum value and the minimum value in the length feature data to obtain maximum length feature and minimum length feature; performing distance value normalization on the length feature data according to the maximum length feature and the minimum length feature to obtain normalized length data; sampling the normalized length data according to a first set sampling interval to obtain the distance feature data of the target fish body; calculating a second centroid according to the contour key points of the target image; calculating the angle between each contour key point and the second centroid to obtain angle data; sampling the angle data according to a second set sampling interval to obtain the angle feature data of the target fish body.

3. The fish species recognition method based on contour key points and attention mechanism according to claim 1, characterized in that, The calculation formula of the loss function is as follows: loss = α * loss1 + β * loss2; Wherein, loss represents a loss function; loss1 represents a loss function of ordinary points, the ordinary points being points in the contour points other than the contour key points; loss2 represents a loss function of contour key points; a represents a weight of the loss function of ordinary points; β represents a weight of the loss function of contour key points; β > a.

4. The fish species recognition method based on contour key points and attention mechanism according to claim 1, characterized in that, The training image of the training fish body of the labeled species is obtained, and specifically includes: An initial image of a training fish body of a labeled species is obtained. The initial image is subjected to denoising enhancement processing to obtain an enhanced image. The enhanced image is subjected to data expansion processing to obtain an expanded image. The expanded image is subjected to data cleaning processing to obtain a training image.

5. The fish species recognition system based on contour key points and attention mechanism, characterized in that, It includes: A target image acquisition module is configured to acquire a target image of a target fish body. A target contour point extraction module is configured to input the target image into a precise contour key point model to obtain contour points and contour key points of the target image. A target feature extraction module is configured to calculate distance feature data of the target fish body according to the contour points of the target image and angle feature data of the target fish body according to the contour key points of the target image by using a sparse sampling method. A target feature fusion module is configured to fuse the distance feature data of the target fish body and the angle feature data of the target fish body to obtain target fusion features. A species identification module is configured to input the target fusion features into a fish species identification model to obtain a species of the target fish body. A contour key point model determination module is configured to determine the precise contour key point model. The contour key point model determination module specifically includes: A training image acquisition unit is configured to acquire a training image of a training fish body of a labeled species. A training contour point extraction unit is configured to extract a contour region of a fish in the training image to obtain contour points of the training image. A key point extraction unit is configured to determine points in the contour points that meet a set condition as contour key points; the set condition includes that a curvature of a contour point is greater than a set curvature value, a distance of the contour point to a centroid is greater than a distance of an adjacent contour point to the centroid, or the distance of the contour point to the centroid is less than the distance of the adjacent contour point to the centroid. A first model training unit is configured to input the training image, the contour points of the training image, and the contour key points of the training image into a deep learning model and train the deep learning model with a minimum loss function as an objective to obtain the precise contour key point model. An identification model determination module is configured to determine the fish species identification model; the identification model determination module specifically includes: A training feature extraction unit is configured to calculate distance feature data of the training fish body according to the contour points of the training image and angle feature data of the training fish body according to the contour key points of the training image by using a sparse sampling method. A training feature fusion unit is configured to fuse the distance feature data of the training fish body and the angle feature data of the training fish body to obtain training fusion features. A second model training unit is configured to input the training fusion feature and the corresponding label data into a deep learning model based on an attention mechanism to train the fish species recognition model.

6. The fish species recognition system based on contour key points and attention mechanism according to claim 5, characterized in that, The target feature extraction module specifically comprises: A first centroid calculation unit is configured to calculate a first centroid according to the contour points of the target image; A length data calculation unit is configured to determine the distance between each contour point calculated in a clockwise direction and the first centroid as length feature data; A maximum and minimum value feature determination unit is configured to determine the maximum value and the minimum value in the length feature data to obtain a maximum length feature and a minimum length feature; A normalization unit is configured to perform distance value normalization on the length feature data according to the maximum length feature and the minimum length feature to obtain normalized length data; A first sampling unit is configured to sample the normalized length data at a first set sampling interval to obtain distance feature data of the target fish body; A second centroid calculation unit is configured to calculate a second centroid according to the contour key points of the target image; An angle data calculation unit is configured to calculate the angle between each contour key point and the second centroid to obtain angle data; A second sampling unit is configured to sample the angle data at a second set sampling interval to obtain angle feature data of the target fish body.

7. The fish species recognition system based on contour key points and attention mechanism according to claim 5, characterized in that, The calculation formula of the loss function in the first model training unit is: loss=α*loss1+β*loss2; Wherein, loss represents the loss function; loss1 represents the loss function of the ordinary point, the ordinary point being the point other than the contour key point in the contour point; loss2 represents the loss function of the contour key point; α represents the weight of the loss function of the ordinary point; β represents the weight of the loss function of the contour key point; β>α.

8. The fish species recognition system based on contour key points and attention mechanism according to claim 5, characterized in that, The training image acquisition unit specifically comprises: An initial image acquisition subunit is configured to acquire an initial image of a training fish body with a labeled species; An image enhancement subunit is configured to perform denoising enhancement processing on the initial image to obtain an enhanced image; A data expansion subunit is configured to perform data expansion processing on the enhanced image to obtain an expanded image; A data cleaning subunit is configured to perform data cleaning processing on the expanded image to obtain a training image.

Images