Fish body length monitoring method based on computer vision

Abstract

The invention discloses a fish body length monitoring method based on computer vision, and relates to the technical field of computer vision. Comprising the steps that data content is acquired, and a YOLOv8 detection model is established; the YOLOv8 detection model is trained; obtaining binocular images of the fish, wherein the binocular images comprise a left image and a right image; calibration is carried out to obtain binocular camera parameters, and binocular correction is carried out on the binocular image of the fish; the trained YOLOv8 detection model identifies the binocular image of the fish, and the head and the tail of the fish are taken as key points; establishing an image coordinate system and a three-dimensional world coordinate system, and correlating a conversion relation between the two; key point detection is carried out on the fish to obtain coordinates of key points in the image coordinate system; re-projecting the key points to a three-dimensional world coordinate system; according to the key points in the three-dimensional world coordinate system, a space straight line is fitted, and the length of the space straight line is the fish body length of the fish. The method can monitor the length of the fish body in real time, is high in accuracy and efficiency, and helps to achieve real-time accurate control of marine ranching operation.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision, and in particular to a fish body length monitoring method based on computer vision. Background Art

[0002] In marine ranches, the length of fish can most directly reflect the growth of fish, which is of great significance for monitoring and evaluating the growth status of fish. Traditional fish length monitoring methods rely on relatively heavy human resources and lack efficiency. Farmers need to catch a part of the fish and measure the size of these samples to reflect the growth status of the entire fish population. This contact measurement method not only causes direct harm to the caught individuals, but also indirectly harms the entire fish population.

[0003] With the rapid development of deep learning and computer vision technology, target monitoring methods based on image recognition have been widely used and studied, especially the YOLO series of models, which have attracted attention for their efficient real-time target monitoring performance. However, the traditional YOLO model lacks a calculation method for the target recognition accuracy and distance of a binocular camera, making it difficult to accurately measure the target, resulting in its limitations in practical applications.

[0004] Fisheries and aquaculture practitioners can also use visual intelligence-based systems to monitor the ecology of farmed fish instead of manual measurements. In length estimation, since the underwater environment used for aquaculture is often affected by floating objects, turbidity and light, these pixel-level matching algorithms often cannot achieve high accuracy. The existing YOLO model will increase the complexity of the background in complex scenes, such as variable lighting conditions, complex textures or dynamic backgrounds, making it difficult for the YOLO model to distinguish between foreground targets and backgrounds. At the same time, occlusion between targets will also seriously affect the accuracy of detection, because the target features of the occluded part are incomplete, resulting in difficulty in recognition; the YOLO model has a large amount of data, high computational complexity, and long inference time. It is difficult to run on edge computing platforms or lower-end devices, resulting in poor practicality of target detection.

[0005] Therefore, it is an urgent problem for technicians in this field to propose a fish length monitoring method based on computer vision to solve the difficulties existing in the prior art. Summary of the invention

[0006] In view of this, the present invention provides a fish length monitoring method based on computer vision, comprising:

[0007] S1, obtain data content and establish a YOLOv8 detection model;

[0008] S2. Train the YOLOv8 detection model to obtain a trained YOLOv8 detection model;

[0009] S3, using a binocular camera to obtain binocular images of the fish, including a left image and a right image;

[0010] S4, calibrating the binocular camera parameters and performing binocular correction on the obtained binocular image of the fish;

[0011] S5. The trained YOLOv8 detection model recognizes the acquired binocular image of the fish, taking the head and tail of the fish as key points;

[0012] S6, establishing an image coordinate system and a three-dimensional world coordinate system, and associating the conversion relationship between them;

[0013] S7, applying a key point detection neural network to detect key points of the fish, and obtaining coordinates of the key points in the image coordinate system;

[0014] S8, applying the binocular camera parameters, reprojecting the key points into the three-dimensional world coordinate system according to the coordinates of each key point in the left and right binocular image coordinate systems;

[0015] S9. According to the key points in the three-dimensional world coordinate system, a spatial straight line is fitted, and the length of the straight line is the length of the fish body.

[0016] In the above method, optionally, in S1, the YOLOv8 detection model established includes: a Backbone layer, a Neck layer, and a Head layer;

[0017] Through the Backbone layer, multi-level feature extraction is performed on the image to be detected to obtain multi-level semantic feature maps;

[0018] The backbone network introduces the SDA module to enhance the sensitivity to the key features of the target, and introduces the LCB module to replace the original convolution module to achieve the hidden connection between channels with lower time complexity;

[0019] The semantic feature maps of multiple levels are further processed through the Neck layer, and the semantic feature maps of different levels are fused to generate fused feature maps of multiple scales;

[0020] The Head layer is responsible for the final target detection task. It inputs the fused feature maps of multiple scales into the Head layer and outputs the category, bounding box position and confidence of each detected target.

[0021] In the above method, optionally, the Backbone layer includes: a first LCB module, a combination of four modules, an SDA module and an SPPF module connected end to end in sequence;

[0022] The module combination includes an LCB module and a C2F module connected end to end;

[0023] The input of the first LCB module is used as the input of the Backbone layer; the outputs of the C2F modules of the second and third module combinations in the four module combinations are output as the semantic feature maps of the first and second levels of the Backbone layer respectively;

[0024] The output of the SPPF module is output as the semantic feature map of the third level of the Backbone layer;

[0025] The LCB module includes: convolution operation, depth-separable convolution operation and pixel shuffle operation, which is used to preserve the hidden connections between channels with lower time complexity;

[0026] The SDA module includes: a spatial attention mechanism and a channel attention mechanism to enhance sensitivity to key features of the target;

[0027] The SPPF module is used to perform a spatial pyramid pooling strategy on the feature map to fuse information of different scales together, including: the first LCB module, three 2D maximum pooling layers, a connection layer, and a second LCB module connected end to end in sequence; wherein, the outputs of the first LCB module and the three 2D maximum pooling layers are all used as inputs of the connection layer, the input of the first LCB module is the input of the SPPF module, and the output of the second LCB module is the output of the SPPF module;

[0028] The C2F module is used to directly connect the feature map of the lower level to the feature map of the higher level, including: the first LCB module, the Split layer, several Thin-neck layers, the connection layer and the second LCB module connected end to end in sequence; the outputs of the first LCB module, the Split layer and the Thin-neck layer are all used as the input of the connection layer, the input of the first LCB module is the input of the C2F module, and the output of the second LCB module is the output of the C2F module;

[0029] The Thin-neck layer includes: the first convolutional layer, the LC bottleneck layer, the connection layer and the second convolutional layer connected end to end in sequence; the output of the LC bottleneck layer passes through the branched convolutional layer and serves as the input of the connection layer; the LCbottleneck layer includes: two LCB modules connected end to end in sequence, the output of the first LCB module passes through the branched convolutional layer and is concatenated with the output of the second LCB module as the output of the LC bottleneck layer; the input of the first convolutional layer is the input of the Thin-neck layer, and the output of the second convolutional layer is the output of the Thin-neck layer.

[0030] In the above method, optionally, in S2, the YOLOv8 detection model is trained, the wildfish dataset is used to train the model, and a dataset with a uniform depth of 40 cm and a number of 450 images is selected to train the model.

[0031] In the above method, optionally, in S4, the binocular camera parameters are calibrated and binocular correction is performed on the obtained binocular image of the fish:

[0032] The Zhang Zhengyou calibration method was used to calibrate the binocular camera using a chessboard calibration plate to obtain the binocular camera parameters, including the camera's intrinsic parameter matrix, extrinsic parameter matrix and distortion coefficient, and the rotation matrix and translation vector between the two cameras; binocular correction was performed on the collected binocular images of fish.

[0033] In the above method, optionally, in S5, the acquired binocular image of the fish is recognized, and the head and tail of the fish are taken as key points;

[0034] Use Stacked Hourglass network for key point prediction;

[0035] In the key point prediction process, the head and tail are selected as the key points of the fish body;

[0036] The Stacked Hourglass network structure is three stacked third-order hourglass structures; each hourglass structure is a stage, and a key point heat map is output at the end of each stage. The jumper part of the hourglass structure is connected by channel;

[0037] When training the Stacked Hourglass network, relay supervision is applied at the end of each stage output, with a weight of 0.1 in the first stage, 0.33 in the second stage, and 1.0 in the third stage.

[0038] In the above method, optionally, in S6, an image coordinate system and a three-dimensional world coordinate system are established, and a conversion relationship between them is associated. For key points on the left and right images of the binocular image, each key point is reprojected using the triangle law of binocular camera imaging; the coordinates P (x w ,y w ,z w ) is calculated as:

[0039]

[0040] Where f is the focal length of the binocular camera, B is the baseline length of the binocular camera, l pix is the length of one pixel in the image coordinate system, x left 、x right are the horizontal coordinates of P on the left and right images of the binocular image in the image coordinate system, yleft is the ordinate of P on the left image of the binocular image in the image coordinate system, and the image coordinate system takes the center of the image as its origin.

[0041] In the above method, optionally, in S7, a key point detection neural network is applied to detect key points of the fish to obtain the coordinates of the key points in the image coordinate system:

[0042] The coordinates of the head key points in the left image of the binocular image are (x 1L ,y 1L ), the tail key point coordinates are (x 2L ,y 2L ),

[0043] The coordinates of the key points of the head in the right picture are (x 1R ,y 1R ), the tail key point coordinates are (x 2R ,y 2R ).

[0044] The above method is optional. In S8, the key points are reprojected into the three-dimensional world coordinate system according to the coordinates of each key point in the left and right binocular image coordinate systems to obtain the three-dimensional world coordinates P1(x w1 ,y w1 ,z w1 ):

[0045]

[0046] The key point of the tail is P2(x w2 ,y w2 ,z w2 ):

[0047]

[0048] In the above method, optionally, in S9, according to the key point of the head and the key point of the tail in the three-dimensional world coordinate system, the length of the straight line between the two points is the length L of the fish body. fish :

[0049]

[0050] It can be seen from the above technical solutions that, compared with the prior art, the present invention provides a fish length monitoring method based on computer vision, which has the following beneficial effects: 1) An adaptive double-layer attention mechanism is introduced in the Backbone layer of the YOLOv8 model to enhance the sensitivity to the key features of the target, solve the challenges of target detection in complex scenes such as illumination changes, complex textures or target occlusion, and help the model to better focus on important features, have strong adaptability to application environments and scenes, and improve the recognition accuracy of targets; 2) The adaptive double-layer attention mechanism can calculate the attention weights of different scales in the multi-layer feature map, and highlight the important scale information, so that the model pays more attention to the features of targets of different scales, thereby further improving the accuracy of target detection; 3) It will not interfere with the ecology of the fish school, and the entire measurement process can be automatically realized by the program; 4) The idea of ​​key point detection is introduced, only focusing on the spatial position of specific key points on the fish, the required equipment is simple, only an underwater binocular camera and a computing backend are required, and the length of multiple fish in the image can be measured in real time, with high accuracy and efficiency, meeting the monitoring needs of fish growth status in the aquaculture industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0052] Figure 1 A flow chart of a fish length monitoring method based on computer vision provided by the present invention;

[0053] Figure 2 The structure diagram of the YOLOv8 detection model provided by the present invention;

[0054] Figure 3 A structural diagram of the LCB module in the YOLOv8 detection model provided by the present invention;

[0055] Figure 4 A structural diagram of the SDA module in the YOLOv8 detection model provided by the present invention;

[0056] Figure 5 This is a structural diagram of the Thin-neck module in the YOLOv8 detection model provided by the present invention;

[0057] Figure 6 A structural diagram of the Stacked Hourglass key point detection network provided by the present invention;

[0058] Figure 7 This is a schematic diagram of the similar triangle principle of binocular imaging provided by the present invention. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] In this application, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of more restrictions, the elements defined by the sentence "comprise one..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0061] Reference Figure 1 As shown, a fish length monitoring method based on computer vision includes:

[0062] S1, obtain data content and establish a YOLOv8 detection model;

[0063] S2. Train the YOLOv8 detection model to obtain a trained YOLOv8 detection model;

[0064] S3, using a binocular camera to obtain binocular images of the fish, including a left image and a right image;

[0065] S4, calibrating the binocular camera parameters and performing binocular correction on the obtained binocular image of the fish;

[0066] S5. The trained YOLOv8 detection model recognizes the acquired binocular image of the fish, taking the head and tail of the fish as key points;

[0067] S6, establishing an image coordinate system and a three-dimensional world coordinate system, and associating the conversion relationship between them;

[0068] S7, applying a key point detection neural network to detect key points of the fish, and obtaining coordinates of the key points in the image coordinate system;

[0069] S8, applying the binocular camera parameters, reprojecting the key points into the three-dimensional world coordinate system according to the coordinates of each key point in the left and right binocular image coordinate systems;

[0070] S9. According to the key points in the three-dimensional world coordinate system, a spatial straight line is fitted, and the length of the straight line is the length of the fish body.

[0071] Further, see Figure 2 As shown, in S1, the established YOLOv8 detection model includes: Backbone layer, Neck layer, and Head layer;

[0072] Through the Backbone layer, multi-level feature extraction is performed on the image to be detected to obtain multi-level semantic feature maps;

[0073] The backbone network introduces the SDA module to enhance the sensitivity to the key features of the target, and introduces the LCB module to replace the original convolution module to achieve the hidden connection between channels with lower time complexity;

[0074] The semantic feature maps of multiple levels are further processed through the Neck layer, and the semantic feature maps of different levels are fused to generate fused feature maps of multiple scales. The Neck layer is an intermediate layer used to fuse the features from the backbone layer to improve the performance of the model.

[0075] The Head layer is responsible for the final target detection task. It inputs the fused feature maps of multiple scales into the Head layer and outputs the category, bounding box position and confidence of each detected target.

[0076] The Head layer is the last layer of the model. Its structure varies depending on the task. In image classification tasks, a softmax classifier is usually used, while in object detection tasks, a bounding box regressor and classifier may be used. The non-maximum suppression (NMS) algorithm is also applied to remove candidate boxes with high overlap and retain the most accurate target box.

[0077] Furthermore, the Backbone layer includes: the first LCB module, four module combinations, the SDA module, and the SPPF module connected end to end in sequence;

[0078] The module combination includes an LCB module and a C2F module connected end to end;

[0079] The input of the first LCB module is used as the input of the Backbone layer; the outputs of the C2F modules of the second and third module combinations in the four module combinations are output as the semantic feature maps of the first and second levels of the Backbone layer respectively;

[0080] The output of the SPPF module is output as the semantic feature map of the third level of the Backbone layer.

[0081] Reference Figure 3 As shown, the LCB module includes: a convolution operation, a depth-separable convolution operation, and a pixel shuffle operation, which is used to preserve the hidden connections between channels with lower time complexity;

[0082] Specifically, a lightweight convolution LCB module is introduced to retain the hidden connections between channels with lower time complexity; first, the lightweight convolution module can reduce the computational complexity of the model while maintaining performance, so that the model can complete the target detection task in a shorter time; secondly, the lightweight convolution module usually has fewer parameters and a smaller model size, which helps to reduce memory usage and enables the model to run on devices with limited resources, thereby improving the practicality of target detection; finally, due to the introduction of the lightweight convolution module, the YOLOv8 model can achieve higher real-time performance while maintaining high accuracy, which is crucial for application scenarios that require fast response.

[0083] Reference Figure 4 As shown, the SDA module includes: a spatial attention mechanism and a channel attention mechanism to enhance the sensitivity to the key features of the target;

[0084] Specifically, an adaptive two-layer attention mechanism is introduced to enhance the sensitivity to the key features of the target. First, the adaptive two-layer attention mechanism solves the challenges of target detection in complex scenes, such as lighting changes, complex textures or target occlusion, by enhancing the sensitivity to the key features of the target. This attention mechanism helps the model to better focus on important features and improve the recognition accuracy of the target; secondly, the adaptive two-layer attention mechanism can calculate the attention weights of different scales in the multi-layer feature map and highlight the important scale information, so that the model pays more attention to the features of targets of different scales, thereby further improving the accuracy of target detection; finally, traditional target detection algorithms may have difficulties in processing multi-scale targets, but the introduction of the adaptive two-layer attention mechanism can solve this problem well, because it can give different attention weights according to different scale information, thereby better detecting targets of different sizes.

[0085] The SPPF module is used to execute the spatial pyramid pooling strategy on the feature map to fuse information of different scales together, including: the first LCB module, three 2D maximum pooling layers, the connection layer and the second LCB module connected end to end in sequence; among them, the outputs of the first LCB module and the three 2D maximum pooling layers are all used as the input of the connection layer, the input of the first LCB module is the input of the SPPF module, and the output of the second LCB module is the output of the SPPF module.

[0086] The C2F module is used to directly connect the feature map of the lower level to the feature map of the higher level, including: the first LCB module, Split layer, several Thin-neck layers, connection layer and the second LCB module connected end to end in sequence; the outputs of the first LCB module, Split layer and Thin-neck layer are all used as the input of the connection layer, the input of the first LCB module is the input of the C2F module, and the output of the second LCB module is the output of the C2F module.

[0087] Reference Figure 5 As shown in the figure, the Thin-neck layer includes: the first convolution layer, the LC bottleneck layer, the connection layer and the second convolution layer connected end to end in sequence; the output of the LC bottleneck layer passes through the branch convolution layer and serves as the input of the connection layer; the LC bottleneck layer includes: two LCB modules connected end to end in sequence, the output of the first LCB module passes through the branch convolution layer and is concatenated with the output of the second LCB module as the output of the LC bottleneck layer; the input of the first convolution layer is the input of the Thin-neck layer, and the output of the second convolution layer is the output of the Thin-neck layer.

[0088] Furthermore, in S2, the YOLOv8 detection model is trained using the wildfish dataset;

[0089] Furthermore, before training, a training data set is constructed by selecting 450 images with a uniform depth of 40 cm and a number of images, that is, selecting training samples and setting the candidate box size for the training samples, and then using the training samples and their candidate boxes as training data; during training, first set the training parameters and initialize the network structure of the target detection model, then load the training data and use it as the input of the target detection model, and continuously optimize the network parameters of the target detection model by calculating the loss function until the model converges, completing the training of the target detection model.

[0090] Furthermore, in S4, the binocular camera parameters are calibrated and binocular correction is performed on the obtained binocular image of the fish:

[0091] The parameters of the binocular camera are obtained by calibration in an underwater environment, including the intrinsic parameter matrix of each camera, the distortion coefficient, the rotation matrix and the translation vector between the two cameras; the Zhang Zhengyou calibration method is used to calibrate through a chessboard image, and a python script is written to obtain the chessboard image and detect the corner points. The binocular camera parameters are calculated by detecting the world coordinate system coordinates of the corner points and the corresponding pixel positions; binocular correction is performed on the collected binocular images of fish.

[0092] Further, in S5, the obtained binocular image of the fish is recognized, and the head and tail of the fish are taken as key points; for a frame of the binocular image, the positions of the bounding boxes in the left image and the right image are predicted, and then the binocular bounding boxes are matched on these bounding boxes to obtain the bounding boxes containing the same fish in the left image and the right image;

[0093] Reference Figure 6 As shown, the Stacked Hourglass network is used for key point prediction. For each pair of successfully matched bounding boxes, the trained Stacked Hourglass network is used to predict the key points of the fish body.

[0094] In the key point prediction process, the head and tail are selected as the key points of the fish body;

[0095] The Stacked Hourglass network structure is three stacked third-order hourglass structures; each hourglass structure is a stage, and a key point heat map is output at the end of each stage. The jumper part of the hourglass structure is connected by channel;

[0096] When training the Stacked Hourglass network, relay supervision is applied at the end of each stage output, with a weight of 0.1 in the first stage, 0.33 in the second stage, and 1.0 in the third stage.

[0097] Further, in S6, the image coordinate system and the three-dimensional world coordinate system are established, and the conversion relationship between them is associated, referring to Figure 7 As shown in the figure, for the key points on the left and right images of the binocular image, each key point is reprojected by the triangle law of binocular camera imaging; the coordinates of the key point P in the three-dimensional world coordinate system are P(x w ,y w ,z w ) is calculated as:

[0098]

[0099] Where f is the focal length of the binocular camera, B is the baseline length of the binocular camera, l pix is the length of one pixel in the image coordinate system, x left 、x right are the horizontal coordinates of P on the left and right images of the binocular image in the image coordinate system, y left is the ordinate of P on the left image of the binocular image in the image coordinate system, and the image coordinate system takes the center of the image as its origin.

[0100] Furthermore, in S7, a key point detection neural network is applied to detect key points of the fish, and the coordinates of the key points in the image coordinate system are obtained:

[0101] The coordinates of the head key points in the left image of the binocular image are (x 1L ,y 1L ), the tail key point coordinates are (x 2L ,y 2L ),

[0102] The coordinates of the key points of the head in the right picture are (x 1R ,y 1R ), the tail key point coordinates are (x 2R ,y 2R ).

[0103] Furthermore, in S8, the key points are reprojected into the three-dimensional world coordinate system according to the coordinates of each key point in the left and right binocular image coordinate systems to obtain the three-dimensional world coordinates P1(x w1 ,y w1 ,z w1 ):

[0104]

[0105] The key point of the tail is P2(x w2 ,y w2 ,z w2 ):

[0106]

[0107] Further, in S9, according to the key point of the head and the key point of the tail in the three-dimensional world coordinate system, the length of the straight line between the two points is the length L of the fish body. fish :

[0108]

[0109] In a specific embodiment, the focal length of the binocular camera used is 1 mm, the baseline length is 60 mm, and the pixel size is 0.003 mm. Calibration is performed using a chessboard image, and a python script is written to obtain the chessboard image and detect corner points. The binocular camera parameters are calculated by detecting the world coordinate system coordinates of the corner points and the corresponding pixel positions. The binocular camera is placed underwater, and sufficient binocular images containing farmed fish are obtained as a data set. These images are annotated, and the bounding box and key point positions of the fish body need to be annotated. The binocular camera parameters are applied, and the key points are reprojected to the three-dimensional world coordinate system through the coordinates of each key point in the left and right binocular image coordinate systems. According to the head key point and the tail key point in the three-dimensional world coordinate system, the straight line length between the two key points is calculated, which is the fish body length L fish .

[0110] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.

[0111] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A fish length monitoring method based on computer vision, characterized in that: include: S1, obtain data content and establish a YOLOv8 detection model; S2. Train the YOLOv8 detection model to obtain a trained YOLOv8 detection model; S3, using a binocular camera to obtain binocular images of the fish, including a left image and a right image; S4, calibrating the binocular camera parameters and performing binocular correction on the obtained binocular image of the fish; S5. The trained YOLOv8 detection model recognizes the acquired binocular image of the fish, taking the head and tail of the fish as key points; S6, establishing an image coordinate system and a three-dimensional world coordinate system, and associating the conversion relationship between them; S7, applying a key point detection neural network to detect key points of the fish, and obtaining coordinates of the key points in the image coordinate system; S8, applying the binocular camera parameters, reprojecting the key points into the three-dimensional world coordinate system according to the coordinates of each key point in the left and right binocular image coordinate systems; S9. According to the key points in the three-dimensional world coordinate system, a spatial straight line is fitted, and the length of the straight line is the length of the fish body.

2. The fish body length monitoring method based on computer vision according to claim 1, characterized in that: In S1, the established YOLOv8 detection model includes: Backbone layer, Neck layer, and Head layer; Through the Backbone layer, multi-level feature extraction is performed on the image to be detected to obtain multi-level semantic feature maps; The semantic feature maps of multiple levels are further processed through the Neck layer, and the semantic feature maps of different levels are fused to generate fused feature maps of multiple scales; The Head layer is responsible for the final target detection task. It inputs the fused feature maps of multiple scales into the Head layer and outputs the category, bounding box position and confidence of each detected target.

3. The fish body length monitoring method based on computer vision according to claim 2 is characterized in that: The Backbone layer includes: the first LCB module, four module combinations, SDA module and SPPF module connected end to end in sequence; The module combination includes an LCB module and a C2F module connected end to end; The input of the first LCB module is used as the input of the Backbone layer; the outputs of the C2F modules of the second and third module combinations in the four module combinations are output as the semantic feature maps of the first and second levels of the Backbone layer respectively; The output of the SPPF module is output as the semantic feature map of the third level of the Backbone layer; The LCB module includes: convolution operation, depth-separable convolution operation and pixel shuffle operation; The SDA module includes: spatial attention mechanism and channel attention mechanism; The SPPF module includes: a first LCB module, three 2D maximum pooling layers, a connection layer, and a second LCB module connected end to end in sequence; wherein the outputs of the first LCB module and the three 2D maximum pooling layers are all used as inputs of the connection layer, the input of the first LCB module is the input of the SPPF module, and the output of the second LCB module is the output of the SPPF module; The C2F module includes: the first LCB module, the Split layer, several Thin-neck layers, the connection layer and the second LCB module connected end to end in sequence; the outputs of the first LCB module, the Split layer and the Thin-neck layer are all used as inputs of the connection layer, the input of the first LCB module is the input of the C2F module, and the output of the second LCB module is the output of the C2F module; The Thin-neck layer includes: the first convolutional layer, the LC bottleneck layer, the connection layer and the second convolutional layer connected end to end in sequence; the output of the LC bottleneck layer passes through the branched convolutional layer and serves as the input of the connection layer; the LC bottleneck layer includes: two LCB modules connected end to end in sequence, the output of the first LCB module passes through the branched convolutional layer and is concatenated with the output of the second LCB module as the output of the LC bottleneck layer; the input of the first convolutional layer is the input of the Thin-neck layer, and the output of the second convolutional layer is the output of the Thin-neck layer.

4. The fish body length monitoring method based on computer vision according to claim 1, characterized in that: In S2, the YOLOv8 detection model is trained using the wildfish dataset. A dataset with a uniform depth of 40 cm and a number of 450 images is selected for training the model.

5. The fish body length monitoring method based on computer vision according to claim 1, characterized in that: In S4, the binocular camera parameters are calibrated and binocular correction is performed on the obtained binocular image of the fish: The Zhang Zhengyou calibration method was used to calibrate the binocular camera using a chessboard calibration plate to obtain the binocular camera parameters, including the camera's intrinsic parameter matrix, extrinsic parameter matrix and distortion coefficient, and the rotation matrix and translation vector between the two cameras; binocular correction was performed on the collected binocular images of fish.

6. The fish body length monitoring method based on computer vision according to claim 1, characterized in that: In S5, the obtained binocular image of the fish is recognized, and the head and tail of the fish are taken as key points; Use Stacked Hourglass network for key point prediction; In the key point prediction process, the head and tail are selected as the key points of the fish body; The Stacked Hourglass network structure is three stacked third-order hourglass structures; each hourglass structure is a stage, and a key point heat map is output at the end of each stage. The jumper part of the hourglass structure is connected by channel; When training the Stacked Hourglass network, relay supervision is applied at the end of each stage output, with a weight of 0.1 in the first stage, 0.33 in the second stage, and 1.0 in the third stage.

7. The fish body length monitoring method based on computer vision according to claim 1, characterized in that: In S6, the image coordinate system and the three-dimensional world coordinate system are established, and the conversion relationship between them is associated. For the key points on the left and right images of the binocular image, each key point is reprojected according to the triangle law of binocular camera imaging; the coordinates P (x w ,y w ,z w ) is calculated as: Where f is the focal length of the binocular camera, B is the baseline length of the binocular camera, l pix is the length of one pixel in the image coordinate system, x left 、x right are the horizontal coordinates of P on the left and right images of the binocular image in the image coordinate system, y left is the ordinate of P on the left image of the binocular image in the image coordinate system, and the image coordinate system takes the center of the image as its origin.

8. The fish body length monitoring method based on computer vision according to claim 1, characterized in that: In S7, the key point detection neural network is used to detect the key points of the fish and obtain the coordinates of the key points in the image coordinate system: The coordinates of the head key points in the left image of the binocular image are (x 1L ,y 1L ), the tail key point coordinates are (x 2L ,y 2L ), the coordinates of the key points of the head in the right figure are (x 1R ,y 1R ), the tail key point coordinates are (x 2R ,y 2R ).

9. The fish body length monitoring method based on computer vision according to claim 1, characterized in that: In S8, the key points are reprojected into the three-dimensional world coordinate system by using the coordinates of each key point in the left and right binocular image coordinate systems to obtain the three-dimensional world coordinates P1(x w1 ,y w1 ,z w1 ): The tail key point 3D world left P2 (x w2 ,y w2 ,z w2 ):

10. The fish body length monitoring method based on computer vision according to claim 1, characterized in that: In S9, according to the key points of the head and the tail in the three-dimensional world coordinate system, the length of the straight line between the two points is the length of the fish body L fish :

Images