Fish body comprehensive detection method and system, computer device and storage medium

Abstract

The application discloses a fish body comprehensive detection method and system, computer equipment and a storage medium. The method comprises the following steps: acquiring a main view image and a top view image of a fish body through different-view-angle depth cameras; adopting a trained target detection model to perform target detection on the main view image and the top view image, and determining initial position information and species information of the fish body; performing sub-pixel edge detection on the initial position information of the fish body to obtain edge position information of the fish body; obtaining coordinate information of the fish body edge in a corresponding camera coordinate system according to the edge position information of the fish body and in combination with depth information of the depth camera, and obtaining fish body size information through calculation of Euclidean distance; and predicting the fish body weight according to the fish body size information and the fish body species information, so as to obtain a fish body growth state. The application improves the accuracy of fish body growth state detection, and can timely find abnormal conditions of fish body growth.

Description

TECHNICAL FIELD

[0001] The application relates to a fish body comprehensive detection method, system, computer device and storage medium, and belongs to the field of fish body measurement. BACKGROUND

[0002] Fish body measurement is of great significance for understanding the growth state of fish and can provide key information for fishery resource management and protection. Traditional fish body measurement techniques include weight measurement and length measurement, which can provide data on fish growth, quantity and population structure, and provide reference for fishery resource management and protection. However, traditional fish body measurement techniques also have some limitations: first, the measurement process may cause harm to fish, especially when large-scale measurement is performed, unnecessary death may occur; second, traditional measurement techniques usually require a lot of manpower and material resources, and the work efficiency is low; in addition, the measurement data may have errors and uncertainties, which affect the accuracy and reliability of the research.

[0003] Therefore, the development of modern fish body measurement technology also brings important opportunities, such as sonar technology and remote sensing technology, which can more accurately obtain fish body feature data and can realize non-invasive measurement, improving the measurement efficiency and accuracy, but there are still problems such as high cost and inconvenient measurement. In recent years, with the continuous development of computer vision technology, various methods for accurately measuring fish body by using high-definition camera to shoot fish body and then using image processing algorithm have emerged, but some of the current methods still have defects such as poor robustness and great influence of fish group overlap shielding. SUMMARY

[0004] Therefore, the application provides a fish body comprehensive detection method, system, computer device and storage medium, which improves the accuracy of fish body growth state detection and can timely find abnormal conditions of fish body growth.

[0005] A first object of the application is to provide a fish body comprehensive detection method

[0006] A second object of the application is to provide a fish body comprehensive detection system.

[0007] A third object of the application is to provide a computer device.

[0008] A fourth object of the application is to provide a storage medium.

[0009] The first object of the application can be achieved by adopting the following technical solutions:

[0010] A fish body comprehensive detection method, the method comprising:

[0011] The main view image and the top view image of the fish body are acquired by different perspective depth cameras, and the different perspective depth cameras include a main view image acquisition depth camera and a top view image acquisition depth camera;

[0012] The trained target detection model is used for target detection on the main view image and the top view image to determine fish initial position information and fish species information;

[0013] Sub-pixel edge detection is performed on the fish initial position information to obtain fish edge position information;

[0014] According to the fish edge position information, combined with the depth information of the depth camera, the coordinate information of the fish edge in the corresponding camera coordinate system is obtained, and the fish size information is obtained by calculating the Euclidean distance;

[0015] According to the fish size information and the fish species information, the fish body weight is predicted, and thus the fish growth state is obtained.

[0016] Further, the trained target detection model is used for target detection on the main view image and the top view image to determine fish initial position information and fish species information, specifically including:

[0017] A backbone network is used for feature extraction on the fish image, and the backbone network is an EfficientNet lightweight network;

[0018] A neck network is used to perform top-down and bottom-up cross-layer connection on the extracted features, so that the features are fused;

[0019] A decoupling head module is used to separate the regression branch and the prediction branch to accelerate the convergence of the target detection model, and the regression branch and the prediction branch are used to process the fused features respectively to obtain fish initial position information and fish species information.

[0020] Further, the sub-pixel edge detection is performed on the fish initial position information to obtain the fish edge position information, specifically including:

[0021] According to the outer frame of the fish initial position information, the main view image or the top view image is cropped;

[0022] Sub-pixel edge detection is performed on the cropped region to determine fish edge information and the position information of the fish edge in the main view image or the top view image.

[0023] Further, the sub-pixel edge detection is performed on the cropped region to determine fish edge information and the position information of the fish edge in the main view image or the top view image, specifically including:

[0024] The pixel-level edge detection is performed on the cropped region using a Canny operator to obtain a fish body pixel-level edge profile;

[0025] The pixel-level edge is convolved with a 7X7 template to obtain multi-order Zernike moments, and the parameters of each edge pixel point are calculated using the Zernike moments;

[0026] An iterative method is used to obtain an optimal threshold of the image, and the optimal threshold of the image is saved as an optimal edge step grayscale height threshold;

[0027] According to the optimal edge step grayscale height threshold, it is judged whether each edge pixel point meets a preset determination condition, if yes, the sub-pixel edge coordinates are calculated, if not, the fish body edge information and the position information of the fish body edge in the main view image or the top view image are obtained.

[0028] Further, the camera coordinate systems of the main view image acquisition depth camera and the top view image acquisition depth camera are a main view camera coordinate system and a top view camera coordinate system, respectively;

[0029] According to the fish body edge position information, the depth information of the depth camera is combined to obtain the coordinate information of the fish body edge in the corresponding camera coordinate system, and the fish body size information is obtained by calculating the Euclidean distance, specifically including:

[0030] According to the fish body edge position information of different views, the coordinates of the upper, lower, left and right extreme points of the fish body edge in the main view image and the top view image are obtained, respectively;

[0031] According to the depth information of the depth camera of different views, the coordinate information of the fish body extreme points in different views is combined to obtain the coordinate information of the fish body extreme points in the main view camera coordinate system and the top view camera coordinate system, respectively;

[0032] The coordinates of the left and right extreme points of the fish body in the top view camera coordinate system are converted into the coordinates in the main view camera coordinate system, and compared with the coordinates of the left and right extreme points of the fish body in the main view camera coordinate system, if the two coordinate positions are the same or within a preset error range, the two coordinates are the position information of the same fish body in different depth camera views, that is, the same fish body appears in different depth camera views, otherwise, the fish body only appears in one depth camera view;

[0033] According to the appearance of the fish body, the Euclidean distance between two points in space is calculated to obtain the fish body size information.

[0034] Further, according to the appearance of the fish body, the Euclidean distance between two points in space is calculated to obtain the fish body size information, specifically including:

[0035] For the same fish body appearing in different depth camera view angles, the average value of the lengths calculated by the main view camera coordinate system and the top view camera coordinate system is taken as the actual length of the fish body, the distance between the upper and lower extreme points in the main view camera coordinate system is taken as the height of the fish body, and the distance between the upper and lower extreme points in the top view camera coordinate system is taken as the width of the fish body;

[0036] For the fish body appearing in only one depth camera view angle, if the fish body appears only in the main view image acquisition depth camera view angle, the distance between the left and right extreme points is taken as the length of the fish body, and the distance between the upper and lower extreme points is taken as the height of the fish body; if the fish body appears in the top view image acquisition depth camera view angle, the distance between the left and right extreme points is taken as the length of the fish body, and the distance between the upper and lower extreme points is taken as the width of the fish body.

[0037] Further, the fish body weight is predicted according to the fish body size information and the fish body species information, so as to obtain the fish body growth state, and specifically includes:

[0038] A regression analysis model is established for different fish species, and regression analysis is performed on the relationships between the length and width and the fish body weight, the relationships between the length and height and the fish body weight, and the relationships between the length, height and width and the fish body weight of different fish species based on existing prior data, so as to obtain three types of function models of different fish species;

[0039] Based on the fish body size information and the fish body species information, the fish body weight is predicted by using the function model;

[0040] By comparing the fish body weights of the same batch of the same species, and comparing the predicted fish body weight with historical fish body growth data of the same period, the fish body with abnormal growth in the same batch of the same species is obtained, and the deviation of the current fish body growth from the normal condition is obtained.

[0041] The second object of the application can be achieved by adopting the following technical scheme:

[0042] A fish body comprehensive detection system, the system comprises:

[0043] An image acquisition unit is configured to acquire a main view image and a top view image of a fish body by using different view angle depth cameras, wherein the different view angle depth cameras comprise a main view camera and a top view camera;

[0044] A target detection unit is configured to perform target detection on the main view image and the top view image by using a trained target detection model, and determine initial position information and species information of the fish body;

[0045] An edge detection unit is configured to perform sub-pixel edge detection on the initial position information of the fish body, and obtain edge position information of the fish body;

[0046] A size calculation unit is configured to obtain coordinate information of the fish body edge in a corresponding camera coordinate system according to the fish body edge position information and depth information of the depth camera, and obtain fish body size information by calculating the Euclidean distance.

[0047] A weight prediction unit is configured to predict the fish body weight according to the fish body size information and the fish body type information, so as to obtain the fish body growth state.

[0048] The third object of the present application can be achieved by adopting the following technical solution:

[0049] A computer device comprises a processor and a memory for storing programs executable by the processor, characterized in that the processor implements the fish body comprehensive detection method when executing the programs stored in the memory.

[0050] The fourth object of the present application can be achieved by adopting the following technical solution:

[0051] A storage medium stores a program, and the program is executed by a processor to implement the fish body comprehensive detection method.

[0052] The present application has the following beneficial effects compared with the prior art:

[0053] 1. The present application uses multi-view cameras to obtain fish body information, effectively reducing the influence of mutual shielding between fish bodies on the accuracy of the results, and through layer-by-layer progressive processing of fish body image data, realizes comprehensive detection of fish body type, fish body size, fish body weight and fish body growth state, and improves the convenience and efficiency of the system.

[0054] 2. The present application uses a target monitoring model based on a lightweight backbone network to detect fish body type and the rough position information of the fish body, which is convenient for subsequent accurate positioning of the fish body position and prediction of the fish body weight, and the lightweight network is also more convenient for deployment and application of the model.

[0055] 3. The present application uses an improved Zernike moment sub-pixel detection method to more accurately detect the initial position information of the fish body, improves the accuracy of the initial position information of the fish body, and makes the subsequent fish body size information and weight information measurement more accurate.

[0056] 4. The present application establishes different fish body weight regression models based on multi-dimensional data such as fish body length and width for different fish species, so that the fish body weight prediction is more accurate. BRIEF DESCRIPTION OF DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those of ordinary skill in the art without creative labor based on these drawings.

[0058] Figure 1 The fish body comprehensive detection method flow chart of the embodiment 1 of the present application.

[0059] Figure 2 The different perspective depth camera arrangement schematic diagram of the embodiment 1 of the present application.

[0060] Figure 3 The sub-pixel edge detection flow chart of the embodiment 1 of the present application.

[0061] Figure 4 The fish body initial position information schematic diagram under the fish body image acquisition depth camera perspective of the embodiment 1 of the present application.

[0062] Figure 5 The fish body initial position information schematic diagram under the fish body image acquisition depth camera perspective of the embodiment 1 of the present application.

[0063] Figure 6 The structure block diagram of the fish body comprehensive detection system of the embodiment 2 of the present application.

[0064] Figure 7 The structure block diagram of the computer device of the embodiment 2 of the present application. DETAILED DESCRIPTION

[0065] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the following will combine the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the protection scope of the present application.

[0066] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as generally understood by those of ordinary skill in the art to which the present application belongs.

[0067] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments of the invention; as used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that the terms “comprising” and “including” and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such processes, methods, products, or apparatus.

[0068] Example 1:

[0069] like Figure 1 As shown in the figure, this embodiment provides a comprehensive fish detection method, which includes the following steps:

[0070] S101. Obtain front view and top view images of the fish body using depth cameras with different perspectives.

[0071] The different perspective depth cameras in this embodiment include a front view image acquisition depth camera and a top view image acquisition depth camera. The arrangement of the two depth cameras is as follows: Figure 2 As shown, the fish are arranged in a cross shape. Two depth cameras capture images of the fish in the front view direction and the top view direction, respectively, which are the front view image and the top view image.

[0072] To facilitate subsequent positioning and calculation, the camera coordinate systems of the depth camera acquiring the front view image and the depth camera acquiring the top view image are divided into a front view camera coordinate system and a top view camera coordinate system. In the front view camera coordinate system, the horizontal direction of the camera center is the X-axis, the vertical direction is the Y-axis, and the direction perpendicular to the lens is the Z-axis. The top view camera coordinate system is simply the front view camera coordinate system rotated 90 degrees around the X-axis. Let the front view camera coordinate system be the main coordinate system, and let the coordinates of the top view camera in the front view camera coordinate system be (X1, Y1, Z1). Let the coordinates of the measured point K in the front view camera coordinate system be (X1, Y1, Z1). K Y K Z K The coordinates of the measured point K in the top-view camera coordinate system are (X' K Y' K Z' K Then, the coordinate transformation relationship between the measured point in the front view camera coordinate system and the top view camera coordinate system is as follows (1):

[0073] (X K ,Y K Z K )=((X1+X' K ),(Z1+Z'K ),(Y1+Y' K )) (1)

[0074] S102. The trained target detection model is used to perform target detection on the front view image and top view image to determine the initial position information and fish species information of the fish.

[0075] The target detection model in this embodiment is a deep learning model based on the YOLOv8 algorithm. The target detection model is trained using a training set, and then the trained target detection model is used to detect targets in the front view image and top view image to determine the initial position information and species information of the fish. The training set is an image dataset from a pre-collected database.

[0076] Furthermore, step S102 specifically includes:

[0077] S1021. A backbone network is used to extract features from fish images. In order to improve the running speed of the model and reduce the number of model parameters, the backbone network uses the EfficientNet lightweight network instead of the original Darknet53 network to extract fish image features.

[0078] S1022. A neck network is used to perform cross-layer connections between the extracted features in a top-down and bottom-up manner, so that the features can be fused more fully.

[0079] S1023. A decoupling head module is used to separate the regression branch and the prediction branch, which accelerates the convergence of the target detection model. The fused features are processed by the regression branch and the prediction branch respectively to obtain the initial position information of the fish and the fish species information.

[0080] S103. Perform sub-pixel edge detection on the initial position information of the fish body to obtain the edge position information of the fish body.

[0081] Furthermore, step S103 specifically includes:

[0082] S1031. Based on the outer border of the fish's initial position information, crop the front view image or top view image.

[0083] S1032. Perform subpixel edge detection on the cropped area to determine the fish body edge information and the position information of the fish body edge in the front view image or top view image.

[0084] Those skilled in the art will readily understand that when the input image is a front view image, the position information of the fish's edge in the front view image can be obtained; when the input image is a top view image, the position information of the fish's edge in the top view image can be obtained.

[0085] likeFigure 3 As shown, step S1032 specifically includes:

[0086] 1) Perform pixel-level edge detection on the cropped region using a Canny operator to obtain a fish body pixel-level edge profile;

[0087] 2) Convolve the pixel-level edge with a 7X7 template to obtain multi-order Zernike moments, and use the Zernike moments to calculate the parameters of each edge pixel point.

[0088] 3) Use an iterative method to obtain an optimal threshold value of the image, and save the optimal threshold value of the image as an optimal edge step grayscale height threshold value.

[0089] 4) According to the optimal edge step grayscale height threshold value, determine whether each edge pixel point satisfies a preset determination condition, if yes, calculate a sub-pixel edge coordinate, if not, eliminate, so as to obtain fish body edge information and position information of the fish body edge in a front view image or a top view image, wherein the preset determination condition is as follows:

[0090] (k≥k t )∩(d≤(2d t / N)) (2)

[0091] Wherein, k represents an edge step grayscale height, k t represents an edge step grayscale height threshold value, d represents a vertical distance from a pixel center to an edge, d t represents a distance threshold value.

[0092] S104, according to the fish body edge position information, combine the depth information of the depth camera to obtain coordinate information of the fish body edge in a corresponding camera coordinate system, and calculate the Euclidean distance to obtain fish body size information.

[0093] Based on the fish body edge position information, the depth information of the depth camera can be obtained, and then the Euclidean distance can be calculated to obtain the fish body size information such as length, width and height. The fish body position information in the front view image under the depth camera view is shown in Figure 4 , and the fish body position information in the top view image under the depth camera view is shown in Figure 5 .

[0094] Further, this step S1041 specifically includes:

[0095] S1041, according to the fish body edge position information of different views, respectively obtain the coordinates of the upper, lower, left and right four extreme points of the fish body edge in the front view image and the top view image. The coordinates of the four extreme points are respectively: 1c , Y1c ), the right extreme point (X 2c , Y 2c ), the upper extreme point (X 1G , Y 1G ), and the lower extreme point (X 2G , Y 2G ); the coordinates of the four extreme points of the fish body edge in the top view are: the left extreme point (X' 1c , Y' 1c ), the right extreme point (X' 2c , Y '2c ), the upper extreme point (X' 1K , Y' 1K ), and the lower extreme point (X' 2K , Y' 2K ).

[0096] S1042, according to the depth information of the different-view-angle depth camera, and in combination with the coordinate information of the fish body extreme points in different views, the fish body extreme point coordinate information in the main view camera coordinate system and the top view camera coordinate system is obtained respectively, the fish body extreme point coordinates in the main view camera coordinate system are: the left extreme point (X 1c , Y 1c , Z 1c ), the right extreme point (X 2c , Y 2c , Z 2c ), the upper extreme point (X 1G , Y 1G , Z 1G ), and the lower extreme point (X 2G , Y 2G , Z 2G ); the fish body extreme point coordinates in the top view camera coordinate system are: the left extreme point (X' 1c , Y' 1c , Z' 1c ), the right extreme point (X' 2c , Y '2c , Z '2c ), the upper extreme point (X' 1K , Y' 1K , Z' 1K ), and the lower extreme point (X' 2K , Y' 2K , Z' 2K ).

[0097] S1043, convert the left and right extreme value point coordinates of the fish body in the top view camera coordinate system into coordinates in the front view camera coordinate system, and compare them with the left and right extreme value point coordinates of the fish body in the front view camera coordinate system. If the two coordinate positions are the same or within a preset error range, then the two coordinates are the position information of the same fish body at different depth camera viewing angles, that is, the same fish body appears at different depth camera viewing angles. Otherwise, the fish body only appears at one depth camera viewing angle.

[0098] S1044, according to the appearance of the fish body, calculate the Euclidean distance of the two points in space to obtain the size information of the fish body.

[0099] 1) For the case where the same fish body appears at different depth camera viewing angles, take the average of the lengths calculated by the front view camera coordinate system and the top view camera coordinate system as the actual length of the fish body, take the distance between the upper and lower extreme points in the front view camera coordinate system as the height of the fish body, and take the distance between the upper and lower extreme points in the top view camera coordinate system as the width of the fish body.

[0100] 2) For the case where the fish body only appears at one depth camera viewing angle, if the fish body only appears at the front view image acquisition depth camera viewing angle, then the distance between the left and right extreme points is taken as the length of the fish body, and the distance between the upper and lower extreme points is taken as the height of the fish body; if the fish body appears at the top view image acquisition depth camera viewing angle, then the distance between the left and right extreme points is taken as the length of the fish body, and the distance between the upper and lower extreme points is taken as the width of the fish body.

[0101] The Euclidean distance calculation formula of this embodiment is as follows:

[0102]

[0103] wherein, l is the distance between two points in space, (x1, y1, z1) and (x2, y2, z2) are the coordinates of the two points.

[0104] S105, according to the size information of the fish body and the species information of the fish body, predict the weight of the fish body, and thus obtain the growth state of the fish body.

[0105] Further, the step S105 specifically comprises:

[0106] S1051, establish a regression analysis model for different fish species, based on existing prior data, perform regression analysis on the relationship between length and width and fish weight, the relationship between length and height and fish weight, and the relationship between length, height and width and fish weight for different fish species, and respectively obtain three types of function models for different fish species.

[0107] S1052, based on the size information of the fish body and the species information of the fish body, use the function model to predict the weight of the fish body.

[0108] S1053. By comparing the body weight of fish of the same species in the same batch, and by comparing the predicted body weight of fish with the historical growth data of fish in the same period, the abnormal growth of fish in the same species in the same batch is obtained, as well as the deviation between the current growth status of fish and the normal status.

[0109] In this embodiment, by comparing the body weights of fish of the same species in the same batch, if the body weight of a fish deviates from the body weight of other fish by a value greater than the deviation threshold, i.e., the deviation is large, then the fish is a fish with abnormal growth, thus identifying fish with abnormal growth in the same species in the same batch; by comparing the predicted body weight of the fish with the historical growth data of fish in the same period, the deviation between the current growth status and the normal status is obtained. If the deviation value is greater than the deviation threshold, i.e. the deviation is large, then the growth status is abnormal.

[0110] Example 2:

[0111] like Figure 6 As shown, this embodiment provides a comprehensive fish detection system, which includes an image acquisition unit 601, a target detection unit 602, an edge detection unit 603, a size calculation unit 604, and a weight prediction unit 605. The specific functions of each unit are as follows:

[0112] The image acquisition unit 601 is used to acquire front view images and top view images of the fish body through different perspective depth cameras, including a front view camera and a top view camera.

[0113] The target detection unit 602 is used to perform target detection on the front view image and top view image using the trained target detection model, and to determine the initial position information and fish species information of the fish.

[0114] The edge detection unit 603 is used to perform sub-pixel edge detection on the initial position information of the fish body to obtain the edge position information of the fish body.

[0115] The size calculation unit 604 is used to obtain the coordinate information of the fish body edge in the corresponding camera coordinate system based on the fish body edge position information and the depth information of the depth camera, and to obtain the fish body size information by calculating the Euclidean distance.

[0116] The weight prediction unit 605 is used to predict the weight of the fish based on the fish size information and fish species information, thereby obtaining the fish's growth status.

[0117] It should be noted that the system provided in this embodiment is only an example of the above-described division of functional units. In practical applications, the above functions can be assigned to different functional units as needed, that is, the internal structure can be divided into different functional units to complete all or part of the functions described above.

[0118] Example 3:

[0119] This embodiment provides a computer device, such as... Figure 7 As shown, it includes a processor 702, a memory, an input device 703, a display device 704, and a network interface 705 connected via a system bus 701. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium 706 and internal memory 707. The non-volatile storage medium 706 stores an operating system, computer programs, and a database. The internal memory 707 provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. When the processor 702 executes the computer programs stored in the memory, it implements the fish comprehensive detection method of Embodiment 1 described above, as follows:

[0120] Front view and top view images of the fish are acquired using depth cameras with different perspectives, including a front view image acquisition depth camera and a top view image acquisition depth camera. A trained target detection model is used to detect targets in the front view and top view images to determine the initial position and species information of the fish. Subpixel edge detection is performed on the initial position information of the fish to obtain the edge position information. Based on the edge position information and the depth information from the depth cameras, the coordinates of the fish edges in the corresponding camera coordinate system are obtained, and the fish size information is obtained by calculating the Euclidean distance. Based on the fish size and species information, the fish weight is predicted, thereby obtaining the fish's growth status.

[0121] Example 4:

[0122] This embodiment provides a storage medium, which is a computer-readable storage medium, storing a computer program. When the computer program is executed by a processor, it implements the comprehensive fish detection method of Embodiment 1 above, as follows:

[0123] Front view and top view images of the fish are acquired using depth cameras with different perspectives, including a front view image acquisition depth camera and a top view image acquisition depth camera. A trained target detection model is used to detect targets in the front view and top view images to determine the initial position and species information of the fish. Subpixel edge detection is performed on the initial position information of the fish to obtain the edge position information. Based on the edge position information and the depth information from the depth cameras, the coordinates of the fish edges in the corresponding camera coordinate system are obtained, and the fish size information is obtained by calculating the Euclidean distance. Based on the fish size and species information, the fish weight is predicted, thereby obtaining the fish's growth status.

[0124] It should be noted that the computer readable storage medium in the embodiments of the present application can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0125] In the embodiments of the present application, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus or device. In the embodiments of the present application, the computer readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer readable programs. Such a propagated data signal can take many forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. The computer readable signal medium can also be any computer readable medium other than the computer readable storage medium that can send, propagate or transmit programs for use by or in connection with an instruction execution system, apparatus or device. The computer programs contained in the computer readable storage medium can be transmitted by any suitable medium, including but not limited to a wire, a cable, a RF (radio frequency) or the like, or any suitable combination of the above.

[0126] The computer readable storage medium described above can be written in one or more programming languages or a combination of the above for executing the computer programs of the embodiments of the present application, including object oriented programming languages such as Java, Python, C++, and conventional procedural programming languages such as C language or similar programming languages. The program can be executed entirely on a user computer, partially on a user computer, as a separate software package, partially on a user computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, through the Internet using an Internet service provider).

[0127] To sum up, the present application adopts the multi-view camera to acquire the fish body information, effectively reduces the influence of mutual shielding between fish bodies on the result accuracy, realizes the comprehensive detection on the fish body type, fish body size, fish body weight and fish body growth state through the layer-by-layer progressive processing on the fish body image data, and improves the convenience and efficiency of the system.

[0128] The above merely describes the preferred embodiments of the present application and is not intended to limit the present application. Although the present application is described in detail with reference to the foregoing embodiments, those skilled in the art can modify the technical solutions described in the foregoing embodiments or equivalently replace some technical features thereof, and any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application. In addition, although some specific terms are used in the present specification, these terms are merely used for convenience and do not constitute any limitation on the present application.

Claims

1. A comprehensive detection method for fish, characterized in that, The method includes: The fish body is captured by a front view image and a top view image using depth cameras with different perspectives, including a front view image acquisition depth camera and a top view image acquisition depth camera. The trained target detection model was used to detect targets in the front view image and top view image to determine the initial position information and fish species information of the fish. Subpixel edge detection is performed on the initial position information of the fish body to obtain the edge position information of the fish body; Based on the position information of the fish's body edge and the depth information of the depth camera, the coordinate information of the fish's body edge in the corresponding camera coordinate system is obtained, and the fish size information is obtained by calculating the Euclidean distance. Based on fish size and species information, predict fish weight to obtain fish growth status; The process of performing sub-pixel edge detection on the initial position information of the fish body to obtain the edge position information of the fish body specifically includes: Based on the outer border of the fish's initial position information, crop the front view image or top view image; Subpixel edge detection is performed on the cropped area to determine the fish body edge information and the position information of the fish body edge in the front view image or top view image; The subpixel edge detection of the cropped area to determine the fish body edge information and the position information of the fish body edge in the front view image or top view image specifically includes: The Canny operator is used to perform pixel-level edge detection on the cropped area to obtain the pixel-level edge contour of the fish body. Convolution is performed between pixel-level edges and a 7x7 template to obtain multi-order Zernike moments, and the parameters of each edge pixel are calculated using Zernike moments. The optimal threshold for the image is obtained using an iterative method, and this optimal threshold is saved as the optimal edge step gray level height threshold. Based on the optimal edge step grayscale height threshold, it is determined whether each edge pixel meets the preset judgment condition. If it meets the condition, the sub-pixel edge coordinates are calculated. If it does not meet the condition, it is removed, thereby obtaining the fish body edge information and the position information of the fish body edge in the front view image or top view image.

2. The comprehensive fish detection method according to claim 1, characterized in that, The training-based target detection model is used to perform target detection on the front view image and top view image to determine the initial position information and fish species information of the fish. Specifically, this includes: Feature extraction of fish images is performed using a backbone network, which is a lightweight version of EfficientNet. The neck network is used to perform cross-layer connections between the extracted features in a top-down and bottom-up manner, so that the features are fused. A decoupled head module is used to separate the regression branch and the prediction branch, which accelerates the convergence of the target detection model. The fused features are processed by the regression branch and the prediction branch respectively to obtain the initial position information of the fish and the fish species information.

3. The comprehensive fish detection method according to claim 1, characterized in that, The camera coordinate systems of the front view image acquisition depth camera and the top view image acquisition depth camera are the front view camera coordinate system and the top view camera coordinate system, respectively. The process involves obtaining the coordinates of the fish's edge in the corresponding camera coordinate system based on the fish's edge position information and the depth information from the depth camera, and then calculating the fish's size information by calculating the Euclidean distance. Specifically, this includes: Based on the position information of the fish body edge in different views, the coordinates of the four extreme points (top, bottom, left, and right) of the fish body edge in the front view image and top view image are obtained respectively. Based on the depth information from depth cameras at different viewpoints, and combined with the coordinate information of the fish's extreme points in different viewpoints, the coordinate information of the fish's extreme points in the front view camera coordinate system and the top view camera coordinate system are obtained respectively. The coordinates of the left and right extreme points of the fish in the top view camera coordinate system are converted into coordinates in the front view camera coordinate system and compared with the coordinates of the left and right extreme points of the fish in the front view camera coordinate system. If the two coordinates are the same or within the preset error range, then the two coordinates are the position information of the same fish in different depth camera views, that is, the same fish appears in different depth camera views. Otherwise, the fish only appears in one depth camera view. Based on the presence of the fish, the Euclidean distance between two points in space is calculated to obtain the fish's size information.

4. The comprehensive fish body detection method according to claim 3, characterized in that, The process of calculating the Euclidean distance between two points in space based on the presence of the fish to obtain the fish size information specifically includes: For the same fish that appears in different depth camera viewpoints, the average length calculated by the front view camera coordinate system and the top view camera coordinate system is taken as the actual length of the fish. The distance between the upper and lower extreme points in the front view camera coordinate system is taken as the height of the fish. The distance between the upper and lower extreme points in the top view camera coordinate system is taken as the width of the fish. If the fish only appears in the perspective of a depth camera, then the distance between the left and right extreme points is taken as the length of the fish, and the distance between the top and bottom extreme points is taken as the height of the fish. If the fish appears in the perspective of the top view image acquisition depth camera, then the distance between the left and right extreme points is taken as the length of the fish, and the distance between the top and bottom extreme points is taken as the width of the fish.

5. The comprehensive fish body detection method according to claim 1, characterized in that, The process of predicting fish weight based on fish size and species information, thereby obtaining the fish's growth status, specifically includes: Regression analysis models were established for different fish species. Based on existing prior data, regression analysis was conducted on the relationships between length and width and fish weight, length and height and fish weight, and length, height and width and fish weight for different fish species, resulting in three types of function models for different fish species. Based on fish size and species information, a function model is used to predict fish weight. By comparing the body weight of fish of the same species in the same batch, and by comparing the predicted body weight of fish with the historical growth data of fish in the same period, we can identify fish with abnormal growth in the same species in the same batch, as well as the deviation between the current growth status of fish and the normal status.

6. A comprehensive fish detection system, characterized in that, The system includes: An image acquisition unit is used to acquire front view images and top view images of the fish body through different perspective depth cameras, including a front view camera and a top view camera. The target detection unit is used to perform target detection on the front view image and top view image using the trained target detection model, and to determine the initial position information and fish species information of the fish. The edge detection unit is used to perform sub-pixel edge detection on the initial position information of the fish body to obtain the edge position information of the fish body. The size calculation unit is used to obtain the coordinate information of the fish body edge in the corresponding camera coordinate system based on the fish body edge position information and the depth information of the depth camera, and to obtain the fish body size information by calculating the Euclidean distance. The weight prediction unit is used to predict the weight of the fish based on the fish size information and fish species information, thereby obtaining the fish's growth status. The process of performing sub-pixel edge detection on the initial position information of the fish body to obtain the edge position information of the fish body specifically includes: Based on the outer border of the fish's initial position information, crop the front view image or top view image; Subpixel edge detection is performed on the cropped area to determine the fish body edge information and the position information of the fish body edge in the front view image or top view image; The subpixel edge detection of the cropped area to determine the fish body edge information and the position information of the fish body edge in the front view image or top view image specifically includes: The Canny operator is used to perform pixel-level edge detection on the cropped area to obtain the pixel-level edge contour of the fish body. Convolution is performed between pixel-level edges and a 7x7 template to obtain multi-order Zernike moments, and the parameters of each edge pixel are calculated using Zernike moments. The optimal threshold for the image is obtained using an iterative method, and this optimal threshold is saved as the optimal edge step gray level height threshold. Based on the optimal edge step grayscale height threshold, it is determined whether each edge pixel meets the preset judgment condition. If it meets the condition, the sub-pixel edge coordinates are calculated. If it does not meet the condition, it is removed, thereby obtaining the fish body edge information and the position information of the fish body edge in the front view image or top view image.

7. A computer device comprising a processor and a memory for storing a processor-executable program, characterized in that, When the processor executes the program stored in the memory, it implements the comprehensive fish detection method according to any one of claims 1-5.

8. A storage medium storing a program, characterized in that, When the program is executed by the processor, it implements the comprehensive fish detection method according to any one of claims 1-5.

Images