Fish body comprehensive detection method and system, computer equipment and storage medium

Abstract

The invention discloses a fish body comprehensive detection method and system, computer equipment and a storage medium. The method comprises the following steps: acquiring a front view image and a top view image of a fish body through depth cameras with different visual angles; performing target detection on the front view image and the top view image by adopting a trained target detection model, and determining fish body initial position information and fish body type information; performing sub-pixel edge detection on the fish body initial position information to obtain fish body edge position information; according to the position information of the fish body edge and depth information of a depth camera, coordinate information of the fish body edge in a corresponding camera coordinate system is obtained, and the Euclidean distance is calculated to obtain fish body size information; and predicting the weight of the fish body according to the fish body size information and the fish body type information so as to obtain the growth state of the fish body. According to the invention, the accuracy of fish body growth state detection is improved, and abnormal conditions of fish body growth can be found in time.

Description

Technical Field

[0001] The invention relates to a comprehensive fish body detection method, system, computer equipment and storage medium, and belongs to the field of fish body measurement. Background Art

[0002] Fish body measurement is of great significance for understanding the growth status of fish and can provide key information for fishery resource management and protection. Traditional fish body measurement techniques include weight measurement and length measurement, which can provide data on fish growth, quantity and population structure, and provide reference for fishery resource management and protection. However, traditional fish body measurement techniques also have some limitations: first, the measurement process may cause harm to fish, especially in large-scale measurements, which may cause unnecessary deaths; second, traditional measurement techniques usually require a lot of manpower and material resources, and have low work efficiency; in addition, the measurement data may have errors and uncertainties, which affects the accuracy and reliability of the research.

[0003] Therefore, the development of modern fish measurement technology has also ushered in important opportunities, such as sonar technology and remote sensing technology. These new technologies can obtain fish feature data more accurately and can achieve non-invasive measurement, improving measurement efficiency and accuracy, but there are still problems such as high cost and inconvenient measurement. In recent years, with the continuous development of computer vision technology, various methods of photographing fish with high-definition cameras and then using image processing algorithms to accurately measure fish have also emerged. However, some current methods still have defects such as poor robustness and being greatly affected by overlapping and shielding of fish schools. Summary of the invention

[0004] In view of this, the present invention provides a comprehensive fish detection method, system, computer equipment and storage medium, which improve the accuracy of fish growth status detection and can promptly detect abnormal conditions of fish growth.

[0005] The first object of the present invention is to provide a comprehensive fish detection method

[0006] The second object of the present invention is to provide a comprehensive fish body detection system.

[0007] A third object of the present invention is to provide a computer device.

[0008] A fourth object of the present invention is to provide a storage medium.

[0009] The first object of the present invention can be achieved by adopting the following technical solutions:

[0010] A comprehensive fish body detection method, the method comprising:

[0011] Acquire a front view image and a top view image of the fish body by using depth cameras with different viewing angles, wherein the depth cameras with different viewing angles include a front view image acquisition depth camera and a top view image acquisition depth camera;

[0012] The trained target detection model is used to perform target detection on the main view image and the top view image to determine the initial position information and fish species information of the fish body;

[0013] Perform sub-pixel edge detection on the initial position information of the fish body to obtain the edge position information of the fish body;

[0014] Based on the fish body edge position information and the depth information of the depth camera, the coordinate information of the fish body edge in the corresponding camera coordinate system is obtained, and the fish body size information is obtained by calculating the Euclidean distance;

[0015] Based on the fish size information and fish species information, the fish weight is predicted to obtain the fish growth status.

[0016] Furthermore, the trained target detection model is used to perform target detection on the main view image and the top view image to determine the initial position information of the fish and the fish species information, specifically including:

[0017] A backbone network is used to extract features from fish images, and the backbone network is an EfficientNet lightweight network;

[0018] The neck network is used to connect the extracted features from top to bottom and from bottom to top across layers to fuse the features;

[0019] The decoupling head module is used to separate the regression branch and the prediction branch to accelerate the convergence of the target detection model. The regression branch and the prediction branch are used to process the fused features respectively to obtain the initial position information and fish species information of the fish.

[0020] Further, performing sub-pixel edge detection on the initial position information of the fish body to obtain the edge position information of the fish body specifically includes:

[0021] The main view image or the top view image is cropped according to the outer frame of the initial position information of the fish body;

[0022] Sub-pixel edge detection is performed on the cropped area to determine the edge information of the fish body and the position information of the edge of the fish body in the main view image or the top view image.

[0023] Furthermore, the sub-pixel edge detection is performed on the cropped area to determine the edge information of the fish body and the position information of the edge of the fish body in the main view image or the top view image, specifically including:

[0024] The cropped area is subjected to pixel-level edge detection using the Canny operator to obtain the pixel-level edge contour of the fish body;

[0025] Convolution is performed on the pixel-level edge and the 7X7 template to obtain multi-order Zernike moments, and the parameters of each edge pixel are calculated using the Zernike moments.

[0026] Use an iterative method to obtain the optimal threshold of the image, and save the optimal threshold of the image as the optimal edge step grayscale height threshold;

[0027] According to the optimal edge step grayscale height threshold, it is judged whether each edge pixel meets the preset judgment condition. If it does, the sub-pixel edge coordinates are calculated. If not, it is eliminated, thereby obtaining the edge information of the fish body and the position information of the fish body edge in the main view image or the top view image.

[0028] Further, the camera coordinate systems of the main view image acquisition depth camera and the top view image acquisition depth camera are the main view camera coordinate system and the top view camera coordinate system respectively;

[0029] The method of obtaining the coordinate information of the fish body edge in the corresponding camera coordinate system based on the fish body edge position information and the depth information of the depth camera, and obtaining the fish body size information by calculating the Euclidean distance, specifically includes:

[0030] According to the edge position information of the fish body in different views, the coordinates of the four extreme points of the upper, lower, left and right edges of the fish body in the main view image and the top view image are obtained respectively;

[0031] According to the depth information of the depth cameras with different viewing angles, combined with the coordinate information of the extreme points of the fish body in different viewing angles, the coordinate information of the extreme points of the fish body in the main view camera coordinate system and the top view camera coordinate system are obtained respectively;

[0032] The coordinates of the left and right extreme points of the fish body in the top view camera coordinate system are converted into the coordinates of the main view camera coordinate system, and compared with the coordinates of the left and right extreme points of the fish body in the main view camera coordinate system. If the two coordinate positions are the same or within the preset error range, then the two coordinates are the position information of the same fish body under different depth camera perspectives, that is, the same fish body appears under different depth camera perspectives, otherwise, the fish body only appears under one depth camera perspective;

[0033] According to the appearance of the fish body, the Euclidean distance between two points in space is calculated to obtain the fish body size information.

[0034] Furthermore, the method of calculating the Euclidean distance between two points in space according to the appearance of the fish body to obtain the fish body size information specifically includes:

[0035] For the case where the same fish body appears in different depth camera perspectives, the average length calculated by the main view camera coordinate system and the top view camera coordinate system is taken as the actual length of the fish body, the distance between the upper and lower extreme points in the main view camera coordinate system is taken as the height of the fish body, and the distance between the upper and lower extreme points in the top view camera coordinate system is taken as the width of the fish body;

[0036] If the fish body only appears in one depth camera perspective, if the fish body only appears in the main view image acquisition depth camera perspective, the distance between the left and right extreme points is used as the length of the fish body, and the distance between the upper and lower extreme points is used as the height of the fish body; if the fish body appears in the top view image acquisition depth camera perspective, the distance between the left and right extreme points is used as the length of the fish body, and the distance between the upper and lower extreme points is used as the width of the fish body.

[0037] Furthermore, the method of predicting the weight of the fish according to the fish size information and the fish species information, thereby obtaining the growth status of the fish, specifically includes:

[0038] Regression analysis models were established for different fish species. Based on the existing prior data, regression analysis was performed on the relationship between the length and width of different fish species and their weight, the relationship between the length and height and their weight, and the relationship between the length, height and width and their weight, and three types of function models were obtained for different fish species.

[0039] Based on the fish size information and fish species information, the fish weight is predicted using a function model;

[0040] By comparing the weights of fish of the same species in the same batch, and comparing the predicted fish weights with the historical fish growth data for the same period, the abnormally growing fish in the same batch of fish of the same species, and the deviation between the current fish growth and the normal situation can be obtained.

[0041] The second object of the present invention can be achieved by adopting the following technical solutions:

[0042] A fish body comprehensive detection system, the system comprising:

[0043] An image acquisition unit, used to acquire a front view image and a top view image of a fish body through different viewing angle depth cameras, wherein the different viewing angle depth cameras include a front view camera and a top view camera;

[0044] The target detection unit is used to perform target detection on the main view image and the top view image using the trained target detection model to determine the initial position information and fish species information of the fish body;

[0045] An edge detection unit is used to perform sub-pixel edge detection on the initial position information of the fish body to obtain the edge position information of the fish body;

[0046] A size calculation unit is used to obtain the coordinate information of the edge of the fish body in the corresponding camera coordinate system according to the edge position information of the fish body and the depth information of the depth camera, and obtain the size information of the fish body by calculating the Euclidean distance;

[0047] The weight prediction unit is used to predict the fish weight according to the fish size information and the fish species information, so as to obtain the growth status of the fish.

[0048] The third object of the present invention can be achieved by adopting the following technical solutions:

[0049] A computer device includes a processor and a memory for storing a program executable by the processor, wherein the processor implements the above-mentioned comprehensive fish body detection method when executing the program stored in the memory.

[0050] The fourth object of the present invention can be achieved by adopting the following technical solutions:

[0051] A storage medium stores a program, and when the program is executed by a processor, the above-mentioned comprehensive fish body detection method is implemented.

[0052] The present invention has the following beneficial effects compared with the prior art:

[0053] 1. The present invention adopts a multi-view camera to obtain fish information, which effectively reduces the influence of mutual shielding between fish bodies on the accuracy of the results. Through the layer-by-layer progressive processing of fish image data, it realizes the comprehensive detection of fish species, fish size, fish weight, and fish growth status, thereby improving the convenience and efficiency of the system.

[0054] 2. The present invention uses a target monitoring model based on a lightweight backbone network to detect fish species and the rough position information of the fish, which is convenient for the subsequent precise positioning of the fish position and the prediction of the fish weight. The lightweight network is also more convenient for the deployment and application of the model.

[0055] 3. The present invention uses an improved Zernike moment sub-pixel detection method to more accurately detect the initial position information of the fish body, thereby improving the accuracy of the initial position information of the fish body and making the subsequent measurement of the fish body size information and weight information more accurate.

[0056] 4. The present invention establishes different fish weight regression models based on multi-dimensional data such as fish length, width and height for different fish species, so that the prediction of fish weight is more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0058] Figure 1 This is a flow chart of the comprehensive fish body detection method of Example 1 of the present invention.

[0059] Figure 2 Schematic diagram of the arrangement of depth cameras with different viewing angles according to Embodiment 1 of the present invention.

[0060] Figure 3 This is a flow chart of sub-pixel edge detection in Example 1 of the present invention.

[0061] Figure 4 This is a schematic diagram of the initial position information of a fish body under the perspective of a depth camera for collecting the main view image of Example 1 of the present invention.

[0062] Figure 5 This is a schematic diagram of the initial position information of a fish body under the perspective of a depth camera for capturing a top view image in Example 1 of the present invention.

[0063] Figure 6 This is a structural block diagram of the comprehensive fish body detection system of embodiment 2 of the present invention.

[0064] Figure 7 This is a structural block diagram of a computer device according to Embodiment 2 of the present invention. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0066] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0067] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention; as used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form. In addition, it should be understood that the terms "include" and "comprises" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or apparatus.

[0068] Embodiment 1:

[0069] like Figure 1 As shown, this embodiment provides a comprehensive fish body detection method, which includes the following steps:

[0070] S101, obtaining a front view image and a top view image of a fish body through depth cameras with different viewing angles.

[0071] The different viewing angle depth cameras of this embodiment include a main view image acquisition depth camera and a top view image acquisition depth camera. The arrangement of the two depth cameras is as follows: Figure 2 As shown, they are arranged in a cross pattern, and the two depth cameras collect images of the fish body in the main view direction and the top view direction, which are the main view image and the top view image, respectively.

[0072] In order to facilitate subsequent positioning and calculation, the camera coordinate systems of the main view image acquisition depth camera and the top view image acquisition depth camera are divided into the main view camera coordinate system and the top view camera coordinate system. In the main view camera coordinate system, the horizontal direction of the camera center is the X axis, the vertical direction is the Y axis, and the direction perpendicular to the lens is the Z axis. The top view camera coordinate system is to rotate the main view camera coordinate system 90 degrees around the X axis; let the main view camera coordinate system be the main coordinate system, the coordinate point of the top view camera in the main view camera coordinate system is (X1, Y1, Z1), and the coordinate of the measured point K in the main view camera coordinate system is (X K , Y K , Z K ), the coordinates of the measured point K in the top view camera coordinate system are (X' K , Y' K , Z' K ), then the coordinate position transformation relationship of the measured point in the main view camera coordinate system and the top view camera coordinate system is as follows (1):

[0073] (X K ,Y K ,Z K )=((X1+X' K ),(Z1+Z'K ),(Y1+Y' K )) (1)

[0074] S102: Use the trained target detection model to perform target detection on the main view image and the top view image to determine the initial position information and fish species information of the fish.

[0075] The target detection model of this embodiment is a deep learning model based on the YOLOV8 algorithm. The target detection model is trained through a training set, and then the trained target detection model is used to perform target detection on the main view image and the top view image to determine the initial position information and fish species information of the fish body, wherein the training set is an image data set in a database collected in advance.

[0076] Furthermore, step S102 specifically includes:

[0077] S1021. The backbone network is used to extract features from fish images. In order to improve the running speed of the model and reduce the number of model parameters, the backbone network uses the EfficientNet lightweight network instead of the original Darknet53 network to extract fish image features.

[0078] S1022. Use the neck network to connect the extracted features from top to bottom and from bottom to top across layers, so that the features can be more fully integrated.

[0079] S1023. Use a decoupling head module to separate the regression branch and the prediction branch to accelerate the convergence of the target detection model. Use the regression branch and the prediction branch to process the fused features respectively to obtain the initial position information and species information of the fish body.

[0080] S103, performing sub-pixel edge detection on the initial position information of the fish body to obtain the edge position information of the fish body.

[0081] Furthermore, step S103 specifically includes:

[0082] S1031. Crop the main view image or the top view image according to the outer frame of the initial position information of the fish body.

[0083] S1032. Perform sub-pixel edge detection on the cropped area to determine the edge information of the fish body and the position information of the edge of the fish body in the main view image or the top view image.

[0084] It is easy for a person skilled in the art to understand that when the input image is a main view image, the position information of the edge of the fish body in the main view image can be obtained, and when the input image is a top view image, the position information of the edge of the fish body in the top view image can be obtained.

[0085] like Figure 3 As shown, step S1032 specifically includes:

[0086] 1) Use the Canny operator to perform pixel-level edge detection on the cropped area to obtain the pixel-level edge contour of the fish body;

[0087] 2) Perform convolution calculation on the pixel-level edge and the 7X7 template to obtain multi-order Zernike moments, and use the Zernike moments to calculate the parameters of each edge pixel.

[0088] 3) Use the iterative method to obtain the optimal threshold of the image, and save the optimal threshold of the image as the optimal edge step grayscale height threshold.

[0089] 4) According to the optimal edge step grayscale height threshold, determine whether each edge pixel meets the preset judgment condition. If it meets the condition, calculate the sub-pixel edge coordinates. If it does not meet the condition, remove it, so as to obtain the edge information of the fish body and the position information of the edge of the fish body in the main view image or the top view image. The preset judgment condition is as follows (2):

[0090] (k≥k t )∩(d≤(2d t / N)) (2)

[0091] Among them, k represents the edge step grayscale height, k t represents the edge step grayscale height threshold, d represents the vertical distance from the pixel center to the edge, d t Indicates the distance threshold.

[0092] S104, according to the fish body edge position information, combined with the depth information of the depth camera, obtain the coordinate information of the fish body edge in the corresponding camera coordinate system, and obtain the fish body size information by calculating the Euclidean distance.

[0093] Based on the fish body edge position information, this embodiment combines the depth information of the depth camera to obtain the coordinate information of the fish body edge in the corresponding camera coordinate system, and then calculates the Euclidean distance to obtain the length, width, height and other size information of the fish body. The schematic diagram of the fish body position information under the perspective of the depth camera in the main view image acquisition is shown in FIG. Figure 4 As shown in the figure, the schematic diagram of the top view image acquisition of the fish body position information from the perspective of the depth camera is as follows Figure 5 shown.

[0094] Furthermore, step S1041 specifically includes:

[0095] S1041, according to the fish body edge position information of different views, respectively obtain the coordinates of the four extreme value points of the upper, lower, left and right edges of the fish body in the main view image and the top view image, the coordinates of the four extreme value points are: left extreme value point (X 1c , Y1c ), right extreme point (X 2c , Y 2c ), the upper extreme point (X 1G , Y 1G ), the lower extreme point (X 2G , Y 2G ); the coordinates of the four extreme points on the edge of the fish body in the top view are: the left extreme point (X' 1c , Y' 1c ), right extreme point (X' 2c , Y '2c ), upper extreme point (X' 1K , Y' 1K ), the lower extreme point (X' 2K , Y' 2K ).

[0096] S1042, according to the depth information of the depth cameras with different viewing angles, combined with the coordinate information of the extreme points of the fish body in different viewing angles, respectively obtain the coordinate information of the extreme points of the fish body in the main view camera coordinate system and the top view camera coordinate system. The coordinates of the extreme points of the fish body in the main view camera coordinate system are: left extreme point (X 1c , Y 1c , Z 1c ), right extreme point (X 2c , Y 2c , Z 2c ), the upper extreme point (X 1G , Y 1G , Z 1G ), the lower extreme point (X 2G , Y 2G , Z 2G ); The coordinates of the extreme points of the fish body in the top view camera coordinate system are: left extreme point (X' 1c , Y' 1c , Z' 1c ), right extreme point (X' 2c , Y '2c , Z '2c ), upper extreme point (X' 1K , Y' 1K , Z' 1K ), the lower extreme point (X' 2K , Y' 2K , Z' 2K ).

[0097] S1043. Convert the coordinates of the left and right extreme points of the fish body in the top view camera coordinate system into coordinates in the main view camera coordinate system, and compare them with the coordinates of the left and right extreme points of the fish body in the main view camera coordinate system. If the two coordinate positions are the same or within a preset error range, the two coordinates are the position information of the same fish body under different depth camera viewing angles, that is, the same fish body appears under different depth camera viewing angles, otherwise, the fish body only appears under one depth camera viewing angle.

[0098] S1044. According to the appearance of the fish body, the Euclidean distance between two points in space is calculated to obtain the size information of the fish body.

[0099] 1) For the case where the same fish body appears in different depth camera perspectives, the average length calculated by the main view camera coordinate system and the top view camera coordinate system is taken as the actual length of the fish body, the distance between the upper and lower extreme points in the main view camera coordinate system is taken as the height of the fish body, and the distance between the upper and lower extreme points in the top view camera coordinate system is taken as the width of the fish body.

[0100] 2) If the fish body only appears in one depth camera perspective, if the fish body only appears in the main view image acquisition depth camera perspective, the distance between the left and right extreme points is taken as the length of the fish body, and the distance between the upper and lower extreme points is taken as the height of the fish body; if the fish body appears in the top view image acquisition depth camera perspective, the distance between the left and right extreme points is taken as the length of the fish body, and the distance between the upper and lower extreme points is taken as the width of the fish body.

[0101] The Euclidean distance calculation formula of this embodiment is as follows (3):

[0102]

[0103] Among them, l is the distance between two points in space, (x1, y1, z1) and (x2, y2, z2) are the coordinates of the two points respectively.

[0104] S105. Predicting the fish weight according to the fish size information and the fish species information, thereby obtaining the fish growth status.

[0105] Furthermore, step S105 specifically includes:

[0106] S1051. Regression analysis models are established for different fish species. Based on the existing prior data, regression analysis is performed on the relationship between the length and width of different fish species and the fish weight, the relationship between the length and height and the weight, and the relationship between the length, height and width and the fish weight, to obtain three types of function models for different fish species.

[0107] S1052. Based on the fish size information and fish species information, use a function model to predict the fish weight.

[0108] S1053. By comparing the weights of fish of the same type and the same batch, and comparing the predicted fish weights with the historical fish growth data of the same period, the fish with abnormal growth in the same batch and the deviation between the current fish growth and the normal situation are obtained.

[0109] In this embodiment, by comparing the weights of fish of the same species in the same batch, if the deviation value between the weight of a fish and the weights of other fish is greater than the deviation threshold, that is, the deviation is large, then the fish is a fish with abnormal growth, thereby obtaining fish with abnormal growth in the same species in the same batch of fish; by comparing the predicted fish weight with the historical fish growth data of the same period, the deviation between the current fish growth situation and the normal situation is obtained, if the deviation value is greater than the deviation threshold, that is, the deviation is large, then the growth situation is abnormal.

[0110] Embodiment 2:

[0111] like Figure 6 As shown, this embodiment provides a comprehensive fish body detection system, which includes an image acquisition unit 601, a target detection unit 602, an edge detection unit 603, a size calculation unit 604 and a weight prediction unit 605. The specific functions of each unit are as follows:

[0112] An image acquisition unit 601 is used to acquire a front view image and a top view image of a fish body through different viewing angle depth cameras, wherein the different viewing angle depth cameras include a front view camera and a top view camera;

[0113] The target detection unit 602 is used to perform target detection on the main view image and the top view image using the trained target detection model to determine the initial position information of the fish and the fish species information;

[0114] The edge detection unit 603 is used to perform sub-pixel edge detection on the initial position information of the fish body to obtain the edge position information of the fish body;

[0115] The size calculation unit 604 is used to obtain the coordinate information of the edge of the fish body in the corresponding camera coordinate system according to the edge position information of the fish body and the depth information of the depth camera, and obtain the size information of the fish body by calculating the Euclidean distance;

[0116] The weight prediction unit 605 is used to predict the weight of the fish according to the fish size information and the fish species information, so as to obtain the growth status of the fish.

[0117] It should be noted that the system provided in this embodiment is only illustrated by the division of the above-mentioned functional units. In actual applications, the above-mentioned functions can be assigned to different functional units as needed, that is, the internal structure can be divided into different functional units to complete all or part of the functions described above.

[0118] Embodiment 3:

[0119] This embodiment provides a computer device, such as Figure 7 As shown, it includes a processor 702, a memory, an input device 703, a display device 704 and a network interface 705 connected through a system bus 701. The processor is used to provide computing and control capabilities. The memory includes a non-volatile storage medium 706 and an internal memory 707. The non-volatile storage medium 706 stores an operating system, a computer program and a database. The internal memory 707 provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. When the processor 702 executes the computer program stored in the memory, the fish body comprehensive detection method of the above-mentioned embodiment 1 is implemented as follows:

[0120] A main view image and a top view image of a fish body are acquired by using depth cameras with different viewing angles, and the depth cameras with different viewing angles include a main view image acquisition depth camera and a top view image acquisition depth camera; a trained target detection model is used to perform target detection on the main view image and the top view image to determine initial position information of the fish body and information on the species of the fish body; sub-pixel edge detection is performed on the initial position information of the fish body to obtain edge position information of the fish body; based on the edge position information of the fish body and in combination with the depth information of the depth camera, coordinate information of the edge of the fish body in a corresponding camera coordinate system is obtained, and the size information of the fish body is obtained by calculating the Euclidean distance; based on the fish body size information and the fish body species information, the weight of the fish body is predicted, thereby obtaining the growth status of the fish body.

[0121] Embodiment 4:

[0122] This embodiment provides a storage medium, which is a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the fish body comprehensive detection method of the above embodiment 1 is implemented as follows:

[0123] A main view image and a top view image of a fish body are acquired by using depth cameras with different viewing angles, and the depth cameras with different viewing angles include a main view image acquisition depth camera and a top view image acquisition depth camera; a trained target detection model is used to perform target detection on the main view image and the top view image to determine initial position information of the fish body and information on the species of the fish body; sub-pixel edge detection is performed on the initial position information of the fish body to obtain edge position information of the fish body; based on the edge position information of the fish body and in combination with the depth information of the depth camera, coordinate information of the edge of the fish body in a corresponding camera coordinate system is obtained, and the size information of the fish body is obtained by calculating the Euclidean distance; based on the fish body size information and the fish body species information, the weight of the fish body is predicted, thereby obtaining the growth status of the fish body.

[0124] It should be noted that the computer-readable storage medium of the present embodiment may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0125] In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device, or device. In this embodiment, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable storage medium other than a computer-readable storage medium, which may send, propagate, or transmit a program used by or in combination with an instruction execution system, device, or device. The computer program contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0126] The computer readable storage medium can be written in one or more programming languages ​​or a combination thereof to execute the computer program of the present embodiment, and the programming language includes an object-oriented programming language, such as Java, Python, C++, and also includes a conventional procedural programming language, such as C language or a similar programming language. The program can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., using an Internet service provider to connect through the Internet).

[0127] To sum up, the present invention adopts a multi-view camera to obtain fish information, which effectively reduces the influence of mutual shading between fish bodies on the accuracy of the results. Through the layer-by-layer progressive processing of fish image data, it realizes the comprehensive detection of fish species, fish size, fish weight, and fish growth status, thereby improving the convenience and efficiency of the system.

[0128] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention is described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention. In addition, although some specific terms are used in this specification, these terms are only for the convenience of description and do not constitute any limitation to the present invention.

Claims

1. A comprehensive fish body detection method, characterized in that: The method comprises: Acquire a front view image and a top view image of the fish body by using depth cameras with different viewing angles, wherein the depth cameras with different viewing angles include a front view image acquisition depth camera and a top view image acquisition depth camera; The trained target detection model is used to perform target detection on the main view image and the top view image to determine the initial position information and fish species information of the fish body; Perform sub-pixel edge detection on the initial position information of the fish body to obtain the edge position information of the fish body; Based on the fish body edge position information and the depth information of the depth camera, the coordinate information of the fish body edge in the corresponding camera coordinate system is obtained, and the fish body size information is obtained by calculating the Euclidean distance; Based on the fish size information and fish species information, the fish weight is predicted to obtain the fish growth status.

2. The comprehensive fish detection method according to claim 1, characterized in that: The trained target detection model is used to perform target detection on the main view image and the top view image to determine the initial position information of the fish body and the fish body species information, specifically including: A backbone network is used to extract features from fish images, and the backbone network is an EfficientNet lightweight network; The neck network is used to connect the extracted features from top to bottom and from bottom to top across layers to fuse the features; The decoupling head module is used to separate the regression branch and the prediction branch to accelerate the convergence of the target detection model. The regression branch and the prediction branch are used to process the fused features respectively to obtain the initial position information and fish species information of the fish.

3. The comprehensive fish body detection method according to claim 1, characterized in that: The sub-pixel edge detection is performed on the initial position information of the fish body to obtain the edge position information of the fish body, specifically including: The main view image or the top view image is cropped according to the outer frame of the initial position information of the fish body; Sub-pixel edge detection is performed on the cropped area to determine the edge information of the fish body and the position information of the edge of the fish body in the main view image or the top view image.

4. The comprehensive fish body detection method according to claim 3, characterized in that: The sub-pixel edge detection is performed on the cropped area to determine the edge information of the fish body and the position information of the edge of the fish body in the main view image or the top view image, specifically including: The cropped area is subjected to pixel-level edge detection using the Canny operator to obtain the pixel-level edge contour of the fish body; Convolution is performed on the pixel-level edge and the 7X7 template to obtain multi-order Zernike moments, and the parameters of each edge pixel are calculated using the Zernike moments. Use an iterative method to obtain the optimal threshold of the image, and save the optimal threshold of the image as the optimal edge step grayscale height threshold; According to the optimal edge step grayscale height threshold, it is judged whether each edge pixel meets the preset judgment condition. If it does, the sub-pixel edge coordinates are calculated. If not, it is eliminated, thereby obtaining the edge information of the fish body and the position information of the fish body edge in the main view image or the top view image.

5. The comprehensive fish body detection method according to claim 1, characterized in that: The camera coordinate systems of the main view image acquisition depth camera and the top view image acquisition depth camera are respectively the main view camera coordinate system and the top view camera coordinate system; The method of obtaining the coordinate information of the fish body edge in the corresponding camera coordinate system based on the fish body edge position information and the depth information of the depth camera, and obtaining the fish body size information by calculating the Euclidean distance, specifically includes: According to the edge position information of the fish body in different views, the coordinates of the four extreme points of the upper, lower, left and right edges of the fish body in the main view image and the top view image are obtained respectively; According to the depth information of the depth cameras with different viewing angles, combined with the coordinate information of the extreme points of the fish body in different viewing angles, the coordinate information of the extreme points of the fish body in the main view camera coordinate system and the top view camera coordinate system are obtained respectively; The coordinates of the left and right extreme points of the fish body in the top view camera coordinate system are converted into the coordinates of the main view camera coordinate system, and compared with the coordinates of the left and right extreme points of the fish body in the main view camera coordinate system. If the two coordinate positions are the same or within the preset error range, then the two coordinates are the position information of the same fish body under different depth camera perspectives, that is, the same fish body appears under different depth camera perspectives, otherwise, the fish body only appears under one depth camera perspective; According to the appearance of the fish body, the Euclidean distance between two points in space is calculated to obtain the fish body size information.

6. The comprehensive fish detection method according to claim 5, characterized in that: According to the appearance of the fish body, the Euclidean distance between two points in space is calculated to obtain the fish body size information, which specifically includes: For the case where the same fish body appears in different depth camera perspectives, the average length calculated by the main view camera coordinate system and the top view camera coordinate system is taken as the actual length of the fish body, the distance between the upper and lower extreme points in the main view camera coordinate system is taken as the height of the fish body, and the distance between the upper and lower extreme points in the top view camera coordinate system is taken as the width of the fish body; If the fish body only appears in one depth camera perspective, if the fish body only appears in the main view image acquisition depth camera perspective, the distance between the left and right extreme points is used as the length of the fish body, and the distance between the upper and lower extreme points is used as the height of the fish body; if the fish body appears in the top view image acquisition depth camera perspective, the distance between the left and right extreme points is used as the length of the fish body, and the distance between the upper and lower extreme points is used as the width of the fish body.

7. The comprehensive fish body detection method according to claim 1, characterized in that: The method of predicting the fish weight according to the fish size information and the fish species information, thereby obtaining the fish growth status, specifically includes: Regression analysis models were established for different fish species. Based on the existing prior data, regression analysis was performed on the relationship between the length and width of different fish species and their weight, the relationship between the length and height and their weight, and the relationship between the length, height and width and their weight, and three types of function models were obtained for different fish species. Based on the fish size information and fish species information, the fish weight is predicted using a function model; By comparing the weights of fish of the same species in the same batch, and comparing the predicted fish weights with the historical fish growth data for the same period, the abnormally growing fish in the same batch of fish of the same species, and the deviation between the current fish growth and the normal situation can be obtained.

8. A comprehensive fish detection system, characterized in that: The system comprises: An image acquisition unit, used to acquire a front view image and a top view image of a fish body through different viewing angle depth cameras, wherein the different viewing angle depth cameras include a front view camera and a top view camera; The target detection unit is used to perform target detection on the main view image and the top view image using the trained target detection model to determine the initial position information and fish species information of the fish body; An edge detection unit is used to perform sub-pixel edge detection on the initial position information of the fish body to obtain the edge position information of the fish body; A size calculation unit is used to obtain the coordinate information of the edge of the fish body in the corresponding camera coordinate system according to the edge position information of the fish body and the depth information of the depth camera, and obtain the size information of the fish body by calculating the Euclidean distance; The weight prediction unit is used to predict the fish weight according to the fish size information and the fish species information, so as to obtain the growth status of the fish.

9. A computer device comprising a processor and a memory for storing a program executable by the processor, characterized in that: When the processor executes the program stored in the memory, the comprehensive fish body detection method described in any one of claims 1-7 is implemented.

10. A storage medium storing a program, characterized in that: When the program is executed by a processor, the comprehensive fish body detection method described in any one of claims 1-7 is implemented.

Images