Marine ranch fish face intelligent identification method and device and storage medium

Abstract

The invention discloses a marine ranch fish face intelligent identification method, a computer device and a storage medium, and the method comprises the steps: carrying out the detection of an underwater monitoring image through a target detection model, obtaining a fish face image and a whole fish image, carrying out the matching detection of each fish face image and each whole fish image, determining the matched fish face image and whole fish image, and carrying out the recognition of the fish face image and whole fish image. And for the fish face image with the matched whole-fish image, performing fish individual identification according to the fish face image, and the like. According to the method, the fish face image and the whole fish image in the underwater monitoring image can be accurately identified by applying the target detection model, and the individual fish individuals can be accurately identified by utilizing the matching property of the fish face image and the whole fish image, so that fish individual identification operations such as specific fish individual tracking and quantity statistics are realized, the production efficiency of marine ranch fishery is expected to be improved, and the method has a wide application prospect. Labor cost is reduced, daily management of modernized marine ranches is better served, and construction of intelligent marine ranches is promoted. The method is widely applied to the technical field of marine ranches.

Description

Technical Field

[0001] The present invention relates to the technical field of marine ranching, and in particular to a method, a computer device and a storage medium for intelligently identifying fish faces in marine ranching. Background Art

[0002] Due to overfishing in the ocean and the deteriorating offshore environment, the marine ecological environment in most coastal waters of the world has been destroyed, which seriously threatens the safety of fishery resources. In order to cope with the trend of fishery decline and promote the coordination of marine resource development and environment, modern marine ranches with the goal of fishery restoration and ecological facility breeding have become an important solution to the depletion of offshore fishery resources and the deterioration of the breeding environment, and are also an important measure to promote the transformation and upgrading of fisheries. In order to ensure the safety of farmed fish, modern marine ranches attach importance to the monitoring and tracking of fish in marine ranches to understand the behavior, growth habits, and health of fish. Moreover, different fish have obvious differences in characteristics, such as living environment, size, shape, texture, etc. Even the same species of fish have certain differences, and large-scale identification is difficult; coupled with the complex underwater environment, poor light in the water, turbid water and other factors, fish image recognition faces great challenges. Therefore, modern marine ranch fish identification and tracking have become key technologies that need to be urgently solved in marine ranch monitoring. Summary of the invention

[0003] In view of the technical problems existing in current marine ranching technology, such as a wide variety of fish species and difficulty in identifying individual fish, the purpose of the present invention is to provide a marine ranching fish face intelligent recognition method, a computer device and a storage medium.

[0004] On the one hand, an embodiment of the present invention includes a method for intelligently identifying fish faces in an ocean ranch, and the method includes the following steps:

[0005] Obtain underwater surveillance images of the ocean ranch;

[0006] Detect the underwater monitoring image using a target detection model to obtain at least one fish face image and at least one whole fish image;

[0007] Performing matching detection on each of the fish face images and each of the whole fish images to determine the fish face images and the whole fish images that match;

[0008] For the fish face image having the matching full fish image, individual fish identification is performed based on the fish face image.

[0009] Further, the use of the target detection model to detect the underwater monitoring image to obtain at least one fish face image and at least one whole fish image includes:

[0010] Establishing the target detection model;

[0011] Get the labeled dataset;

[0012] Using the labeled data set to train the target detection model;

[0013] Input the underwater monitoring image into the trained target detection model for processing, and obtain the fish face bounding box and the whole fish bounding box output by the target detection model;

[0014] Determining the corresponding fish face image in the underwater monitoring image according to the fish face bounding box;

[0015] According to the whole fish bounding box, the corresponding whole fish image is determined in the underwater monitoring image.

[0016] Furthermore, the using the labeled data set to train the target detection model includes:

[0017] Expanding the labeled data set by an active learning method;

[0018] The target detection model is trained using the expanded labeled data set.

[0019] Further, the matching detection of each of the fish face images and each of the whole fish images to determine the matching fish face images and the whole fish images includes:

[0020] Obtain a first fish face bounding box corresponding to a first fish face image, and a first whole fish bounding box corresponding to a first whole fish image; wherein the first fish face image is any one of the fish face images, and the first whole fish image is any one of the whole fish images;

[0021] Determine an IoMA value according to the first fish face bounding box and the first whole fish bounding box;

[0022] When the IoMA value is equal to the threshold, it is determined that the first fish face image matches the first whole fish image; otherwise, it is determined that the first fish face image does not match the first whole fish image.

[0023] Further, determining the IoMA value according to the first fish face bounding box and the first whole fish bounding box includes:

[0024] Obtaining an intersection area of ​​the first fish face bounding box and the first whole fish bounding box;

[0025] Obtaining the minimum area of ​​the first fish face bounding box and the first whole fish bounding box; the minimum area is the minimum area between the first fish face bounding box and the first whole fish bounding box;

[0026] The ratio obtained by dividing the intersection area by the smallest area is determined as the IoMA value.

[0027] Further, for the fish face image having the matching whole fish image, performing fish individual identification according to the fish face image includes:

[0028] The fish face image having the matching full fish image is used as a target fish face image and searched in an image base library; the image base library includes at least one pre-stored fish face image and ID information corresponding to the pre-stored fish face image;

[0029] When the pre-stored fish face image having the same features as the target fish face image is searched from the image base, the target fish face image is marked with the ID information of the searched pre-stored fish face image;

[0030] When the pre-stored fish face image having the same features as the target fish face image is not found in the image base library, new ID information is generated to mark the target fish face image, and the image base library is updated using the target fish face image.

[0031] Furthermore, the step of obtaining underwater monitoring images of the marine ranch includes:

[0032] During the feeding period of the ocean ranch, shooting underwater monitoring videos at the feeding point of the ocean ranch;

[0033] The underwater monitoring image is acquired from the underwater monitoring video.

[0034] Furthermore, the step of obtaining underwater monitoring images of the marine ranch includes:

[0035] When the target fish face image is marked with the ID information of the pre-stored fish face image that is searched, the area of ​​the target fish face image in the underwater monitoring image is marked as a repeated appearance area;

[0036] Obtaining the cumulative number of times each area in the underwater monitoring image is marked as a repeated area;

[0037] For any area in the underwater monitoring image, when the accumulated marking times of the area is greater than a times threshold, the area is deleted from the underwater monitoring image.

[0038] On the other hand, an embodiment of the present invention also includes a computer device, including a memory and a processor, the memory is used to store at least one program, and the processor is used to load at least one program to execute the marine ranch fish face intelligent recognition method in the embodiment.

[0039] On the other hand, an embodiment of the present invention also includes a computer-readable storage medium, which stores a program executable by a processor. When the program executable by the processor is executed by the processor, it is used to execute the marine ranch fish face intelligent recognition method in the embodiment.

[0040] The beneficial effects of the present invention are: the marine ranch fish face intelligent recognition method in the embodiment applies the cutting-edge technologies of computer vision, machine learning and fishery technology, and is oriented to the modern marine ranch fish monitoring big data. It can apply the target detection model to accurately identify the fish face image and the whole fish image in the underwater monitoring image, and use the matching of the fish face image and the whole fish image to accurately identify the individual fish, thereby realizing the fish individual identification operations such as specific fish individual tracking and quantity counting, which is expected to improve the marine ranch fishery production efficiency, reduce labor costs, better serve the daily management of modern marine ranches, and promote the construction of smart marine ranches. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 Schematic diagram of the steps of the intelligent identification method of fish faces in ocean ranches in the embodiment;

[0042] Figure 2 Schematic diagram of the process of the intelligent identification method of fish faces in the marine ranch in the embodiment;

[0043] Figure 3 Schematic diagram of the principle of the intelligent identification method of fish faces in the marine ranch in the embodiment;

[0044] Figure 4 A schematic diagram of obtaining a labeled data set in an embodiment;

[0045] Figure 5 A schematic diagram of the principle of using an active learning method to train a target detection model in an embodiment;

[0046] Figure 6 A schematic diagram of the result obtained by processing an underwater monitoring image by the YOLOv8 model in the embodiment;

[0047] Figure 7 Schematic diagram of the principle of the step of matching and detecting each fish face image and each full fish image in the embodiment;

[0048] Figure 8 Schematic diagram of the moving trajectory of the target fish face image in the embodiment. DETAILED DESCRIPTION

[0049] Fish image recognition can be performed using technologies such as SIFT (Scale-invariant feature transform), HOG (Histogram of Oriented Gradient) or shape contours, but these fish image recognition technologies usually require researchers to manually specify features. Moreover, these methods rely on the experience of researchers and have certain difficulties in solving recognition tasks with feature similarity or a large number of categories. In addition, traditional technical solutions have some limitations, such as certain requirements for the background of the collected image, which are difficult to apply to fish recognition in complex seawater backgrounds. In particular, as the number and type of fish to be identified increase, the probability of feature overlap also increases, making it more difficult to manually analyze and set feature classification thresholds. Especially in the context of modern marine ranches, the breeding density is high, the monitoring range is significantly smaller than the breeding range, and the fish species in the same breeding cage are single. The application of traditional fish recognition technology is extremely limited only by distinguishing between the features of different categories.

[0050] In view of the limitations of technologies such as SIFT and HOG, we can consider designing a fish face recognition method and system based on coupled surface texture features and geometric features. This method is based on a small sample learning framework. On the basis of considering the influence of fish tail swing and oxygen concentration on fish body morphology, the improved Resnet network and YOLOv8 network are used to extract the surface texture features and geometric features of fish respectively, and then the identity of individual fish of the same species is recognized through the coupled analysis of surface texture features and geometric features.

[0051] Based on the above principles, in this embodiment, various steps in a method for intelligently identifying fish faces in an ocean ranch are provided. Figure 1 ,The ,marine ranch fish face intelligent recognition ,method includes the following steps:

[0052] S1. Obtain underwater monitoring images of the ocean ranch;

[0053] S2. Use the target detection model to detect the underwater surveillance image to obtain at least one fish face image and at least one full fish image;

[0054] S3. Matching detection is performed on each fish face image and each whole fish image to determine matching fish face images and whole fish images;

[0055] S4. For a fish face image with a matching full fish image, individual fish identification is performed based on the fish face image.

[0056] The process of steps S1-S4 is as follows Figure 2 As shown, the principles of steps S1-S4 are as follows Figure 3 shown.

[0057] Reference Figure 2 In step S1, an underwater camera may be installed at the location of the fish in the marine ranch. The field of view of the underwater camera includes locations where the fish often appear, such as covering the inner space of a cage in the marine ranch. The underwater monitoring video is obtained by shooting with the underwater camera, and the underwater monitoring image is extracted from the underwater monitoring video. The underwater monitoring image includes an image of at least one fish in the marine ranch.

[0058] In this embodiment, when executing step S2, that is, using the target detection model to detect the underwater monitoring image to obtain at least one fish face image and at least one whole fish image, the following steps can be specifically performed:

[0059] S201. Establishing a target detection model;

[0060] S202. Obtaining a labeled data set;

[0061] S203. Train the target detection model using the labeled data set;

[0062] S204. Input the underwater monitoring image into the trained target detection model for processing, and obtain the fish face bounding box and the whole fish bounding box output by the target detection model;

[0063] S205. Determine a corresponding fish face image in the underwater monitoring image according to the fish face bounding box;

[0064] S206. Determine a corresponding whole fish image in the underwater monitoring image according to the whole fish boundary box.

[0065] In step S201, YOLO can be used as the target detection model. In this embodiment, YOLOv8 is used as an example of the target detection model for description.

[0066] In step S201, the trained target detection model can be used directly, so there is no need to perform training steps such as steps S202-S203. If the target detection model has not been trained, steps S202-S203 can be performed to train the target detection model.

[0067] In step S202, a labeled dataset may be obtained from a public database. In this embodiment, the labeled dataset may be obtained from the DeepFish dataset. The DeepFish dataset is a large-scale benchmark dataset that contains approximately 40,000 images of fish taken underwater, which are taken from 20 fish habitats in the tropical marine environment of Australia.

[0068] Specifically, a small number of images (e.g., 1,000) can be obtained from the DeepFish dataset and manually annotated so that each image has a label. The label of the image indicates the position of the fish face (excluding the body and tail) and the whole fish (including the face, body, and tail) in the image. Specifically, Figure 4 The diagram is represented by a bounding box. Figure 4 In the figure, the green bounding box indicates the position of the fish face contained in the image, and the blue bounding box indicates the position of the whole fish contained in the image.

[0069] By obtaining public dataset images and manually annotating them, we obtained multiple pictures containing fish and labels used to indicate the location of the fish face and the whole fish in the pictures. These pictures (training samples) and corresponding labels constitute the annotated dataset.

[0070] In step S203, the target detection model is trained using the annotated data set obtained in step S202. Specifically, the training of the YOLOv8 model includes the following steps:

[0071] (1) Data enhancement: Data enhancement techniques (such as rotation, scaling, flipping, color transformation, etc.) are used to increase the diversity of training samples and improve the generalization ability of the model.

[0072] (2) Model configuration: Define the YOLOv8 model architecture, including the input layer, feature extraction layer, detection head, etc., and set hyperparameters such as learning rate, batch size, number of iterations, etc.

[0073] (3) Model training: Use the back-propagation algorithm to update model parameters through the optimization algorithm. After each epoch, use the validation set to evaluate the model performance and adjust the hyperparameters or perform early stopping based on the results.

[0074] (4) Save the model. After training is completed, save the model weights for subsequent use.

[0075] In this embodiment, when executing steps S202-S203, an active learning method may also be used to train the target detection model.

[0076] In this embodiment, the active learning method can be performed using the MQS (Membership Query Synthesis) method. Specifically, a small number of pictures (e.g., 1,000 pictures) can be obtained from the DeepFish dataset and manually annotated. In this way, there are still quite a few (more than 30,000) pictures in the DeepFish dataset that are not annotated and can be taken out as unlabeled training samples. The unlabeled pictures in the DeepFish dataset can also be transformed (e.g., by adding random noise), or pictures containing fish can be generated using a picture generation model, thereby obtaining more unlabeled training samples.

[0077] In this embodiment, the principle of the active learning method is as follows Figure 5 As shown, after obtaining the unlabeled training samples, the unlabeled training samples can be input into a target detection model that has undergone preliminary training (for example, training with training samples with manually annotated labels) for processing. The target detection model that has undergone preliminary training can identify the training samples with initial labels from the unlabeled training samples, where the initial labels are also used to represent the bounding boxes of the fish faces and the positions of the entire fish in the picture. However, the positions represented by the initial labels are not necessarily the exact positions of the fish faces and the entire fish in the picture. Therefore, based on the initial labels, the initial labels can be adjusted to the exact positions through manual correction to obtain the corrected labels.

[0078] Since the training samples with corrected labels can be used to train the target detection model just like the training samples directly taken from the DeepFish dataset and manually annotated, which is equivalent to an increase in the number of training samples in the annotated dataset, the active learning method can expand the annotated dataset. For example, starting from 1,000 pictures taken from the DeepFish dataset, the training samples in the annotated dataset can be expanded to 10,000. Moreover, since the initially trained target detection model outputs the initial labels, the workload required for manual correction is less than manual annotation. After more and more training samples are trained, the initial labels identified by the target detection model are actually close to the accurate positions of the fish face and the whole fish. The workload of manual correction is further reduced, and even the manual correction process may not be required, thereby improving learning efficiency.

[0079] After the training of steps S202-S203, in step S204, the trained target detection model can be used to process the underwater monitoring image obtained in step S1. Specifically, the processing of the underwater monitoring image using the YOLOv8 model includes the following steps:

[0080] (1) Image preprocessing: Before detection, the underwater monitoring image is preprocessed, for example, the size of the underwater monitoring image is adjusted to a multiple of 32;

[0081] (2) Feature extraction: The preprocessed image is input into the trained YOLOv8 model and the image features are extracted through the convolutional neural network;

[0082] (3) Anchor box generation and classification: Generate multiple anchor boxes on the feature map, classify the target category and regress the bounding box coordinates for each anchor box;

[0083] (4) Non-maximum suppression: Apply the non-maximum suppression algorithm to all predicted boxes, remove redundant boxes, and only retain the box with the highest confidence;

[0084] (5) Output result: The model returns the recognition result.

[0085] In step S204, the result obtained by the YOLOv8 model processing an underwater monitoring image is as follows: Figure 6 shown. Figure 6 In the , the recognition results of the YOLOv8 model for underwater surveillance images include the target category and the location of its bounding box. Specifically, the recognition results of the YOLOv8 model for underwater surveillance images include the fish face bounding box and the whole fish bounding box, where the fish face bounding box determines a specific area in the underwater surveillance image, within which the face of a fish is contained; the whole fish bounding box determines a specific area in the underwater surveillance image, within which the face, body and tail of a fish are contained as a whole.

[0086] In step S204, if the underwater surveillance image contains multiple fish, the YOLOv8 model may identify multiple fish face bounding boxes and multiple whole fish bounding boxes.

[0087] In step S205, the area in the underwater monitoring image defined by each fish face boundary frame is defined as a fish face image. In step S206, the area in the underwater monitoring image defined by each full fish boundary frame is defined as a full fish image.

[0088] By executing steps S201-S206, the target detection model can be used to efficiently and accurately detect fish face images and whole fish images in underwater monitoring images.

[0089] Due to the high density of breeding in the cages of the marine ranch and the limited video monitoring range, there is basically no situation where there is only one fish in the actual monitoring screen of the marine ranch. Often, many fish appear in the screen at the same time and there may be overlap between the fish. If only the YOLOv8 model is used, it can only identify the fish, but cannot distinguish the individuals of the same species in the cage. This is meaningless for the fish school monitoring work of modern marine ranches. Therefore, step S3 is executed to match and detect each fish face image and each whole fish image, determine the matching fish face image and whole fish image, and determine the matching fish face image and whole fish image as a corresponding fish individual.

[0090] In this embodiment, when executing step S3, that is, performing matching detection on each fish face image and each whole fish image to determine the matching fish face image and whole fish image, the following steps can be specifically performed:

[0091] S301. Obtain a first fish face bounding box corresponding to the first fish face image, and a first whole fish bounding box corresponding to the first whole fish image;

[0092] S302. Determine an IoMA value according to the first fish face bounding box and the first whole fish bounding box;

[0093] S303. When the IoMA value is equal to the threshold, it is determined that the first fish face image matches the first whole fish image; otherwise, it is determined that the first fish face image does not match the first whole fish image.

[0094] When executing step S301, each fish face image and each whole fish image identified in the underwater monitoring image can be traversed (whole fish images that have been determined to have matching fish face images and fish face images that have been determined to have matching whole fish images will no longer participate in subsequent traversals), so that each fish face image and each whole fish image has the possibility of being traversed and forming a combination.

[0095] The principle of steps S301-S303 is as follows Figure 7 shown.

[0096] In step S301, for any fish face image that is traversed, in this embodiment, it is called the first fish face image, and its corresponding fish face bounding box is the first fish face bounding box, such as Figure 7 As shown in the red bounding box in the middle right (the bounding box marked as fish_face); for any whole fish image that is traversed, it is called the first whole fish image in this embodiment, and its corresponding whole fish bounding box is the first whole fish bounding box, such as Figure 7 This is shown in the blue bounding box in the middle right (marked as the bounding box of fish).

[0097] In step S302, the IoMA value is determined according to the first fish face bounding box and the first whole fish bounding box. In this embodiment, the IoMA values ​​of the first fish face bounding box and the first whole fish bounding box are calculated as follows:

[0098] (1) obtaining the intersection of the first fish face bounding box and the first whole fish bounding box, that is, the overlapping portion of the first fish face bounding box and the first whole fish bounding box, and obtaining the area of ​​this portion, namely, the intersection area;

[0099] (2) Obtaining the minimum area corresponding to the first fish face bounding box and the first whole fish bounding box. Specifically, obtaining the area Area A of the first fish face bounding box and the area Area B of the first whole fish bounding box, respectively, and obtaining the minimum value min(Area A, Area B) therebetween, thereby obtaining the minimum area;

[0100] (3) The ratio of the intersection area divided by the minimum area min(Area A, Area B) is determined as the IoMA value, that is,

[0101]

[0102] In this embodiment, the value range of the IoMA value is 0-1, and a relatively large threshold value can be set within this interval. In step S303, the IoMA value is compared with the threshold value. If the IoMA value is greater than the threshold value, it can be determined that the first fish face image matches the first full fish image. Conversely, if the IoMA value is less than or equal to the threshold value, it can be determined that the first fish face image does not match the first full fish image.

[0103] In this embodiment, the principle of executing steps S301-S303 is that steps S301-S303 implement an optimized Hungarian algorithm, and the calculated IoMA value has a corresponding weight relationship between the first fish face image and the first whole fish image, so the IoMA value is equivalent to a fish face matching weight, and a connection between the first fish face image (fish face) and the first whole fish image (fish) is established; in an ideal case, Figure 7 As shown in the red bounding box in the middle left, the IoMA value calculated in this way is equal to 1, which means that the first fish face image (the first fish face bounding box, e.g. Figure 7 The small red bounding box in the middle left) is completely in the first whole fish image (the first whole fish bounding box, e.g. Figure 7If the first fish face image and the first whole fish image are within the large red bounding box on the middle left), it means that the first fish face image and the first whole fish image are taken of the same fish, and the first fish face image is a part of the first whole fish image, that is, the first fish face image and the first whole fish image belong to the same fish individual; similarly, if the calculated IoMA value is equal to 0, this indicates that the first fish face image (the first fish face bounding box) has no intersection with the first whole fish image, that is, the first fish face image and the first whole fish image belong to different fish individuals, and the two fish individuals do not overlap in the underwater monitoring image. If the calculated IoMA value is greater than 0 and less than 1, this indicates that the first fish face image (the first fish face bounding box, for example Figure 7 The black bounding box in the middle left) and the first whole fish image (the first whole fish bounding box, e.g. Figure 7 The first fish face image and the first whole fish image have some intersection but not complete overlap, which indicates that the first fish face image and the first whole fish image belong to different fish individuals, and the two fish individuals partially overlap in the underwater monitoring image. Therefore, ideally, an IoMA value equal to 1 indicates that the first fish face image and the first whole fish image correspond to the same fish individual, and it can be determined that the first fish face image matches the first whole fish image; an IoMA value less than 1 indicates that the first fish face image and the first whole fish image correspond to different fish individuals, and it can be determined that the first fish face image does not match the first whole fish image. In actual processing, a threshold close to 1 (for example, 0.8 or 0.9) can be set. When the IoMA value is greater than or equal to the threshold, it is determined that the first fish face image matches the first whole fish image, and when the IoMA value is less than the threshold, it is determined that the first fish face image does not match the first whole fish image.

[0104] By executing the maximum weight matching of the optimized Hungarian algorithm implemented by executing steps S301-S303, the fish face image in the underwater monitoring image can be matched with the whole fish image. A matched fish face image and a whole fish image indicate that an individual fish has been identified, thereby using the fish face as a feature to distinguish different individuals of the same fish school.

[0105] By executing steps S1-S3, different fish individuals contained in the underwater monitoring image can be identified and distinguished. Therefore, when executing step S4, fish individual identification operations such as tracking the trajectory of specific fish individuals, counting the number of fish individuals (total number, density, etc.), and identifying the volume of fish individuals can be performed.

[0106] Since the video surveillance range of the marine ranch is generally much smaller than its vast breeding area, in the actual surveillance video, fish often appear in the camera's field of view, disappear from the field of view, and reappear in the field of view. This situation may even occur multiple times, which can easily lead to the situation where an already identified fish is identified as a new fish, resulting in repeated counting and inaccurate counting. Therefore, it is necessary to determine whether the fish face image and the whole fish image matched by executing steps S1-S3 correspond to the fish that enters the video surveillance range for the first time (newly identified fish) or the fish that enters the video surveillance range repeatedly (previously identified fish).

[0107] In this embodiment, when executing step S4, that is, for the fish face image with a matching full fish image, performing fish individual identification based on the fish face image, the following steps may be specifically performed:

[0108] S401. With a fish face image having a matching full fish image as the target fish face image, search in the image base library; the image base library includes at least one pre-stored fish face image, and the ID information corresponding to the pre-stored fish face image;

[0109] S402. When a pre-stored fish face image having the same features as the target fish face image is searched from the image base, the target fish face image is marked with the ID information of the pre-stored fish face image searched;

[0110] S403. When no pre-stored fish face image having the same features as the target fish face image is found in the image base, new ID information is generated to mark the target fish face image, and the image base is updated using the target fish face image.

[0111] Steps S401-S403 implement fish face re-identification. The following is a detailed description of the process of using fish face re-identification:

[0112] In step S401, an image gallery is pre-established, in which a plurality of pre-stored fish face images and ID information corresponding to the pre-stored fish face images are stored. Specifically, the first frame or multiple frames of underwater monitoring images of the underwater monitoring video can be taken, and fish face images with matching full fish images (that is, fish face images identified as belonging to individual fish) identified from these underwater monitoring images can be used as pre-stored fish face images to establish the image gallery.

[0113] When executing step S401, for a pair of matching whole fish images and fish face images determined by executing steps S1-S3, the fish face image is used as the target fish face image, and the target fish face image is processed using the trained Resnet50 model. After processing, a set of data that can be used for comparison with the image gallery is obtained.

[0114] In step S401, when searching, the process of feature extraction and similarity measurement is executed: features are extracted from all pre-stored fish face images in the image base library through a deep learning algorithm, including global features and local features, wherein the global features extract the overall information of each pre-stored fish face image, while the local features focus on the specific areas of the pre-stored fish face images; next, in the similarity measurement stage, the algorithm maps the learned features to a new space, and calculates the Euclidean distance between the features of the target fish face image and the features of each pre-stored fish face image.

[0115] If the Euclidean distance between the feature of the target fish face image and the feature of a pre-stored fish face image is less than the distance threshold, it can be considered that the feature of the target fish face image is the same as the feature of the pre-stored fish face image, and the target fish face image and the pre-stored fish face image are taken of the same individual fish, and step S402 can be executed to mark the target fish face image with the ID information corresponding to the pre-stored fish face image, so that the target fish face image and the pre-stored fish face image have the same ID information, indicating that they correspond to the same individual fish. The target fish face image can also be stored in the image base as a new pre-stored fish face image, thereby updating the image base, so as to achieve the purpose of continuously updating the image base data and expanding the data set.

[0116] If the Euclidean distance between the feature of the target fish face image and the feature of a pre-stored fish face image is greater than the distance threshold, it can be considered that the feature of the target fish face image is different from the feature of the pre-stored fish face image, and the target fish face image and the pre-stored fish face image are obtained by photographing different fish individuals. Step S403 can be performed to generate new ID information, and the new ID information is different from the ID information of all current pre-stored fish face images. The target fish face image is marked with the new ID information, indicating that the target fish face image corresponds to a newly identified fish individual. The target fish face image is stored in the image base as a new pre-stored fish face image, thereby updating the image base, and achieving the purpose of continuously updating the image base data and expanding the data set.

[0117] If for a target fish face image, no pre-stored fish face image with the same features is found in the image base library, then Figure 8 As shown, the moving trajectory of the target fish face image is tracked and stored, thereby realizing the trajectory tracking of a specific fish individual.

[0118] In this embodiment, by executing steps S401-S403, fish face re-recognition is achieved, and it can be determined whether the latest recognized fish face image and the matching whole fish image correspond to fish individuals that are recognized for the first time or fish individuals that have been recognized before; since executing steps S401-S403 will only generate new ID information for fish individuals that are recognized for the first time, and fish individuals that are repeatedly recognized will only be assigned existing ID information, it is possible to accurately count fish individuals by counting the number of ID information.

[0119] In this embodiment, when executing step S1, that is, the step of obtaining the underwater monitoring image of the ocean ranch, the following steps may be specifically performed:

[0120] S101A. During the feeding period of the marine ranch, take underwater surveillance video at the feeding point of the marine ranch;

[0121] S102A. Obtain an underwater monitoring image from the underwater monitoring video.

[0122] Steps S101A-S102A are the first method of acquiring underwater monitoring images.

[0123] Marine ranches generally feed fish at fixed feeding points during specific feeding periods every day. In step S101A, an underwater camera can be set at the feeding point of the marine ranch, and the underwater camera can be started during the feeding period to shoot underwater monitoring videos, so as to perform step S102A to extract underwater monitoring images.

[0124] In this embodiment, by executing steps S101A-S102A, the fish can be guided to perform characteristic behaviors during daily characteristic time periods by feeding the fish. The fish individuals identified by subsequent execution of steps S2-S4 will reflect the daily characteristic time period behaviors of the fish in the marine ranch, such as fish distribution patterns or movement patterns, etc., which can be used as a reference for breeders or fish researchers.

[0125] In this embodiment, by executing steps S101A-S102A, the fish can also be guided to perform specific behaviors by feeding the fish, so as to meet the needs of executing the intelligent recognition method for fish faces in marine ranches.

[0126] For example, the feeding device can deliver feed to a specific location within the field of view of the underwater camera, thereby guiding the fish individuals to gather at the specific location, so that the distribution of fish individuals at other locations within the field of view of the underwater camera will be sparser, that is, there will be fewer overlapping different fish individuals, and there will be more combinations of fish face images and whole fish images that can be matched in the underwater monitoring image, so the speed of executing steps S301-S303 can be faster (this is because when executing steps S301-S303, the fish face images and whole fish images that can be matched will no longer participate in the subsequent traversal process, so the higher the matching success rate of the fish face images and the whole fish images, the fewer the number of fish face images and whole fish images that need to be traversed subsequently, and the faster the steps S301-S303 can be executed).

[0127] For example, the feeding equipment can deliver feed to a specific location outside the field of view of the underwater camera, thereby guiding individual fish to gather at the specific location. In this way, individual fish that enter the field of view of the underwater camera will tend to leave the field of view and no longer enter the field of view. This can reduce the possibility of individual fish repeatedly appearing in the field of view of the underwater camera, and reduce the need to execute steps S401-S403 for fish face re-identification. It can even ensure accurate individual fish identification without executing steps S401-S403, thereby reducing the amount of data processing and improving identification efficiency.

[0128] In this embodiment, when executing step S1, that is, the step of obtaining the underwater monitoring image of the ocean ranch, the following steps may be specifically performed:

[0129] S101B. When the target fish face image is marked with the ID information of the pre-stored fish face image searched, the target fish face image in the underwater monitoring image is marked as a recurring area;

[0130] S102B obtains the cumulative number of times each area in the underwater surveillance image is marked as a repeated region;

[0131] S103B. For any area in the underwater monitoring image, when the cumulative marking times of the area is greater than the times threshold, the area is deleted from the underwater monitoring image.

[0132] Steps S101B-S103B are a second method for acquiring underwater monitoring images.

[0133] In this embodiment, since the underwater monitoring image is obtained from the underwater monitoring video, that is, the underwater monitoring image is a frame in the underwater monitoring video, the processing of steps S1-S4 is performed for each underwater monitoring image, so steps S1-S4 are actually repeatedly performed. At this time, steps S101B-S103B are performed on the basis of previously executing steps S1-S4 multiple times.

[0134] Each time steps S1-S4 are executed, if step S402 is executed, a pre-stored fish face image with the same features as the target fish face image is searched from the image database, and the target fish face image is marked with the ID information of the searched pre-stored fish face image. Then, the area of ​​the target fish face image in the underwater monitoring image can be marked as a "repeatedly appearing area", and the cumulative number of times this area is marked as a "repeatedly appearing area" is increased by 1.

[0135] In step S102B, the field of view of the underwater camera is divided into a plurality of areas, so that the underwater monitoring image captured by the underwater camera is also divided into a plurality of areas, and the cumulative number of times each area is marked as a "repeated area" is counted.

[0136] In step S103B, for any area in the underwater monitoring image, the cumulative marking times of this area is compared with the number threshold (for example, 100 times). If the cumulative marking times of this area is greater than the number threshold, then the area is segmented and deleted from the underwater monitoring image, that is, this area is no longer included in the underwater monitoring image to be processed when the subsequent steps S2-S4 are executed.

[0137] In this embodiment, the principle of executing steps S101B-S103B is that the cumulative number of markings of an area in the underwater monitoring image reflects the number of times the same fish individual has repeatedly appeared in this area through re-identification in the past. If the cumulative number of markings of this area is greater than the number threshold, it can be determined that the number of times the same fish individual has repeatedly appeared in this area through re-identification in the past is large, and this area in the underwater monitoring image can be deleted. Therefore, when currently executing steps S2-S4, the number of times steps S401-S403 are executed can be reduced or there is no need to execute steps S401-S403, and the possibility of repeated identification of fish individuals contained in the underwater monitoring image can also be reduced, thereby reducing the amount of data processing and improving recognition efficiency.

[0138] The marine ranch fish face intelligent recognition method in this embodiment applies cutting-edge technologies of computer vision, machine learning and fishery technology, and is oriented towards modern marine ranch fish monitoring big data. It can use target detection models to accurately identify fish face images and whole fish images in underwater monitoring images, and use the matching of fish face images and whole fish images to accurately identify individual fish, thereby realizing fish individual recognition operations such as specific fish individual tracking and quantity counting, which is expected to improve marine ranch fishery production efficiency, reduce labor costs, better serve the daily management of modern marine ranches, and promote the construction of smart marine ranches.

[0139] A computer program for executing the ocean ranch fish face intelligent recognition method in this embodiment can be written and written into a computer device or storage medium. When the computer program is read out and executed, the ocean ranch fish face intelligent recognition method in this embodiment is executed, thereby achieving the same technical effect as the ocean ranch fish face intelligent recognition method in the embodiment.

[0140] It should be noted that, unless otherwise specified, when a feature is referred to as being "fixed" or "connected" to another feature, it may be directly fixed or connected to the other feature, or it may be indirectly fixed or connected to the other feature. In addition, the descriptions of up, down, left, right, etc. used in the present disclosure are only relative to the relative positional relationship of the components of the present disclosure in the accompanying drawings. The singular forms of "a", "" and "the" used in the present disclosure are also intended to include the plural forms, unless the context clearly indicates other meanings. In addition, unless otherwise defined, all technical and scientific terms used in this embodiment have the same meaning as those generally understood by those skilled in the art. The terms used in the specification of this embodiment are only for describing specific embodiments and are not intended to limit the present invention. The term "and / or" used in this embodiment includes any combination of one or more related listed items.

[0141] It should be understood that, although the term first, second, third etc. may be adopted to describe various elements in the present disclosure, these elements should not be limited to these terms. These terms are only used to distinguish the same type of elements from each other. For example, without departing from the scope of the present disclosure, the first element may also be referred to as the second element, and similarly, the second element may also be referred to as the first element. The use of any and all examples or exemplary language ("for example", "such as" etc.) provided by the present embodiment is only intended to better illustrate embodiments of the present invention, and unless otherwise required, the scope of the present invention will not be limited.

[0142] It should be appreciated that embodiments of the present invention may be implemented or enforced by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable memory. The method may be implemented in a computer program using standard programming techniques - including a non-transitory computer-readable storage medium configured with a computer program, wherein the storage medium so configured causes the computer to operate in a specific and predefined manner - according to the methods and drawings described in the specific embodiments. Each program may be implemented in a high-level procedural or object-oriented programming language to communicate with a computer system. However, if desired, the program may be implemented in assembly or machine language. In any case, the language may be a compiled or interpreted language. In addition, the program may be run on a programmed dedicated integrated circuit for this purpose.

[0143] In addition, the operations of the process described in this embodiment may be performed in any suitable order, unless otherwise indicated in this embodiment or otherwise clearly contradicted by the context. The process described in this embodiment (or variations and / or combinations thereof) may be performed under the control of one or more computer systems configured with executable instructions, and may be implemented as a code (e.g., executable instructions, one or more computer programs, or one or more applications) executed on one or more processors in common, by hardware or a combination thereof. A computer program includes a plurality of instructions that may be executed by one or more processors.

[0144] Further, the method can be implemented in any type of computing platform that is operably connected to a suitable computer, including but not limited to a personal computer, a minicomputer, a mainframe, a workstation, a network or distributed computing environment, a separate or integrated computer platform, or in communication with a charged particle tool or other imaging device, etc. Various aspects of the present invention can be implemented in machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, an optical read and / or write storage medium, a RAM, a ROM, etc., so that it can be read by a programmable computer, and when the storage medium or device is read by the computer, it can be used to configure and operate the computer to perform the process described herein. In addition, the machine-readable code, or part thereof, can be transmitted via a wired or wireless network. When such media includes instructions or programs that implement the above steps in conjunction with a microprocessor or other data processor, the invention of this embodiment includes these and other different types of non-transitory computer-readable storage media. When programmed according to the methods and techniques of the present invention, the present invention also includes the computer itself.

[0145] The computer program can be applied to input data to perform the functions of the present embodiment, thereby converting the input data to generate output data stored in a non-volatile memory. The output information can also be applied to one or more output devices such as a display. In a preferred embodiment of the present invention, the converted data represents a physical and tangible object, including a specific visual depiction of the physical and tangible object produced on the display.

[0146] The above are only preferred embodiments of the present invention. The present invention is not limited to the above embodiments. As long as the technical effects of the present invention are achieved by the same means, any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the scope of protection of the present invention. Within the scope of protection of the present invention, its technical solutions and / or implementation methods may have various modifications and changes.

Claims

1. A method for intelligent recognition of fish faces in marine ranches, characterized in that: The marine ranch fish face intelligent recognition method comprises: Obtain underwater surveillance images of the ocean ranch; Detect the underwater monitoring image using a target detection model to obtain at least one fish face image and at least one whole fish image; Performing matching detection on each of the fish face images and each of the whole fish images to determine the fish face images and the whole fish images that match; For the fish face image having the matching full fish image, individual fish identification is performed based on the fish face image.

2. The method for intelligently identifying fish faces in marine ranching according to claim 1, characterized in that: The method of detecting the underwater monitoring image using the target detection model to obtain at least one fish face image and at least one whole fish image includes: Establishing the target detection model; Get the labeled dataset; Using the labeled data set to train the target detection model; Input the underwater monitoring image into the trained target detection model for processing, and obtain the fish face bounding box and the whole fish bounding box output by the target detection model; Determining the corresponding fish face image in the underwater monitoring image according to the fish face bounding box; According to the whole fish bounding box, the corresponding whole fish image is determined in the underwater monitoring image.

3. The method for intelligently identifying fish faces in marine ranching according to claim 2, characterized in that: The using the labeled data set to train the target detection model includes: Expanding the labeled data set by an active learning method; The target detection model is trained using the expanded labeled data set.

4. The method for intelligently identifying fish faces in marine ranching according to claim 1, characterized in that: The matching detection of each of the fish face images and each of the whole fish images to determine the matching fish face images and the whole fish images includes: Obtain a first fish face bounding box corresponding to a first fish face image, and a first whole fish bounding box corresponding to a first whole fish image; wherein the first fish face image is any one of the fish face images, and the first whole fish image is any one of the whole fish images; Determine an IoMA value according to the first fish face bounding box and the first whole fish bounding box; When the IoMA value is equal to the threshold, it is determined that the first fish face image matches the first whole fish image; otherwise, it is determined that the first fish face image does not match the first whole fish image.

5. The method for intelligently identifying fish faces in marine ranching according to claim 4, characterized in that: The determining the IoMA value according to the first fish face bounding box and the first whole fish bounding box comprises: Obtaining an intersection area of ​​the first fish face bounding box and the first whole fish bounding box; Obtaining the minimum area of ​​the first fish face bounding box and the first whole fish bounding box; the minimum area is the minimum area between the first fish face bounding box and the first whole fish bounding box; The ratio obtained by dividing the intersection area by the smallest area is determined as the IoMA value.

6. The method for intelligently identifying fish faces in marine ranching according to any one of claims 1 to 5, characterized in that: The method of identifying individual fishes according to the fish face image having the matching whole fish image comprises: The fish face image having the matching full fish image is used as a target fish face image and searched in an image base library; the image base library includes at least one pre-stored fish face image and ID information corresponding to the pre-stored fish face image; When the pre-stored fish face image having the same features as the target fish face image is searched from the image base, the target fish face image is marked with the ID information of the searched pre-stored fish face image; When the pre-stored fish face image having the same features as the target fish face image is not found in the image base library, new ID information is generated to mark the target fish face image, and the image base library is updated using the target fish face image.

7. The method for intelligently identifying fish faces in marine ranching according to claim 1, characterized in that: The method of obtaining underwater monitoring images of the ocean ranch includes: During the feeding period of the ocean ranch, shooting underwater monitoring videos at the feeding point of the ocean ranch; The underwater monitoring image is acquired from the underwater monitoring video.

8. The method for intelligently identifying fish faces in marine ranching according to claim 6, characterized in that: The method of obtaining underwater monitoring images of the ocean ranch includes: When the target fish face image is marked with the ID information of the pre-stored fish face image that is searched, the area of ​​the target fish face image in the underwater monitoring image is marked as a repeated appearance area; Obtaining the cumulative number of times each area in the underwater monitoring image is marked as a repeated area; For any area in the underwater monitoring image, when the accumulated marking times of the area is greater than a times threshold, the area is deleted from the underwater monitoring image.

9. A computer device, characterized in that: It includes a memory and a processor, the memory is used to store at least one program, and the processor is used to load at least one program to execute the marine ranch fish face intelligent recognition method described in any one of claims 1-8.

10. A computer-readable storage medium storing a program executable by a processor, characterized in that: The program executable by the processor is used to execute the marine ranch fish face intelligent recognition method described in any one of claims 1-8 when executed by the processor.

Images