Fish detection methods, devices, systems and storage media

Abstract

This invention discloses a fish detection method, apparatus, system, and storage medium, relating to the field of underwater detection technology. The fish detection method includes the following steps: inputting a set of fish keyframe images into a Gaussian Mixture Model (GMM) for moving target detection, obtaining GMM fish target bounding boxes; inputting the set of fish keyframe images into a YOLO neural network model for moving target detection, obtaining YOLO fish target bounding boxes; determining whether the GMM fish target bounding boxes and / or YOLO fish target bounding boxes are true fish target bounding boxes according to a preset judgment logic, and outputting the corresponding true fish target bounding boxes. Utilizing the high detection speed and accuracy of the YOLO neural network model, and integrating the GMM model to fully exploit the characteristics of fish moving target information, the fish target bounding box fusion judgment method effectively avoids false detections and missed detections, improves the model's detection accuracy, and achieves real-time and accurate detection of fish feeding intensity, feeding distribution, and other state information.

Description

Technical Field

[0001] This invention relates to the field of underwater detection technology, and in particular to a method, apparatus, system and storage medium for fish detection. Background Technology

[0002] Currently, my country's aquaculture technology is developing from nearshore to deep-sea areas. Deep-sea aquaculture is characterized by its distance from the shore and complex aquatic environment, which increases the difficulty of daily aquaculture operations, monitoring of aquaculture anomalies, and safety maintenance.

[0003] In deep-sea cage aquaculture, overfeeding can easily lead to uneaten feed, wasting resources and polluting the aquatic environment; underfeeding can result in uneven fish growth, affecting aquaculture efficiency. How to achieve real-time monitoring of fish feeding intensity and distribution, considering the characteristics of large fish populations, rapid changes in fish populations, fish obstruction, and changes in fish position and body shape, is a crucial technical problem that needs to be solved in the process of realizing intelligent deep-sea aquaculture. Summary of the Invention

[0004] The main objective of this invention is to provide a fish detection method that enables real-time detection of fish feeding intensity, feeding distribution, and other status information.

[0005] To achieve the above objectives, the fish detection method provided by the present invention includes the following steps:

[0006] The keyframe image set of fish is input into the Gaussian Mixture Model (GMM) for moving target detection to obtain the GMM fish target bounding box.

[0007] The set of fish keyframe images is input into the YOLO neural network model for moving target detection to obtain the YOLO fish target bounding box.

[0008] Based on a preset judgment logic, determine whether the GMM fish target bounding box and / or the YOLO fish target bounding box are real fish target bounding boxes, and output the corresponding real fish target bounding box.

[0009] Further, the step of determining whether the GMM fish target bounding box and / or the YOLO fish target bounding box are real fish target bounding boxes according to a preset judgment logic, and outputting the corresponding real fish target bounding box, includes:

[0010] Determine whether a first undetermined border exists in the GMM fish target border; the first undetermined border does not appear in the YOLO fish target border;

[0011] If the first undetermined border exists, determine whether the duration of the existence of the first undetermined border reaches a preset time threshold.

[0012] If the duration of the first undetermined border reaches the preset time threshold, then the first undetermined border is determined to be a real fish target border and output.

[0013] Furthermore, the YOLO fish target bounding box includes the YOLO target presence confidence level;

[0014] The step of determining whether the GMM fish target bounding box and / or the YOLO fish target bounding box are real fish target bounding boxes according to a preset judgment logic, and outputting the corresponding real fish target bounding box, includes:

[0015] Determine whether a second undetermined border exists in the YOLO fish target border; the second undetermined border does not appear in the GMM fish target border;

[0016] If a second undetermined bounding box exists, then determine whether the confidence level of the YOLO target of the second undetermined bounding box reaches a preset confidence threshold.

[0017] If the confidence level of the YOLO target of the second undetermined bounding box reaches the preset confidence threshold, then the second undetermined bounding box is determined to be a real fish target bounding box and output.

[0018] Furthermore, the YOLO fish target bounding box includes the YOLO target position x-coordinate, YOLO target position y-coordinate, YOLO target width, and YOLO target height; the GMM fish target bounding box includes the GMM target position x-coordinate, GMM target position y-coordinate, GMM target width, and GMM target height.

[0019] The step of determining whether the GMM fish target bounding box and / or the YOLO fish target bounding box are real fish target bounding boxes according to a preset judgment logic, and outputting the corresponding real fish target bounding box, includes:

[0020] Determine whether a third undetermined border exists within the YOLO fish target border; the third undetermined border appears within the GMM fish target border;

[0021] If the third undetermined border exists, then obtain the first difference between the horizontal coordinate of the YOLO target position and the horizontal coordinate of the GMM target position of the third undetermined border, obtain the second difference between the vertical coordinate of the YOLO target position and the vertical coordinate of the GMM target position of the third undetermined border, obtain the third difference between the YOLO target width and the GMM target width of the third undetermined border, and obtain the fourth difference between the YOLO target height and the GMM target height of the third undetermined border.

[0022] Determine whether the first difference, the second difference, the third difference, and the fourth difference are less than a preset difference threshold;

[0023] If at least one of the first difference, the second difference, the third difference, and the fourth difference is less than the preset difference threshold, then the third undetermined bounding box is determined to be the real fish target bounding box and output.

[0024] Further, after the step of determining whether the first difference, the second difference, the third difference, and the fourth difference are less than a preset difference threshold, the method includes:

[0025] If the first difference, the second difference, the third difference, and the fourth difference are all greater than or equal to the preset difference threshold, then it is determined whether the straight-line distance between the center point of the third undetermined border in the YOLO fish target border and the center point of the third undetermined border in the GMM fish target border is less than the preset distance threshold.

[0026] If the straight-line distance between the center point of the third undetermined border in the YOLO fish target border and the center point of the third undetermined border in the GMM fish target border is less than the preset distance threshold, then the third undetermined border is determined to be the real fish target border and output.

[0027] Further, after the step of determining whether the straight-line distance between the center point of the third undetermined border in the YOLO fish target border and the center point of the third undetermined border in the GMM fish target border is less than a preset distance threshold, the method includes:

[0028] If the straight-line distance between the center point of the third undetermined border in the YOLO fish target border and the center point of the third undetermined border in the GMM fish target border is greater than or equal to the preset distance threshold, then it is determined whether the ratio of the enclosed area of ​​the third undetermined border in the GMM fish target border to the enclosed area of ​​the third undetermined border in the YOLO fish target border is greater than or equal to the preset ratio threshold.

[0029] If the ratio of the enclosed area of ​​the third undetermined border in the GMM fish target border to the enclosed area of ​​the third undetermined border in the YOLO fish target border is greater than or equal to the preset ratio threshold, then the third undetermined border is determined to be the real fish target border and output.

[0030] Further, after the step of outputting the corresponding real fish target bounding box, the following steps are included:

[0031] The number and location of detected fish targets are obtained from the actual fish target bounding box.

[0032] The fish targets in the fish keyframe image set are counted by manual measurement to obtain the actual number and location of the fish targets.

[0033] The number of detected fish targets is compared with the number of actual fish targets, and the location of the detected fish targets is compared with the location of the actual fish targets to obtain the model accuracy evaluation index of the combination of the GMM Gaussian mixture model and the YOLO neural network model.

[0034] Further, the step of comparing the number of detected fish targets with the number of actually measured fish targets, and comparing the positions of the detected fish targets with the positions of the actually measured fish targets, to obtain the model accuracy evaluation index of the combination of the GMM Gaussian Mixture Model and the YOLO Neural Network Model, includes:

[0035] Count the actual number of fish targets among the detected fish targets;

[0036] The model accuracy is calculated using the following formula:

[0037] P = T R / T

[0038] Where P is the accuracy of the model, and T R T represents the actual number of fish targets, and T represents the number of detected fish targets.

[0039] And / or, calculate the model recall using the following formula:

[0040] R = T R / N

[0041] Where R is the recall rate of the model, and T R N represents the actual number of fish targets, and N represents the measured number of fish targets.

[0042] And / or, calculate the false positive rate according to the following formula:

[0043] P f =(TT) R ) / T

[0044] Among them, P f Let T be the false detection rate. R T represents the actual number of fish targets, and T represents the number of detected fish targets.

[0045] And / or, calculate the false negative rate according to the following formula:

[0046] P m =(NT) R) / T

[0047] Among them, P m Let T be the false negative rate. R Where N is the actual number of fish targets, T is the number of fish targets measured, and T is the number of fish targets detected.

[0048] And / or, calculate the fish target position error for each frame in the fish keyframe image set according to the following formula:

[0049] E i =∑(|S T -S A | / S A )

[0050] Among them, E i S represents the target position error of the fish. T For the detection of the target fish location, S A The measured target location of the fish;

[0051] And / or, calculate the expected value of the fish target position error in the fish keyframe image set according to the following formula:

[0052] D = 1 / n∑(1 / T) R ∑(|S T -S A | / S A ))

[0053] Where D is the expected value of the target position error of the fish, n is the number of image frames in the keyframe image set of the fish, and T R S represents the actual target number of fish. T For the detection of the target fish location, S A The measured target location of the fish is shown.

[0054] Further, before the step of inputting the fish keyframe image set into the Gaussian Mixture Model (GMM) for moving target detection to obtain the GMM fish target bounding box, the following steps are included:

[0055] Perform GMM background modeling operation on the first frame background image of the fish keyframe image set;

[0056] And / or, before the step of inputting the fish keyframe image set into the YOLO neural network model for moving target detection to obtain the YOLO fish target bounding box, the following steps are included:

[0057] Fish target training samples are input into the YOLO neural network model for autonomous training.

[0058] Furthermore, to achieve the above objectives, the present invention also provides a fish detection device, the fish detection device comprising:

[0059] The GMM detection module is used to input the fish keyframe image set into the GMM Gaussian mixture model to perform moving target detection operations, so as to obtain the GMM fish target bounding box.

[0060] The YOLO detection module is used to input the fish keyframe image set into the YOLO neural network model to perform moving target detection operations, so as to obtain the YOLO fish target bounding box.

[0061] The judgment module is used to determine whether the GMM fish target border and / or the YOLO fish target border are real fish target borders according to a preset judgment logic, and output the corresponding real fish target border.

[0062] In addition, to achieve the above objectives, the present invention also provides a fish detection system, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps of the fish detection method as described above.

[0063] In addition, to achieve the above objectives, the present invention also provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores a fish detection program, which, when executed by a processor, implements the steps of the fish detection method as described above.

[0064] The fish detection method provided by this invention combines a YOLO neural network model with a Gaussian mixture model (GMM) to detect fish feeding behavior. Leveraging the high speed and accuracy of the YOLO neural network model, and combining it with the GMM model to fully extract information about moving fish targets, the method uses a fish target bounding box fusion judgment method to compare the fish target detection results of the GMM model with those of the YOLO neural network model. This limits the fish target detection information of the YOLO neural network model, thereby eliminating false targets in dynamic backgrounds and detecting occluded fish targets, effectively avoiding false detections and missed detections. Compared to using only the YOLO neural network model with detection thresholds, this method improves the detection accuracy of the model while maintaining the high accuracy of the YOLO neural network model, thus achieving real-time and accurate detection of fish feeding intensity, feeding distribution, and other state information. Attached Figure Description

[0065] Figure 1 This is a schematic flowchart of an embodiment of the fish detection method of the present invention;

[0066] Figure 2This is a schematic diagram of the preset judgment logic in one embodiment of the fish detection method of the present invention;

[0067] Figure 3 This is a schematic diagram of model accuracy evaluation in one embodiment of the fish detection method of the present invention;

[0068] Figure 4 This is a schematic diagram of the structure of an embodiment of the fish detection device of the present invention;

[0069] Figure 5 This is a schematic diagram of the system structure of the hardware operating environment involved in the embodiments of the present invention.

[0070] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0071] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0072] In deep-sea cage aquaculture, overfeeding can easily lead to uneaten feed, wasting resources and polluting the aquatic environment; underfeeding can result in uneven fish growth, affecting aquaculture efficiency. How to achieve real-time monitoring of fish feeding intensity and distribution, considering the characteristics of large fish populations, rapid changes in fish populations, fish obstruction, and changes in fish position and body shape, is a crucial technical problem that needs to be solved in the process of realizing intelligent deep-sea aquaculture.

[0073] The technical solution of the present invention includes: inputting a set of fish keyframe images into a Gaussian mixture model (GMM) to perform moving target detection to obtain a GMM fish target bounding box; inputting a set of fish keyframe images into a YOLO neural network model to perform moving target detection to obtain a YOLO fish target bounding box; determining whether the GMM fish target bounding box and / or the YOLO fish target bounding box are real fish target bounding boxes according to a preset judgment logic, and outputting the corresponding real fish target bounding box.

[0074] Reference Figure 5 , Figure 5 This is a schematic diagram of the fish detection system structure in the hardware operating environment involved in the embodiments of the present invention.

[0075] like Figure 5As shown, the fish detection system may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM), such as a disk drive. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0076] Those skilled in the art will understand that Figure 5 The structure shown does not constitute a limitation on the fish detection system and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0077] like Figure 5 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a data storage module, a network communication module, a user interface module, and a fish detection program.

[0078] exist Figure 5 In the electronic device shown, the network interface 1004 is mainly used for data communication with other devices; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the electronic device of the present invention can be set in the electronic device, and the electronic device calls the fish detection program stored in the memory 1005 through the processor 1001 and executes the fish detection method provided in the embodiment of the present invention.

[0079] This invention provides a method for detecting fish, referring to... Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of a fish detection system according to the present invention.

[0080] In this embodiment, the fish detection method includes the following steps:

[0081] S1, input the fish keyframe image set into the GMM Gaussian mixture model to perform moving target detection operation to obtain the GMM fish target bounding box;

[0082] S2, input the fish keyframe image set into the YOLO neural network model to perform moving target detection operation to obtain the YOLO fish target bounding box;

[0083] S3, determine whether the GMM fish target bounding box and / or YOLO fish target bounding box are real fish target bounding boxes according to the preset judgment logic, and output the corresponding real fish target bounding box.

[0084] The fish keyframe image set is extracted from videos of fish activity in net cages captured by image sensing devices using keyframe technology and after noise reduction processing. It includes multiple frames that can characterize the main distribution of fish targets.

[0085] In step S2, the YOLO neural network model based on deep learning can label the color, shape, outline and other image texture features of fish targets in the video sequence images and train them to form a feature extraction convolutional network, thereby realizing the detection of fish targets and outputting the distribution of fish targets and the quantitative intensity of fish feeding behavior.

[0086] However, YOLO neural network models typically set detection thresholds to avoid false detections of stationary objects resembling fish. But if the threshold is set too high, false negatives will occur. In other words, the detection algorithm of a single YOLO neural network model cannot adequately extract the motion information of fish targets to avoid false negatives in the case of fish transients and fish occlusion, nor can it adapt to changes in fish orientation and body shape to extract target features and avoid false positives.

[0087] To address the aforementioned shortcomings of the single YOLO neural network model, step S1 of this embodiment introduces a Gaussian Mixture Model (GMM) to combine with the YOLO neural network model for fish feeding behavior detection. The GMM can more fully exploit the characteristics of fish target motion information, and can necessaryly replace the detection threshold set by the YOLO neural network model, thereby effectively avoiding false detections and false negatives.

[0088] In practical implementation, after the fish keyframe image set is input into the YOLO neural network model, each frame image is divided into S×S grids. The YOLO detection algorithm detects fish target bounding boxes for each grid. The RGB frame spots in the grid are extracted and passed to the fish YOLO classifier, thereby detecting the fish target bounding boxes in each grid. Here, the fish target bounding box refers to the outline of the fish identified by the corresponding detection algorithm. Based on the identified fish target bounding boxes, information such as the number and distribution of fish targets in any grid can be determined. Combining the fish target bounding boxes of all grids in a frame yields all the fish target bounding boxes in that frame. By combining the fish target bounding boxes of each frame, the distribution of fish targets and the quantitative intensity of fish feeding behavior in a certain time period in the aquaculture area (such as aquaculture cages) can be obtained. It can be understood that the YOLO fish target bounding box can refer to a single fish target bounding box or a set of fish target bounding boxes detected by the YOLO neural network model within any grid, within all grids of any frame, or across multiple frames. Its definition can be flexibly adjusted according to the application scenario, and is not limited here.

[0089] Similarly, after the fish keyframe image set is input into the Gaussian Mixture Model (GMM), each frame is divided into S×S grids. The GMM detection algorithm detects fish target bounding boxes for each grid. RGB frame spots in the grid are extracted and passed to the fish ResNet-50 classifier, thus detecting the fish target bounding boxes in each grid. Based on the identified fish target bounding boxes, the quantity and distribution of fish targets in any grid can be determined. Combining the fish target bounding boxes of all grids in a frame yields all fish target bounding boxes in that frame. Finally, combining the fish target bounding boxes of each frame provides the distribution of fish targets in a farming area (such as a fish cage) over a certain time period and the quantitative intensity of fish feeding behavior. It can be understood that the GMM fish target bounding box can refer to a single fish target bounding box or a set of fish target bounding boxes detected by the GMM within any grid, all grids in any frame, or multiple frames. Its definition can be flexibly adjusted according to the application scenario, and is not limited here.

[0090] For any one of the fish target bounding boxes, at least the following scenarios are possible: 1. It is detected by the Gaussian Mixture Model (GMM) but not by the YOLO Neural Network Model (YNN); 2. It is detected by the YOLO Neural Network Model but not by the GMM; or 3. It is detected by both the GMM and YOLO Neural Network Models. For these scenarios, a judgment and selection can be made using the preset judgment logic in step S3. Specifically, this judgment and selection operation can be automatically executed by a judge. For the same fish target bounding box, the judge analyzes various detection parameters of the GMM and YOLO fish target bounding boxes and compares these parameters with preset thresholds to determine whether the accuracy of the detected GMM and / or YOLO fish target bounding boxes meets preset requirements. The judge then selects the GMM or YOLO fish target bounding box whose accuracy meets the preset requirements as the true fish target bounding box output. The same operation applies to other fish target bounding boxes, thus obtaining more accurate true fish target bounding boxes compared to a single data source.

[0091] Therefore, this embodiment combines the YOLO neural network model with the Gaussian mixture model (GMM) to detect fish feeding behavior. Leveraging the YOLO neural network model's high detection speed and accuracy, and combining it with the GMM model allows for the full extraction of fish movement target information, the fish target bounding box fusion judgment method compares the fish target detection results of the GMM model with the YOLO neural network model's fish target detection information. This effectively eliminates false targets in dynamic backgrounds and detects occluded fish targets, thus avoiding false detections and missed detections. Compared to using only the YOLO neural network model with detection thresholds, this method improves the model's detection accuracy while maintaining the high accuracy of the YOLO neural network model, thereby achieving real-time and accurate detection of fish feeding intensity, feeding distribution, and other state information.

[0092] Optionally, before step S1, the following steps are included:

[0093] S01, perform GMM background modeling on the first frame background image of the fish keyframe image set; to provide a basis for image and foreground segmentation in subsequent GMM fish target detection operations; specifically, the GMM Gaussian mixture model uses the Gaussian method to characterize the pixel feature value of each pixel in each frame image and matches it with the background; if the match is successful, the pixel is determined to be a background pixel, otherwise it is determined to be a foreground pixel, thereby achieving the separation of foreground target and background.

[0094] Optionally, before step S2, the following steps are included:

[0095] S02, input the fish target training samples into the YOLO neural network model for autonomous training; the fish target training samples include the inherent image texture features such as fish outlines and structures. The feature extraction convolutional layer of the YOLO neural network model can form a fish feature extraction network by learning the rules of the labeled fish outlines, structures and other inherent image texture features, which can be used for subsequent YOLO fish target detection operations.

[0096] Regarding the specific judgment logic in step S3, one or more specific solutions described in the following embodiments can be adopted:

[0097] Optionally, refer to Figure 2 Step S3 includes:

[0098] S301, determine whether there is a first undetermined bounding box in the GMM fish target bounding box; the first undetermined bounding box does not appear in the YOLO fish target bounding box; that is, for one of the fish target bounding boxes, it is detected by the GMM Gaussian mixture model, but not by the YOLO neural network model, then it is used as the first undetermined bounding box to be determined.

[0099] S302, if a first undetermined border exists, determine whether the duration of the existence of the first undetermined border reaches a preset time threshold; specifically, whether the duration of the existence of the first undetermined border reaches the preset time threshold can be determined by whether the first undetermined border is continuously detected in 60 key frame images, or whether the inter-frame duration of the first undetermined border exceeds 3 seconds.

[0100] S303, if the duration of the existence of the first undetermined bounding box reaches a preset time threshold, then the first undetermined bounding box is determined to be a real fish target bounding box and output; that is, for the determined fish target, it will be represented by the GMM fish target bounding box in the future.

[0101] Optionally, refer to Figure 2 The YOLO fish bounding box includes the YOLO target presence confidence score. Specifically, during the detection process, the YOLO neural network model divides each input image frame into S×S grids. The YOLO detection algorithm detects fish bounding boxes for each grid and provides five parameters of the detected fish bounding boxes (YOLO target position x-coordinate, YOLO target position y-coordinate, YOLO target width, YOLO target height, and YOLO target presence confidence score). Assuming that B fish bounding boxes are detected in each grid, the final S×S×(B×5+C) dimensional feature vector map of the fish keyframe image set can be obtained through the YOLO neural network model. Here, 5 represents the five parameters corresponding to the fish bounding boxes, and C represents the number of fish categories.

[0102] Similarly, the bounding box of the GMM fish target also includes the above five parameters (GMM target position x-coordinate, GMM target position y-coordinate, GMM target width, GMM target height, and GMM target presence confidence).

[0103] Step S3 includes:

[0104] S304, determine whether there is a second undetermined bounding box in the YOLO fish target bounding box; the second undetermined bounding box does not appear in the GMM fish target bounding box; that is, for one of the fish target bounding boxes, it is detected by the YOLO neural network model but not by the GMM Gaussian mixture model, then it is used as the second undetermined bounding box to be determined.

[0105] S305, if a second undetermined bounding box exists, determine whether the YOLO target existence confidence of the second undetermined bounding box reaches a preset confidence threshold; specifically, determine whether the YOLO target existence confidence of the second undetermined bounding box is greater than or equal to 0.95;

[0106] S306, if the confidence level of the YOLO target of the second undetermined bounding box reaches the preset confidence threshold, then the second undetermined bounding box is determined to be a real fish target bounding box and output; that is, for the judged fish target, it will be represented by the YOLO fish target bounding box in the future.

[0107] Optionally, refer to Figure 2 Step S3 includes:

[0108] S307, determine whether there is a third undetermined bounding box in the YOLO fish target bounding box; the third undetermined bounding box appears in the GMM fish target bounding box; that is, for one fish target bounding box, if it is detected by both the YOLO neural network model and the GMM Gaussian mixture model, then it is used as the third undetermined bounding box to be determined.

[0109] S308, if there is a third undetermined border, then obtain the first difference between the horizontal coordinate of the YOLO target position and the horizontal coordinate of the GMM target position of the third undetermined border, obtain the second difference between the vertical coordinate of the YOLO target position and the vertical coordinate of the GMM target position of the third undetermined border, obtain the third difference between the YOLO target width and the GMM target width of the third undetermined border, and obtain the fourth difference between the YOLO target height and the GMM target height of the third undetermined border.

[0110] S309, determine whether the first difference, the second difference, the third difference, and the fourth difference are less than a preset difference threshold; the preset difference threshold can be set according to actual needs and is not limited here;

[0111] S310, if at least one of the first difference, the second difference, the third difference, and the fourth difference is less than a preset difference threshold, then the third undetermined bounding box is determined as the real fish target bounding box and output; that is, for the determined fish target, it will be represented by the YOLO fish target bounding box in the future.

[0112] Optionally, refer to Figure 2 Step S309 includes:

[0113] S3091, if the first difference, the second difference, the third difference, and the fourth difference are all greater than or equal to the preset difference threshold, then determine whether the straight-line distance between the center point of the third undetermined border in the YOLO fish target border and the center point of the third undetermined border in the GMM fish target border is less than the preset distance threshold; the preset distance threshold can be set according to actual needs and is not limited here.

[0114] S3092, if the straight-line distance between the center point of the third undetermined bounding box in the YOLO fish target bounding box and the center point of the third undetermined bounding box in the GMM fish target bounding box is less than a preset distance threshold, then the third undetermined bounding box is determined to be the real fish target bounding box and output; that is, for the judged fish target, it will be represented by the YOLO fish target bounding box in the future.

[0115] Optionally, refer to Figure 2 Step S3091 includes the following:

[0116] S30911, if the straight-line distance between the center point of the third undetermined border in the YOLO fish target border and the center point of the third undetermined border in the GMM fish target border is greater than or equal to a preset distance threshold, then determine whether the ratio of the enclosed area of ​​the third undetermined border in the GMM fish target border to the enclosed area of ​​the third undetermined border in the YOLO fish target border is greater than or equal to a preset ratio threshold; the preset ratio threshold can be set to 0.95.

[0117] S30912, if the ratio of the enclosed area of ​​the third undetermined border in the GMM fish target border to the enclosed area of ​​the third undetermined border in the YOLO fish target border is greater than or equal to a preset ratio threshold, then the third undetermined border is determined to be the real fish target border and output; that is, for the judged fish target, it will be represented by the YOLO fish target border in the future.

[0118] Furthermore, referring to Figure 3 In one exemplary embodiment, step S3 is followed by:

[0119] S4. Obtain the number and location of detected fish targets from the real fish target bounding boxes; that is, after completing the fish target detection of the fish keyframe image set, count the total number of real fish target bounding boxes and record the corresponding fish target locations (which can be represented by the parameters related to fish target locations among the five parameters of the YOLO fish target bounding box or the GMM fish target bounding box mentioned above).

[0120] S5, the fish targets in the fish keyframe image set are counted by manual measurement to obtain the actual number and location of the fish targets; where manual measurement means manual counting without the help of automatic detection functions of models and algorithms.

[0121] S6 compares the number of detected fish targets with the actual number of fish targets, and compares the location of detected fish targets with the actual location of fish targets to obtain the model accuracy evaluation index of the combination of GMM Gaussian Mixture Model and YOLO Neural Network Model. That is, the actual number and location of fish targets measured by manual measurement are used as the actual number and location of fish targets in the aquaculture area (such as aquaculture cages). The deviation of the detected number and location of fish targets is evaluated based on this standard, so as to obtain relevant parameters that can characterize the model accuracy after the combination of GMM Gaussian Mixture Model and YOLO Neural Network Model, which can provide a reference for subsequent model adjustment and improvement.

[0122] Optionally, refer to Figure 3 Step S6 includes:

[0123] S601, Statistically detect the actual number of fish targets among the fish target counts;

[0124] The model accuracy is calculated using the following formula:

[0125] P = T R / T

[0126] Where P is the model accuracy, T R T represents the actual number of fish targets, and T represents the number of detected fish targets.

[0127] S602, calculate the model recall using the following formula:

[0128] R = T R / N

[0129] Where R is the model recall rate, and T is the model recall rate. R N represents the actual number of fish targets, and N represents the measured number of fish targets.

[0130] S603, calculate the false detection rate according to the following formula:

[0131] Pf =(TT) R ) / T

[0132] Among them, P f T represents the false detection rate. R T represents the actual number of fish targets, and T represents the number of detected fish targets.

[0133] S604, calculate the false detection rate according to the following formula:

[0134] P m =(NT) R ) / T

[0135] Among them, P m T represents the false negative rate. R N represents the actual number of fish targets, T represents the number of fish targets measured, and T represents the number of fish targets detected.

[0136] S605, calculate the fish target position error for each frame in the fish keyframe image set according to the following formula:

[0137] E i =∑(|S T -S A | / S A )

[0138] Among them, E i S represents the target position error for the fish. T To detect the location of fish targets, S A To measure the target location of the fish;

[0139] S606, calculate the expected value of the fish target position error in the fish keyframe image set according to the following formula:

[0140] D = 1 / n∑(1 / T) R ∑(|S T -S A | / S A ))

[0141] Where D is the expected value of the target position error of the fish, n is the number of image frames in the fish keyframe image set, and T R S represents the actual target number of fish. T To detect the location of fish targets, S A To measure the target location of the fish.

[0142] Correspondingly, refer to Figure 4 This invention also provides a fish detection device, which includes:

[0143] The GMM detection module 10 is used to input the fish keyframe image set into the GMM Gaussian mixture model to perform moving target detection operations, so as to obtain the GMM fish target bounding box.

[0144] YOLO detection module 20 is used to input the fish keyframe image set into the YOLO neural network model to perform moving target detection operation in order to obtain the YOLO fish target bounding box;

[0145] The judgment module 30 is used to determine whether the GMM fish target bounding box and / or YOLO fish target bounding box are real fish target bounding boxes according to the preset judgment logic, and output the corresponding real fish target bounding box.

[0146] Correspondingly, the present invention also provides a computer-readable storage medium storing a fish detection program, which, when executed by a processor, implements the steps of the above-described fish detection method.

[0147] In this embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, any type of disk (including floppy disk, hard disk, optical disk, CD-ROM, and magneto-optical disk), ROM (Read-Only Memory), RAM (Random Access Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic cards, or optical cards, and other media capable of storing program code.

[0148] The aforementioned fish detection device, fish detection system, and computer-readable storage medium can all be configured to correspond to a fish detection method, the specific steps of which are described in the above embodiments. Since the aforementioned fish detection device, fish detection system, and computer-readable storage medium employ all the technical solutions corresponding to all the above embodiments, they possess at least all the beneficial effects brought about by the technical solutions of the above embodiments, which will not be elaborated upon here.

[0149] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0150] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0151] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0152] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for detecting fish, characterized in that, The fish detection method includes the following steps: The keyframe image set of fish is input into the Gaussian Mixture Model (GMM) for moving target detection to obtain the GMM fish target bounding box. The set of fish keyframe images is input into the YOLO neural network model for moving target detection to obtain the YOLO fish target bounding box. Based on a preset judgment logic, determine whether the GMM fish target bounding box and / or the YOLO fish target bounding box are real fish target bounding boxes, and output the corresponding real fish target bounding box; The YOLO fish target bounding box includes the YOLO target presence confidence score; The step of determining whether the GMM fish target bounding box and / or the YOLO fish target bounding box are real fish target bounding boxes according to a preset judgment logic, and outputting the corresponding real fish target bounding box, includes: Determine whether a second undetermined border exists in the YOLO fish target border; the second undetermined border does not appear in the GMM fish target border; If a second undetermined bounding box exists, then determine whether the confidence level of the YOLO target of the second undetermined bounding box reaches a preset confidence threshold. If the confidence level of the YOLO target of the second undetermined bounding box reaches the preset confidence threshold, then the second undetermined bounding box is determined to be a real fish target bounding box and output; The YOLO fish target bounding box includes the YOLO target position x-coordinate, YOLO target position y-coordinate, YOLO target width, and YOLO target height; the GMM fish target bounding box includes the GMM target position x-coordinate, GMM target position y-coordinate, GMM target width, and GMM target height. The step of determining whether the GMM fish target bounding box and / or the YOLO fish target bounding box are real fish target bounding boxes according to a preset judgment logic, and outputting the corresponding real fish target bounding box, includes: Determine whether a third undetermined border exists within the YOLO fish target border; the third undetermined border appears within the GMM fish target border; If the third undetermined border exists, then obtain the first difference between the horizontal coordinate of the YOLO target position and the horizontal coordinate of the GMM target position of the third undetermined border, obtain the second difference between the vertical coordinate of the YOLO target position and the vertical coordinate of the GMM target position of the third undetermined border, obtain the third difference between the YOLO target width and the GMM target width of the third undetermined border, and obtain the fourth difference between the YOLO target height and the GMM target height of the third undetermined border. Determine whether the first difference, the second difference, the third difference, and the fourth difference are less than a preset difference threshold; If at least one of the first difference, the second difference, the third difference, and the fourth difference is less than the preset difference threshold, then the third undetermined bounding box is determined to be the real fish target bounding box and output.

2. The fish detection method according to claim 1, characterized in that, The step of determining whether the GMM fish target bounding box and / or the YOLO fish target bounding box are real fish target bounding boxes according to a preset judgment logic, and outputting the corresponding real fish target bounding box, includes: Determine whether a first undetermined border exists in the GMM fish target border; the first undetermined border does not appear in the YOLO fish target border; If the first undetermined border exists, determine whether the duration of the existence of the first undetermined border reaches a preset time threshold. If the duration of the first undetermined border reaches the preset time threshold, then the first undetermined border is determined to be a real fish target border and output. And / or, before the step of inputting the fish keyframe image set into the Gaussian Mixture Model (GMM) for moving target detection to obtain the GMM fish target bounding box, the following steps are included: Perform GMM background modeling operation on the first frame background image of the fish keyframe image set; And / or, before the step of inputting the fish keyframe image set into the YOLO neural network model for moving target detection to obtain the YOLO fish target bounding box, the following steps are included: Fish target training samples are input into the YOLO neural network model for autonomous training.

3. The fish detection method according to claim 1, characterized in that, After the step of determining whether the first difference, the second difference, the third difference, and the fourth difference are less than a preset difference threshold, the method includes: If the first difference, the second difference, the third difference, and the fourth difference are all greater than or equal to the preset difference threshold, then it is determined whether the straight-line distance between the center point of the third undetermined border in the YOLO fish target border and the center point of the third undetermined border in the GMM fish target border is less than the preset distance threshold. If the straight-line distance between the center point of the third undetermined border in the YOLO fish target border and the center point of the third undetermined border in the GMM fish target border is less than the preset distance threshold, then the third undetermined border is determined to be the real fish target border and output.

4. The fish detection method according to claim 3, characterized in that, After the step of determining whether the straight-line distance between the center point of the third undetermined border in the YOLO fish target border and the center point of the third undetermined border in the GMM fish target border is less than a preset distance threshold, the following steps are included: If the straight-line distance between the center point of the third undetermined border in the YOLO fish target border and the center point of the third undetermined border in the GMM fish target border is greater than or equal to the preset distance threshold, then it is determined whether the ratio of the enclosed area of ​​the third undetermined border in the GMM fish target border to the enclosed area of ​​the third undetermined border in the YOLO fish target border is greater than or equal to the preset ratio threshold. If the ratio of the enclosed area of ​​the third undetermined border in the GMM fish target border to the enclosed area of ​​the third undetermined border in the YOLO fish target border is greater than or equal to the preset ratio threshold, then the third undetermined border is determined to be the real fish target border and output.

5. The fish detection method according to claim 1, characterized in that, After the step of outputting the corresponding real fish target bounding box, the following steps are included: The number and location of detected fish targets are obtained from the actual fish target bounding box. The fish targets in the fish keyframe image set are counted by manual measurement to obtain the actual number and location of the fish targets. The number of detected fish targets is compared with the number of actual fish targets, and the location of the detected fish targets is compared with the location of the actual fish targets to obtain the model accuracy evaluation index of the combination of the GMM Gaussian mixture model and the YOLO neural network model.

6. The fish detection method according to claim 5, characterized in that, The step of comparing the number of detected fish targets with the number of actually measured fish targets, and comparing the positions of the detected fish targets with the positions of the actually measured fish targets, to obtain the model accuracy evaluation index of the combination of the Gaussian Mixture Model (GMM) and the YOLO Neural Network Model, includes: Count the actual number of fish targets among the detected fish targets; The model accuracy is calculated using the following formula: in, The accuracy of the model is... The actual target number of fish. The number of fish targets to be detected; And / or, calculate the model recall using the following formula: in, The recall rate of the model is... The actual target number of fish. The actual number of fish targets; And / or, calculate the false positive rate according to the following formula: in, The false detection rate is... The actual target number of fish. The number of fish targets to be detected; And / or, calculate the false negative rate according to the following formula: in, The false negative rate is... The actual target number of fish. The measured target number of fish, The number of fish targets to be detected; And / or, calculate the fish target position error for each frame in the fish keyframe image set according to the following formula: in, The target position error of the fish is given. The location of the detected fish target. The measured target location of the fish; And / or, calculate the expected value of the fish target position error in the fish keyframe image set according to the following formula: in, Let the expected value of the target position error of the fish be denoted as . The number of image frames in the fish keyframe image set. The actual target number of fish. The location of the detected fish target. The measured target location of the fish is shown.

7. A fish detection device for implementing the fish detection method as described in any one of claims 1 to 6, characterized in that, The fish detection device includes: The GMM detection module is used to input the fish keyframe image set into the GMM Gaussian mixture model to perform moving target detection operations, so as to obtain the GMM fish target bounding box. The YOLO detection module is used to input the fish keyframe image set into the YOLO neural network model to perform moving target detection operations, so as to obtain the YOLO fish target bounding box. The judgment module is used to determine whether the GMM fish target border and / or the YOLO fish target border are real fish target borders according to a preset judgment logic, and output the corresponding real fish target border.

8. A fish detection system, characterized in that, The fish detection system includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When executed by the processor, the computer program implements the steps of the fish detection method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a fish detection program, which, when executed by a processor, implements the steps of the fish detection method as described in any one of claims 1 to 6.

Images