A method and system for monitoring the state of fish in deep - sea cage aquaculture

Abstract

The present invention discloses a method and system for monitoring the state of fish in deep - water cage aquaculture. The method includes using an underwater drone to collect fish swimming videos of a target deep - water aquaculture area to obtain each fish swimming image frame; using an AI algorithm to perform reflected - light correlation enhancement processing on the fish swimming image frames to obtain enhanced deep - water fish swimming images, obtaining each fish target, dividing each fish group within each enhanced deep - water fish swimming image, and calculating the ring - free group index of each fish group; analyzing the morphological similarity characteristics of all fish groups to determine each similar fish group of each fish group; and generating a fish state monitoring result of the target deep - water aquaculture area based on the difference in swimming characteristics and the ring - free group index between each fish group within each enhanced deep - water fish swimming image and the determined similar fish groups. The method and system for monitoring the state of fish in deep - water cage aquaculture provided by the embodiments of the present invention improve the accuracy of fish state monitoring.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular, to a method and system for monitoring the state of fish in deep - sea cage aquaculture. Background Art

[0002] Deep - sea aquaculture refers to aquaculture activities carried out in waters far from the coast and with relatively deep water depths. Monitoring the swimming state of fish in deep - sea aquaculture areas not only helps to deeply understand fish behavior patterns, optimize aquaculture management, improve production efficiency and economic benefits, but also can timely warn of diseases, evaluate environmental adaptability, and support ecological impact analysis and sustainable resource management, thus promoting the healthy development of the aquaculture industry and environmental protection.

[0003] With the development of artificial intelligence technology, existing technologies usually first record the swimming state of fish in deep - sea aquaculture areas through underwater drones, and then use machine vision to analyze the swimming characteristics of fish, so as to realize the monitoring of the state of fish in deep - sea aquaculture areas. However, in the photos collected in the deep - water shooting environment, there are bubbles, light spots, and various types of fish swimming interspersed, resulting in difficult accurate fine - grained identification and analysis of each fish in the image, and further leading to inaccurate monitoring of the state of fish.

[0004] It can be seen that how to improve the accuracy of monitoring the state of fish in deep - sea aquaculture areas has become a technical problem that needs to be urgently solved by those skilled in the art. Summary of the Invention

[0005] The present invention provides a method and system for monitoring the state of fish in deep - sea cage aquaculture to solve the technical problem that the monitoring of the state of fish is affected by various factors, resulting in low accuracy of monitoring the state of fish in deep - sea aquaculture areas, and to improve the accuracy of monitoring the state of fish in deep - sea aquaculture areas.

[0006] To solve the above - mentioned technical problem, an embodiment of the present invention provides a method for monitoring the state of fish in deep - sea cage aquaculture.

[0007] Use an underwater drone to collect the fish swimming video of the target deep - sea aquaculture area, and process the fish swimming video to obtain each fish swimming image frame;

[0008] Perform reflected - light correlation enhancement processing on each fish swimming image frame to obtain each deep - sea fish swimming enhanced image; the reflected - light correlation enhancement processing is designed to map the value range result of the reflected - light intensity smoothing factor of each pixel point in each deep - sea fish swimming enhanced image as the standard deviation of the Gaussian filter of the retina - cerebral cortex theory algorithm to achieve image enhancement processing, where the reflected - light intensity smoothing factor reflects the gray - level distribution characteristics within the neighborhood range of each pixel point;

[0009] Perform object detection on each enhanced deep - sea fish swimming image to obtain each fish target within each enhanced deep - sea fish swimming image;

[0010] Determine each fish school within each enhanced deep - sea fish swimming image according to the collective swimming characteristics of the fish targets in adjacent enhanced deep - sea fish swimming images; Based on the dispersion characteristics of the position distributions of all the fish targets in each fish school extracted, determine the ring - leaving outlier index of each fish school;

[0011] Obtain each homogeneous fish school of each fish school based on the morphological similarity characteristics of all the fish schools within each enhanced deep - sea fish swimming image; Generate the fish status monitoring result of the target deep - sea aquaculture area based on the difference in swimming characteristics between each fish school and the determined homogeneous fish school within each enhanced deep - sea fish swimming image, and the ring - leaving outlier index.

[0012] As one of the preferred solutions, the specific method for obtaining the reflected light intensity smoothing factor is as follows:

[0013] Construct a neighborhood window for each pixel point, and take the pixel point with the largest pixel value within the neighborhood window as the high - light intensity point;

[0014] Take several circular boundaries centered on the high - light intensity point as the neighborhood halo boundaries for each pixel point, and take the absolute value of the difference between the pixel value of each pixel point on each neighborhood halo boundary and the pixel value of the high - light intensity point as the centrifugal light intensity attenuation value of each pixel point on each neighborhood halo boundary;

[0015] Take the range of the centrifugal light intensity attenuation values of all pixel points on each neighborhood halo boundary as the centrifugal attenuation discrete index of each neighborhood halo boundary, and take the average value of the centrifugal light intensity attenuation values of all pixel points on each neighborhood halo boundary as the centrifugal attenuation average index of each neighborhood halo boundary;

[0016] Calculate the reflected light intensity smoothing factor for each pixel point based on the linear relationship between the centrifugal attenuation discrete index and the corresponding centrifugal attenuation average index of all the neighborhood halo boundaries of each pixel point.

[0017] As one of the preferred solutions, the step of determining each fish school within each enhanced deep - sea fish swimming image according to the collective swimming characteristics of the fish targets in adjacent enhanced deep - sea fish swimming images includes:

[0018] Using an AI algorithm to perform matching analysis on each fish target in the adjacent enhanced deep - sea fish circular - swimming images, to obtain the circular - swimming position time series of each fish target;

[0019] Taking the Euclidean distance between the circular - swimming position time series of two fish targets in each enhanced deep - sea fish circular - swimming image as the corresponding circular - swimming curve dissimilarity index; taking the calculation result of the exponential function with the natural constant as the base and the opposite number of the circular - swimming curve dissimilarity index as the exponent as the corresponding group - swimming similarity;

[0020] Taking the group - swimming similarity as the similarity evaluation criterion of the clustering algorithm, and using the clustering algorithm to perform clustering analysis on all fish targets in each enhanced deep - sea fish circular - swimming image, to obtain each fish school in each enhanced deep - sea fish circular - swimming image.

[0021] As one of the preferred solutions, the step of using an AI algorithm to perform matching analysis on each fish target in the adjacent enhanced deep - sea fish circular - swimming images, to obtain the circular - swimming position time series of each fish target, includes:

[0022] Using a corner - detection algorithm to extract all corners of each enhanced deep - sea fish circular - swimming image; constructing the histogram of oriented gradients and the color histogram of each fish target;

[0023] Taking each fish target in the current enhanced deep - sea fish circular - swimming image as the fish target to be measured; taking all fish targets in the remaining enhanced deep - sea fish circular - swimming images as the fish targets to be matched in turn;

[0024] Constructing the first histogram of oriented gradients, the first color histogram of the fish target to be measured, the second histogram of oriented gradients, and the second color histogram of the fish target to be matched;

[0025] Based on the difference features between the first histogram of oriented gradients and the second histogram of oriented gradients, and the difference features between the first color histogram and the second color histogram, calculating the chromaticity matching factor between the fish target to be measured and each fish target to be matched;

[0026] Performing corner matching on the fish target to be measured and all fish targets to be matched, to obtain the number of corner matches between the fish target to be measured and each fish target to be matched; calculating the morphological matching factor between the fish target to be measured and each fish target to be matched based on the number of corner matches and the chromaticity matching factor;

[0027] Sort and analyze the morphological matching factors of the target fish to be measured and all the target fish to be measured in each enhanced deep - water fish tour image to obtain the matched fish targets of the target fish to be measured in each enhanced deep - water fish tour image;

[0028] Arrange the positions of all the matched fish targets of each fish target in the corresponding enhanced deep - water fish tour image in ascending order according to time to obtain the tour position time series of each fish target.

[0029] As one of the preferred solutions, determining the ring - free group index of each fish school based on the dispersion characteristics of the position distribution of all the fish targets of each fish school extracted includes:

[0030] Take the center point of the minimum circumscribed rectangle of each fish school in each enhanced deep - water fish tour image as the fish school center of each fish school;

[0031] Take the average value of the Euclidean distances between all the fish targets of each fish school and the corresponding fish school center as the ring - free group index of each fish school.

[0032] As one of the preferred solutions, generating the fish state monitoring result of the target deep - water aquaculture area based on the difference in swimming characteristics between each fish school in each enhanced deep - water fish tour image and the determined same - type fish schools, and the ring - free group index includes:

[0033] Obtain the swimming speed time series of the corresponding fish target based on the change characteristics of the position data in each tour position time series;

[0034] Perform differential processing on the swimming speed time series of each fish target to obtain the swimming speed progressive series of each fish target, and take the range of all the element values in the swimming speed progressive series of all the fish targets in each fish school as the swimming speed fluctuation factor of the corresponding fish school;

[0035] Take the average value of all the element values of the swimming speed time series of all the fish targets in each fish school as the swimming speed mean value of the corresponding fish school;

[0036] Stitch the swimming speed fluctuation factor and the corresponding swimming speed mean value of each fish school to obtain the swimming speed distribution feature vector of each fish school;

[0037] Based on the difference characteristics of the swimming speed distribution feature vectors of each fish group in each deep - sea fish circular - tour enhanced image and the corresponding swimming speed distribution feature vectors of all the same - type fish groups, and the circular - tour outlier index, calculate the state anomaly index of each fish group to evaluate the degree of abnormal fish state in the target deep - sea aquaculture area.

[0038] As one of the preferred solutions, the shape matching factor is used to evaluate the morphological similarity characteristics between the fish targets;

[0039] Obtaining each same - type fish group of each fish group based on the morphological similarity characteristics of all the fish groups in each deep - sea fish circular - tour enhanced image extracted includes:

[0040] Take each fish group in the current deep - sea fish circular - tour enhanced image as the fish group to be measured, and take the remaining fish groups in the current deep - sea fish circular - tour enhanced image as the fish groups to be matched in turn;

[0041] Take the maximum value of the shape matching factors between each fish target in the fish group to be measured and all the fish targets in the fish group to be matched as the cluster integration factor of each fish target in the fish group to be measured and the fish group to be matched;

[0042] Take the average value of the cluster integration factors of all the fish targets in the fish group to be measured and the fish group to be matched as the similar - ethnic - group similarity index between the fish group to be matched and the fish group to be matched;

[0043] Perform normalization processing on all the similar - ethnic - group similarity indexes, and compare and analyze the results of the normalization processing with a preset similarity threshold to obtain each same - type fish group of each fish group.

[0044] As one of the preferred solutions, calculating the shape matching factor between the fish target to be measured and each fish target to be matched based on the number of corner matches and the chromaticity matching factor includes:

[0045] Take the maximum value of the number of corner matches between the fish target to be measured and all the fish targets to be matched as the reference number of corner matches of the fish target to be measured;

[0046] Divide the number of corner matches between the fish target to be measured and each fish target to be matched by the reference number of corner matches of the fish target to be measured as the corner - matching score between the fish target to be measured and each fish target to be matched;

[0047] Based on the linear relationship between the corner matching scores and the corresponding chromaticity matching factors, the morphological matching factors of the fish target to be measured and each of the fish targets to be matched are calculated.

[0048] As one preferred solution, calculating the state anomaly index of each fish school based on the difference features of the swimming speed distribution feature vectors of each fish school in each enhanced deep-sea fish swimming-around image and the swimming speed distribution feature vectors of all the same-kind fish schools corresponding thereto, and the outlier index of swimming-around includes:

[0049] Inputting the swimming speed distribution feature vectors, the outlier index of swimming-around of each fish school in each enhanced deep-sea fish swimming-around image and the swimming speed distribution feature vectors of all the same-kind fish schools corresponding thereto into the abnormal state evaluation expression, and calculating the state anomaly index of each fish school in each enhanced deep-sea fish swimming-around image;

[0050] The abnormal state evaluation expression is:

[0051]

[0052] Wherein, is the state anomaly index of the fish school m in the i-th enhanced deep-sea fish swimming-around image, is the normalization function, is the outlier index of swimming-around of the fish school m in the i-th enhanced deep-sea fish swimming-around image, is the exponential function with the natural constant as the base, is the number of the same-kind fish schools of the fish school m in the i-th enhanced deep-sea fish swimming-around image, is the cosine similarity between two vectors, is the swimming speed distribution feature vector of the fish school m in the i-th enhanced deep-sea fish swimming-around image, is the swimming speed distribution feature vector of the s-th same-kind fish school of the fish school m in the i-th enhanced deep-sea fish swimming-around image.

[0053] Another embodiment of the present invention provides a fish state monitoring system for deep-sea cage culture, the system includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, the above-mentioned fish state monitoring method for deep-sea cage culture is implemented.

[0054] Compared with the prior art, the beneficial effects of the embodiments of the present invention are at least one of the following:

[0055] An underwater drone is used to collect fish swimming videos in the target deep-water aquaculture area, and the fish swimming videos are processed to obtain individual fish swimming image frames. Since deep-water images often have color distortion due to the absorption and scattering of light of different wavelengths by water bodies, in order to avoid the influence of bubbles and light spots on the recognition of fish status, each fish swimming image frame is subjected to reflected light correlation enhancement processing to obtain each deep-water fish swimming enhanced image. The reflected light correlation enhancement processing is designed to map the value range of the reflected light intensity smoothing factor of each pixel point in each deep-water fish swimming enhanced image to the standard deviation of the Gaussian filter of the retinal cerebral cortex theoretical algorithm to achieve image enhancement processing, wherein the reflected light intensity smoothing factor reflects the grayscale distribution characteristics within the neighborhood range of each pixel point, and the different light intensity characteristics around each pixel point in the fish swimming image frame are targeted for image enhancement. While retaining the detailed information in the fish swimming image frame, the reflected light is smoothed as much as possible, which helps to eliminate the problem of uneven illumination and improves the accuracy of subsequent fish status monitoring.

[0056] Considering that most deep-water farmed fishes swim in clusters, the fish state may be abnormal when the fish deviate from the group swimming. Target detection is performed on each deep-water fish circumferential enhanced image to obtain each fish target in each deep-water fish circumferential enhanced image. The fish schools in each deep-water fish circumferential enhanced image are determined based on the cluster swimming characteristics of the fish targets extracted in the adjacent deep-water fish circumferential enhanced images. The circumferential outlier index of each fish school is determined based on the dispersed characteristics of the position distribution of all fish targets extracted from each fish school. The fish schools are first obtained through the cluster swimming characteristics of the fish targets in the adjacent deep-water fish circumferential enhanced images, rather than dividing the fish schools through a single deep-water fish circumferential enhanced image, to avoid misdivision. Then, the dispersed characteristics of the position distribution of all fish targets in each fish school in each deep-water fish circumferential enhanced image are analyzed. The fish schools with higher overall fish school swimming dispersion are more likely to be abnormal, thereby improving the reliability of fish state monitoring.

[0057] Since there may be many kinds of fish raised in deep-water aquaculture areas, the morphology, living habits and swimming characteristics of different fish are different. However, various fish swim interspersedly. In order to avoid the interference of the swimming of various types of fish schools, which leads to misjudgment of the fish status, the morphological similarity characteristics of all fish schools in each deep-water fish circumnavigation enhanced image are first analyzed to obtain each similar fish school of each fish school. Then, the difference in swimming characteristics between each fish school and the corresponding similar fish school in each deep-water fish circumnavigation enhanced image is analyzed. The state abnormality index of each fish school is calculated in combination with the circumnavigation outlier index, which further improves the accuracy of fish status monitoring.

[0058] By performing reflected light correlation enhancement processing on each fish round-trip image frame, each pixel point in each fish round-trip image frame is enhanced separately according to different light intensities, so as to smooth the reflected light as much as possible while retaining the detailed information in the fish round-trip image frame, and further analyze the differences in the swimming characteristics of each fish group and the same-kind fish groups and the clustering characteristics of each fish group, obtain the state anomaly index of each fish group, and comprehensively improve the accuracy of fish state monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 FIG. is a schematic flowchart of a method for monitoring the state of fish in a deep-water cage culture according to one embodiment of the present invention;

[0060] Figure 2 FIG. is a schematic diagram of the neighborhood halo boundary according to one embodiment of the present invention;

[0061] Figure 3 FIG. is a schematic diagram for obtaining the state anomaly index according to one embodiment of the present invention;

[0062] Figure 4 FIG. is an architecture diagram of a system for monitoring the state of fish in a deep-water cage culture according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0063] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. The purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0064] In the description of the present application, the terms "first", "second", "third", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first", "second", "third", etc. may explicitly or implicitly include one or more of such features. In the description of the present application, unless otherwise specified, the meaning of "a plurality" is two or more.

[0065] In the description of this application, it should be noted that unless otherwise clearly defined and limited, the terms "installation", "connection", and "coupling" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, and it can be the communication inside two components. The terms "vertical", "horizontal", "left", "right", "up", "down" and similar expressions used herein are only for the purpose of illustration, rather than indicating or implying that the device or component referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus cannot be construed as a limitation on the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0066] In the description of this application, it should be noted that unless otherwise defined, all technical and scientific terms used in this invention have the same meaning as those commonly understood by those skilled in the technical field to which this technology belongs. The terms used in the specification of this invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0067] Since the photos collected in the deep - water shooting environment are filled with bubbles, light spots, and various kinds of fish swimming through, the impacts of bubbles and light spots on fish state recognition are all caused by the scattering, reflection, etc. of light.

[0068] In view of this, the embodiments of the present invention adopt the following related technical means to enhance the images of the collected deep - water aquaculture areas, making the details and features in the images easy to identify, ensuring the accuracy and reliability of the images, and providing strong support for subsequent scientific research, exploration, aquaculture and other activities.

[0069] An embodiment of the present invention provides a method for monitoring the state of fish in deep - water cage aquaculture. Specifically, please refer to Figure 1 , Figure 1 which shows a schematic flow chart of a method for monitoring the state of fish in deep - water cage aquaculture in one of the embodiments of the present invention.

[0070] Step S1: Use an underwater drone to collect the fish - swimming - around video of the target deep - water aquaculture area, and process the fish - swimming - around video to obtain each fish - swimming - around image frame.

[0071] Specifically, set up an underwater drone at each monitoring point in the target deep - water aquaculture area, and use the underwater drone to shoot the target deep - water aquaculture area to obtain the fish - swimming - around video of the target deep - water aquaculture area.

[0072] It should be noted that a video is composed of consecutive frames, and each frame is a static image. By decomposing the video into frames, a series of clear static images can be obtained, which can be used to more precisely analyze the state of fish, such as behavior patterns, appearance characteristics, etc.

[0073] Specifically, the fish swimming videos are processed to obtain individual fish swimming image frames.

[0074] Step S2: Perform reflected light correlation enhancement processing on each fish swimming image frame to obtain each enhanced deep - water fish swimming image; perform object detection on each enhanced deep - water fish swimming image to obtain each fish target within each enhanced deep - water fish swimming image.

[0075] It should be noted that there are a large number of fish in the deep - water aquaculture area. The respiration of fish will produce bubbles, and the images collected in the deep - water area often suffer from color distortion due to the absorption and scattering of light of different wavelengths by the water body. In order to avoid the influence of bubbles generated by fish respiration and the scattering of light by the water body on the monitoring of fish state, it is necessary to first enhance the collected original images. The following table shows the classification of each fish target in the deep - water aquaculture area:

[0076]

[0077] Specifically, perform reflected light correlation enhancement processing on each fish swimming image frame to obtain each enhanced deep - water fish swimming image.

[0078] The reflected light correlation enhancement processing is designed to map the value range of the reflected light intensity smoothing factor of each pixel point in each enhanced deep - water fish swimming image as the standard deviation of the Gaussian filter of the retina - cortex theory algorithm to achieve image enhancement processing, where the reflected light intensity smoothing factor reflects the gray - level distribution characteristics within the neighborhood range of each pixel point.

[0079] It should be noted that the core idea of the retina - cortex theory algorithm is that the color of an object is determined by the object's reflection ability of long - wave, medium - wave, and short - wave light, rather than by the absolute value of the reflected light intensity. At the same time, the color of an object is not affected by the non - uniformity of illumination and has consistency, that is, color constancy. The value range of the reflected light intensity smoothing factor is mapped to the value range of the standard deviation of the Gaussian filter of the retina - cortex theory algorithm through value range mapping.

[0080] In this step, the specific calculation method of the reflected light intensity smoothing factor is as follows:

[0081] Construct the neighborhood window for each pixel point, and take the pixel point with the largest pixel value within the neighborhood window as the highlight intensity point; take several circular boundaries centered on the highlight intensity point as the neighborhood halo boundaries for each pixel point, and take the absolute value of the difference between the pixel value of each pixel point on each neighborhood halo boundary and the pixel value of the highlight intensity point as the centrifugal light intensity attenuation value for each pixel point on each neighborhood halo boundary.

[0082] Take the range of the centrifugal light intensity attenuation values of all pixel points on each neighborhood halo boundary as the centrifugal attenuation discrete index for each neighborhood halo boundary, and take the average value of the centrifugal light intensity attenuation values of all pixel points on each neighborhood halo boundary as the centrifugal attenuation average index for each neighborhood halo boundary.

[0083] Based on the linear relationship between the centrifugal attenuation discrete index and the corresponding centrifugal attenuation average index of all neighborhood halo boundaries of each pixel point, calculate the reflected light intensity smoothing factor for each pixel point.

[0084] It should be noted that when the centrifugal attenuation discrete index of the neighborhood halo boundary of a pixel point is larger and the corresponding centrifugal attenuation average index is larger, it indicates that there is more likely a light spot or bubble with gradually decreasing pixel value around the central point within the neighborhood range of this pixel point, and higher-degree smoothing should be performed on it to avoid affecting the monitoring of the fish state.

[0085] As an embodiment of the present application, take the product of the centrifugal attenuation discrete index and the corresponding centrifugal attenuation average index of each neighborhood halo boundary as the corresponding light intensity scattering characteristic factor, and take the average value of the light intensity scattering characteristic factors of all neighborhood halo boundaries of each pixel point as the corresponding reflected light intensity smoothing factor.

[0086] Specifically, please refer to Figure 2 , Figure 2 which shows a schematic diagram of the neighborhood halo boundary in one embodiment of the present invention.

[0087] Furthermore, perform object detection on each enhanced image of deep-sea fish swimming around to obtain each fish target within each enhanced image of deep-sea fish swimming around.

[0088] It should be noted that object detection is a core task in the field of computer vision, aiming to find all interesting targets in an image or video. Object detection is a well-known technology, and it will not be elaborated in this embodiment.

[0089] As an embodiment of the present application, use the Yolo model to perform object detection on each enhanced image of deep-sea fish swimming around.

[0090] Step S3: Determine each fish school in each deep - water fish round - trip enhanced image according to the collective swimming characteristics of fish targets in the adjacent deep - water fish round - trip enhanced images; Use the AI algorithm to determine the round - trip outlier index of each fish school based on the dispersion characteristics of the position distribution of all fish targets in each extracted fish school.

[0091] It should be noted that considering that most deep - water cultured fish swim in a collective manner, when fish deviate from collective swimming, their states may be abnormal. By analyzing the collective characteristics of fish swimming, the abnormal states of fish can be preliminarily monitored.

[0092] Specifically, perform matching analysis on each fish target in adjacent deep - water fish round - trip enhanced images to obtain the round - trip position time series of each fish target; Take the Euclidean distance between the round - trip position time series of two fish targets in each deep - water fish round - trip enhanced image as the corresponding round - trip curve dissimilarity index; Take the calculation result of the exponential function with the natural constant as the base and the opposite number of the round - trip curve dissimilarity index as the exponent as the corresponding collective swimming similarity; Take the collective swimming similarity as the similarity evaluation criterion of the clustering algorithm, and perform clustering analysis on all fish targets in each deep - water fish round - trip enhanced image with the clustering algorithm to obtain each fish school in each deep - water fish round - trip enhanced image.

[0093] It should be noted that the calculations of the clustering algorithm and the Euclidean distance are both well - known technologies, and are not elaborated in this embodiment. The round - trip position time series represents the swimming curve of the fish target. When the swimming curves of fish targets are more different, they are more likely to belong to different fish schools, and the collective swimming similarity is lower.

[0094] As an embodiment of the present application, use the K - means clustering algorithm to perform clustering analysis on all fish targets in each deep - water fish round - trip enhanced image. As other implementation manners, the implementer can select by himself / herself, and the present application does not make special restrictions on this.

[0095] Furthermore, take the center point of the minimum circumscribed rectangle of each fish school in each deep - water fish round - trip enhanced image as the fish school center of each fish school; Take the average value of the Euclidean distances between all fish targets in each fish school and the corresponding fish school center as the round - trip outlier index of each fish school.

[0096] It should be noted that the fish school is obtained by enhancing the collective swimming characteristics of the fish school swimming in the image through multiple deep-sea fish roundabouts, rather than dividing the fish school through a single deep-sea fish roundabout enhanced image, so as to avoid misclassification. Then, the dispersion characteristics of the position distribution of all fish targets in each fish school in each deep-sea fish roundabout enhanced image are analyzed. The fish school with a higher overall swimming dispersion degree of the fish school is more likely to be abnormal, improving the reliability of fish state monitoring.

[0097] In this step, each fish target in adjacent deep-sea fish roundabout enhanced images is subjected to matching analysis to obtain the roundabout position time series of each fish target, including:

[0098] The corner detection algorithm is used to extract all corners of each deep-sea fish roundabout enhanced image; the orientation gradient histogram and color histogram of each fish target are constructed;

[0099] Each fish target in the current deep-sea fish roundabout enhanced image is used as the fish target to be measured; all fish targets in the remaining deep-sea fish roundabout enhanced images are sequentially used as the fish targets to be matched; the first orientation gradient histogram, the first color histogram of the fish target to be measured, the second orientation gradient histogram, and the second color histogram of the fish target to be matched are constructed; based on the difference characteristics between the first orientation gradient histogram and the second orientation gradient histogram, and the difference characteristics between the first color histogram and the second color histogram, the chromaticity matching factor between the fish target to be measured and each fish target to be matched is calculated;

[0100] The fish target to be measured and all fish targets to be matched are subjected to corner matching to obtain the number of corner matches between the fish target to be measured and each fish target to be matched; based on the number of corner matches and the chromaticity matching factor, the morphological matching factor between the fish target to be measured and each fish target to be matched is calculated;

[0101] The morphological matching factors of the fish target to be measured and all fish targets to be measured in each deep-sea fish roundabout enhanced image are sorted and analyzed to obtain the matched fish targets of the fish target to be measured in each deep-sea fish roundabout enhanced image; the positions of all matched fish targets of each fish target in the corresponding deep-sea fish roundabout enhanced image are arranged in ascending order according to time to obtain the roundabout position time series of each fish target.

[0102] It should be noted that the corner detection algorithm is an important technology in the fields of computer vision and image processing, which is used to find the pixels in an image that may represent the intersection points or corners of object edges. As important feature points in the image, corners can be used for object recognition and tracking; the histogram of oriented gradients captures the shape features of the image by calculating the gradient direction histogram of the local area of the image, and the color histogram reflects the proportion of different colors in the whole image. The construction of the corner detection algorithm, the histogram of oriented gradients, and the color histogram are all well-known technologies, and will not be elaborated in this embodiment.

[0103] Furthermore, it should be noted that by corner matching and comparing the histograms of oriented gradients and color histograms between fish targets, the similarities of the contour features, shape features, and color features between fish targets are respectively analyzed, improving the accuracy of matching.

[0104] In this step, the morphological matching factor between the fish target to be measured and each fish target to be matched is calculated based on the number of corner matches and the chromaticity matching factor, including: taking the maximum value of the number of corner matches between the fish target to be measured and all fish targets to be matched as the reference number of corner matches for the fish target to be measured; dividing the number of corner matches between the fish target to be measured and each fish target to be matched by the reference number of corner matches for the fish target to be measured as the corner matching score between the fish target to be measured and each fish target to be matched; and calculating the morphological matching factor between the fish target to be measured and each fish target to be matched based on the linear relationship between the corner matching score and the corresponding chromaticity matching factor.

[0105] It should be noted that when the corner matching score and the chromaticity matching factor between fish targets are larger, it indicates that the morphological similarity features between the fish targets are more obvious and they are more likely to be the same fish target.

[0106] Step S4: Obtain each homogeneous fish group of each fish group based on the morphological similarity features of all fish groups in each enhanced image of deep-sea fish swimming; generate the fish status monitoring result of the target deep-sea aquaculture area based on the difference in swimming characteristics and the ring free-group index between each fish group in each enhanced image of deep-sea fish swimming and the determined homogeneous fish group.

[0107] Specifically, please refer to Figure 3 , Figure 3 which shows the schematic diagram for obtaining the status anomaly index in one embodiment of the present invention.

[0108] It should be noted that the morphological matching factor is used to evaluate the morphological similarity features between fish targets.

[0109] Specifically, each fish school in the current enhanced deep - sea fish round - trip image is regarded as a fish school to be measured, and the remaining fish schools in the current enhanced deep - sea fish round - trip image are successively regarded as fish schools to be matched; the maximum value of the morphological matching factors between each fish target in the fish school to be measured and all fish targets in the fish school to be matched is used as the cluster integration factor of each fish target in the fish school to be measured and the fish school to be matched.

[0110] The average value of the cluster integration factors of all fish targets in the fish school to be measured and the fish school to be matched is used as the similar - ethnic - group similarity index between the fish school to be matched and the fish school to be matched; all the similar - ethnic - group similarity indexes are normalized, and the normalized results are compared and analyzed with the preset similarity threshold to obtain each fish school's respective similar fish schools.

[0111] Furthermore, based on the change characteristics of the position data in each round - trip position time series, the swimming speed time series of the corresponding fish targets is obtained, and the swimming speed time series of each fish target is differentiated to obtain the swimming - speed progressive series of each fish target.

[0112] The range of all element values in the swimming - speed progressive series of all fish targets in each fish school is used as the swimming - speed fluctuation factor of the corresponding fish school; the average value of all element values in the swimming speed time series of all fish targets in each fish school is used as the swimming - speed mean value of the corresponding fish school; the swimming - speed fluctuation factor and the corresponding swimming - speed mean value of each fish school are spliced to obtain the swimming - speed distribution feature vector of each fish school;

[0113] The swimming - speed distribution feature vectors, the round - trip outlier index, and the swimming - speed distribution feature vectors of all corresponding similar fish schools of each fish school in each enhanced deep - sea fish round - trip image are input into the abnormal - state evaluation expression, and the state abnormal index of each fish school in each enhanced deep - sea fish round - trip image is calculated.

[0114] The abnormal - state evaluation expression is:

[0115]

[0116] Among them, is the state abnormal index of the fish school m in the i - th enhanced deep - sea fish round - trip image, is the normalization function, is the round - trip outlier index of the fish school m in the i - th enhanced deep - sea fish round - trip image, is the exponential function with the natural constant as the base, is the number of similar fish schools of the fish school m in the i - th enhanced deep - sea fish round - trip image, is the cosine similarity between two vectors, is the swimming - speed distribution feature vector of the fish school m in the i - th enhanced deep - sea fish round - trip image, It is the swimming speed distribution feature vector of the s-th conspecific fish group of the fish group m in the i-th deep-sea fish round-trip enhanced image.

[0117] It should be noted that to evaluate the degree of abnormality of fish status in the target deep-sea aquaculture area, when the round-trip outlier index of the target fish group is larger, it indicates that the target fish group is swimming. The more obvious the dispersibility characteristics of the position distribution of the fish targets in the target fish group are, the more likely there are some fish targets with abnormal status in the fish group, resulting in a decrease in the clustering characteristics of the fish group; when the cosine similarity between the swimming speed distribution feature vector of the target fish group and the swimming speed distribution feature vector of the corresponding conspecific fish group is smaller, it indicates that the difference in swimming characteristics between the target fish group and the similar fish group is larger, and it is more likely that the entire fish group has an abnormality.

[0118] Compared with the prior art, the beneficial effects of the embodiments of the present invention are at least one of the following:

[0119] An underwater drone is used to collect the fish round-trip video of the target deep-sea aquaculture area, and the fish round-trip video is processed to obtain each fish round-trip image frame; since deep-sea images often suffer from color distortion due to the absorption and scattering of light of different wavelengths by water bodies, in order to avoid the influence of bubbles and light spots on the recognition of fish status, each fish round-trip image frame is subjected to reflected light correlation enhancement processing to obtain each deep-sea fish round-trip enhanced image. The reflected light correlation enhancement processing is designed to map the value range result of the reflected light intensity smoothing factor of each pixel point in each deep-sea fish round-trip enhanced image as the standard deviation of the Gaussian filter of the retina-cerebral cortex theory algorithm to achieve image enhancement processing. Among them, the reflected light intensity smoothing factor reflects the gray distribution characteristics within the neighborhood range of each pixel point, and performs targeted image enhancement on different pixel points according to different light intensity characteristics around each pixel point in the fish round-trip image frame. While retaining the detail information in the fish round-trip image frame, the reflected light is smoothed as much as possible, which helps to eliminate the problem of uneven illumination and improves the accuracy of subsequent fish status monitoring;

[0120] Considering that most deep - water cultured fish swim in a clustered manner, when fish deviate from the group swimming, their states may become abnormal. Target detection is performed on each enhanced image of deep - water fish circum - navigation to obtain each fish target within each enhanced image of deep - water fish circum - navigation; each fish school within each enhanced image of deep - water fish circum - navigation is determined according to the clustered swimming characteristics of the fish targets in adjacent enhanced images of deep - water fish circum - navigation; based on the dispersion characteristics of the position distributions of all fish targets in each fish school, the circum - navigation outlier index of each fish school is determined. First, fish schools are obtained through the clustered swimming characteristics of the fish targets in adjacent enhanced images of deep - water fish circum - navigation, rather than dividing fish schools through a single enhanced image of deep - water fish circum - navigation, avoiding misclassification. Then, the dispersion characteristics of the position distributions of all fish targets in each fish school within each enhanced image of deep - water fish circum - navigation are analyzed. The higher the overall dispersion degree of the fish school swimming, the more likely it is to be abnormal, improving the reliability of fish state monitoring;

[0121] Since there may be multiple types of fish raised in the deep - water aquaculture area, the morphologies, living habits, and swimming characteristics of different fish are different. However, various fish swim interspersed. To avoid the swimming of various fish schools interfering with each other and causing misjudgment of fish states, first, the morphological similarity characteristics of all fish schools within each enhanced image of deep - water fish circum - navigation are analyzed to obtain each homogeneous fish school within each fish school. Then, the difference in swimming characteristics between each fish school and its corresponding homogeneous fish school within each enhanced image of deep - water fish circum - navigation is analyzed, and the state anomaly index of each fish school is calculated in combination with the circum - navigation outlier index, further improving the accuracy of fish state monitoring;

[0122] By performing reflected - light correlation enhancement processing on each fish - circum - navigation image frame, each pixel point in each fish - circum - navigation image frame is enhanced separately according to different light intensities, so as to smooth the reflected light as much as possible while retaining the detailed information in the fish - circum - navigation image frame. Further, the difference in swimming characteristics between each fish school and its homogeneous fish school and the clustered characteristics of each fish school are analyzed to obtain the state anomaly index of each fish school, comprehensively improving the accuracy of fish state monitoring.

[0123] Another embodiment of the present invention provides a fish state monitoring system for deep - water cage aquaculture. The system includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements a fish state monitoring method for deep - water cage aquaculture as described above.

[0124] Specifically, please refer to Figure 4 , Figure 4 which shows the architecture diagram of a fish state monitoring system for deep - water cage aquaculture in one of the embodiments of the present invention.

[0125] The above-described embodiments merely represent several implementation manners of the present invention. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the scope of the patent for the present invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several variations and improvements can still be made, and these all fall within the protection scope of the present invention. Therefore, the protection scope of the patent for the present invention shall be subject to the appended claims.

Claims

1. A method for monitoring the status of fish cultured in deep water cages, characterized in that: include: An underwater drone is used to collect fish swimming videos in a target deep-water aquaculture area, and the fish swimming videos are processed to obtain image frames of each fish swimming; Performing reflected light correlation enhancement processing on each of the fish swimming image frames to obtain enhanced images of each deep-water fish swimming image; The reflected light association enhancement processing is designed to use the value range mapping result of the reflected light intensity smoothing factor of each pixel point in each of the fish swimming image frames as the standard deviation of the Gaussian filter of the retinal cerebral cortex theory algorithm to achieve image enhancement processing, wherein the reflected light intensity smoothing factor reflects the grayscale distribution characteristics within the neighborhood range of each pixel point; Performing target detection on each of the deep-water fish circumnavigation enhanced images to obtain each fish target in each of the deep-water fish circumnavigation enhanced images; Determine the individual schools of fish in each of the deep-water fish circumnavigation enhanced images according to the extracted clustered swimming characteristics of the fish targets in the adjacent deep-water fish circumnavigation enhanced images; determine the circumnavigation outlier index of each of the fish schools based on the extracted dispersion characteristics of the position distribution of all the fish targets in each of the fish schools; Based on the morphological similarity features of all the fish schools in each of the deep-water fish circumnavigation enhanced images extracted, the various similar fish schools of each of the fish schools are obtained; based on the differences in swimming characteristics between each of the fish schools in each of the deep-water fish circumnavigation enhanced images and the circumnavigation outlier index, the fish status monitoring results of the target deep-water aquaculture area are generated.

2. A method for monitoring the status of fish cultured in deep water cages according to claim 1, characterized in that: The specific method for obtaining the reflected light intensity smoothing factor is: Constructing a neighborhood window for each pixel point, and taking the pixel point with the largest pixel value in the neighborhood window as a high light intensity point; Using a plurality of circular boundaries with the high light intensity point as the center as the neighborhood light ring boundary of each pixel point, and using the absolute value of the difference between the pixel value of each pixel point on each neighborhood light ring boundary and the pixel value of the high light intensity point as the centrifugal light intensity attenuation value of each pixel point on each neighborhood light ring boundary; The range of the centrifugal light intensity attenuation values ​​of all the pixels on each of the neighborhood halo boundaries is taken as the centrifugal attenuation discrete index of each of the neighborhood halo boundaries, and the average value of the centrifugal light intensity attenuation values ​​of all the pixels on each of the neighborhood halo boundaries is taken as the centrifugal attenuation average index of each of the neighborhood halo boundaries; Based on the linear relationship between the centrifugal attenuation discrete indexes of all the neighborhood halo boundaries of each pixel and the corresponding centrifugal attenuation average indexes, the reflected light intensity smoothing factor of each pixel is calculated.

3. The method for monitoring the status of fish cultured in deep water cages according to claim 1, characterized in that: The method of determining each school of fish in each of the enhanced deep-water fish swimming images according to the extracted clustered swimming characteristics of the fish targets in the adjacent enhanced deep-water fish swimming images comprises: Using an AI algorithm to perform matching analysis on each of the fish targets in the adjacent enhanced deep-water fish swimming images, and obtaining a time series of the swimming positions of each of the fish targets; The Euclidean distance between the time series of the swimming positions of the two fish targets in each of the deep-water fish swimming enhanced images is used as the corresponding swimming curve dissimilarity index; the calculation result of an exponential function with a natural constant as the base and the opposite number of the swimming curve dissimilarity index as the exponent is used as the corresponding cluster swimming similarity; The cluster swimming similarity is used as a similarity evaluation standard of the clustering algorithm, and the clustering algorithm is used to perform cluster analysis on all the fish targets in each of the deep-water fish swimming enhanced images to obtain the various fish schools in each of the deep-water fish swimming enhanced images.

4. A method for monitoring the status of fish cultured in deep water cages as claimed in claim 3, characterized in that: The AI ​​algorithm is used to perform matching analysis on each of the fish targets in the adjacent enhanced deep-water fish swimming images to obtain a time series of the swimming positions of each of the fish targets, including: A corner detection algorithm is used to extract all corner points of each of the deep-water fish swimming enhanced images; a directional gradient histogram and a color histogram of each of the fish targets are constructed; Each of the fish targets in the current enhanced image of deep-water fish swimming around is used as a fish target to be detected; all of the fish targets in the remaining enhanced images of deep-water fish swimming around are used as fish targets to be matched in sequence; Constructing a first directional gradient histogram and a first color histogram of the fish target to be detected and a second directional gradient histogram and a second color histogram of the fish target to be matched; Based on the difference characteristics of the first oriented gradient histogram and the second oriented gradient histogram, and the difference characteristics of the first color histogram and the second color histogram, a chromaticity matching factor between the fish target to be detected and each of the fish targets to be matched is calculated; Performing corner point matching on the fish target to be tested and all the fish targets to be matched, obtaining the number of corner point matches between the fish target to be tested and each of the fish targets to be matched; calculating the morphological matching factor between the fish target to be tested and each of the fish targets to be matched based on the number of corner point matches and the chromaticity matching factor; Sorting and analyzing the morphological matching factors of the fish target to be detected and all the fish targets to be detected in each of the enhanced deep-water fish circumnavigation images, to obtain the matched fish targets of the fish target to be detected in each of the enhanced deep-water fish circumnavigation images; The positions of all the matched fish targets of each fish target in the corresponding enhanced deep-water fish swimming image are arranged in ascending order according to time sequence to obtain a time series of the swimming positions of each fish target.

5. The method for monitoring the status of fish cultured in deep water cages according to claim 1, characterized in that: The step of determining the circumstellar outlier index of each of the fish schools based on the extracted dispersion characteristics of the position distribution of all the fish targets of each of the fish schools comprises: Taking the center point of the minimum circumscribed rectangle of each of the fish schools in each of the deep-water fish swimming enhanced images as the school center of each of the fish schools; The average value of the Euclidean distances between all the fish targets of each fish school and the corresponding center of the fish school is taken as the circumstantial outlier index of each fish school.

6. A method for monitoring the status of fish cultured in deep water cages as claimed in claim 4, characterized in that: The method of generating the fish status monitoring result of the target deep-water aquaculture area based on the difference in swimming characteristics between each of the fish schools in each of the deep-water fish circumnavigation enhanced images and the determined similar fish schools and the circumnavigation outlier index comprises: Acquire the swimming speed time series of the corresponding fish target based on the change characteristics of the position data in each of the circumnavigation position time series; Performing differential processing on the swimming speed time series of each of the fish targets to obtain a progressive swimming speed sequence of each of the fish targets, and taking the extreme difference of all element values ​​in the progressive swimming speed sequence of all the fish targets in each of the fish schools as the swimming speed fluctuation factor of the corresponding fish school; Taking the average value of all element values ​​of the swimming speed time series of all the fish targets in each of the fish schools as the mean swimming speed of the corresponding fish school; splicing the swimming speed fluctuation factor of each of the fish schools and the corresponding swimming speed mean value to obtain a swimming speed distribution feature vector of each of the fish schools; Based on the difference characteristics of the swimming speed distribution feature vector of each fish school in each deep-water fish circumnavigation enhanced image and the corresponding swimming speed distribution feature vectors of all the same fish schools, and the circumnavigation outlier index, the state abnormality index of each fish school is calculated to evaluate the degree of abnormality of the fish state in the target deep-water aquaculture area.

7. A method for monitoring the status of fish cultured in deep water cages as claimed in claim 4, characterized in that: The morphological matching factor is used to evaluate the morphological similarity characteristics between the fish targets; The method of obtaining each similar school of fish of each school of fish based on the morphological similarity features of all the schools of fish in each of the deep-water fish swimming enhanced images extracted comprises: Each of the fish schools in the current enhanced image of deep-water fish swimming around is used as a fish school to be tested, and the remaining fish schools in the current enhanced image of deep-water fish swimming around are used as fish schools to be matched in sequence; The maximum value of the morphological matching factors between each of the fish targets in the fish school to be tested and all of the fish targets in the fish school to be matched is used as the cluster integration factor of each of the fish targets in the fish school to be tested and the fish school to be matched; The average value of the cluster integration factors of all the fish targets in the fish school to be tested and the fish school to be matched is used as the similarity index of the fish school to be matched and the same group of the fish school to be matched; The similarity indexes of all the similar groups are normalized, and the results of the normalization are compared and analyzed with a preset similarity threshold to obtain the similar groups of each of the fish schools.

8. A method for monitoring the status of fish cultured in deep water cages as claimed in claim 4, characterized in that: The step of calculating the morphological matching factor of the fish target to be detected and each of the fish targets to be matched based on the number of corner point matches and the chromaticity matching factor comprises: The maximum value of the number of corner point matches between the fish target to be detected and all the fish targets to be matched is used as a reference number of corner point matches of the fish target to be detected; The number of corner point matches between the fish target to be tested and each of the fish targets to be matched is divided by the reference number of corner point matches of the fish target to be tested, as a corner point match score between the fish target to be tested and each of the fish targets to be matched; Based on the linear relationship between the corner point matching score and the corresponding chromaticity matching factor, the morphological matching factor of the fish target to be detected and each of the fish targets to be matched is calculated.

9. A method for monitoring the status of fish cultured in deep water cages as claimed in claim 6, characterized in that: The calculating of the abnormal state index of each fish school based on the difference characteristics of the swimming speed distribution feature vector of each fish school in each deep-water fish circumnavigation enhanced image and the swimming speed distribution feature vectors of all corresponding fish schools of the same type and the circumnavigation outlier index comprises: Input the swimming speed distribution feature vector of each of the fish schools in each of the deep-water fish circumnavigation enhanced images, the circumnavigation outlier index and the swimming speed distribution feature vectors of all corresponding fish schools of the same type into an abnormal state evaluation expression, and calculate the state abnormality index of each of the fish schools in each of the deep-water fish circumnavigation enhanced images; The abnormal state evaluation expression is: ； in, is the abnormal state index of the fish school m in the i-th deep-water fish swimming enhanced image, is the normalization function, is the swimming outlier index of the fish school m in the i-th deep-water fish swimming enhanced image, is an exponential function with a natural constant as base, is the number of the same fish schools as the fish school m in the i-th deep-water fish swimming enhanced image, is the cosine similarity between two vectors, is the swimming speed distribution feature vector of the fish school m in the i-th deep-water fish swimming enhanced image, is the swimming speed distribution feature vector of the sth fish school of the same type as the fish school m in the i-th deep-water fish swimming enhanced image.

10. A deep-water cage culture fish status monitoring system, characterized in that: The system includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, the method for monitoring the status of fish in deep-water cage culture as described in any one of claims 1 to 9 is implemented.

Images