Method and system for monitoring fish state of deepwater cage culture

Abstract

The invention discloses a fish state monitoring method and system for deepwater cage culture, and the method comprises the steps: collecting a fish round-trip video of a target deepwater culture region through employing an underwater unmanned plane, and obtaining all fish round-trip image frames; an AI algorithm is adopted to carry out reflected light correlation enhancement processing on the fish round-swimming image frames, deepwater fish round-swimming enhanced images are obtained, all fish targets are obtained, all fish schools in each deepwater fish round-swimming enhanced image are divided, and the round-free group index of each fish school is calculated; analyzing the morphological similarity characteristics of all fish schools, and determining the same kind of fish schools of each fish school; and generating a fish state monitoring result of the target deepwater culture area based on the swimming characteristic difference between each fish school in each deepwater fish round swimming enhancement image and the determined fish school of the same kind and the round free group index. According to the fish state monitoring method and system for deepwater cage culture provided by the embodiment of the invention, the accuracy of fish state monitoring is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method and system for monitoring the status of fish cultured in deep-water cages. Background Art

[0002] Deep-water aquaculture refers to aquaculture activities carried out in deep waters far away from the coast. Monitoring the fish migration status in deep-water aquaculture areas not only helps to gain a deeper understanding of fish behavior patterns and optimize aquaculture management, improve production efficiency and economic benefits, but also can provide timely warnings of diseases, assess environmental adaptability, and support ecological impact analysis and sustainable resource management, thereby promoting the healthy development of the aquaculture industry and environmental protection.

[0003] With the development of artificial intelligence technology, existing technologies usually record the swimming status of fish in deep-water aquaculture areas through underwater drones, and then use machine vision to analyze the swimming characteristics of fish, so as to realize the monitoring of the fish status in deep-water aquaculture areas. However, in the photos collected in the deep-water shooting environment, bubbles, light spots, and various types of fish are interspersed, making it difficult to accurately identify and analyze each fish in the image, and thus making it impossible to accurately monitor the status of fish.

[0004] It can be seen that how to improve the accuracy of fish status monitoring in deep-water aquaculture areas has become a technical problem that technical personnel in this field need to solve urgently. Summary of the invention

[0005] The present invention provides a method and system for monitoring the status of fish in deep-water cage culture, so as to solve the technical problem that the monitoring of the status of fish is affected by various factors, resulting in low accuracy of the monitoring of the status of fish in deep-water culture areas, and improve the accuracy of the monitoring of the status of fish in deep-water culture areas.

[0006] In order to solve the above technical problems, an embodiment of the present invention provides a method for monitoring the status of fish cultured in deep water cages.

[0007] An underwater drone is used to collect fish swimming videos in a target deep-water aquaculture area, and the fish swimming videos are processed to obtain image frames of each fish swimming; Performing reflected light correlation enhancement processing on each of the fish swimming image frames to obtain each deep-water fish swimming enhanced image; the reflected light correlation enhancement processing is designed to use the value range mapping result of the reflected light intensity smoothing factor of each pixel point in each of the deep-water fish swimming enhanced images as the standard deviation of the Gaussian filter of the retinal cerebral cortex theoretical algorithm to achieve image enhancement processing, wherein the reflected light intensity smoothing factor reflects the grayscale distribution characteristics within the neighborhood range of each pixel point; Performing target detection on each of the deep-water fish circumnavigation enhanced images to obtain each fish target in each of the deep-water fish circumnavigation enhanced images; Determine the individual schools of fish in each of the deep-water fish circumnavigation enhanced images according to the extracted clustered swimming characteristics of the fish targets in the adjacent deep-water fish circumnavigation enhanced images; determine the circumnavigation outlier index of each of the fish schools based on the extracted dispersion characteristics of the position distribution of all the fish targets in each of the fish schools; Based on the morphological similarity features of all the fish schools in each of the deep-water fish circumnavigation enhanced images extracted, the various similar fish schools of each of the fish schools are obtained; based on the differences in swimming characteristics between each of the fish schools in each of the deep-water fish circumnavigation enhanced images and the circumnavigation outlier index, the fish status monitoring results of the target deep-water aquaculture area are generated.

[0008] As one of the preferred solutions, the specific method for obtaining the reflected light intensity smoothing factor is: Constructing a neighborhood window for each pixel point, and taking the pixel point with the largest pixel value in the neighborhood window as a high light intensity point; Using a plurality of circular boundaries with the high light intensity point as the center as the neighborhood light ring boundary of each pixel point, and using the absolute value of the difference between the pixel value of each pixel point on each neighborhood light ring boundary and the pixel value of the high light intensity point as the centrifugal light intensity attenuation value of each pixel point on each neighborhood light ring boundary; The range of the centrifugal light intensity attenuation values ​​of all the pixels on each of the neighborhood halo boundaries is taken as the centrifugal attenuation discrete index of each of the neighborhood halo boundaries, and the average value of the centrifugal light intensity attenuation values ​​of all the pixels on each of the neighborhood halo boundaries is taken as the centrifugal attenuation average index of each of the neighborhood halo boundaries; Based on the linear relationship between the centrifugal attenuation discrete indexes of all the neighborhood halo boundaries of each pixel and the corresponding centrifugal attenuation average indexes, the reflected light intensity smoothing factor of each pixel is calculated.

[0009] As one of the preferred solutions, the method of determining each school of fish in each of the enhanced deep-water fish swimming images according to the extracted clustered swimming characteristics of the fish targets in the adjacent enhanced deep-water fish swimming images includes: Using an AI algorithm to perform matching analysis on each of the fish targets in the adjacent enhanced deep-water fish swimming images, and obtaining a time series of the swimming positions of each of the fish targets; The Euclidean distance between the time series of the swimming positions of the two fish targets in each of the deep-water fish swimming enhanced images is used as the corresponding swimming curve dissimilarity index; the calculation result of an exponential function with a natural constant as the base and the opposite number of the swimming curve dissimilarity index as the exponent is used as the corresponding cluster swimming similarity; The cluster swimming similarity is used as a similarity evaluation standard of the clustering algorithm, and the clustering algorithm is used to perform cluster analysis on all the fish targets in each of the deep-water fish swimming enhanced images to obtain the various fish schools in each of the deep-water fish swimming enhanced images.

[0010] As one of the preferred solutions, the AI ​​algorithm is used to perform matching analysis on each of the fish targets in the adjacent enhanced deep-water fish swimming images to obtain the swimming position time series of each of the fish targets, including: A corner detection algorithm is used to extract all corner points of each of the deep-water fish swimming enhanced images; a directional gradient histogram and a color histogram of each of the fish targets are constructed; Each of the fish targets in the current enhanced image of deep-water fish swimming around is used as a fish target to be detected; all of the fish targets in the remaining enhanced images of deep-water fish swimming around are used as fish targets to be matched in sequence; Constructing a first directional gradient histogram and a first color histogram of the fish target to be detected and a second directional gradient histogram and a second color histogram of the fish target to be matched; Based on the difference characteristics of the first oriented gradient histogram and the second oriented gradient histogram, and the difference characteristics of the first color histogram and the second color histogram, a chromaticity matching factor between the fish target to be detected and each of the fish targets to be matched is calculated; Performing corner point matching on the fish target to be tested and all the fish targets to be matched, obtaining the number of corner point matches between the fish target to be tested and each of the fish targets to be matched; calculating the morphological matching factor between the fish target to be tested and each of the fish targets to be matched based on the number of corner point matches and the chromaticity matching factor; Sorting and analyzing the morphological matching factors of the fish target to be detected and all the fish targets to be detected in each of the enhanced deep-water fish circumnavigation images, to obtain the matched fish targets of the fish target to be detected in each of the enhanced deep-water fish circumnavigation images; The positions of all the matched fish targets of each fish target in the corresponding enhanced deep-water fish swimming image are arranged in ascending order according to time sequence to obtain a time series of the swimming positions of each fish target.

[0011] As one of the preferred solutions, the method of determining the circumstellar outlier index of each of the fish schools based on the extracted dispersion characteristics of the position distribution of all the fish targets of each of the fish schools comprises: Taking the center point of the minimum circumscribed rectangle of each of the fish schools in each of the deep-water fish swimming enhanced images as the school center of each of the fish schools; The average value of the Euclidean distances between all the fish targets of each fish school and the corresponding center of the fish school is taken as the circumstantial outlier index of each fish school.

[0012] As one of the preferred solutions, the fish status monitoring result of the target deep-water aquaculture area is generated based on the difference in swimming characteristics between each of the fish schools in each of the deep-water fish circumnavigation enhanced images and the determined similar fish schools and the circumnavigation outlier index, including: Acquire the swimming speed time series of the corresponding fish target based on the change characteristics of the position data in each of the circumnavigation position time series; Performing differential processing on the swimming speed time series of each of the fish targets to obtain a progressive swimming speed sequence of each of the fish targets, and taking the extreme difference of all element values ​​in the progressive swimming speed sequence of all the fish targets in each of the fish schools as the swimming speed fluctuation factor of the corresponding fish school; Taking the average value of all element values ​​of the swimming speed time series of all the fish targets in each of the fish schools as the mean swimming speed of the corresponding fish school; splicing the swimming speed fluctuation factor of each of the fish schools and the corresponding swimming speed mean value to obtain a swimming speed distribution feature vector of each of the fish schools; Based on the difference characteristics of the swimming speed distribution feature vector of each fish school in each deep-water fish circumnavigation enhanced image and the corresponding swimming speed distribution feature vectors of all the same fish schools, and the circumnavigation outlier index, the state abnormality index of each fish school is calculated to evaluate the degree of abnormality of the fish state in the target deep-water aquaculture area.

[0013] As one preferred solution, the morphological matching factor is used to evaluate the morphological similarity characteristics between the fish targets; The method of obtaining each similar school of fish of each school of fish based on the morphological similarity features of all the schools of fish in each of the deep-water fish swimming enhanced images extracted comprises: Each of the fish schools in the current enhanced image of deep-water fish swimming around is used as a fish school to be tested, and the remaining fish schools in the current enhanced image of deep-water fish swimming around are used as fish schools to be matched in sequence; The maximum value of the morphological matching factors between each of the fish targets in the fish school to be tested and all of the fish targets in the fish school to be matched is used as the cluster integration factor of each of the fish targets in the fish school to be tested and the fish school to be matched; The average value of the cluster integration factors of all the fish targets in the fish school to be tested and the fish school to be matched is used as the similarity index of the fish school to be matched and the same group of the fish school to be matched; The similarity indexes of all the similar groups are normalized, and the results of the normalization are compared and analyzed with a preset similarity threshold to obtain the similar groups of each of the fish schools.

[0014] As one preferred solution, the morphological matching factor of the fish target to be detected and each of the fish targets to be matched is calculated based on the number of corner point matches and the chromaticity matching factor, including: The maximum value of the number of corner point matches between the fish target to be detected and all the fish targets to be matched is used as a reference number of corner point matches of the fish target to be detected; The number of corner point matches between the fish target to be tested and each of the fish targets to be matched is divided by the reference number of corner point matches of the fish target to be tested, as a corner point match score between the fish target to be tested and each of the fish targets to be matched; Based on the linear relationship between the corner point matching score and the corresponding chromaticity matching factor, the morphological matching factor of the fish target to be detected and each of the fish targets to be matched is calculated.

[0015] As one of the preferred solutions, the state abnormality index of each fish school is calculated based on the difference characteristics of the swimming speed distribution feature vector of each fish school in each deep-water fish circumnavigation enhanced image and the corresponding swimming speed distribution feature vectors of all the same fish schools, and the circumnavigation outlier index, including: Input the swimming speed distribution feature vector of each of the fish schools in each of the deep-water fish circumnavigation enhanced images, the circumnavigation outlier index and the swimming speed distribution feature vectors of all corresponding fish schools of the same type into an abnormal state evaluation expression, and calculate the state abnormality index of each of the fish schools in each of the deep-water fish circumnavigation enhanced images; The abnormal state evaluation expression is:

[0016] in, is the abnormal state index of the fish school m in the i-th deep-water fish swimming enhanced image, is the normalization function, is the swimming outlier index of the fish school m in the i-th deep-water fish swimming enhanced image, is an exponential function with a natural constant as base, is the number of the same fish schools as the fish school m in the i-th deep-water fish swimming enhanced image, is the cosine similarity between two vectors, is the swimming speed distribution feature vector of the fish school m in the i-th deep-water fish swimming enhanced image, is the swimming speed distribution feature vector of the sth fish school of the same type as the fish school m in the i-th deep-water fish swimming enhanced image.

[0017] Yet another embodiment of the present invention provides a system for monitoring the status of fish cultured in deep-water cages, the system comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, the method for monitoring the status of fish cultured in deep-water cages as described above is implemented.

[0018] Compared with the prior art, the embodiments of the present invention have the following advantages: An underwater drone is used to collect fish swimming videos in the target deep-water aquaculture area, and the fish swimming videos are processed to obtain individual fish swimming image frames. Since deep-water images often have color distortion due to the absorption and scattering of light of different wavelengths by water bodies, in order to avoid the influence of bubbles and light spots on the recognition of fish status, each fish swimming image frame is subjected to reflected light correlation enhancement processing to obtain each deep-water fish swimming enhanced image. The reflected light correlation enhancement processing is designed to map the value range of the reflected light intensity smoothing factor of each pixel point in each deep-water fish swimming enhanced image to the standard deviation of the Gaussian filter of the retinal cerebral cortex theoretical algorithm to achieve image enhancement processing, wherein the reflected light intensity smoothing factor reflects the grayscale distribution characteristics within the neighborhood range of each pixel point, and the different light intensity characteristics around each pixel point in the fish swimming image frame are targeted for image enhancement. While retaining the detailed information in the fish swimming image frame, the reflected light is smoothed as much as possible, which helps to eliminate the problem of uneven illumination and improves the accuracy of subsequent fish status monitoring. Considering that most deep-water farmed fishes swim in clusters, the fish state may be abnormal when the fish deviate from the group swimming. Target detection is performed on each deep-water fish circumferential enhanced image to obtain each fish target in each deep-water fish circumferential enhanced image. The fish schools in each deep-water fish circumferential enhanced image are determined based on the cluster swimming characteristics of the fish targets extracted in the adjacent deep-water fish circumferential enhanced images. The circumferential outlier index of each fish school is determined based on the dispersed characteristics of the position distribution of all fish targets extracted from each fish school. The fish schools are first obtained through the cluster swimming characteristics of the fish targets in the adjacent deep-water fish circumferential enhanced images, rather than dividing the fish schools through a single deep-water fish circumferential enhanced image, to avoid misdivision. Then, the dispersed characteristics of the position distribution of all fish targets in each fish school in each deep-water fish circumferential enhanced image are analyzed. The fish schools with higher overall fish school swimming dispersion are more likely to be abnormal, thereby improving the reliability of fish state monitoring. Since there may be many kinds of fish raised in deep-water aquaculture areas, the morphology, living habits and swimming characteristics of different fish are different. However, various fish swim interspersedly. In order to avoid the interference of the swimming of various types of fish schools, which leads to misjudgment of the fish status, the morphological similarity characteristics of all fish schools in each deep-water fish circumnavigation enhanced image are first analyzed to obtain each similar fish school of each fish school. Then, the difference in swimming characteristics between each fish school and the corresponding similar fish school in each deep-water fish circumnavigation enhanced image is analyzed. The state abnormality index of each fish school is calculated in combination with the circumnavigation outlier index, which further improves the accuracy of fish status monitoring. By performing reflected light correlation enhancement processing on each fish swimming image frame, each pixel in each fish swimming image frame is enhanced separately according to the different light intensities, so that the reflected light is smoothed as much as possible while retaining the detail information in the fish swimming image frame. The differences in the swimming characteristics of each fish school and the same fish school and the clustering characteristics of each fish school are further analyzed to obtain the state abnormality index of each fish school, thereby comprehensively improving the accuracy of fish state monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a schematic flow chart of a method for monitoring the status of fish cultured in deep water cages in one embodiment of the present invention; Figure 2 is a schematic diagram of a neighborhood halo boundary in one embodiment of the present invention; Figure 3 is a schematic diagram of obtaining a state abnormality index in one embodiment of the present invention; Figure 4 It is an architectural diagram of a fish status monitoring system for deep-water cage culture in one embodiment of the present invention. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] In the description of this application, the terms "first", "second", "third", etc. are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first", "second", "third", etc. may explicitly or implicitly include one or more of the feature. In the description of this application, unless otherwise specified, "plurality" means two or more.

[0022] In the description of the present application, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be a connection between the two elements. The terms "vertical", "horizontal", "left", "right", "upper", "lower" and similar expressions used herein are only for illustrative purposes, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0023] In the description of this application, it should be noted that, unless otherwise defined, all technical and scientific terms used in the present invention have the same meaning as those commonly understood by those skilled in the art. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood by specific circumstances.

[0024] Since the photos collected in deep-water shooting environments are full of bubbles, light spots and various types of fish swimming around, the effects of bubbles and light spots on fish status identification are caused by the scattering and reflection of light.

[0025] In view of this, the embodiment of the present invention adopts the following related technical means to realize image enhancement of the collected deep-water aquaculture area, so that the details and features in the image are easy to identify, ensuring the accuracy and reliability of the image, and providing strong support for subsequent scientific research, detection, aquaculture and other activities.

[0026] An embodiment of the present invention provides a method for monitoring the status of fish cultured in deep water cages. For details, see Figure 1 , Figure 1 Shown is a flow chart of a method for monitoring the status of fish cultured in deep-water cages in one embodiment of the present invention.

[0027] Step S1: Use an underwater drone to collect fish swimming videos in the target deep-water aquaculture area, process the fish swimming videos, and obtain individual fish swimming image frames.

[0028] Specifically, underwater drones are set at various monitoring points in the target deep-water aquaculture area, and the target deep-water aquaculture area is photographed by the underwater drones to obtain a video of fish swimming around the target deep-water aquaculture area.

[0029] It should be noted that the video is composed of continuous frames, each frame is a static image. By deframing the video, a series of clear static images can be obtained. These images can be used to more accurately analyze the status of fish, such as behavioral patterns, appearance characteristics, etc.

[0030] Specifically, the fish swimming video is processed to obtain individual fish swimming image frames.

[0031] Step S2: performing reflected light correlation enhancement processing on each fish swimming image frame to obtain each deep-water fish swimming enhanced image; performing target detection on each deep-water fish swimming enhanced image to obtain each fish target in each deep-water fish swimming enhanced image.

[0032] It should be noted that there are a large number of fish in deep-water aquaculture areas. Fish breathing will produce bubbles, and the images collected in deep-water areas often have color distortion due to the absorption and scattering of light of different wavelengths by the water body. In order to avoid the bubbles produced by fish breathing and the scattering of light by the water body affecting the monitoring of fish status, the collected original images need to be enhanced first. The following table shows the classification of various fish targets in deep-water aquaculture areas:

[0033] Specifically, each fish swimming image frame is subjected to reflected light correlation enhancement processing to obtain each deep-water fish swimming enhanced image.

[0034] The reflected light association enhancement processing is designed to map the value range of the reflected light intensity smoothing factor of each pixel point in each deep-water fish swimming enhanced image as the standard deviation of the Gaussian filter of the retinal cortex theoretical algorithm to achieve image enhancement processing, where the reflected light intensity smoothing factor reflects the grayscale distribution characteristics within the neighborhood range of each pixel point.

[0035] It should be noted that the core idea of ​​the retinal cortex theory algorithm is that the color of an object is determined by the object's ability to reflect long-wave, medium-wave, and short-wave light, rather than by the absolute value of the reflected light intensity. At the same time, the color of an object is not affected by the non-uniformity of illumination and has consistency, that is, color constancy. The range of the reflected light intensity smoothing factor is mapped to the range of the standard deviation of the Gaussian filter of the retinal cortex theory algorithm through range mapping.

[0036] In this step, the specific calculation method of the reflected light intensity smoothing factor is: A neighborhood window is constructed for each pixel point, and the pixel point with the largest pixel value in the neighborhood window is taken as the high light intensity point; several circular boundaries with the high light intensity point as the center are taken as the neighborhood halo boundaries of each pixel point, and the absolute value of the difference between the pixel value of each pixel point on each neighborhood halo boundary and the pixel value of the high light intensity point is taken as the centrifugal light intensity attenuation value of each pixel point on each neighborhood halo boundary.

[0037] The range of the centrifugal light intensity attenuation values ​​of all pixels on the boundary of each neighborhood halo is taken as the centrifugal attenuation discrete index of each neighborhood halo boundary, and the average of the centrifugal light intensity attenuation values ​​of all pixels on the boundary of each neighborhood halo is taken as the centrifugal attenuation average index of each neighborhood halo boundary.

[0038] Based on the linear relationship between the centrifugal attenuation discrete index of all neighborhood halo boundaries of each pixel and the corresponding centrifugal attenuation average index, the reflected light intensity smoothing factor of each pixel is calculated.

[0039] It should be noted that the larger the centrifugal attenuation discrete index of the neighborhood halo boundary of a pixel point and the larger the corresponding centrifugal attenuation average index, the more likely it is that there is a light spot or bubble with a gradually weakening pixel value around the center point in the neighborhood of the pixel point, and a higher degree of smoothing should be performed on it to avoid affecting the monitoring of fish status.

[0040] As an embodiment of the present application, the product of the centrifugal attenuation discrete index of each neighborhood halo boundary and the corresponding centrifugal attenuation average index is used as the corresponding light intensity scattering characteristic factor, and the average value of the light intensity scattering characteristic factors of all neighborhood halo boundaries of each pixel point is used as the corresponding reflected light intensity smoothing factor.

[0041] For details, see Figure 2 , Figure 2 A schematic diagram showing a neighborhood halo boundary in one embodiment of the present invention is shown.

[0042] Furthermore, target detection is performed on each enhanced image of deep-water fish swimming around, and each fish target in each enhanced image of deep-water fish swimming around is obtained.

[0043] It should be noted that target detection is a core task in the field of computer vision, which aims to find all targets of interest in an image or video. Target detection is a well-known technology and will not be described in detail in this embodiment.

[0044] As an embodiment of the present application, the Yolo model is used to perform target detection on each deep-water fish swimming enhanced image.

[0045] Step S3: Determine the individual fish schools in each deep-water fish enhanced image based on the extracted clustered swimming characteristics of the fish targets in the adjacent deep-water fish enhanced images; and determine the swimming outlier index of each fish school based on the extracted dispersed characteristics of the position distribution of all fish targets in each fish school using an AI algorithm.

[0046] It should be noted that, considering that most deep-water farmed fish swim in a clustering pattern, the fish state may become abnormal when they deviate from the group swimming pattern. By analyzing the clustering characteristics of fish swimming, it is possible to conduct preliminary monitoring of the abnormal state of fish.

[0047] Specifically, a matching analysis is performed on each fish target in adjacent deep-water fish enhanced images to obtain the time series of the swimming positions of each fish target; the Euclidean distance between the swimming position time series of two fish targets in each deep-water fish enhanced image is used as the corresponding swimming curve difference index; the calculation result of an exponential function with a natural constant as the base and the opposite number of the swimming curve difference index as the exponent is used as the corresponding cluster swimming similarity; the cluster swimming similarity is used as the similarity evaluation standard of the clustering algorithm, and the clustering algorithm is used to perform cluster analysis on all fish targets in each deep-water fish enhanced image to obtain each fish school in each deep-water fish enhanced image.

[0048] It should be noted that the clustering algorithm and the calculation of the Euclidean distance are both well-known technologies, and this embodiment will not be described in detail. The time series of the circumnavigation position represents the swimming curve of the fish target. When the swimming curves of the fish targets are more different, they are more likely to belong to different fish schools, and the similarity of the cluster swimming is lower.

[0049] As an embodiment of the present application, a K-means clustering algorithm is used to perform cluster analysis on all fish targets in each deep-water fish swimming enhanced image. As other implementation methods, implementers can choose them at their own discretion, and the present application does not impose any special restrictions on this.

[0050] Furthermore, the center point of the minimum circumscribed rectangle of each fish school in each deep-water fish circumnavigation enhanced image is taken as the school center of each fish school; the average value of the Euclidean distances between all fish targets of each fish school and the corresponding school center is taken as the circumnavigation outlier index of each fish school.

[0051] It should be noted that the fish schools are first obtained through the clustered swimming characteristics of the fish schools in multiple deep-water fish enhanced images, rather than dividing the fish schools through a single deep-water fish enhanced image, to avoid misdivision. Then, the dispersion characteristics of the position distribution of all fish targets of each fish school in each deep-water fish enhanced image are analyzed. The fish schools with a higher degree of overall fish movement dispersion are more likely to be abnormal, thereby improving the reliability of fish status monitoring.

[0052] In this step, matching analysis is performed on each fish target in the adjacent deep-water fish swimming enhanced image to obtain the swimming position time series of each fish target, including: A corner detection algorithm is used to extract all corners of each deep-water fish swimming enhanced image; a directional gradient histogram and a color histogram of each fish target are constructed; Each fish target in the current deep-water fish circumnavigation enhanced image is used as a fish target to be tested; all fish targets in the remaining deep-water fish circumnavigation enhanced images are used as fish targets to be matched in turn; a first directional gradient histogram and a first color histogram of the fish target to be tested and a second directional gradient histogram and a second color histogram of the fish target to be matched are constructed; based on the difference characteristics of the first directional gradient histogram and the second directional gradient histogram, and the difference characteristics of the first color histogram and the second color histogram, a chromaticity matching factor between the fish target to be tested and each fish target to be matched is calculated; Corner point matching is performed on the fish target to be tested and all the fish targets to be matched, and the number of corner point matches between the fish target to be tested and each fish target to be matched is obtained; and the morphological matching factor of the fish target to be tested and each fish target to be matched is calculated based on the number of corner point matches and the chromaticity matching factor; The morphological matching factors of the fish target to be tested and all the fish targets to be tested in each deep-water fish circumnavigation enhanced image are sorted and analyzed to obtain the matched fish targets of the fish target to be tested in each deep-water fish circumnavigation enhanced image; the positions of all the matched fish targets of each fish target in the corresponding deep-water fish circumnavigation enhanced image are arranged in ascending order according to chronological order to obtain the circumnavigation position time series of each fish target.

[0053] It should be noted that the corner detection algorithm is an important technology in the field of computer vision and image processing. It is used to find pixels in an image that may represent the intersection points or corners of object edges. Corner points, as important feature points in an image, can be used for target recognition and tracking. The directional gradient histogram captures the shape features of an image by calculating the gradient direction histogram of a local area of ​​the image. The color histogram reflects the proportion of different colors in the entire image. The construction of the corner detection algorithm, the directional gradient histogram, and the color histogram are all well-known technologies, and will not be described in detail in this embodiment.

[0054] It should be further explained that by matching corner points and comparing the directional gradient histogram and color histogram between fish targets, the similarities of contour features, shape features and color features between fish targets were analyzed respectively, thereby improving the matching accuracy.

[0055] In this step, the morphological matching factor of the fish target to be tested and each fish target to be matched is calculated based on the number of corner point matches and the chromaticity matching factor, including: taking the maximum number of corner point matches between the fish target to be tested and all fish targets to be matched as the reference number of corner point matches of the fish target to be tested; dividing the number of corner point matches between the fish target to be tested and each fish target to be matched by the reference number of corner point matches of the fish target to be tested as the corner point matching score of the fish target to be tested and each fish target to be matched; based on the linear relationship between the corner point matching score and the corresponding chromaticity matching factor, the morphological matching factor of the fish target to be tested and each fish target to be matched is calculated.

[0056] It should be noted that when the corner point matching score and the chromaticity matching factor between fish targets are larger, the morphological similarity characteristics between the cumulative targets are more obvious, and the target is more likely to be the same fish target.

[0057] Step S4: based on the morphological similarity features of all fish schools in each deep-water fish circumnavigation enhanced image extracted, the various similar fish schools of each fish school are obtained; based on the difference in swimming characteristics between each fish school in each deep-water fish circumnavigation enhanced image and the determined similar fish school, and the circumnavigation outlier index, the fish status monitoring results of the target deep-water aquaculture area are generated.

[0058] For details, see Figure 3 , Figure 3 It is a schematic diagram showing the acquisition of the abnormal state index in one embodiment of the present invention.

[0059] It should be noted that the morphological matching factor is used to evaluate the morphological similarity characteristics between fish targets.

[0060] Specifically, each fish school in the current deep-water fish swimming enhanced image is used as the fish school to be tested, and the remaining fish schools in the current deep-water fish swimming enhanced image are used as the fish schools to be matched in turn; the maximum value of the morphological matching factor between each fish target in the fish school to be tested and all the fish targets in the fish school to be matched is used as the cluster integration factor of each fish target in the fish school to be tested and the fish school to be matched.

[0061] The average value of the cluster integration factor of all fish targets in the fish school to be tested and the fish school to be matched is taken as the similarity index of the fish school to be matched and the same group of fish schools to be matched; all the similarity indexes of the same group of fish schools are normalized, and the normalization results are compared and analyzed with the preset similarity threshold to obtain the various similar fish schools of each fish school.

[0062] Furthermore, based on the change characteristics of the position data in each circumferential position time series, the swimming speed time series of the corresponding fish target is obtained, and the swimming speed time series of each fish target is differentially processed to obtain a progressive swimming speed sequence of each fish target.

[0063] The extreme difference of all element values ​​in the progressive sequence of swimming speed of all fish targets in each fish school is taken as the swimming speed fluctuation factor of the corresponding fish school; the average value of all element values ​​in the swimming speed time series of all fish targets in each fish school is taken as the mean swimming speed of the corresponding fish school; the swimming speed fluctuation factor of each fish school and the corresponding mean swimming speed are spliced ​​to obtain the swimming speed distribution characteristic vector of each fish school; The swimming speed distribution feature vector of each fish school in each deep-water fish circumnavigation enhanced image, the circumnavigation outlier index and the swimming speed distribution feature vectors of all corresponding fish schools of the same type are input into the abnormal state evaluation expression, and the state abnormality index of each fish school in each deep-water fish circumnavigation enhanced image is calculated; The abnormal state evaluation expression is:

[0064] in, is the abnormal state index of the fish school m in the i-th deep-water fish swimming enhanced image, is the normalization function, is the swimming outlier index of the fish school m in the i-th deep-water fish swimming enhanced image, is an exponential function with a natural constant as base, is the number of the same fish schools as the fish school m in the i-th deep-water fish swimming enhanced image, is the cosine similarity between two vectors, is the swimming speed distribution feature vector of the fish school m in the i-th deep-water fish swimming enhanced image, is the swimming speed distribution feature vector of the sth fish school of the same type as the fish school m in the i-th deep-water fish swimming enhanced image.

[0065] It should be noted that the method used to evaluate the degree of abnormal fish status in the target deep-water aquaculture area is as follows: when the circum-grouping index of the target fish school is larger, the more obvious the dispersion characteristics of the position distribution of the fish targets in the target fish school are, and the more likely it is that there are some fish targets with abnormal status in the fish school, resulting in a decrease in the clustering characteristics of the fish school; when the cosine similarity between the swimming speed distribution feature vector of the target fish school and the corresponding swimming speed distribution feature vector of the same fish school is smaller, the greater the difference in the swimming characteristics between the target fish school and the similar fish school is, and the fish school as a whole is more likely to be abnormal.

[0066] Compared with the prior art, the embodiments of the present invention have the following advantages: An underwater drone is used to collect fish swimming videos in the target deep-water aquaculture area, and the fish swimming videos are processed to obtain individual fish swimming image frames. Since deep-water images often have color distortion due to the absorption and scattering of light of different wavelengths by water bodies, in order to avoid the influence of bubbles and light spots on the recognition of fish status, each fish swimming image frame is subjected to reflected light correlation enhancement processing to obtain each deep-water fish swimming enhanced image. The reflected light correlation enhancement processing is designed to map the value range of the reflected light intensity smoothing factor of each pixel point in each deep-water fish swimming enhanced image to the standard deviation of the Gaussian filter of the retinal cerebral cortex theoretical algorithm to achieve image enhancement processing, wherein the reflected light intensity smoothing factor reflects the grayscale distribution characteristics within the neighborhood range of each pixel point, and the different light intensity characteristics around each pixel point in the fish swimming image frame are targeted for image enhancement. While retaining the detailed information in the fish swimming image frame, the reflected light is smoothed as much as possible, which helps to eliminate the problem of uneven illumination and improves the accuracy of subsequent fish status monitoring. Considering that most deep-water farmed fishes swim in clusters, the fish state may be abnormal when the fish deviate from the group swimming. Target detection is performed on each deep-water fish circumferential enhanced image to obtain each fish target in each deep-water fish circumferential enhanced image. The fish schools in each deep-water fish circumferential enhanced image are determined based on the cluster swimming characteristics of the fish targets extracted in the adjacent deep-water fish circumferential enhanced images. The circumferential outlier index of each fish school is determined based on the dispersed characteristics of the position distribution of all fish targets extracted from each fish school. The fish schools are first obtained through the cluster swimming characteristics of the fish targets in the adjacent deep-water fish circumferential enhanced images, rather than dividing the fish schools through a single deep-water fish circumferential enhanced image, to avoid misdivision. Then, the dispersed characteristics of the position distribution of all fish targets in each fish school in each deep-water fish circumferential enhanced image are analyzed. The fish schools with higher overall fish school swimming dispersion are more likely to be abnormal, thereby improving the reliability of fish state monitoring. Since there may be many kinds of fish raised in deep-water aquaculture areas, the morphology, living habits and swimming characteristics of different fish are different. However, various fish swim interspersedly. In order to avoid the interference of the swimming of various types of fish schools, which leads to misjudgment of the fish status, the morphological similarity characteristics of all fish schools in each deep-water fish circumnavigation enhanced image are first analyzed to obtain each similar fish school of each fish school. Then, the difference in swimming characteristics between each fish school and the corresponding similar fish school in each deep-water fish circumnavigation enhanced image is analyzed. The state abnormality index of each fish school is calculated in combination with the circumnavigation outlier index, which further improves the accuracy of fish status monitoring. By performing reflected light correlation enhancement processing on each fish swimming image frame, each pixel in each fish swimming image frame is enhanced separately according to the different light intensities, so that the reflected light is smoothed as much as possible while retaining the detail information in the fish swimming image frame. The differences in the swimming characteristics of each fish school and the same fish school and the clustering characteristics of each fish school are further analyzed to obtain the state abnormality index of each fish school, thereby comprehensively improving the accuracy of fish state monitoring.

[0067] Yet another embodiment of the present invention provides a system for monitoring the status of fish cultured in deep-water cages, the system comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, the method for monitoring the status of fish cultured in deep-water cages as described above is implemented.

[0068] For details, see Figure 4 , Figure 4 Shown is an architectural diagram of a fish status monitoring system for deep-water cage farming in one embodiment of the present invention.

[0069] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A method for monitoring the status of fish cultured in deep water cages, characterized in that: include: An underwater drone is used to collect fish swimming videos in a target deep-water aquaculture area, and the fish swimming videos are processed to obtain image frames of each fish swimming; Performing reflected light correlation enhancement processing on each of the fish swimming image frames to obtain enhanced images of each deep-water fish swimming image; The reflected light association enhancement processing is designed to use the value range mapping result of the reflected light intensity smoothing factor of each pixel point in each of the deep-water fish swimming enhanced images as the standard deviation of the Gaussian filter of the retinal cerebral cortex theory algorithm to achieve image enhancement processing, wherein the reflected light intensity smoothing factor reflects the grayscale distribution characteristics within the neighborhood range of each pixel point; Performing target detection on each of the deep-water fish circumnavigation enhanced images to obtain each fish target in each of the deep-water fish circumnavigation enhanced images; Determine the individual schools of fish in each of the deep-water fish circumnavigation enhanced images according to the extracted clustered swimming characteristics of the fish targets in the adjacent deep-water fish circumnavigation enhanced images; determine the circumnavigation outlier index of each of the fish schools based on the extracted dispersion characteristics of the position distribution of all the fish targets in each of the fish schools; Based on the morphological similarity features of all the fish schools in each of the deep-water fish circumnavigation enhanced images extracted, the various similar fish schools of each of the fish schools are obtained; based on the differences in swimming characteristics between each of the fish schools in each of the deep-water fish circumnavigation enhanced images and the circumnavigation outlier index, the fish status monitoring results of the target deep-water aquaculture area are generated.

2. A method for monitoring the status of fish cultured in deep water cages according to claim 1, characterized in that: The specific method for obtaining the reflected light intensity smoothing factor is: Constructing a neighborhood window for each pixel point, and taking the pixel point with the largest pixel value in the neighborhood window as a high light intensity point; Using a plurality of circular boundaries with the high light intensity point as the center as the neighborhood light ring boundary of each pixel point, and using the absolute value of the difference between the pixel value of each pixel point on each neighborhood light ring boundary and the pixel value of the high light intensity point as the centrifugal light intensity attenuation value of each pixel point on each neighborhood light ring boundary; The range of the centrifugal light intensity attenuation values ​​of all the pixels on each of the neighborhood halo boundaries is taken as the centrifugal attenuation discrete index of each of the neighborhood halo boundaries, and the average value of the centrifugal light intensity attenuation values ​​of all the pixels on each of the neighborhood halo boundaries is taken as the centrifugal attenuation average index of each of the neighborhood halo boundaries; Based on the linear relationship between the centrifugal attenuation discrete indexes of all the neighborhood halo boundaries of each pixel and the corresponding centrifugal attenuation average indexes, the reflected light intensity smoothing factor of each pixel is calculated.

3. The method for monitoring the status of fish cultured in deep water cages according to claim 1, characterized in that: The method of determining each school of fish in each of the enhanced deep-water fish swimming images according to the extracted clustered swimming characteristics of the fish targets in the adjacent enhanced deep-water fish swimming images comprises: Using an AI algorithm to perform matching analysis on each of the fish targets in the adjacent enhanced deep-water fish swimming images, and obtaining a time series of the swimming positions of each of the fish targets; The Euclidean distance between the time series of the swimming positions of the two fish targets in each of the deep-water fish swimming enhanced images is used as the corresponding swimming curve dissimilarity index; the calculation result of an exponential function with a natural constant as the base and the opposite number of the swimming curve dissimilarity index as the exponent is used as the corresponding cluster swimming similarity; The cluster swimming similarity is used as a similarity evaluation standard of the clustering algorithm, and the clustering algorithm is used to perform cluster analysis on all the fish targets in each of the deep-water fish swimming enhanced images to obtain the various fish schools in each of the deep-water fish swimming enhanced images.

4. A method for monitoring the status of fish cultured in deep water cages as claimed in claim 3, characterized in that: The AI ​​algorithm is used to perform matching analysis on each of the fish targets in the adjacent enhanced deep-water fish swimming images to obtain a time series of the swimming positions of each of the fish targets, including: A corner detection algorithm is used to extract all corner points of each of the deep-water fish swimming enhanced images; a directional gradient histogram and a color histogram of each of the fish targets are constructed; Each of the fish targets in the current enhanced image of deep-water fish swimming around is used as a fish target to be detected; all of the fish targets in the remaining enhanced images of deep-water fish swimming around are used as fish targets to be matched in sequence; Constructing a first directional gradient histogram and a first color histogram of the fish target to be detected and a second directional gradient histogram and a second color histogram of the fish target to be matched; Based on the difference characteristics of the first oriented gradient histogram and the second oriented gradient histogram, and the difference characteristics of the first color histogram and the second color histogram, a chromaticity matching factor between the fish target to be detected and each of the fish targets to be matched is calculated; Performing corner point matching on the fish target to be tested and all the fish targets to be matched, obtaining the number of corner point matches between the fish target to be tested and each of the fish targets to be matched; calculating the morphological matching factor between the fish target to be tested and each of the fish targets to be matched based on the number of corner point matches and the chromaticity matching factor; Sorting and analyzing the morphological matching factors of the fish target to be detected and all the fish targets to be detected in each of the enhanced deep-water fish circumnavigation images, to obtain the matched fish targets of the fish target to be detected in each of the enhanced deep-water fish circumnavigation images; The positions of all the matched fish targets of each fish target in the corresponding enhanced deep-water fish swimming image are arranged in ascending order according to time sequence to obtain a time series of the swimming positions of each fish target.

5. The method for monitoring the status of fish cultured in deep water cages according to claim 1, characterized in that: The step of determining the circumstellar outlier index of each of the fish schools based on the extracted dispersion characteristics of the position distribution of all the fish targets of each of the fish schools comprises: Taking the center point of the minimum circumscribed rectangle of each of the fish schools in each of the deep-water fish swimming enhanced images as the school center of each of the fish schools; The average value of the Euclidean distances between all the fish targets of each fish school and the corresponding center of the fish school is taken as the circumstantial outlier index of each fish school.

6. A method for monitoring the status of fish cultured in deep water cages as claimed in claim 4, characterized in that: The method of generating the fish status monitoring result of the target deep-water aquaculture area based on the difference in swimming characteristics between each of the fish schools in each of the deep-water fish circumnavigation enhanced images and the determined similar fish schools and the circumnavigation outlier index comprises: Acquire the swimming speed time series of the corresponding fish target based on the change characteristics of the position data in each of the circumnavigation position time series; Performing differential processing on the swimming speed time series of each of the fish targets to obtain a progressive swimming speed sequence of each of the fish targets, and taking the extreme difference of all element values ​​in the progressive swimming speed sequence of all the fish targets in each of the fish schools as the swimming speed fluctuation factor of the corresponding fish school; Taking the average value of all element values ​​of the swimming speed time series of all the fish targets in each of the fish schools as the mean swimming speed of the corresponding fish school; splicing the swimming speed fluctuation factor of each of the fish schools and the corresponding swimming speed mean value to obtain a swimming speed distribution feature vector of each of the fish schools; Based on the difference characteristics of the swimming speed distribution feature vector of each fish school in each deep-water fish circumnavigation enhanced image and the corresponding swimming speed distribution feature vectors of all the same fish schools, and the circumnavigation outlier index, the state abnormality index of each fish school is calculated to evaluate the degree of abnormality of the fish state in the target deep-water aquaculture area.

7. A method for monitoring the status of fish cultured in deep water cages as claimed in claim 4, characterized in that: The morphological matching factor is used to evaluate the morphological similarity characteristics between the fish targets; The method of obtaining each similar school of fish of each school of fish based on the morphological similarity features of all the schools of fish in each of the deep-water fish swimming enhanced images extracted comprises: Each of the fish schools in the current enhanced image of deep-water fish swimming around is used as a fish school to be tested, and the remaining fish schools in the current enhanced image of deep-water fish swimming around are used as fish schools to be matched in sequence; The maximum value of the morphological matching factors between each of the fish targets in the fish school to be tested and all of the fish targets in the fish school to be matched is used as the cluster integration factor of each of the fish targets in the fish school to be tested and the fish school to be matched; The average value of the cluster integration factors of all the fish targets in the fish school to be tested and the fish school to be matched is used as the similarity index of the fish school to be matched and the same group of the fish school to be matched; The similarity indexes of all the similar groups are normalized, and the results of the normalization are compared and analyzed with a preset similarity threshold to obtain the similar groups of each of the fish schools.

8. A method for monitoring the status of fish cultured in deep water cages as claimed in claim 4, characterized in that: The step of calculating the morphological matching factor of the fish target to be detected and each of the fish targets to be matched based on the number of corner point matches and the chromaticity matching factor comprises: The maximum value of the number of corner point matches between the fish target to be detected and all the fish targets to be matched is used as a reference number of corner point matches of the fish target to be detected; The number of corner point matches between the fish target to be tested and each of the fish targets to be matched is divided by the reference number of corner point matches of the fish target to be tested, as a corner point match score between the fish target to be tested and each of the fish targets to be matched; Based on the linear relationship between the corner point matching score and the corresponding chromaticity matching factor, the morphological matching factor of the fish target to be detected and each of the fish targets to be matched is calculated.

9. A method for monitoring the status of fish cultured in deep water cages as claimed in claim 6, characterized in that: The calculating of the abnormal state index of each fish school based on the difference characteristics of the swimming speed distribution feature vector of each fish school in each deep-water fish circumnavigation enhanced image and the swimming speed distribution feature vectors of all corresponding fish schools of the same type and the circumnavigation outlier index comprises: Input the swimming speed distribution feature vector of each of the fish schools in each of the deep-water fish circumnavigation enhanced images, the circumnavigation outlier index and the swimming speed distribution feature vectors of all corresponding fish schools of the same type into an abnormal state evaluation expression, and calculate the state abnormality index of each of the fish schools in each of the deep-water fish circumnavigation enhanced images; The abnormal state evaluation expression is: ； in, is the abnormal state index of the fish school m in the i-th deep-water fish swimming enhanced image, is the normalization function, is the swimming outlier index of the fish school m in the i-th deep-water fish swimming enhanced image, is an exponential function with a natural constant as base, is the number of the same fish schools as the fish school m in the i-th deep-water fish swimming enhanced image, is the cosine similarity between two vectors, is the swimming speed distribution feature vector of the fish school m in the i-th deep-water fish swimming enhanced image, is the swimming speed distribution feature vector of the sth fish school of the same type as the fish school m in the i-th deep-water fish swimming enhanced image.

10. A deep-water cage culture fish status monitoring system, characterized in that: The system includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, the method for monitoring the status of fish in deep-water cage culture as described in any one of claims 1 to 9 is implemented.

Images