A color restoration method, device and apparatus for underwater fish detection images

Abstract

The present application discloses a color restoration method, equipment and device for underwater fish detection images, and relates to the field of image processing technology. The color restoration method for underwater fish detection images includes: obtaining an underwater image of underwater fish; the underwater image includes a first time-series image and a second time-series image; based on an edge algorithm, a plurality of first edge chain codes are obtained from the first time-series image. When restoring the color of fish in underwater images, the present application utilizes the characteristic that underwater fish like to live in groups, and obtains the image of the fish closest to the camera by calculating similarity and symmetry, and then restores the color of other fish of the same species based on the principle that the color of the fish closest to the camera is more realistic. Compared with the prior art, the restoration method of the present application is not easy to cause the color ratio to change, that is, the enhanced image is not prone to color distortion.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and specifically to a method, equipment and device for color restoration of underwater fish detection images. Background Art

[0002] When taking photos of underwater fish, due to the low underwater light level, the contrast of the underwater fish images taken is too low and the fish details are not obvious. Therefore, it is generally necessary to enhance the underwater fish images to increase the image contrast, thereby facilitating the extraction of more image information.

[0003] In the prior art, the contrast of an image is generally enhanced by histogram equalization. The specific method is to enhance the values ​​of each pixel in the RGB channels separately, and then obtain the enhanced image. Since water has different absorption rates for light of different wavelengths, there is a problem with the proportional relationship of the three channel values ​​at each fish pixel in the image to be enhanced. In addition, the color of seawater is almost blue, so after enhancing the values ​​under each channel separately, the enhanced image is prone to color distortion due to the change in color ratio. Summary of the invention

[0004] The purpose of the present application is to provide a method, device and apparatus for color restoration of underwater fish detection images, so as to solve the technical problem of color distortion after enhancing underwater images of fish in the prior art.

[0005] To achieve the above objectives, this application provides the following technical solutions:

[0006] In a first aspect, the present application proposes a technical solution for a color restoration method of an underwater fish detection image, the color restoration method of an underwater fish detection image comprising:

[0007] Acquire underwater images of underwater fish; the underwater images include a first time-series image and a second time-series image; the time sequence of the first time-series image and the second time-series image is the same; the first time-series image is pre-acquired by a first camera; the second time-series image is pre-acquired by a second camera; the first camera and the second camera are arranged facing each other;

[0008] Acquire a plurality of first edge chain codes from the first time-series image based on an edge algorithm;

[0009] Acquire a plurality of second edge chain codes from the second time series image based on an edge algorithm;

[0010] Based on the multiple first edge chain codes, a third edge chain code is obtained; the third edge chain code is any edge chain code among the first edge chain codes;

[0011] Based on the multiple second edge chain codes, a fourth edge chain code is obtained; the fourth edge chain code is any edge chain code among the second edge chain codes;

[0012] Based on the third edge chain code and the fourth edge chain code, obtaining a first similarity;

[0013] If the first similarity is greater than a first preset value, a first distance and a second distance are obtained; the first distance is the distance between the fish corresponding to the third edge chain code and the fourth edge chain code and the first camera; the second distance is the distance between the fish corresponding to the third edge chain code and the fourth edge chain code and the second camera;

[0014] Based on the first distance and the second distance, obtaining a minimum distance;

[0015] The color of each fish in the first time-series image and the second time-series image is restored based on the color of the fish that is at the shortest distance from the first camera or the second camera.

[0016] As a specific solution in the technical solution of the present application, the step of obtaining a third edge chain code based on multiple first edge chain codes includes:

[0017] Based on the plurality of first edge chain codes, obtaining a plurality of shared edge chain codes;

[0018] Based on multiple shared edge chain codes, a first shared edge chain code and a second shared edge chain code are obtained; the first shared edge chain code is any shared edge chain code among the shared edge chain codes; the second shared edge chain code is any shared edge chain code among the shared edge chain codes that is different from the first shared edge chain code;

[0019] Based on the first common edge chain code and the second common edge chain code, obtaining a second similarity;

[0020] If the second similarity is greater than a second preset value, the first shared edge chain code is included in the data set;

[0021] Based on the data set, the third edge chain code is obtained; the third edge chain code is any edge chain code in the data set.

[0022] As a specific solution in the technical solution of the present application, the obtaining of the second similarity based on the first common edge chain code and the second common edge chain code includes:

[0023] Based on a dynamic time rule algorithm, obtaining a dynamic distance between the first common edge chain code and the second common edge chain code;

[0024] Based on the dynamic distance, the second similarity is acquired.

[0025] As a specific solution in the technical solution of the present application, the obtaining of the second similarity based on the dynamic distance includes:

[0026] Based on the first common edge chain code, a first mean and a second mean are obtained; the first mean is the mean of the B channel pixels in each pixel point in the first common edge chain code; the second mean is the mean of the G channel pixels in each pixel point in the first common edge chain code;

[0027] Based on the second common edge chain code, a third mean and a fourth mean are obtained; the third mean is the mean of the B channel pixels in each pixel point in the second common edge chain code; the fourth mean is the mean of the G channel pixels in each pixel point in the second common edge chain code;

[0028] The second similarity is obtained based on the first mean, the second mean, the third mean, the fourth mean and the dynamic distance.

[0029] As a specific solution in the technical solution of the present application, the calculation formula for obtaining the second similarity based on the first mean, the second mean, the third mean, the fourth mean and the dynamic distance is as follows:

[0030] ;

[0031] represents the second similarity of the ath shared edge chain code; n represents the number of shared edge chain codes; represents the mean value of the B channel pixels in each pixel point in the i-th common edge chain code; It represents the mean value of the G channel pixels in each pixel point in the i-th common edge chain code and its value is not 0; represents the mean value of the B channel pixels in each pixel point in the ath common edge chain code; It represents the mean value of the G channel pixels in each pixel point in the ath common edge chain code and its value is not 0; Indicates the absolute value. represents the weight normalization function; represents the dynamic distance between the a-th common edge chain code and the i-th common edge chain code; Represents an exponential function with a natural constant as its base.

[0032] As a specific solution in the technical solution of the present application, the obtaining of the first similarity based on the third edge chain code and the fourth edge chain code includes:

[0033] Based on the third edge chain code, a first distance sequence is acquired; the first distance sequence is the distance between each pixel point in the third edge chain code and the centroid of the first time series image;

[0034] Based on the fourth edge chain code, a second distance sequence is acquired; the second distance sequence is the distance between a plurality of pixel points in the fourth edge chain code and the centroid of the second time series image; the second distance sequence corresponds to the data value of the first distance sequence one by one;

[0035] Based on the first distance sequence and the second distance sequence, a ratio sequence is obtained; a data value in the ratio sequence is equal to a ratio of a data value in the first distance sequence to a corresponding data value in the second distance sequence;

[0036] Based on the ratio sequence, obtaining information entropy;

[0037] Based on the information entropy, the first similarity is obtained.

[0038] As a specific solution in the technical solution of the present application, the calculation formula for obtaining the first similarity based on the information entropy is as follows:

[0039] ;

[0040] in, represents the first similarity between the third edge chain code and the fourth edge chain code; represents the second similarity of the third edge chain code; represents the second similarity of the fourth edge chain code; represents information entropy; Represents a mapping function, which is used to map the value in the brackets to the interval [0, 1].

[0041] As a specific solution in the technical solution of the present application, the color of each fish in the first time-series image and the second time-series image is restored based on the color of the fish that is the shortest distance from the first camera or the second camera, including:

[0042] Based on the third edge chain code, obtaining a first fish image from the first time-series image;

[0043] Based on the fourth edge chain code, acquiring a second fish image from the second time-series image;

[0044] Based on the first fish image and the second fish image, a third fish image and a fourth fish image are acquired; the third fish image is a fish image corresponding to the minimum distance; the fourth fish image is another fish image of the first fish image and the second fish image that is different from the third fish image;

[0045] Based on the third fish image, the fourth fish image is enlarged in proportion to obtain a fifth fish image;

[0046] Based on the third fish image, the color of the fifth fish image is restored.

[0047] In a second aspect, the present application proposes a technical solution of a computer-readable storage medium for color restoration of underwater fish detection images. The color restoration device for underwater fish detection images includes a reading module and a processing module, wherein:

[0048] A reading module, used for acquiring underwater images of underwater fish; the underwater images include a first time-series image and a second time-series image; the time sequence of the first time-series image and the second time-series image is the same; the first time-series image is pre-acquired by a first camera; the second time-series image is pre-acquired by a second camera; the first camera and the second camera are arranged facing each other;

[0049] A processing module, configured to obtain a plurality of first edge chain codes from the first time-series image based on an edge algorithm;

[0050] and, obtaining a plurality of second edge chain codes from the second time series images based on an edge algorithm;

[0051] And, based on the multiple first edge chain codes, a third edge chain code is obtained; the third edge chain code is any edge chain code among the first edge chain codes;

[0052] And, based on the plurality of second edge chain codes, a fourth edge chain code is obtained; the fourth edge chain code is any edge chain code among the second edge chain codes;

[0053] and, based on the third edge chain code and the fourth edge chain code, obtaining a first similarity;

[0054] And, if the first similarity is greater than a first preset value, a first distance and a second distance are obtained; the first distance is the distance between the fish corresponding to the third edge chain code and the fourth edge chain code and the first camera; the second distance is the distance between the fish corresponding to the third edge chain code and the fourth edge chain code and the second camera;

[0055] and, based on the first distance and the second distance, obtaining a minimum distance;

[0056] And, based on the color of the fish that is at the shortest distance from the first camera or the second camera, the color of each fish in the first time-series image and the second time-series image is restored.

[0057] As a specific solution in the technical solution of the present application, the processing module is further used to obtain multiple shared edge chain codes based on multiple first edge chain codes;

[0058] And, based on the multiple shared edge chain codes, a first shared edge chain code and a second shared edge chain code are obtained; the first shared edge chain code is any shared edge chain code among the shared edge chain codes; the second shared edge chain code is any shared edge chain code among the shared edge chain codes that is different from the first shared edge chain code;

[0059] and, based on the first common edge chain code and the second common edge chain code, obtaining a second similarity;

[0060] and, if the second similarity is greater than a second preset value, incorporating the first shared edge chain code into the data set;

[0061] And, based on the data set, the third edge chain code is obtained; the third edge chain code is any edge chain code in the data set.

[0062] As a specific solution in the technical solution of the present application, the processing module is further used to obtain the dynamic distance between the first common edge chain code and the second common edge chain code based on a dynamic time rule algorithm;

[0063] And, based on the dynamic distance, the second similarity is obtained.

[0064] As a specific solution in the technical solution of the present application, the processing module is further used to obtain a first mean and a second mean based on the first common edge chain code; the first mean is the mean of the B channel pixels in each pixel point in the first common edge chain code; the second mean is the mean of the G channel pixels in each pixel point in the first common edge chain code;

[0065] And, based on the second common edge chain code, a third mean and a fourth mean are obtained; the third mean is the mean of the B channel pixels in each pixel point in the second common edge chain code; the fourth mean is the mean of the G channel pixels in each pixel point in the second common edge chain code;

[0066] And, based on the first mean, the second mean, the third mean, the fourth mean and the dynamic distance, the second similarity is obtained.

[0067] As a specific solution in the technical solution of the present application, the processing module obtains the calculation formula of the second similarity based on the first mean, the second mean, the third mean, the fourth mean and the dynamic distance as follows:

[0068] ;

[0069] represents the second similarity of the ath shared edge chain code; n represents the number of shared edge chain codes; represents the mean value of the B channel pixels in each pixel point in the i-th common edge chain code; It represents the mean value of the G channel pixels in each pixel point in the i-th common edge chain code and its value is not 0; represents the mean value of the B channel pixels in each pixel point in the ath common edge chain code; It represents the mean value of the G channel pixels in each pixel point in the ath common edge chain code and its value is not 0; Indicates the absolute value. represents the weight normalization function; represents the dynamic distance between the a-th common edge chain code and the i-th common edge chain code; Represents an exponential function with a natural constant as its base.

[0070] As a specific solution in the technical solution of the present application, the processing module is further used to obtain a first distance sequence based on the third edge chain code; the first distance sequence is the distance between each pixel point in the third edge chain code and the centroid of the first time series image;

[0071] And, based on the fourth edge chain code, a second distance sequence is obtained; the second distance sequence is the distance between a plurality of pixel points in the fourth edge chain code and the centroid of the second time series image; the second distance sequence has a one-to-one correspondence with the data values ​​of the first distance sequence;

[0072] And, based on the first distance sequence and the second distance sequence, a ratio sequence is obtained; the data value in the ratio sequence is equal to the ratio of the data value in the first distance sequence to the corresponding data value in the second distance sequence;

[0073] And, based on the ratio sequence, obtaining information entropy;

[0074] And, based on the information entropy, the first similarity is obtained.

[0075] As a specific solution in the technical solution of the present application, the processing module obtains the calculation formula of the first similarity based on the information entropy as follows:

[0076] ;

[0077] in, represents the first similarity between the third edge chain code and the fourth edge chain code; represents the second similarity of the third edge chain code; represents the second similarity of the fourth edge chain code; represents information entropy; Represents a mapping function, which is used to map the value in the brackets to the interval [0, 1].

[0078] As a specific solution in the technical solution of the present application, the processing module is further used to obtain a first fish image from the first time-series image based on the third edge chain code;

[0079] And, based on the fourth edge chain code, acquiring a second fish image from the second time-series image;

[0080] And, based on the first fish image and the second fish image, a third fish image and a fourth fish image are acquired; the third fish image is a fish image corresponding to the minimum distance; the fourth fish image is another fish image of the first fish image and the second fish image that is different from the third fish image;

[0081] And, based on the third fish image, the fourth fish image is enlarged in equal proportion to obtain a fifth fish image;

[0082] And, based on the third fish image, the color of the fifth fish image is restored.

[0083] In a third aspect, the present application proposes a technical solution for a color restoration device for underwater fish detection images, the color restoration device for underwater fish detection images comprising:

[0084] A reader, used for acquiring underwater images of underwater fish; the underwater images include a first time-sequence image and a second time-sequence image; the time sequence of the first time-sequence image and the second time-sequence image is the same; the first time-sequence image is pre-acquired by a first camera; the second time-sequence image is pre-acquired by a second camera; the first camera and the second camera are arranged facing each other;

[0085] A processor, configured to obtain a plurality of first edge chain codes from the first time-series image based on an edge algorithm;

[0086] and, obtaining a plurality of second edge chain codes from the second time series images based on an edge algorithm;

[0087] And, based on the multiple first edge chain codes, a third edge chain code is obtained; the third edge chain code is any edge chain code among the first edge chain codes;

[0088] And, based on the plurality of second edge chain codes, a fourth edge chain code is obtained; the fourth edge chain code is any edge chain code among the second edge chain codes;

[0089] and, based on the third edge chain code and the fourth edge chain code, obtaining a first similarity;

[0090] And, if the first similarity is greater than a first preset value, a first distance and a second distance are obtained; the first distance is the distance between the fish corresponding to the third edge chain code and the fourth edge chain code and the first camera; the second distance is the distance between the fish corresponding to the third edge chain code and the fourth edge chain code and the second camera;

[0091] and, based on the first distance and the second distance, obtaining a minimum distance;

[0092] And, based on the color of the fish that is at the shortest distance from the first camera or the second camera, the color of each fish in the first time-series image and the second time-series image is restored.

[0093] As a specific solution in the technical solution of the present application, the processor is further used to obtain multiple shared edge chain codes based on multiple first edge chain codes;

[0094] And, based on the multiple shared edge chain codes, a first shared edge chain code and a second shared edge chain code are obtained; the first shared edge chain code is any shared edge chain code among the shared edge chain codes; the second shared edge chain code is any shared edge chain code among the shared edge chain codes that is different from the first shared edge chain code;

[0095] and, based on the first common edge chain code and the second common edge chain code, obtaining a second similarity;

[0096] and, if the second similarity is greater than a second preset value, incorporating the first shared edge chain code into the data set;

[0097] And, based on the data set, the third edge chain code is obtained; the third edge chain code is any edge chain code in the data set.

[0098] As a specific solution in the technical solution of the present application, the processor is further used to obtain a dynamic distance between the first common edge chain code and the second common edge chain code based on a dynamic time rule algorithm;

[0099] And, based on the dynamic distance, the second similarity is obtained.

[0100] As a specific solution in the technical solution of the present application, the processor is further used to obtain a first mean and a second mean based on the first common edge chain code; the first mean is the mean of the B channel pixels in each pixel point in the first common edge chain code; the second mean is the mean of the G channel pixels in each pixel point in the first common edge chain code;

[0101] And, based on the second common edge chain code, a third mean and a fourth mean are obtained; the third mean is the mean of the B channel pixels in each pixel point in the second common edge chain code; the fourth mean is the mean of the G channel pixels in each pixel point in the second common edge chain code;

[0102] And, based on the first mean, the second mean, the third mean, the fourth mean and the dynamic distance, the second similarity is obtained.

[0103] As a specific solution in the technical solution of the present application, the processor obtains the calculation formula of the second similarity based on the first mean, the second mean, the third mean, the fourth mean and the dynamic distance as follows:

[0104] ;

[0105] represents the second similarity of the ath shared edge chain code; n represents the number of shared edge chain codes; represents the mean value of the B channel pixels in each pixel point in the i-th common edge chain code; It represents the mean value of the G channel pixels in each pixel point in the i-th common edge chain code and its value is not 0; represents the mean value of the B channel pixels in each pixel point in the ath common edge chain code; It represents the mean value of the G channel pixels in each pixel point in the ath common edge chain code and its value is not 0; Indicates the absolute value. represents the weight normalization function; represents the dynamic distance between the a-th common edge chain code and the i-th common edge chain code; Represents an exponential function with a natural constant as its base.

[0106] As a specific solution in the technical solution of the present application, the processor is further used to obtain a first distance sequence based on the third edge chain code; the first distance sequence is the distance between each pixel point in the third edge chain code and the centroid of the first time series image;

[0107] And, based on the fourth edge chain code, a second distance sequence is obtained; the second distance sequence is the distance between a plurality of pixel points in the fourth edge chain code and the centroid of the second time series image; the second distance sequence has a one-to-one correspondence with the data values ​​of the first distance sequence;

[0108] And, based on the first distance sequence and the second distance sequence, a ratio sequence is obtained; the data value in the ratio sequence is equal to the ratio of the data value in the first distance sequence to the corresponding data value in the second distance sequence;

[0109] And, based on the ratio sequence, obtaining information entropy;

[0110] And, based on the information entropy, the first similarity is obtained.

[0111] As a specific solution in the technical solution of the present application, the processor obtains the calculation formula of the first similarity based on the information entropy as follows:

[0112] ;

[0113] in, represents the first similarity between the third edge chain code and the fourth edge chain code; represents the second similarity of the third edge chain code; represents the second similarity of the fourth edge chain code; represents information entropy; Represents a mapping function, which is used to map the value in the brackets to the interval [0, 1].

[0114] As a specific solution in the technical solution of the present application, the processor is further used to obtain a first fish image from the first time-series image based on the third edge chain code;

[0115] And, based on the fourth edge chain code, acquiring a second fish image from the second time-series image;

[0116] And, based on the first fish image and the second fish image, a third fish image and a fourth fish image are acquired; the third fish image is a fish image corresponding to the minimum distance; the fourth fish image is another fish image of the first fish image and the second fish image that is different from the third fish image;

[0117] And, based on the third fish image, the fourth fish image is enlarged in equal proportion to obtain a fifth fish image;

[0118] And, based on the third fish image, the color of the fifth fish image is restored.

[0119] Compared with the prior art, the beneficial effects of this application are:

[0120] When restoring the colors of fish in underwater images, the present application utilizes the characteristic that underwater fish like to live in groups, obtains the image of the fish closest to the camera by calculating similarity and symmetry, and then restores the colors of other fish of the same species based on the principle that the colors of the fish closest to the camera are more realistic. Compared with the prior art, the restoration method of the present application is less likely to change the color ratio, that is, the enhanced image is less likely to have color distortion. BRIEF DESCRIPTION OF THE DRAWINGS

[0121] Figure 1 A schematic diagram of a flow chart of a color restoration method for underwater fish detection images proposed in an embodiment of the present application;

[0122] Figure 2 A schematic diagram of the structure of a color restoration device for underwater fish detection images proposed in an embodiment of the present application;

[0123] Figure 3 A schematic diagram of the structure of a color restoration device for underwater fish detection images proposed in an embodiment of the present application;

[0124] Figure 4 This is a schematic diagram of the installation of the first camera and the second camera proposed in the embodiment of the present application, which are located under the water surface;

[0125] Figure 5 An underwater image of a fish proposed in an embodiment of the present application;

[0126] Figure 6 Based on Figure 5 The acquired edge chain code image;

[0127] Figure 7 A schematic diagram of an edge chain code image of a fish proposed in an embodiment of the present application;

[0128] Figure 8 A schematic diagram of the distance between a pixel point in an image and the image centroid proposed in an embodiment of the present application.

[0129] In the figure: 1, water surface; 2, first camera; 3, second camera; 41, outer contour line; 42, inner contour line; 43, fish back contour; 44, fish belly contour; 51, image centroid; 52, pixel point. DETAILED DESCRIPTION

[0130] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0131] The terms "first", "second", etc. in the description of the embodiments of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence, such as the first time-series image and the second time-series image proposed below, which belong to different time-series images. It should be understood that the time-series images used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules that are clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products or devices. The division of modules that appear in the embodiments of the present application is only a logical division. There may be other division methods when implemented in actual applications. For example, multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, and the indirect coupling or communication connection between modules can be electrical or other similar forms, which are not limited in the embodiments of the present application. In addition, the modules or submodules described as separate components may or may not be physically separated, may or may not be physical modules, or may be distributed in multiple circuit modules, and some or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment of the present application.

[0132] A color restoration method for underwater fish detection images, such as Figure 1 As shown, the color restoration method of the underwater fish detection image includes steps S100 to S900.

[0133] Step S100: Acquire underwater images of underwater fish.

[0134] In an embodiment of the present application, the underwater image includes a first time-series image and a second time-series image. The time sequence of the first time-series image and the second time-series image is the same, the first time-series image is pre-acquired by the first camera 2, and the second time-series image is pre-acquired by the second camera 3. Figure 4 As shown, the first camera 2 and the second camera 3 are arranged facing each other, wherein the vertical direction is as shown in FIG. Figure 4 That is, in the embodiment of the present application, the fish located between the first camera 2 and the second camera 3 are mainly photographed. In other words, if both the first time-series image and the second time-series image have a certain fish, the first time-series image must have one side of the fish; and the second time-series image must have the other side of the fish.

[0135] Step S200: obtaining a plurality of first edge chain codes from the first time-series image based on an edge algorithm.

[0136] It should be clear that it is a mature technology to obtain multiple edge chain codes from a certain image based on edge algorithms. Figure 5 and Figure 6 As shown, Figure 5 is an underwater image of an underwater fish. Figure 6 Based on Figure 5 Obtained edge chain code image.

[0137] Step S300: Acquire a plurality of second edge chain codes from the second time-series image based on an edge algorithm.

[0138] It should be noted that most fish are gregarious animals, that is, if the first time sequence image or the second time sequence image has a certain fish, then the first time sequence image or the second time sequence image should also have other fish of the same species as the fish. It is easy to understand that if a certain fish is closer to the camera, the color of the fish is more realistic, that is, in the embodiment of the present application, the color of other fish of the same species can be restored based on the color of a certain fish that is closest to the camera (for example, the first camera 2 or the second camera 3).

[0139] Step S400: Based on the plurality of first edge chain codes, a third edge chain code is obtained.

[0140] It should be clear that the third edge chain code is any edge chain code among the first edge chain codes. In the embodiment of the present application, the purpose of obtaining the third edge chain code is to determine whether the outline corresponding to the third edge chain code is a fish. If it is a fish, the color of the fish corresponding to the outline can be restored later.

[0141] It should be clear that Figure 7 The contour line is obtained based on a certain fish. Generally, due to the large difference between the color of the fish belly and the color of the fish back, the contour line obtained based on a certain fish generally includes an outer contour line 41 and an inner contour line 42 as shown in the figure. The inner contour line 42 is the dividing line between the fish back and the fish belly. Figure 7 As shown, the inner contour line 42 divides the outer contour line 41 into the fish back contour 43 and the fish belly contour 44. It is easy to understand that in the process of obtaining edge chain codes based on the edge algorithm, since the inner contour line 42 belongs to both the fish back contour 43 and the fish belly contour 44, the edge chain code corresponding to the inner contour line 42 is defined as a common edge chain code. In other words, if the two fish are of the same species, the edge chain codes corresponding to the inner contour lines 42 of the two fish must be similar. That is, Figure 6 As shown, if Figure 6If there are multiple similar shared edge chain codes in the dataset, the fish corresponding to these shared edge chain codes should be of the same species.

[0142] In order to be able to quickly identify all the fish in the first time-series image, in one embodiment of the present application, step S400, based on multiple first edge chain codes, obtains a third edge chain code, including steps S410 to S450.

[0143] Step S410: Based on the multiple first edge chain codes, multiple shared edge chain codes are obtained.

[0144] It should be clear that, as can be seen from the foregoing, the shared edge chain code is the edge chain code corresponding to the contour line shared by any two adjacent contours. Based on multiple first edge chain codes, obtaining the shared edge chain code among them is a mature technology and will not be described in detail here.

[0145] Step S420: Based on the multiple shared edge chain codes, obtain a first shared edge chain code and a second shared edge chain code.

[0146] In an embodiment of the present application, the first shared edge chain code is any shared edge chain code among the shared edge chain codes; the second shared edge chain code is any shared edge chain code among the shared edge chain codes that is different from the first shared edge chain code.

[0147] Step S430: Acquire a second similarity based on the first common edge chain code and the second common edge chain code.

[0148] As can be seen from the foregoing, the shared edge chain codes corresponding to the inner contour lines 42 of the same type of fish are similar. In other words, if a shared edge chain code has a high similarity with other shared edge chain codes among all shared edge chain codes, then this shared edge chain code is likely to be the edge chain code corresponding to the inner contour line 42 of a certain fish. In other words, if a shared edge chain code is not similar to other shared edge chain codes among all shared edge chain codes, then this shared edge chain code is likely not the edge chain code corresponding to the inner contour line 42 of a certain fish.

[0149] In an embodiment of the present application, any reasonable method can be used to obtain the second similarity based on the first shared edge chain code and the second shared edge chain code. For example, the Euclidean distance algorithm or the cosine similarity algorithm can be used. It is easy to understand that in underwater images, the size of the fish and the distance from the camera can cause the lengths of the first shared edge chain code and the second shared edge chain code to be different. In other words, it is difficult to accurately calculate the second similarity of the first shared edge chain code and the second shared edge chain code through a general algorithm.

[0150] In order to accurately calculate the second similarity between the first common edge chain code and the second common edge chain code, in one embodiment of the present application, step S430, based on the first common edge chain code and the second common edge chain code, obtains the second similarity, including step S431 and step S432.

[0151] Step S431: Based on the dynamic time rule algorithm, obtain the dynamic distance between the first common edge chain code and the second common edge chain code.

[0152] It should be noted that the Dynamic Time Warping (DTW) algorithm is an algorithm for comparing the similarity of two sequences, which can be applied to the similarity calculation of data of two sequences of different lengths. The dynamic time warping algorithm minimizes the cumulative distance between two sequences through nonlinear mapping, thereby evaluating their similarity. In other words, the dynamic time warping algorithm can be applied to the application scenarios of the embodiments of the present application.

[0153] Step S432: Based on the dynamic distance, obtain the second similarity.

[0154] It should be clear that in the embodiments of the present application, the dynamic distance can be directly used as the second similarity.

[0155] As can be seen from the foregoing, if the fish is small or the fish is far away from the camera, the image corresponding to the fish is small; if the image corresponding to the fish is small, the number of pixels corresponding to its shared edge chain code is also small. If the number of pixels corresponding to its shared edge chain code is also small, the error of the second similarity obtained may also be large. In order to lower the second similarity weight corresponding to the shared edge chain code of smaller fish or fish far away from the camera, in one embodiment of the present application, step S432, based on the dynamic distance, obtains the second similarity, including steps S433 to S435.

[0156] Step S433: Based on the first common edge chain code, obtain a first mean and a second mean.

[0157] In this embodiment, the first mean value is the mean value of the B channel pixels in each pixel point in the first common edge chain code; the second mean value is the mean value of the G channel pixels in each pixel point in the first common edge chain code.

[0158] Step S434: Based on the second common edge chain code, obtain a third mean and a fourth mean.

[0159] In this embodiment, the third mean is the mean of the B channel pixels in each pixel point in the second common edge chain code; the fourth mean is the mean of the G channel pixels in each pixel point in the second common edge chain code.

[0160] Step S435: Based on the first mean, the second mean, the third mean, the fourth mean and the dynamic distance, the second similarity is obtained.

[0161] It should be clear that in the embodiments of the present application, any reasonable method can be used to obtain the second similarity based on the first mean, the second mean, the third mean, the fourth mean and the dynamic distance. For example, in one embodiment of the present application, step S435, based on the first mean, the second mean, the third mean, the fourth mean and the dynamic distance, the calculation formula for obtaining the second similarity is as follows:

[0162] ;

[0163] represents the second similarity of the ath shared edge chain code; n represents the number of shared edge chain codes; represents the mean value of the B channel pixels in each pixel point in the i-th common edge chain code; It represents the mean value of the G channel pixels in each pixel point in the i-th common edge chain code and its value is not 0; represents the mean value of the B channel pixels in each pixel point in the ath common edge chain code; It represents the mean value of the G channel pixels in each pixel point in the ath common edge chain code and its value is not 0; Indicates the absolute value. represents the weight normalization function; represents the dynamic distance between the a-th common edge chain code and the i-th common edge chain code; Represents an exponential function with a natural constant as its base.

[0164] It should be noted that water has different absorption rates for light of different wavelengths, that is, the attenuation rates of red light, blue light and green light in water are different, and the color of each pixel in the shared edge chain code is determined by the pixel value of the R channel, the pixel value of the G channel and the pixel value of the B channel. In other words, The ratio reflects the The distance between the edge chain code and the camera. It is easy to understand that the absorption rate of seawater to green light is greater than that to blue light. That is to say, if The larger the ratio, the The farther the shared edge chain code is from the camera. The smaller the value of The distance between the edge chain code and the camera is the same as the The distances between the shared edge chain codes and the camera are similar, so the dynamic distance In other words, the larger the second similarity obtained later, the greater the weight. The shared edge chain code and The more similar the shared edge chain codes are.

[0165] Step S440: If the second similarity is greater than a second preset value, the first shared edge chain code is included in the data set.

[0166] It should be noted that, each first common edge chain code in the data set of this embodiment is similar. In other words, each first common edge chain code in the data set of this embodiment may be an edge chain code corresponding to the inner contour line 42 of the fish, and the fish corresponding to each first common edge chain code should also be similar.

[0167] In an embodiment of the present application, a second preset value can be set as required. It is easy to understand that if the second preset value is larger, the number of first common edge chain codes in the subsequently acquired data set will be smaller; if the second preset value is smaller, the number of first common edge chain codes in the subsequently acquired data set will be larger. It is easy to understand that if the number of first common edge chain codes is small, the colors of many fish cannot be restored subsequently; if the number of first common edge chain codes is large, the colors of some fish may not be correctly restored subsequently. In order to enable the colors of most fish in the first time series image to be restored and the restoration accuracy to be high, in an embodiment of the present application, the second preset value can be greater than or equal to 0.6 and less than or equal to 0.9.

[0168] Step S450: Based on the data set, obtain the third edge chain code.

[0169] It should be understood that, in the embodiment of the present application, the third edge chain code is any edge chain code in the data set.

[0170] It should be noted that in the embodiment of the present application, if the fish is symmetrical, the color of the other side of the fish can be restored based on one side of the fish, that is, the color of the fish of the same type as the fish in the second time sequence image can be restored based on the color of the fish in the first time sequence image. If the fish is asymmetrical, restoration cannot be performed.

[0171] Step S500: Based on the plurality of second edge chain codes, a fourth edge chain code is obtained.

[0172] It should be clear that the fourth edge chain code is any edge chain code among the second edge chain codes. In the embodiment of the present application, the fourth edge chain code may be a shared edge chain code, and the method for obtaining the fourth edge chain code may refer to step S400, which will not be described in detail here.

[0173] Step S600: Acquire a first similarity based on the third edge chain code and the fourth edge chain code.

[0174] In the embodiment of the present application, the third edge chain code and the fourth edge chain code belong to edge chain codes corresponding to the inner contour lines 42 on different sides of the same fish. That is, the first similarity is at least used to indicate whether the two sides of the same fish are similar. If the third edge chain code and the fourth edge chain code are similar, it means that the two sides of the fish are symmetrical; if the third edge chain code and the fourth edge chain code are not similar, it means that the two sides of the fish are asymmetrical.

[0175] In an embodiment of the present application, the first similarity can be obtained based on the third edge chain code and the fourth edge chain code in any suitable manner. For example, in an embodiment of the present application, the first similarity between the third edge chain code and the fourth edge chain code can be obtained based on a cosine distance algorithm or a Euclidean distance algorithm.

[0176] It should be noted that, due to the different distances between the same fish and the first camera 2 and the second camera 3, the edge chain codes (i.e., the third edge chain code and the fourth edge chain code) corresponding to the inner contour lines 42 on both sides of the same fish are also different in size. In other words, when the distance between a fish and the camera changes, the size of the fish in the image and the distance from the edge of the image will also change accordingly, that is, the pixel points on the edge chain code corresponding to the fish (i.e., Figure 8 The pixel point 52 shown in FIG. Figure 8 The distance between the image centroid 51 shown in FIG. Figure 8 The distance S shown in the figure will change. It is easy to understand that since the fish is located between the first camera 2 and the second camera 3, no matter how the position of the fish changes, if the third edge chain code and the fourth edge chain code belong to the same fish, the distance between each pixel point on the third edge chain code and the image center of mass and the distance between each corresponding pixel point on the fourth edge chain code and the image center of mass must be proportional. That is to say, in an embodiment of the present application, whether two edge chain codes are similar can be determined by the distance between each pixel point in the third edge chain code and the image center of mass. Specifically, in one embodiment of the present application, step S600, based on the third edge chain code and the fourth edge chain code, obtains a first similarity, including steps S610 to S650.

[0177] Step S610: Based on the third edge chain code, a first distance sequence is acquired.

[0178] It should be clear that the first distance sequence is the distance between each pixel point in the third edge chain code and the centroid of the first time-series image. It should be clear that obtaining the distance between two points is a mature technology and will not be described in detail here.

[0179] Step S620: Based on the fourth edge chain code, a second distance sequence is acquired.

[0180] It should be clear that the second distance sequence is the distance between a plurality of pixel points in the fourth edge chain code and the centroid of the second time series image; the data values ​​of the second distance sequence and the first distance sequence correspond one to one.

[0181] It should be noted that, in the embodiment of the present application, if the number of pixels in the third edge chain code and the fourth edge chain code is inconsistent, it is necessary to select a number of similar points from the edge chain code with more pixels to make the number of pixels of the two consistent. Specifically, in the embodiment of the present application, assuming that in the third edge chain code and the fourth edge chain code, the number of pixels of one edge chain code is j, and the number of pixels of the other edge chain code is g, where j is greater than g; then the 1st pixel point, the [2j / g]th pixel point, the [3j / g]th pixel point... the [(g-1)j / g]th pixel point and the jth pixel point can be selected from the j edge chain codes with more pixels, respectively, so that the number of pixels of the subsequent two edge chain codes is consistent. In this embodiment, [ ] means rounding the values ​​in brackets.

[0182] Step S630: Acquire a ratio sequence based on the first distance sequence and the second distance sequence.

[0183] It should be clear that the data values ​​in the ratio sequence are equal to the ratios of the data values ​​in the first distance sequence to the corresponding data values ​​in the second distance sequence.

[0184] Step S640: Obtain information entropy based on the ratio sequence.

[0185] It should be clear that obtaining the information entropy corresponding to the sequence based on the sequence is a mature technology and will not be described here. As can be seen from the previous text, if the third edge chain code and the fourth edge chain code are edge chain codes on the same fish, the values ​​in the ratio sequence are similar, that is, the information entropy is small; if the third edge chain code and the fourth edge chain code are not edge chain codes on the same fish, the values ​​in the ratio sequence are quite different, that is, the information entropy is large.

[0186] Step S650: Based on the information entropy, obtain the first similarity.

[0187] It should be clear that in the embodiment of the present application, the information entropy can be directly used as the first similarity. In order to accurately characterize the first similarity, in one embodiment of the present application, step S650, based on the information entropy, obtains the calculation formula of the first similarity as follows:

[0188] ;

[0189] in, represents the first similarity between the third edge chain code and the fourth edge chain code; represents the second similarity of the third edge chain code; represents the second similarity of the fourth edge chain code; represents information entropy; Represents a mapping function, which is used to map the value in the brackets to the interval [0, 1].

[0190] It is easy to understand that in the embodiment of the present application, if the first similarity The larger the similarity, the more likely the fish photographed in the current time sequence is symmetrical. The smaller it is, the more likely the fish photographed in the current time sequence is asymmetrical.

[0191] Step S700: If the first similarity is greater than a first preset value, obtaining a first distance and a second distance.

[0192] It should be clear that in this embodiment, the first distance is the distance between the fish corresponding to the third edge chain code and the fourth edge chain code and the first camera 2; the second distance is the distance between the fish corresponding to the third edge chain code and the fourth edge chain code and the second camera 3.

[0193] In the embodiment of the present application, the first preset value can be reasonably set. It is easy to understand that if the first preset value is large, it means that the third edge chain code and the fourth edge chain code need to be very similar, so that the color of other fish can be restored by the fish corresponding to the third edge chain code or the fourth edge chain code. It is easy to understand that under this condition, the similarity and symmetry between the individual fish photographed are required to be extremely high. If some fish belong to the same species but the individual fish are relatively different, it will be difficult to restore the color of these fish later. If the first preset value is small, it means that the third edge chain code and the fourth edge chain code need to be relatively similar, so that the color of other fish can be restored by the fish corresponding to the third edge chain code or the fourth edge chain code. It is easy to understand that under this condition, the similarity and symmetry between the individual fish photographed are required to be low. If some fish do not belong to the same species, but the similarity between the two is high, the color of some fish may also be restored incorrectly. After a large number of experimental verifications by the inventor, it was found that the first preset value can be greater than or equal to 0.6 and less than or equal to 0.8.

[0194] As we can see from the previous text, the closer a fish is to the camera, the larger its corresponding edge chain code is in the image, that is, the more edge chain code pixels it has in the image. Figure 4 As shown, the distance between the first camera 2 and the second camera 3 is L; the third edge chain code and the fourth edge chain code are two edge chain codes on the same fish; and the number of pixels of the third edge chain code is X1; the number of pixels of the fourth edge chain code is X2; the distance between the fish and the first camera 2 is Y1, and the distance from the second camera 3 is Y2, then the following set of equations can be obtained.

[0195] X1 / Y2=X2 / Y1;

[0196] Y1+Y2=L;

[0197] It should be clear that in the above equations, X1, X2 and L are all known. Solving two unknowns Y1 and Y2 through two equations is a mature technology and will not be described in detail here.

[0198] Step S800: Acquire a minimum distance based on the first distance and the second distance.

[0199] It should be clear that obtaining the minimum value from two values ​​is a mature technology.

[0200] Step S900: restoring the color of each fish in the first time-series image and the second time-series image based on the color of the fish that is at the shortest distance from the first camera 2 or the second camera 3 .

[0201] Specifically, in an embodiment of the present application, step S900, restoring the color of each fish in the first time-series image and the second time-series image based on the color of the fish at the shortest distance from the first camera 2 or the second camera 3, includes steps S910 to S950.

[0202] Step S910: Based on the third edge chain code, a first fish image is acquired from the first time-series image.

[0203] It should be clear that obtaining a fish image corresponding to an edge chain code from a certain image is a mature technology and will not be described in detail here.

[0204] Step S920: Based on the fourth edge chain code, a second fish image is acquired from the second time-series image.

[0205] It should be clear that obtaining a fish image corresponding to an edge chain code from a certain image is a mature technology and will not be described in detail here.

[0206] Step S930: acquiring a third fish image and a fourth fish image based on the first fish image and the second fish image.

[0207] It should be understood that the third fish image is a fish image corresponding to the minimum distance; and the fourth fish image is another fish image of the first fish image and the second fish image that is different from the third fish image.

[0208] Step S940: Based on the third fish image, the fourth fish image is enlarged in proportion to obtain a fifth fish image.

[0209] It should be noted that scaling an image is a mature technology and will not be described in detail here.

[0210] Step S950: based on the third fish image, restore the color of the fifth fish image.

[0211] As can be seen from the foregoing, the third fish image is a fish image that is closer to the camera, and its color is more realistic. After scaling, the size of the fifth fish image is basically the same as the third fish image, that is, in the subsequent restoration process, the restoration accuracy can be further improved.

[0212] It is easy to understand that in the embodiment of the present application, the third fish image can also be used to restore the images of other fish of the same species. For example, the image of the fish whose distance from the camera is close to 0 can be used to restore the color of all other images of the same fish, which will not be described in detail here.

[0213] It should be clear that the embodiment of the color restoration method of underwater fish detection images proposed in the present application utilizes the characteristic that underwater fish like to live in groups when restoring the colors of fish in underwater images. By calculating similarity and symmetry, the image of the fish closest to the camera is obtained, and then based on the principle that the color of the fish closest to the camera is more realistic, the colors of other fish of the same species are restored. Compared with the prior art, the restoration method of the present application is not easy to change the color ratio, that is, the enhanced image is not easy to have color distortion.

[0214] After introducing an embodiment of a color restoration method for underwater fish detection images proposed in the present application, the following introduces a color restoration device for underwater fish detection images proposed in the present application. Figure 2 As shown, the color restoration device 10 for underwater fish detection images includes:

[0215] The reading module 11 is used to obtain underwater images of underwater fish; the underwater images include a first time-series image and a second time-series image; the time sequence of the first time-series image and the second time-series image is the same; the first time-series image is pre-acquired by the first camera 2; the second time-series image is pre-acquired by the second camera 3; the first camera 2 and the second camera 3 are arranged opposite to each other;

[0216] A processing module 12 is used to obtain a plurality of first edge chain codes from the first time-series image based on an edge algorithm;

[0217] and, obtaining a plurality of second edge chain codes from the second time series images based on an edge algorithm;

[0218] And, based on the multiple first edge chain codes, a third edge chain code is obtained; the third edge chain code is any edge chain code among the first edge chain codes;

[0219] And, based on the plurality of second edge chain codes, a fourth edge chain code is obtained; the fourth edge chain code is any edge chain code among the second edge chain codes;

[0220] and, based on the third edge chain code and the fourth edge chain code, obtaining a first similarity;

[0221] And, if the first similarity is greater than a first preset value, a first distance and a second distance are obtained; the first distance is the distance between the fish corresponding to the third edge chain code and the fourth edge chain code and the first camera 2; the second distance is the distance between the fish corresponding to the third edge chain code and the fourth edge chain code and the second camera 3;

[0222] and, based on the first distance and the second distance, obtaining a minimum distance;

[0223] And, based on the color of the fish that is the shortest distance from the first camera 2 or the second camera 3, the color of each fish in the first time-series image and the second time-series image is restored.

[0224] As a specific embodiment of the present application, the processing module 12 is further configured to obtain a plurality of shared edge chain codes based on a plurality of first edge chain codes;

[0225] And, based on the multiple shared edge chain codes, a first shared edge chain code and a second shared edge chain code are obtained; the first shared edge chain code is any shared edge chain code among the shared edge chain codes; the second shared edge chain code is any shared edge chain code among the shared edge chain codes that is different from the first shared edge chain code;

[0226] and, based on the first common edge chain code and the second common edge chain code, obtaining a second similarity;

[0227] and, if the second similarity is greater than a second preset value, incorporating the first shared edge chain code into the data set;

[0228] And, based on the data set, the third edge chain code is obtained; the third edge chain code is any edge chain code in the data set.

[0229] As a specific embodiment of the present application, the processing module 12 is further used to obtain a dynamic distance between the first common edge chain code and the second common edge chain code based on a dynamic time rule algorithm;

[0230] And, based on the dynamic distance, the second similarity is obtained.

[0231] As a specific embodiment of the present application, the processing module 12 is further used to obtain a first mean and a second mean based on the first common edge chain code; the first mean is the mean of the B channel pixels in each pixel point in the first common edge chain code; the second mean is the mean of the G channel pixels in each pixel point in the first common edge chain code;

[0232] And, based on the second common edge chain code, a third mean and a fourth mean are obtained; the third mean is the mean of the B channel pixels in each pixel point in the second common edge chain code; the fourth mean is the mean of the G channel pixels in each pixel point in the second common edge chain code;

[0233] And, based on the first mean, the second mean, the third mean, the fourth mean and the dynamic distance, the second similarity is obtained.

[0234] As a specific embodiment of the present application, the processing module 12 obtains the calculation formula of the second similarity based on the first mean, the second mean, the third mean, the fourth mean and the dynamic distance as follows:

[0235] ;

[0236] represents the second similarity of the ath shared edge chain code; n represents the number of shared edge chain codes; represents the mean value of the B channel pixels in each pixel point in the i-th common edge chain code; It represents the mean value of the G channel pixels in each pixel point in the i-th common edge chain code and its value is not 0; represents the mean value of the B channel pixels in each pixel point in the ath common edge chain code; It represents the mean value of the G channel pixels in each pixel point in the ath common edge chain code and its value is not 0; Indicates the absolute value. represents the weight normalization function; represents the dynamic distance between the a-th common edge chain code and the i-th common edge chain code; Represents an exponential function with a natural constant as its base.

[0237] As a specific embodiment of the present application, the processing module 12 is further used to obtain a first distance sequence based on the third edge chain code; the first distance sequence is the distance between each pixel point in the third edge chain code and the centroid of the first time series image;

[0238] And, based on the fourth edge chain code, a second distance sequence is obtained; the second distance sequence is the distance between a plurality of pixel points in the fourth edge chain code and the centroid of the second time series image; the second distance sequence has a one-to-one correspondence with the data values ​​of the first distance sequence;

[0239] And, based on the first distance sequence and the second distance sequence, a ratio sequence is obtained; the data value in the ratio sequence is equal to the ratio of the data value in the first distance sequence to the corresponding data value in the second distance sequence;

[0240] And, based on the ratio sequence, obtaining information entropy;

[0241] And, based on the information entropy, the first similarity is obtained.

[0242] As a specific embodiment of the present application, the processing module 12 obtains the calculation formula of the first similarity based on the information entropy as follows:

[0243] ;

[0244] in, represents the first similarity between the third edge chain code and the fourth edge chain code; represents the second similarity of the third edge chain code; represents the second similarity of the fourth edge chain code; represents information entropy; Represents a mapping function, which is used to map the value in the brackets to the interval [0, 1].

[0245] As a specific embodiment of the present application, the processing module 12 is further used to obtain a first fish image from the first time-series image based on the third edge chain code;

[0246] And, based on the fourth edge chain code, acquiring a second fish image from the second time-series image;

[0247] And, based on the first fish image and the second fish image, a third fish image and a fourth fish image are acquired; the third fish image is a fish image corresponding to the minimum distance; the fourth fish image is another fish image of the first fish image and the second fish image that is different from the third fish image;

[0248] And, based on the third fish image, the fourth fish image is enlarged in equal proportion to obtain a fifth fish image;

[0249] And, based on the third fish image, the color of the fifth fish image is restored.

[0250] It should be clear that the embodiment of the color restoration device for underwater fish detection images proposed in the present application utilizes the characteristic that underwater fish like to live in groups when restoring the colors of fish in underwater images. By calculating similarity and symmetry, the image of the fish closest to the camera is obtained, and then based on the principle that the color of the fish closest to the camera is more realistic, the colors of other fish of the same species are restored. Compared with the prior art, the restoration method of the present application is not easy to change the color ratio, that is, the enhanced image is not easy to have color distortion.

[0251] After introducing an embodiment of a color restoration device for underwater fish detection images proposed in the present application, the following introduces a color restoration device for underwater fish detection images proposed in the present application, such as Figure 3 As shown, the color restoration device 20 for underwater fish detection images includes:

[0252] The reader 21 is used to obtain underwater images of underwater fish; the underwater images include a first time-series image and a second time-series image; the time sequence of the first time-series image and the second time-series image is the same; the first time-series image is pre-acquired by the first camera 2; the second time-series image is pre-acquired by the second camera 3; the first camera 2 and the second camera 3 are arranged facing each other;

[0253] The processor 22 is configured to obtain a plurality of first edge chain codes from the first time-series image based on an edge algorithm;

[0254] and, obtaining a plurality of second edge chain codes from the second time series images based on an edge algorithm;

[0255] And, based on the multiple first edge chain codes, a third edge chain code is obtained; the third edge chain code is any edge chain code among the first edge chain codes;

[0256] And, based on the plurality of second edge chain codes, a fourth edge chain code is obtained; the fourth edge chain code is any edge chain code among the second edge chain codes;

[0257] and, based on the third edge chain code and the fourth edge chain code, obtaining a first similarity;

[0258] And, if the first similarity is greater than a first preset value, a first distance and a second distance are obtained; the first distance is the distance between the fish corresponding to the third edge chain code and the fourth edge chain code and the first camera 2; the second distance is the distance between the fish corresponding to the third edge chain code and the fourth edge chain code and the second camera 3;

[0259] and, based on the first distance and the second distance, obtaining a minimum distance;

[0260] And, based on the color of the fish that is the shortest distance from the first camera 2 or the second camera 3, the color of each fish in the first time-series image and the second time-series image is restored.

[0261] As a specific embodiment of the present application, the processor 22 is further configured to obtain a plurality of shared edge chain codes based on a plurality of first edge chain codes;

[0262] And, based on the multiple shared edge chain codes, a first shared edge chain code and a second shared edge chain code are obtained; the first shared edge chain code is any shared edge chain code among the shared edge chain codes; the second shared edge chain code is any shared edge chain code among the shared edge chain codes that is different from the first shared edge chain code;

[0263] and, based on the first common edge chain code and the second common edge chain code, obtaining a second similarity;

[0264] and, if the second similarity is greater than a second preset value, incorporating the first shared edge chain code into the data set;

[0265] And, based on the data set, the third edge chain code is obtained; the third edge chain code is any edge chain code in the data set.

[0266] As a specific embodiment of the present application, the processor 22 is further configured to obtain a dynamic distance between the first common edge chain code and the second common edge chain code based on a dynamic time rule algorithm;

[0267] And, based on the dynamic distance, the second similarity is obtained.

[0268] As a specific embodiment of the present application, the processor 22 is further used to obtain a first mean and a second mean based on the first common edge chain code; the first mean is the mean of the B channel pixels in each pixel point in the first common edge chain code; the second mean is the mean of the G channel pixels in each pixel point in the first common edge chain code;

[0269] And, based on the second common edge chain code, a third mean and a fourth mean are obtained; the third mean is the mean of the B channel pixels in each pixel point in the second common edge chain code; the fourth mean is the mean of the G channel pixels in each pixel point in the second common edge chain code;

[0270] And, based on the first mean, the second mean, the third mean, the fourth mean and the dynamic distance, the second similarity is obtained.

[0271] As a specific embodiment of the present application, the processor 22 obtains the calculation formula of the second similarity based on the first mean, the second mean, the third mean, the fourth mean and the dynamic distance as follows:

[0272] ;

[0273] represents the second similarity of the ath shared edge chain code; n represents the number of shared edge chain codes; represents the mean value of the B channel pixels in each pixel point in the i-th common edge chain code; It represents the mean value of the G channel pixels in each pixel point in the i-th common edge chain code and its value is not 0; represents the mean value of the B channel pixels in each pixel point in the ath common edge chain code; It represents the mean value of the G channel pixels in each pixel point in the ath common edge chain code and its value is not 0; Indicates the absolute value. represents the weight normalization function; represents the dynamic distance between the a-th common edge chain code and the i-th common edge chain code; Represents an exponential function with a natural constant as its base.

[0274] As a specific embodiment of the present application, the processor 22 is further configured to obtain a first distance sequence based on the third edge chain code; the first distance sequence is the distance between each pixel point in the third edge chain code and the centroid of the first time series image;

[0275] And, based on the fourth edge chain code, a second distance sequence is obtained; the second distance sequence is the distance between a plurality of pixel points in the fourth edge chain code and the centroid of the second time series image; the second distance sequence has a one-to-one correspondence with the data values ​​of the first distance sequence;

[0276] And, based on the first distance sequence and the second distance sequence, a ratio sequence is obtained; the data value in the ratio sequence is equal to the ratio of the data value in the first distance sequence to the corresponding data value in the second distance sequence;

[0277] And, based on the ratio sequence, obtaining information entropy;

[0278] And, based on the information entropy, the first similarity is obtained.

[0279] As a specific embodiment of the present application, the processor 22 obtains the calculation formula of the first similarity based on the information entropy as follows:

[0280] ;

[0281] in, represents the first similarity between the third edge chain code and the fourth edge chain code; represents the second similarity of the third edge chain code; represents the second similarity of the fourth edge chain code; represents information entropy; Represents a mapping function, which is used to map the value in the brackets to the interval [0, 1].

[0282] As a specific embodiment of the present application, the processor 22 is further configured to obtain a first fish image from the first time-series image based on the third edge chain code;

[0283] And, based on the fourth edge chain code, acquiring a second fish image from the second time-series image;

[0284] And, based on the first fish image and the second fish image, a third fish image and a fourth fish image are acquired; the third fish image is a fish image corresponding to the minimum distance; the fourth fish image is another fish image of the first fish image and the second fish image that is different from the third fish image;

[0285] And, based on the third fish image, the fourth fish image is enlarged in equal proportion to obtain a fifth fish image;

[0286] And, based on the third fish image, the color of the fifth fish image is restored.

[0287] It should be clear that the embodiment of the color restoration device for underwater fish detection images proposed in the present application utilizes the characteristic that underwater fish like to live in groups when restoring the colors of fish in underwater images. By calculating similarity and symmetry, the image of the fish closest to the camera is obtained, and then based on the principle that the color of the fish closest to the camera is more realistic, the colors of other fish of the same species are restored. Compared with the prior art, the restoration method of the present application is not easy to change the color ratio, that is, the enhanced image is not easy to have color distortion.

[0288] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0289] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the methods, devices and equipment described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0290] In the several embodiments provided in the embodiments of the present application, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0291] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0292] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0293] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0294] The computer program product includes one or more computer instructions. When the computer program is loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disk), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0295] Although the embodiments of the present application have been shown and described, it will be appreciated by those skilled in the art that many changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles of the present application.

Claims

1. A color restoration method for underwater fish detection images, characterized in that: include: Get underwater images of underwater fish; The underwater image includes a first time-series image and a second time-series image; The time sequence of the first time-series image and the second time-series image is the same; the first time-series image is pre-acquired by a first camera (2); The second time-series image is pre-acquired by a second camera (3); the first camera (2) and the second camera (3) are arranged facing each other; Acquire a plurality of first edge chain codes from the first time-series image based on an edge algorithm; Acquire a plurality of second edge chain codes from the second time series image based on an edge algorithm; Based on the multiple first edge chain codes, a third edge chain code is obtained; the third edge chain code is any edge chain code among the first edge chain codes; Based on the multiple second edge chain codes, a fourth edge chain code is obtained; the fourth edge chain code is any edge chain code among the second edge chain codes; Based on the third edge chain code and the fourth edge chain code, obtaining a first similarity; If the first similarity is greater than a first preset value, a first distance and a second distance are obtained; the first distance is the distance between the fish corresponding to the third edge chain code and the fourth edge chain code and the first camera (2); the second distance is the distance between the fish corresponding to the third edge chain code and the fourth edge chain code and the second camera (3); Based on the first distance and the second distance, obtaining a minimum distance; The color of each fish in the first time-series image and the second time-series image is restored based on the color of the fish that is at the shortest distance from the first camera (2) or the second camera (3).

2. The color restoration method of underwater fish detection images according to claim 1, characterized in that: The step of obtaining a third edge chain code based on the plurality of first edge chain codes includes: Based on the plurality of first edge chain codes, obtaining a plurality of shared edge chain codes; Based on multiple shared edge chain codes, a first shared edge chain code and a second shared edge chain code are obtained; the first shared edge chain code is any shared edge chain code among the shared edge chain codes; the second shared edge chain code is any shared edge chain code among the shared edge chain codes that is different from the first shared edge chain code; Based on the first common edge chain code and the second common edge chain code, obtaining a second similarity; If the second similarity is greater than a second preset value, the first shared edge chain code is included in the data set; Based on the data set, the third edge chain code is obtained; the third edge chain code is any edge chain code in the data set.

3. The color restoration method of underwater fish detection image according to claim 2, characterized in that: The acquiring a second similarity based on the first common edge chain code and the second common edge chain code includes: Based on a dynamic time rule algorithm, obtaining a dynamic distance between the first common edge chain code and the second common edge chain code; Based on the dynamic distance, the second similarity is acquired.

4. The color restoration method of underwater fish detection images according to claim 3, characterized in that: The acquiring the second similarity based on the dynamic distance includes: Based on the first common edge chain code, a first mean and a second mean are obtained; the first mean is the mean of the B channel pixels in each pixel point in the first common edge chain code; the second mean is the mean of the G channel pixels in each pixel point in the first common edge chain code; Based on the second common edge chain code, a third mean and a fourth mean are obtained; the third mean is the mean of the B channel pixels in each pixel point in the second common edge chain code; the fourth mean is the mean of the G channel pixels in each pixel point in the second common edge chain code; The second similarity is obtained based on the first mean, the second mean, the third mean, the fourth mean and the dynamic distance.

5. The color restoration method of underwater fish detection image according to claim 4, characterized in that: The calculation formula for obtaining the second similarity based on the first mean, the second mean, the third mean, the fourth mean and the dynamic distance is as follows: ； represents the second similarity of the ath shared edge chain code; n represents the number of shared edge chain codes; represents the mean value of the B channel pixels in each pixel point in the i-th common edge chain code; It represents the mean value of the G channel pixels in each pixel point in the i-th common edge chain code and its value is not 0; represents the mean value of the B channel pixels in each pixel point in the ath common edge chain code; It represents the mean value of the G channel pixels in each pixel point in the ath common edge chain code and its value is not 0; Indicates the absolute value. represents the weight normalization function; represents the dynamic distance between the ath common edge chain code and the ith common edge chain code; Represents an exponential function with a natural constant as its base.

6. The color restoration method of underwater fish detection images according to any one of claims 2 to 5, characterized in that: The obtaining of the first similarity based on the third edge chain code and the fourth edge chain code includes: Based on the third edge chain code, a first distance sequence is acquired; the first distance sequence is the distance between each pixel point in the third edge chain code and the centroid of the first time series image; Based on the fourth edge chain code, a second distance sequence is obtained; the second distance sequence is the distance between each pixel point in the fourth edge chain code and the centroid of the second time series image; the second distance sequence corresponds to the data value of the first distance sequence one by one; Based on the first distance sequence and the second distance sequence, a ratio sequence is obtained; a data value in the ratio sequence is equal to a ratio of a data value in the first distance sequence to a corresponding data value in the second distance sequence; Based on the ratio sequence, obtaining information entropy; Based on the information entropy, the first similarity is obtained.

7. The color restoration method of underwater fish detection images according to claim 6, characterized in that: The calculation formula for obtaining the first similarity based on the information entropy is as follows: ； in, represents the first similarity between the third edge chain code and the fourth edge chain code; represents the second similarity of the third edge chain code; represents the second similarity of the fourth edge chain code; represents information entropy; Represents a mapping function, which is used to map the value in the brackets to the interval [0, 1].

8. The color restoration method of underwater fish detection images according to any one of claims 1 to 5, characterized in that: The method restores the color of each fish in the first time-series image and the second time-series image based on the color of the fish that is the shortest distance from the first camera (2) or the second camera (3), including: Based on the third edge chain code, obtaining a first fish image from the first time-series image; Based on the fourth edge chain code, acquiring a second fish image from the second time-series image; Based on the first fish image and the second fish image, a third fish image and a fourth fish image are acquired; the third fish image is a fish image corresponding to the minimum distance; the fourth fish image is another fish image of the first fish image and the second fish image that is different from the third fish image; Based on the third fish image, the fourth fish image is enlarged in proportion to obtain a fifth fish image; Based on the third fish image, the color of the fifth fish image is restored.

9. A computer-readable storage medium for color restoration of underwater fish detection images, characterized in that: It includes a reading module and a processing module, wherein: A reading module, used for acquiring underwater images of underwater fish; the underwater images include a first time-series image and a second time-series image; the time sequence of the first time-series image and the second time-series image is the same; the first time-series image is pre-acquired by a first camera (2); the second time-series image is pre-acquired by a second camera (3); the first camera (2) and the second camera (3) are arranged facing each other; A processing module, configured to obtain a plurality of first edge chain codes from the first time-series image based on an edge algorithm; and, obtaining a plurality of second edge chain codes from the second time series images based on an edge algorithm; And, based on the multiple first edge chain codes, a third edge chain code is obtained; the third edge chain code is any edge chain code among the first edge chain codes; And, based on the plurality of second edge chain codes, a fourth edge chain code is obtained; the fourth edge chain code is any edge chain code among the second edge chain codes; and, based on the third edge chain code and the fourth edge chain code, obtaining a first similarity; And, if the first similarity is greater than a first preset value, a first distance and a second distance are obtained; the first distance is the distance between the fish corresponding to the third edge chain code and the fourth edge chain code and the first camera (2); the second distance is the distance between the fish corresponding to the third edge chain code and the fourth edge chain code and the second camera (3); and, based on the first distance and the second distance, obtaining a minimum distance; And, based on the color of the fish that is the shortest distance from the first camera (2) or the second camera (3), the color of each fish in the first time-series image and the second time-series image is restored.

10. A color restoration device for underwater fish detection images, characterized in that: include: a reader for acquiring underwater images of underwater fish; The underwater image includes a first time-series image and a second time-series image; The time sequence of the first time-series image and the second time-series image is the same; the first time-series image is pre-acquired by a first camera (2); and the second time-series image is pre-acquired by a second camera (3); The first camera (2) and the second camera (3) are arranged facing each other; A processor, configured to obtain a plurality of first edge chain codes from the first time-series image based on an edge algorithm; and, obtaining a plurality of second edge chain codes from the second time series images based on an edge algorithm; And, based on the multiple first edge chain codes, a third edge chain code is obtained; the third edge chain code is any edge chain code among the first edge chain codes; And, based on the plurality of second edge chain codes, a fourth edge chain code is obtained; the fourth edge chain code is any edge chain code among the second edge chain codes; and, based on the third edge chain code and the fourth edge chain code, obtaining a first similarity; And, if the first similarity is greater than a first preset value, a first distance and a second distance are obtained; the first distance is the distance between the fish corresponding to the third edge chain code and the fourth edge chain code and the first camera (2); the second distance is the distance between the fish corresponding to the third edge chain code and the fourth edge chain code and the second camera (3); and, based on the first distance and the second distance, obtaining a minimum distance; And, based on the color of the fish that is the shortest distance from the first camera (2) or the second camera (3), the color of each fish in the first time-series image and the second time-series image is restored.

Images