A marine plankton automatic detection method, system, device and medium based on real-time image recognition

Abstract

The application discloses a kind of ocean plankton automatic detection method, system, equipment and medium based on real-time image recognition, mainly related to image processing technical field, to solve the problem that existing scheme adopts single frame independent analysis strategy, unable to effectively model the motion trajectory and morphological evolution law of plankton in time series, cannot dynamically adjust sensitivity according to image noise level.The application comprises: adjusting the amplitude of preset frequency domain detection signal using image noise level;Extract image frequency from image frame;Using adjusted preset frequency domain detection signal, image frequency, calculate preset plankton target existence;When preset plankton target existence is greater than preset value, output cropped plankton local feature subgraph;Plankton local feature subgraph is stacked as multiple frames of input according to time sequence to form time sequence feature sequence;Based on time sequence feature sequence, obtain plankton class probability using lightweight convolution network.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an automatic detection method, system, device and medium for marine plankton based on real-time image recognition. Background Technology

[0002] Current automated detection of marine plankton largely relies on static image processing techniques, primarily employing traditional image enhancement methods (such as Retinex and dark channel dehazing) to compensate for illumination in underwater microscopic images. However, these methods cannot achieve adaptive grayscale normalization based on water depth and spectral attenuation characteristics, resulting in severe contrast distortion and blurred texture details in deep-water areas, making it difficult to reliably extract target contours. Furthermore, feature extraction is often based on standard LBP operators or manually designed shape features, lacking robustness to the multi-angle rotational morphology of plankton, and prone to false responses under strong scattering and suspended particle interference, leading to a higher false detection rate.

[0003] Existing systems generally employ a single-frame independent analysis strategy, failing to effectively model the movement trajectories and morphological evolution patterns of plankton over time. This results in significant fluctuations in the identification results of the same species under different postures or lighting conditions, making it difficult to achieve continuous and stable category determination. Furthermore, traditional detection frameworks do not incorporate noise adaptation mechanisms; the frequency domain detection signal remains fixed, making it impossible to dynamically adjust sensitivity based on image noise levels. This further exacerbates classification failures in low signal-to-noise ratio environments, hindering the reliability and accuracy of in-situ real-time monitoring. Summary of the Invention

[0004] This application provides an automatic detection method, system, device, and medium for marine plankton based on real-time image recognition, to solve the problems of existing solutions that adopt a single-frame independent analysis strategy, which fails to effectively model the motion trajectory and morphological evolution of plankton in time series and cannot dynamically adjust the sensitivity according to the image noise level.

[0005] In a first aspect, this application provides an automatic detection method for marine plankton based on real-time image recognition, the method comprising: Real-time acquisition of in-situ underwater microscopic images of the ocean; elimination of depth illumination attenuation through adaptive grayscale normalization; output of homogenized images corresponding to the microscopic images; extraction of texture feature vectors from the homogenized images using an improved rotation-invariant LBP operator; construction of the covariance matrix corresponding to the texture feature vectors within a sliding window; obtaining the covariance matrix through feature decomposition to obtain image frames containing effective plankton in a preset high feature value region. A no-reference noise estimation method is used to calculate the image noise level of the image frame; the amplitude of the preset frequency domain detection signal is adjusted using the image noise level; the image frequency is extracted from the image frame; and the existence of the preset floating target is calculated using the adjusted preset frequency domain detection signal and the image frequency. When the existence of a pre-defined planktonic target is greater than a pre-defined value, a cropped local feature map of planktonic organisms is output. The local feature maps of planktonic organisms are stacked into multiple frames as input in time series to form a temporal feature sequence. Based on the temporal feature sequence, a lightweight convolutional network is used to obtain the probability of planktonic organism category.

[0006] In one implementation of this application, in-situ underwater microscopic images of the ocean are acquired in real time. Adaptive grayscale normalization is used to eliminate depth illumination attenuation, outputting a homogenized image corresponding to the microscopic image. An improved rotation-invariant LBP operator is used to extract texture feature vectors from the homogenized image, and a covariance matrix corresponding to the texture feature vectors is constructed within a sliding window. The covariance matrix is ​​obtained through eigenvalue decomposition to obtain image frames containing effective plankton in a preset high-eigenvalue region. Specifically, this includes: To eliminate depth-based illumination attenuation through adaptive grayscale normalization, the red light attenuation coefficient at different depths is first estimated based on a physical model of water light attenuation. Then, each grayscale image frame is divided into local regions, and the mean and variance of each block are calculated to construct a normalization function. ,in, Represents a homogenized image; This represents the original grayscale value of the image at pixel position (x, y). This represents the average gray value within a local region centered at (x, y). This represents the standard deviation of grayscale values ​​in a local area. This indicates a pre-defined minimum positive number. This represents the global grayscale mean at the reference depth. This represents the global grayscale standard deviation at the reference depth. Based on the homogenized image, an improved rotation-invariant LBP operator is used to perform binary encoding on the 8-neighborhood of each pixel, and the statistical equivalent pattern histogram is used as the texture feature vector. Construct the covariance matrix C within the sliding window: ; Where N represents the total number of pixels involved in the calculation within the sliding window. This represents the LBP local texture feature vector of the i-th pixel. Indicates all within the sliding window The mean vector; The largest eigenvalue of the covariance matrix is ​​obtained through eigenvalue decomposition. Based on the distribution characteristics of the principal eigenvalue, a dynamic threshold is set. ,in, The mean and standard deviation of the feature values ​​of the current frame sequence are given by k, where k represents a preset empirical coefficient. The regions corresponding to the largest feature values ​​less than the dynamic threshold are removed to obtain image frames containing effective plankton in the preset high feature value regions.

[0007] In one implementation of this application, the amplitude of a preset frequency domain detection signal is adjusted using the image noise level; the image frequency is extracted from the image frame; and the existence of a preset floating target is calculated using the adjusted preset frequency domain detection signal and the image frequency, specifically including: Perform a two-dimensional fast Fourier transform on the image frame to convert the spatial domain grayscale distribution into a spectrum; center the spectrum so that the zero-frequency component is located at the center of the image, and extract the image spectrum; The injected signal is a preset frequency domain detection signal with an amplitude of 1 / 10 of the image noise level and a frequency set to 120 Hz outside the background resonant frequency band; Through the formula: Calculate the existence of a pre-defined floating target; among which, Represents the image spectrum. This indicates the preset frequency domain detection signal. This indicates the existence of a pre-defined floating target.

[0008] In one implementation of this application, when the existence of a preset planktonic target is greater than a preset value, a cropped local feature sub-image of planktonic organisms is output, specifically including: Perform a two-dimensional fast Fourier transform on the current image frame to obtain the frequency domain response spectrum, and apply a notch filter in the background resonant frequency band to generate a binary mask. ;in, τ represents the preset value. Represents the image spectrum. This indicates the preset frequency domain detection signal. Indicates the existence of a pre-defined floating target; Using a binary mask as input, all isolated target regions are extracted using a connected component analysis algorithm. The minimum bounding rectangle boundary of each connected component is calculated. The connected component with the largest area and the center closest to the geometric center of the image is selected, and its boundary is extended to a preset fixed size and normalized into a local feature sub-image of plankton.

[0009] In one implementation of this application, local feature sub-images of plankton are stacked into multiple frames as input in a time sequence to form a temporal feature sequence; based on the temporal feature sequence, a lightweight convolutional network is used to obtain the probability of plankton category, specifically including: The local feature images of plankton are stacked into multiple frames as input in a time series. Each frame is processed by a pre-trained shallow feature extractor, which outputs a local feature vector of a preset dimension. The vector is then averaged along the spatial dimension to obtain a scalar feature value for each frame, thus forming a temporal feature sequence. The sequence is divided into N non-overlapping sub-blocks at equal intervals, with each block corresponding to T / N frames. Input the time-series feature sequence into a preset objective function: To obtain the smoothed feature sequence; in, This represents the mean of the i sub-values. λ is the preset smoothing weight The sub-value allocation boundary is optimized by gradient descent to minimize L; the optimization process uses the Adam algorithm, which starts from a preset initial equal division boundary, iteratively updates the boundary offset until convergence, and outputs the optimal sub-value sequence. The optimal sub-value sequence is input into a lightweight convolutional network as a one-dimensional temporal input. The temporal pattern is extracted by 1D convolution, and the dimensionality is reduced by global average pooling. Finally, the probability of planktonic category is output through a fully connected layer.

[0010] Secondly, this application provides an automatic detection system for marine plankton based on real-time image recognition, the system comprising: The acquisition module is used to acquire in-situ underwater microscopic images of the ocean in real time. It eliminates depth illumination attenuation through adaptive grayscale normalization and outputs a homogenized image corresponding to the microscopic image. An improved rotation-invariant LBP operator is used to extract texture feature vectors from the homogenized image and construct the covariance matrix corresponding to the texture feature vectors within a sliding window. The covariance matrix is ​​obtained through feature decomposition to obtain image frames containing effective plankton in the preset high feature value region. The calculation module is used to calculate the image noise level of an image frame using a no-reference noise estimation method; adjust the amplitude of a preset frequency domain detection signal using the image noise level; extract the image frequency from the image frame; and calculate the existence of a preset floating target using the adjusted preset frequency domain detection signal and the image frequency. The category module is used to output cropped local feature maps of planktonic organisms when the existence of a preset planktonic target is greater than a preset value. The local feature maps of planktonic organisms are stacked into multiple frames as input in time sequence to form a temporal feature sequence. Based on the temporal feature sequence, a lightweight convolutional network is used to obtain the probability of planktonic organism category.

[0011] In one implementation of this application, a module is obtained to eliminate depth illumination attenuation through adaptive grayscale normalization. First, based on a physical model of water light attenuation, the red light attenuation coefficient at different depths is estimated. Then, each frame of grayscale image is divided into local regions, and the mean and variance of each block are calculated to construct a normalization function. ,in, Represents a homogenized image; This represents the original grayscale value of the image at pixel position (x, y). This represents the average gray value within a local region centered at (x, y). This represents the standard deviation of grayscale values ​​in a local area. This indicates a pre-defined minimum positive number. This represents the global grayscale mean at the reference depth. This represents the global grayscale standard deviation at the reference depth. Based on the homogenized image, an improved rotation-invariant LBP operator is used to perform binary encoding on the 8-neighborhood of each pixel, and the statistical equivalent pattern histogram is used as the texture feature vector. Construct the covariance matrix C within the sliding window: ; Where N represents the total number of pixels involved in the calculation within the sliding window. This represents the LBP local texture feature vector of the i-th pixel. Indicates all within the sliding window The mean vector; The largest eigenvalue of the covariance matrix is ​​obtained through eigenvalue decomposition. Based on the distribution characteristics of the principal eigenvalue, a dynamic threshold is set. ,in, The mean and standard deviation of the feature values ​​of the current frame sequence are given by k, where k represents a preset empirical coefficient. The regions corresponding to the largest feature values ​​less than the dynamic threshold are removed to obtain image frames containing effective plankton in the preset high feature value regions.

[0012] In one implementation of this application, a calculation module is used to perform a two-dimensional fast Fourier transform on the image frame to convert the spatial domain grayscale distribution into a spectrum; the spectrum is centered so that the zero-frequency component is located at the center of the image, and the image spectrum is extracted. The injected signal is a preset frequency domain detection signal with an amplitude of 1 / 10 of the image noise level and a frequency set to 120 Hz outside the background resonant frequency band; Through the formula: Calculate the existence of a pre-defined floating target; among which, Represents the image spectrum. This indicates the preset frequency domain detection signal. This indicates the existence of a pre-defined floating target.

[0013] Thirdly, this application provides an automatic detection device for marine plankton based on real-time image recognition, the device comprising: processor; And a memory containing executable code, which, when executed, causes the processor to perform an automatic detection method for marine plankton based on real-time image recognition, as described above.

[0014] Fourthly, this application provides a non-volatile computer storage medium storing computer instructions, which, when executed, implement an automatic detection method for marine plankton based on real-time image recognition as described above.

[0015] As can be seen from the above technical solutions, this application has the following advantages: By introducing an adaptive grayscale normalization mechanism, the brightness distribution of underwater microscopic images is dynamically corrected according to water depth and spectral attenuation characteristics. This eliminates uneven illumination and contrast distortion caused by depth variations, allowing the outline details of plankton in deep water areas to be clearly preserved, thus improving image uniformity and structural discernibility.

[0016] By employing an improved rotation-invariant LBP operator combined with covariance matrix eigenvalue decomposition, robust texture representation of multi-angle morphology of plankton is achieved, suppressing false responses caused by suspended particles and strong scattering interference. By extracting high eigenvalue regions to construct effective image frames, the true target region is focused, reducing the false detection rate and improving the reliability of feature extraction and the accuracy of target localization in complex aquatic environments.

[0017] The system calculates the image noise level in real time using a no-reference noise estimation method and dynamically adjusts the amplitude of the frequency domain detection signal accordingly. This allows the system sensitivity to adaptively match the current signal-to-noise ratio environment, avoiding classification failure caused by a fixed threshold under low signal-to-noise ratio. At the same time, local feature sub-images are stacked into a temporal feature sequence according to the time series. A lightweight convolutional network is then used to model the motion trajectory and morphological evolution of plankton, achieving continuous and stable class probability output across frames. Attached Figure Description

[0018] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart of an automatic detection method for marine plankton based on real-time image recognition, provided in an embodiment of this application.

[0020] Figure 2 This is a schematic diagram of the internal structure of an automatic marine plankton detection system based on real-time image recognition, provided in an embodiment of this application.

[0021] Figure 3 This is a schematic diagram of the internal structure of an automatic marine plankton detection device based on real-time image recognition, provided in an embodiment of this application. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Those skilled in the art should understand that the embodiments described below are merely preferred embodiments of this disclosure and do not imply that this disclosure can only be implemented through these preferred embodiments. These preferred embodiments are merely used to explain the technical principles of this disclosure and are not intended to limit the scope of protection of this disclosure. Based on the preferred embodiments provided by this disclosure, all other embodiments obtained by those skilled in the art without creative effort should still fall within the scope of protection of this disclosure.

[0024] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0025] The technical solutions proposed in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0026] The embodiment provides an automatic detection method for marine plankton based on real-time image recognition, such as Figure 1 As shown in the embodiments of this application, the method mainly includes the following steps: Step 110: Real-time acquisition of in-situ underwater microscopic images of the ocean; elimination of depth illumination attenuation through adaptive grayscale normalization; output of homogenized images corresponding to the microscopic images; extraction of texture feature vectors from the homogenized images using an improved rotation-invariant LBP operator; construction of the covariance matrix corresponding to the texture feature vectors within a sliding window; obtaining the covariance matrix through feature decomposition to obtain image frames containing effective plankton in preset high eigenvalue regions.

[0027] In some embodiments, the specific components include: To eliminate depth-based illumination attenuation through adaptive grayscale normalization, the red light attenuation coefficient at different depths is first estimated based on a physical model of water light attenuation. Then, each grayscale image frame is divided into local regions, and the mean and variance of each block are calculated to construct a normalization function. ,in, Represents a homogenized image; This represents the original grayscale value of the image at pixel position (x, y). This represents the average gray value within a local region centered at (x, y). This represents the standard deviation of grayscale values ​​in a local area. This indicates a pre-defined minimum positive number. This represents the global grayscale mean at the reference depth. This represents the global grayscale standard deviation at the reference depth. Based on the homogenized image, an improved rotation-invariant LBP operator is used to perform binary encoding on the 8-neighborhood of each pixel, and the statistical equivalent pattern histogram is used as the texture feature vector. Construct the covariance matrix C within the sliding window: ; Where N represents the total number of pixels involved in the calculation within the sliding window. This represents the LBP local texture feature vector of the i-th pixel. Indicates all within the sliding window The mean vector; The largest eigenvalue of the covariance matrix is ​​obtained through eigenvalue decomposition. Based on the distribution characteristics of the principal eigenvalue, a dynamic threshold is set. ,in, The mean and standard deviation of the feature values ​​of the current frame sequence are given by k, where k represents a preset empirical coefficient. The regions corresponding to the largest feature values ​​less than the dynamic threshold are removed to obtain image frames containing effective plankton in the preset high feature value regions.

[0028] It should be added that, in the adaptive grayscale normalization process, each frame of image is divided into 8×8 pixel non-overlapping blocks. Based on the red light attenuation coefficient estimated by the water body light attenuation model, the local mean and standard deviation of each block are calculated. Combined with the global statistics at the reference depth, the normalization mapping is completed to ensure that the grayscale distribution of different depth regions tends to be consistent.

[0029] In the improved rotation-invariant LBP operator, a circular neighborhood with a radius of 1 and 8 sampling points is used to perform binary encoding on the neighborhood grayscale of each pixel, and rotation-invariant encoding is obtained through the rotation minimum method. Only 58 equivalent patterns with no more than two 0-1 transitions in the binary mode are counted to construct texture feature vectors. The sliding window size is set to 16×16 pixels with a step size of 8 pixels, and the image is traversed in a 50% overlap manner to calculate the covariance matrix of the LBP feature vectors of all pixels in the window. The maximum eigenvalue is obtained through feature decomposition, and a dynamic threshold is set to the mean of the feature values ​​of the current frame sequence minus 1.5 times the standard deviation. Regions below this threshold are removed. Then, the opening operation of 3×3 circular structuring elements is performed on the remaining binary regions to remove isolated noise points. Then, 8-neighborhood connected component analysis is performed, and only connected regions with an area greater than 15 pixels are retained as valid planktonic image frames.

[0030] Step 120: Calculate the image noise level of the image frame using a no-reference noise estimation method; adjust the amplitude of the preset frequency domain detection signal using the image noise level; extract the image frequency from the image frame; and calculate the existence of the preset floating target using the adjusted preset frequency domain detection signal and the image frequency.

[0031] In some embodiments, the amplitude of a preset frequency domain detection signal is adjusted using the image noise level; the image frequency is extracted from the image frame; and the existence of a preset floating target is calculated using the adjusted preset frequency domain detection signal and the image frequency. Specifically, this can be as follows: Perform a two-dimensional fast Fourier transform on the image frame to convert the spatial domain grayscale distribution into a spectrum; center the spectrum so that the zero-frequency component is located at the center of the image, and extract the image spectrum; The injected signal is a preset frequency domain detection signal with an amplitude of 1 / 10 of the image noise level and a frequency set to 120 Hz outside the background resonant frequency band; Through the formula: Calculate the existence of a pre-defined floating target; among which, Represents the image spectrum. This indicates the preset frequency domain detection signal. This indicates the existence of a pre-defined floating target.

[0032] It should be noted that in the no-reference noise estimation, the edge region after the image spectrum is centered (outside 15 pixels from the center and outside the 45° fan-shaped area in the corner) is selected as the noise-dominant region. The median of the amplitude of all frequency points in this region is calculated as the estimated value of the image noise level to avoid local energy interference caused by the target structure.

[0033] In the frequency domain detection signal injection stage, a pure sine wave with an amplitude of 1 / 10 of the noise level is injected at the radial position of 120 Hz of the centered spectrum (corresponding to the 64th pixel in the horizontal direction and the 128th pixel in the vertical direction, with an image size of 512×512 pixels). This position is located outside the main energy distribution area (low-frequency core area) of the image and avoids the known water background resonant frequency band (80–100 Hz). After injection, the rest of the spectrum remains unchanged, and a single frequency component is superimposed only at the specified coordinate point.

[0034] In determining the existence of a pre-defined floating target, the amplitude of all frequency points within a 3×3 neighborhood is extracted with the injected signal location as the center, and the median is calculated as the local noise baseline. If the amplitude of the injected point exceeds twice the median, and the point is an isolated peak in the spectrum (with no other points in its 8 neighborhood whose amplitude exceeds 1.5 times the baseline), then the pre-defined floating target is determined to exist; otherwise, it is determined to be noise fluctuation.

[0035] Step 130: When the existence of the preset planktonic target is greater than the preset value, output the cropped planktonic local feature sub-image; stack the planktonic local feature sub-images into multiple frames as input in time series to form a temporal feature sequence; based on the temporal feature sequence, use a lightweight convolutional network to obtain the planktonic category probability.

[0036] In some embodiments, when the presence of a preset planktonic target is greater than a preset value, a cropped local feature sub-image of planktonic organisms is output, specifically including: Perform a two-dimensional fast Fourier transform on the current image frame to obtain the frequency domain response spectrum, and apply a notch filter in the background resonant frequency band to generate a binary mask. ;in, τ represents the preset value. Represents the image spectrum. This indicates the preset frequency domain detection signal. Indicates the existence of a pre-defined floating target; Using a binary mask as input, all isolated target regions are extracted using a connected component analysis algorithm. The minimum bounding rectangle boundary of each connected component is calculated. The connected component with the largest area and the center closest to the geometric center of the image is selected, and its boundary is extended to a preset fixed size and normalized into a local feature sub-image of plankton.

[0037] In some embodiments, local feature sub-images of plankton are stacked into multiple frames as input in a time series manner to form a temporal feature sequence; based on the temporal feature sequence, a lightweight convolutional network is used to obtain the probability of plankton category, specifically including: The local feature images of plankton are stacked into multiple frames as input in a time series. Each frame is processed by a pre-trained shallow feature extractor, which outputs a local feature vector of a preset dimension. The vector is then averaged along the spatial dimension to obtain a scalar feature value for each frame, thus forming a temporal feature sequence. The sequence is divided into N non-overlapping sub-blocks at equal intervals, with each block corresponding to T / N frames. Input the time-series feature sequence into a preset objective function: To obtain the smoothed feature sequence; in, This represents the mean of the i sub-values. λ is the preset smoothing weight The sub-value allocation boundary is optimized by gradient descent to minimize L; the optimization process uses the Adam algorithm, which starts from a preset initial equal division boundary, iteratively updates the boundary offset until convergence, and outputs the optimal sub-value sequence. The optimal sub-value sequence is input into a lightweight convolutional network as a one-dimensional temporal input. The temporal pattern is extracted by 1D convolution, and the dimensionality is reduced by global average pooling. Finally, the probability of planktonic category is output through a fully connected layer.

[0038] It should be noted that when the existence of the pre-defined planktonic target exceeds the pre-defined threshold, the binary mask of the current image frame is used as input to perform 8-neighborhood connected component analysis to extract all isolated target regions; the minimum bounding rectangle of each connected component is calculated, and the region with the largest area and a center coordinate Euclidean distance of less than 30 pixels from the geometric center of the image is selected; the boundary of this region is extended outward by 16 pixels (truncated if it is insufficient to the image boundary), and after cropping, it is uniformly scaled to 64×64 pixels, and the grayscale value is normalized to the [0,255] interval. The output is a local feature sub-image of planktonic organisms.

[0039] Continuously collected local feature images of plankton are stacked in chronological order to form a temporal sequence of up to 16 frames. Each frame is input into a pre-trained ShuffleNetV2x0.5 network, and its penultimate layer's 128-dimensional feature vector is extracted. Global average pooling is then performed along the spatial dimension to obtain a scalar value for each frame, forming a 16-dimensional temporal feature sequence. This sequence is divided into four non-overlapping sub-blocks, each containing four frames. The median value within each sub-block is used to replace the corresponding frame value in the original sequence, completing temporal smoothing. The smoothed 16-dimensional sequence is then used as a one-dimensional input and fed into a lightweight 1D convolutional network containing three 1D convolutional layers (kernel sizes of 5, 3, and 3, and channel numbers of 16, 32, and 64, respectively). Each layer is followed by ReLU and max pooling (pooling window of 2). Finally, global average pooling is used to reduce the dimensionality to 64 dimensions, and the softmax probability distribution of six common plankton species is output through a fully connected layer.

[0040] In addition, this application Figure 2 This application provides an automatic detection system for marine plankton based on real-time image recognition. For example... Figure 2 As shown in the embodiments of this application, the system mainly includes: The acquisition module 210 is used to acquire in-situ underwater microscopic images of the ocean in real time. It eliminates depth illumination attenuation through adaptive grayscale normalization and outputs a homogenized image corresponding to the microscopic image. It uses an improved rotation-invariant LBP operator to extract texture feature vectors from the homogenized image and constructs the covariance matrix corresponding to the texture feature vectors within a sliding window. It obtains the covariance matrix through feature decomposition and obtains an image frame containing effective plankton in a preset high feature value region.

[0041] Module 210 is used to eliminate depth illumination attenuation through adaptive grayscale normalization. First, based on a physical model of water light attenuation, the red light attenuation coefficient at different depths is estimated. Then, each grayscale image frame is divided into local regions, and the mean and variance of each block are calculated to construct a normalization function. ,in, Represents a homogenized image; This represents the original grayscale value of the image at pixel position (x, y). This represents the average gray value within a local region centered at (x, y). This represents the standard deviation of grayscale values ​​in a local area. This indicates a pre-defined minimum positive number. This represents the global grayscale mean at the reference depth. This represents the global grayscale standard deviation at the reference depth. Based on the homogenized image, an improved rotation-invariant LBP operator is used to perform binary encoding on the 8-neighborhood of each pixel, and the statistical equivalent pattern histogram is used as the texture feature vector. Construct the covariance matrix C within the sliding window: ; Where N represents the total number of pixels involved in the calculation within the sliding window. This represents the LBP local texture feature vector of the i-th pixel. Indicates all within the sliding window The mean vector; The largest eigenvalue of the covariance matrix is ​​obtained through eigenvalue decomposition. Based on the distribution characteristics of the principal eigenvalue, a dynamic threshold is set. ,in, The mean and standard deviation of the feature values ​​of the current frame sequence are given by k, where k represents a preset empirical coefficient. The regions corresponding to the largest feature values ​​less than the dynamic threshold are removed to obtain image frames containing effective plankton in the preset high feature value regions.

[0042] The calculation module 220 is used to calculate the image noise level of the image frame using a no-reference noise estimation method; adjust the amplitude of the preset frequency domain detection signal using the image noise level; extract the image frequency from the image frame; and calculate the existence of the preset floating target using the adjusted preset frequency domain detection signal and the image frequency.

[0043] The calculation module 220 is used to perform a two-dimensional fast Fourier transform on the image frame, converting the spatial domain grayscale distribution into a spectrum; and to perform a centering process on the spectrum so that the zero-frequency component is located at the center of the image, thereby extracting the image spectrum. The injected signal is a preset frequency domain detection signal with an amplitude of 1 / 10 of the image noise level and a frequency set to 120 Hz outside the background resonant frequency band; Through the formula: Calculate the existence of a pre-defined floating target; among which, Represents the image spectrum. This indicates the preset frequency domain detection signal. This indicates the existence of a pre-defined floating target.

[0044] Category module 230 is used to output cropped local feature sub-images of planktonic organisms when the existence of a preset planktonic target is greater than a preset value; the local feature sub-images of planktonic organisms are stacked into multiple frames as input in time sequence to form a temporal feature sequence; based on the temporal feature sequence, a lightweight convolutional network is used to obtain the probability of planktonic organism category.

[0045] The above are method embodiments of this application. Based on the same inventive concept, this application also provides an automatic detection device for marine plankton based on real-time image recognition. Figure 3 As shown, the device includes: a processor; and a memory storing executable code thereon, which, when executed, causes the processor to perform an automatic detection method for marine plankton based on real-time image recognition as described in the above embodiments.

[0046] Specifically, the server acquires in-situ underwater microscopic images of the ocean in real time. Adaptive grayscale normalization is used to eliminate depth illumination attenuation, outputting a homogenized image corresponding to the microscopic image. An improved rotation-invariant LBP operator is used to extract texture feature vectors from the homogenized image, and a covariance matrix corresponding to the texture feature vectors is constructed within a sliding window. The covariance matrix is ​​obtained through feature decomposition to acquire image frames containing valid plankton in a preset high-eigenvalue region. A no-reference noise estimation method is used to calculate the image noise level of the image frame. The amplitude of a preset frequency domain detection signal is adjusted using the image noise level. Image frequencies are extracted from the image frame. The presence of a preset plankton target is calculated using the adjusted preset frequency domain detection signal and image frequencies. When the presence of the preset plankton target is greater than a preset value, a cropped local feature map of plankton is output. The local feature maps of plankton are stacked in time series as multi-frame input to form a temporal feature sequence. Based on the temporal feature sequence, a lightweight convolutional network is used to obtain the probability of plankton category. In addition, this application embodiment also provides a non-volatile computer storage medium storing executable instructions, which, when executed, implement the above-described automatic detection method for marine plankton based on real-time image recognition.

[0047] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An automatic detection method for marine plankton based on real-time image recognition, characterized in that, The method includes: Real-time acquisition of in-situ underwater microscopic images of the ocean; elimination of depth illumination attenuation through adaptive grayscale normalization; output of homogenized images corresponding to the microscopic images; extraction of texture feature vectors from the homogenized images using an improved rotation-invariant LBP operator; construction of the covariance matrix corresponding to the texture feature vectors within a sliding window; obtaining the covariance matrix through feature decomposition to obtain image frames containing effective plankton in a preset high feature value region. A no-reference noise estimation method is used to calculate the image noise level of the image frame; the amplitude of the preset frequency domain detection signal is adjusted using the image noise level; the image frequency is extracted from the image frame; and the existence of the preset floating target is calculated using the adjusted preset frequency domain detection signal and the image frequency. When the existence of a pre-defined planktonic target is greater than a pre-defined value, a cropped local feature map of planktonic organisms is output. The local feature maps of planktonic organisms are stacked into multiple frames as input in time series to form a temporal feature sequence. Based on the temporal feature sequence, a lightweight convolutional network is used to obtain the probability of planktonic organism category.

2. The automatic detection method for marine plankton based on real-time image recognition according to claim 1, characterized in that, Real-time acquisition of in-situ underwater microscopic images of the ocean; elimination of depth illumination attenuation through adaptive grayscale normalization; output of homogenized images corresponding to the microscopic images; extraction of texture feature vectors from homogenized images using an improved rotation-invariant LBP operator; construction of the covariance matrix corresponding to the texture feature vectors within a sliding window. Image frames containing valid plankton in a predetermined high eigenvalue region are obtained by calculating the covariance matrix through eigenvalue decomposition. Specifically, this includes: To eliminate depth-based illumination attenuation through adaptive grayscale normalization, the red light attenuation coefficient at different depths is first estimated based on a physical model of water light attenuation. Then, each grayscale image frame is divided into local regions, and the mean and variance of each block are calculated to construct a normalization function. ,in, Represents a homogenized image; This represents the original grayscale value of the image at pixel position (x, y). This represents the average gray value within a local region centered at (x,y). This represents the standard deviation of grayscale values ​​in a local area. This indicates a pre-defined minimum positive number. This represents the global grayscale mean at the reference depth. This represents the global grayscale standard deviation at the reference depth. Based on the homogenized image, an improved rotation-invariant LBP operator is used to perform binary encoding on the 8-neighborhood of each pixel, and the statistical equivalent pattern histogram is used as the texture feature vector. Construct the covariance matrix C within the sliding window: ； Where N represents the total number of pixels involved in the calculation within the sliding window. This represents the LBP local texture feature vector of the i-th pixel. Indicates all within the sliding window The mean vector; The largest eigenvalue of the covariance matrix is ​​obtained through eigenvalue decomposition. Based on the distribution characteristics of the principal eigenvalue, a dynamic threshold is set. ,in, The mean and standard deviation of the feature values ​​of the current frame sequence are given by k, where k represents a preset empirical coefficient. The regions corresponding to the largest feature values ​​less than the dynamic threshold are removed to obtain image frames containing effective plankton in the preset high feature value regions.

3. The automatic detection method for marine plankton based on real-time image recognition according to claim 1, characterized in that, Adjust the amplitude of the preset frequency domain detection signal based on the image noise level; extract the image frequency from the image frame; Using the adjusted preset frequency domain detection signal and image frequency, the existence of the preset floating target is calculated, specifically including: Perform a two-dimensional fast Fourier transform on the image frame to convert the spatial domain grayscale distribution into a spectrum; center the spectrum so that the zero-frequency component is located at the center of the image, and extract the image spectrum; The injected signal is a preset frequency domain detection signal with an amplitude of 1 / 10 of the image noise level and a frequency set to 120 Hz outside the background resonant frequency band; Through the formula: Calculate the existence of a pre-defined floating target; among which, Represents the image spectrum. This indicates the preset frequency domain detection signal. This indicates the existence of a pre-defined floating target.

4. The automatic detection method for marine plankton based on real-time image recognition according to claim 1, characterized in that, When the pre-defined presence of planktonic targets is greater than a pre-defined value, the cropped local feature sub-image of planktonic organisms is output, specifically including: Perform a two-dimensional fast Fourier transform on the current image frame to obtain the frequency domain response spectrum, and apply a notch filter in the background resonant frequency band to generate a binary mask. ;in, τ represents the preset value. Represents the image spectrum. This indicates the preset frequency domain detection signal. Indicates the existence of a pre-defined floating target; Using a binary mask as input, all isolated target regions are extracted using a connected component analysis algorithm. The minimum bounding rectangle boundary of each connected component is calculated. The connected component with the largest area and the center closest to the geometric center of the image is selected, and its boundary is extended to a preset fixed size and normalized into a local feature sub-image of plankton.

5. The automatic detection method for marine plankton based on real-time image recognition according to claim 1, characterized in that, The local feature sub-images of planktonic organisms are stacked into multiple frames of input in a time series to form a temporal feature sequence; Based on temporal feature sequences, a lightweight convolutional network is used to obtain the probability of planktonic organism categories, specifically including: The local feature images of plankton are stacked into multiple frames as input in a time series. Each frame is processed by a pre-trained shallow feature extractor, which outputs a local feature vector of a preset dimension. The vector is then averaged along the spatial dimension to obtain a scalar feature value for each frame, thus forming a temporal feature sequence. The sequence is divided into N non-overlapping sub-blocks at equal intervals, with each block corresponding to T / N frames. Input the time-series feature sequence into a preset objective function: To obtain the smoothed feature sequence; in, This represents the mean of the i sub-values. λ is the preset smoothing weight The sub-value allocation boundary is optimized by gradient descent to minimize L; the optimization process uses the Adam algorithm, which starts from a preset initial equal division boundary, iteratively updates the boundary offset until convergence, and outputs the optimal sub-value sequence. The optimal sub-value sequence is input into a lightweight convolutional network as a one-dimensional temporal input. The temporal pattern is extracted by 1D convolution, and the dimensionality is reduced by global average pooling. Finally, the probability of planktonic category is output through a fully connected layer.

6. An automatic detection system for marine plankton based on real-time image recognition, characterized in that, The system includes: The acquisition module is used to acquire in-situ underwater microscopic images of the ocean in real time. It eliminates depth illumination attenuation through adaptive grayscale normalization and outputs a homogenized image corresponding to the microscopic image. An improved rotation-invariant LBP operator is used to extract texture feature vectors from the homogenized image and construct the covariance matrix corresponding to the texture feature vectors within a sliding window. The covariance matrix is ​​obtained through feature decomposition to obtain image frames containing effective plankton in the preset high feature value region. The calculation module is used to calculate the image noise level of an image frame using a no-reference noise estimation method; adjust the amplitude of a preset frequency domain detection signal using the image noise level; extract the image frequency from the image frame; and calculate the existence of a preset floating target using the adjusted preset frequency domain detection signal and the image frequency. The category module is used to output cropped local feature maps of planktonic organisms when the existence of a preset planktonic target is greater than a preset value. The local feature maps of planktonic organisms are stacked into multiple frames as input in time sequence to form a temporal feature sequence. Based on the temporal feature sequence, a lightweight convolutional network is used to obtain the probability of planktonic organism category.

7. The automatic marine plankton detection system based on real-time image recognition according to claim 6, characterized in that, The module obtains information to eliminate depth-based illumination attenuation through adaptive grayscale normalization. First, it estimates the red light attenuation coefficient at different depths based on a physical model of water light attenuation. Then, it divides each grayscale image frame into local regions, calculates the mean and variance of each block, and constructs a normalization function. ,in, Represents a homogenized image; This represents the original grayscale value of the image at pixel position (x, y). This represents the average gray value within a local region centered at (x,y). This represents the standard deviation of grayscale values ​​in a local area. This indicates a pre-defined minimum positive number. This represents the global grayscale mean at the reference depth. This represents the global grayscale standard deviation at the reference depth. Based on the homogenized image, an improved rotation-invariant LBP operator is used to perform binary encoding on the 8-neighborhood of each pixel, and the statistical equivalent pattern histogram is used as the texture feature vector. Construct the covariance matrix C within the sliding window: ； Where N represents the total number of pixels involved in the calculation within the sliding window. This represents the LBP local texture feature vector of the i-th pixel. Indicates all within the sliding window The mean vector; The largest eigenvalue of the covariance matrix is ​​obtained through eigenvalue decomposition. Based on the distribution characteristics of the principal eigenvalue, a dynamic threshold is set. ,in, The mean and standard deviation of the feature values ​​of the current frame sequence are given by k, where k represents a preset empirical coefficient. The regions corresponding to the largest feature values ​​less than the dynamic threshold are removed to obtain image frames containing effective plankton in the preset high feature value regions.

8. The automatic marine plankton detection system based on real-time image recognition according to claim 6, characterized in that, The calculation module is used to perform a two-dimensional fast Fourier transform on the image frame, converting the spatial domain grayscale distribution into a spectrum; and to center the spectrum so that the zero-frequency component is located at the center of the image, thereby extracting the image spectrum. The injected signal is a preset frequency domain detection signal with an amplitude of 1 / 10 of the image noise level and a frequency set to 120 Hz outside the background resonant frequency band; Through the formula: Calculate the existence of a pre-defined floating target; among which, Represents the image spectrum. This indicates the preset frequency domain detection signal. This indicates the existence of a pre-defined floating target.

9. An automatic detection device for marine plankton based on real-time image recognition, characterized in that, The device includes: processor; And a memory having executable code stored thereon, which, when executed, causes the processor to perform an automatic detection method for marine plankton based on real-time image recognition as described in any one of claims 1-5.

10. A non-volatile computer storage medium, characterized in that, It stores computer instructions, which, when executed, implement an automatic detection method for marine plankton based on real-time image recognition as described in any one of claims 1-5.

Images