Water body rare species detection method and system based on underwater image analysis

Abstract

The present application relates to image analysis technology, disclose a kind of water rare biological detection method and system based on underwater image analysis, comprising: monitoring target water area in preset time period in light signal data stream and spectrum data stream, identify the light signal data stream light emitting event, obtain light emitting event timestamp sequence, based on light emitting event timestamp sequence obtain light emitting image sequence in light signal data stream and spectrum data sequence in spectrum data stream, to light emitting image sequence and spectrum data sequence is carried out denoising correction and space-time synchronization processing, obtain after alignment image block sequence and after alignment spectrum curve, according to after alignment spectrum curve constructs spectrum attention weight, to after alignment image block sequence is carried out multi-scale morphological feature extraction, obtain multi-scale morphological feature map, to multi-scale morphological feature map is carried out rare biological target detection, obtain target detection result.The present application can improve the accuracy of water rare biological detection.

Description

Technical Field

[0001] This invention relates to the field of image analysis technology, and in particular to a method and system for detecting rare aquatic organisms based on underwater image analysis. Background Technology

[0002] The monitoring and identification of rare underwater organisms is of vital importance for marine research, ecosystem assessment, and the conservation of rare species. However, due to the absorption and scattering effects of water on light, the complex and variable underwater lighting environment, and the low frequency of occurrence, diverse morphologies, and the fact that rare organisms often exhibit camouflage or mimicry, vision-based detection of rare underwater organisms is an extremely challenging task.

[0003] Currently, mainstream underwater biological detection methods mainly rely on the following technical routes, but all of them have obvious limitations. For example, detection methods based on active light source illumination and high-definition cameras, detection methods based on pure visual deep learning models, and technologies based on sonar or environmental DNA are all problematic. These methods struggle to achieve effective, reliable, and real-time detection of rare underwater organisms without disturbing the organisms, with low energy consumption and high accuracy. Therefore, there is an urgent need in this field for an innovative detection solution that can overcome the aforementioned shortcomings. Summary of the Invention

[0004] This invention provides a method and system for detecting rare aquatic organisms based on underwater image analysis, the main purpose of which is to solve the problem of low accuracy in existing methods for detecting rare aquatic organisms.

[0005] To achieve the above objectives, the present invention provides a method for detecting rare aquatic organisms based on underwater image analysis, comprising:

[0006] The photon flux and spectral information of the target water area within a preset time period are monitored using a preset low-light imager and spectrometer to obtain optical signal data stream and spectral data stream;

[0007] Based on a preset threshold, the emission events of the optical signal data stream are identified to obtain a timestamp sequence of emission events;

[0008] Based on the timestamp sequence of the emission events, obtain the emission image sequence in the optical signal data stream and the spectral data sequence in the spectral data stream;

[0009] The luminescent image sequence and the spectral data sequence are subjected to denoising correction and spatiotemporal synchronization processing to obtain an aligned image block sequence and an aligned spectral curve.

[0010] Based on the aligned spectral curves, spectral attention weights are constructed, and multi-scale morphological features are extracted from the aligned image patch sequence to obtain multi-scale morphological feature maps.

[0011] Rare biological targets are detected based on the spectral attention weights of the multi-scale morphological feature map, and the target detection results are obtained.

[0012] Optionally, the step of using a preset low-light imager and spectrometer to monitor the photon flux and spectral information of the target water area within a preset time period to obtain optical signal data stream and spectral data stream includes:

[0013] The low-light imager continuously acquires images at a preset acquisition frequency to obtain an image data stream;

[0014] The image data stream is converted into a grayscale image to obtain a grayscale image data stream;

[0015] Based on pixel intensity calculation, the photon flux of each frame in the grayscale image data stream is identified to obtain the optical signal data stream;

[0016] The spectrometer continuously acquires spectra at a preset acquisition frequency to obtain a spectral sequence;

[0017] Calculate the signal-to-noise ratio of the spectral sequence, and identify low-quality data in the spectral sequence based on a preset signal-to-noise ratio threshold;

[0018] The low-quality data in the spectral sequence are averaged to obtain a spectral data stream.

[0019] Optionally, the step of identifying the emission events of the optical signal data stream based on a preset threshold to obtain the emission event timestamp sequence includes:

[0020] The optical signal data stream is smoothed and denoised to obtain a smoothed optical signal data stream.

[0021] Based on the pre-acquired background light data and the smoothed light signal data stream, a dynamic threshold is calculated to obtain a dynamic threshold sequence.

[0022] The smoothed optical signal data stream is used to identify luminescence events using the dynamic threshold sequence to obtain a luminescence event timestamp sequence.

[0023] Optionally, the step of calculating a dynamic threshold based on pre-acquired background light data and the smoothed light signal data stream to obtain a dynamic threshold sequence includes:

[0024] Background light estimation is performed based on the smoothed optical signal data stream to obtain a real-time background light estimation sequence;

[0025] Calculate the standard deviation of the background light data to obtain a background light standard deviation sequence;

[0026] Each standard deviation in the background light standard deviation sequence is multiplied by a preset sensitivity coefficient to obtain the product result;

[0027] Each product result is added to each corresponding background light estimate in the real-time background light estimate sequence to obtain the dynamic threshold sequence.

[0028] Optionally, obtaining the emission image sequence in the optical signal data stream and the spectral data sequence in the spectral data stream based on the emission event timestamp sequence includes:

[0029] Confirm the time extraction range based on the preset time window parameters;

[0030] Construct a data extraction time window based on the time extraction range;

[0031] High-resolution image data from the optical signal data stream is extracted using the data extraction time window and the luminescence event timestamp sequence to obtain a luminescence image sequence.

[0032] High-resolution spectral data is extracted from the spectral data stream using the data extraction time window and the luminescence event timestamp sequence to obtain a spectral data sequence.

[0033] Optionally, the step of performing denoising correction and spatiotemporal synchronization processing on the luminescent image sequence and the spectral data sequence to obtain an aligned image patch sequence and an aligned spectral curve includes:

[0034] The luminescent image sequence is subjected to mean filtering to obtain a preliminary denoised image sequence;

[0035] The preliminary denoised image sequence is subjected to image flat-field correction to obtain the corrected image sequence;

[0036] The baseline of the spectral data sequence is corrected using the asymmetric least squares method to obtain the corrected spectral sequence.

[0037] The corrected image sequence and the corrected spectral sequence are time-synchronized and aligned to obtain an aligned image block sequence and an aligned spectral curve.

[0038] Optionally, constructing spectral attention weights based on the aligned spectral curves includes:

[0039] Extract the spectral features of the aligned spectral curve to obtain the spectral feature vector;

[0040] The spectral feature vector is matched with a preset rare biological spectral library to obtain a similarity score;

[0041] An initial spectral response scalar is generated based on the similarity score;

[0042] The initial spectral response scalar is mapped to an initial spectral attention weight map;

[0043] The initial spectral attention weight map is weighted and formatted for output to obtain the spectral attention weights.

[0044] Optionally, the step of extracting multi-scale morphological features from the aligned image patch sequence to obtain a multi-scale morphological feature map includes:

[0045] The aligned image block sequence is then normalized to obtain a normalized image block sequence.

[0046] Extract the low-level feature map of the standardized image patch sequence, and perform contour enhancement on the low-level feature map to obtain a contour-enhanced feature map;

[0047] The intermediate feature map of the standardized image patch sequence is extracted, and the intermediate feature map is upsampled to obtain the sampled feature map;

[0048] The sampled feature map and the contour-enhanced feature map are added element-wise to obtain a multi-scale fused feature map;

[0049] A multi-scale morphological feature map is generated based on the multi-scale fusion feature map.

[0050] Optionally, the step of performing rare biological target detection on the multi-scale morphological feature map based on the spectral attention weights to obtain target detection results includes:

[0051] The spectral attention weights and the multi-scale morphological feature maps are fused using feature weighting to obtain a weighted multi-scale feature map.

[0052] A lightweight region proposal network is used to generate a list of region proposals based on the weighted multi-scale feature map;

[0053] Based on the region proposal list, the weighted multi-scale feature map is aligned and classified with regions of interest to obtain region of interest feature vectors.

[0054] Based on the feature vector of the region of interest, target classification and bounding box regression are performed to obtain the target detection result.

[0055] To address the aforementioned problems, the present invention also provides a rare aquatic organism detection system based on underwater image analysis, the system comprising:

[0056] The data acquisition module is used to monitor the photon flux and spectral information of the target water area within a preset time period using a preset low-light imager and spectrometer, and to obtain optical signal data stream and spectral data stream.

[0057] The sequence data recognition module is used to identify the emission events of the optical signal data stream based on a preset threshold, obtain the emission event timestamp sequence, and obtain the emission image sequence in the optical signal data stream and the spectral data sequence in the spectral data stream based on the emission event timestamp sequence.

[0058] The noise reduction and correction module is used to perform noise reduction and correction and spatiotemporal synchronization processing on the luminescent image sequence and the spectral data sequence to obtain an aligned image block sequence and an aligned spectral curve.

[0059] The feature extraction module is used to construct spectral attention weights based on the aligned spectral curves, and to perform multi-scale morphological feature extraction on the aligned image patch sequence to obtain a multi-scale morphological feature map.

[0060] The target detection module is used to perform rare biological target detection on the multi-scale morphological feature map based on the spectral attention weights, and obtain the target detection result.

[0061] This invention utilizes a pre-set low-light imager and spectrometer to monitor photon flux and spectral information of a target water area within a preset time period, obtaining optical signal data streams and spectral data streams. Based on a preset threshold, emission events in the optical signal data streams are identified, resulting in a timestamp sequence of emission events. Based on this timestamp sequence, emission image sequences from the optical signal data streams and spectral data sequences from the spectral data streams are obtained. Noise reduction and spatiotemporal synchronization processing are performed on the emission image sequences and spectral data sequences to obtain aligned image patch sequences and aligned spectral curves. Spectral attention weights are constructed based on the aligned spectral curves, and multi-scale morphological feature extraction is performed on the aligned image patch sequences to obtain multi-scale morphological feature maps. Rare biological targets are detected based on the multi-scale morphological feature maps using the spectral attention weights, yielding target detection results. Therefore, the underwater image analysis-based method and system for detecting rare aquatic organisms proposed in this invention can solve the problem of low accuracy in existing methods for detecting rare aquatic organisms. Attached Figure Description

[0062] Figure 1 This is a flowchart illustrating a method for detecting rare aquatic organisms based on underwater image analysis, provided in an embodiment of the present invention.

[0063] Figure 2 This is a functional block diagram of a rare aquatic organism detection system based on underwater image analysis, provided in an embodiment of the present invention.

[0064] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0065] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0066] This application provides a method for detecting rare aquatic organisms based on underwater image analysis. The execution entity of this method includes, but is not limited to, at least one electronic device that can be configured to execute the method provided in this application, such as a server or a terminal. In other words, the method can be executed by software or hardware installed on a terminal or server device, and the software may be a blockchain platform. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cluster of cloud servers. The server can be an independent server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0067] Reference Figure 1 The diagram shown is a flowchart illustrating a method for detecting rare aquatic organisms based on underwater image analysis, according to an embodiment of the present invention. In this embodiment, the method for detecting rare aquatic organisms based on underwater image analysis includes:

[0068] S1. Use a preset low-light imager and spectrometer to monitor the photon flux and spectral information of the target water area within a preset time period to obtain optical signal data stream and spectral data stream.

[0069] In this embodiment of the invention, the step of using a preset low-light imager and spectrometer to monitor the photon flux and spectral information of the target water area within a preset time period to obtain optical signal data stream and spectral data stream includes:

[0070] The low-light imager continuously acquires images at a preset acquisition frequency to obtain an image data stream;

[0071] The image data stream is converted into a grayscale image to obtain a grayscale image data stream;

[0072] Based on pixel intensity calculation, the photon flux of each frame in the grayscale image data stream is identified to obtain the optical signal data stream;

[0073] The spectrometer continuously acquires spectra at a preset acquisition frequency to obtain a spectral sequence;

[0074] Calculate the signal-to-noise ratio of the spectral sequence, and identify low-quality data in the spectral sequence based on a preset signal-to-noise ratio threshold;

[0075] The low-quality data in the spectral sequence are averaged to obtain a spectral data stream.

[0076] Specifically, the preset sampling frequency can be once per second.

[0077] In detail, converting the image data stream into a grayscale image to obtain a grayscale image data stream involves removing the hue and saturation information of the image and retaining only the brightness value of each pixel.

[0078] In detail, the step of calculating and identifying the photon flux of each frame in the grayscale image data stream based on pixel intensity to obtain the optical signal data stream is achieved by calculating the average intensity (or sum) of all pixels or pixels in a specific region of interest in each frame, and converting the visual brightness information of the frame into a single, quantifiable value, namely "photon flux".

[0079] In detail, the mean-filling of low-quality data in the spectral sequence is performed because directly removing low-quality data would cause breakpoints in the spectral sequence, potentially resulting in the loss of information at critical moments. By employing a "mean-filling" strategy (e.g., replacing the current low-quality data with the average of adjacent high-quality spectral data), the temporal continuity of the spectral data stream can be maintained without significantly introducing bias.

[0080] S2. Identify the light emission events of the optical signal data stream based on a preset threshold, and obtain the light emission event timestamp sequence.

[0081] In this embodiment of the invention, the step of identifying the emission events of the optical signal data stream based on a preset threshold to obtain the emission event timestamp sequence includes:

[0082] The optical signal data stream is smoothed and denoised to obtain a smoothed optical signal data stream.

[0083] Based on the pre-acquired background light data and the smoothed light signal data stream, a dynamic threshold is calculated to obtain a dynamic threshold sequence.

[0084] The smoothed optical signal data stream is used to identify luminescence events using the dynamic threshold sequence to obtain a luminescence event timestamp sequence.

[0085] In detail, the smoothing and noise reduction processing of the optical signal data stream to obtain a smoothed optical signal data stream can be achieved by calculating the weighted average of each data point and its neighboring points within a time window, effectively suppressing random fluctuations caused by sensor electronic noise and environmental transient interference, while preserving the true trend of bioluminescence signal, and finally outputting a smoother and more stable smoothed optical signal data stream.

[0086] In this embodiment of the invention, the step of calculating a dynamic threshold based on pre-acquired background light data and the smoothed light signal data stream to obtain a dynamic threshold sequence includes:

[0087] Background light estimation is performed based on the smoothed optical signal data stream to obtain a real-time background light estimation sequence;

[0088] Calculate the standard deviation of the background light data to obtain a background light standard deviation sequence;

[0089] Each standard deviation in the background light standard deviation sequence is multiplied by a preset sensitivity coefficient to obtain the product result;

[0090] Each product result is added to each corresponding background light estimate in the real-time background light estimate sequence to obtain the dynamic threshold sequence.

[0091] Specifically, the sensitivity coefficient can be taken as 3.5.

[0092] S3. Based on the timestamp sequence of the emission event, obtain the emission image sequence in the optical signal data stream and the spectral data sequence in the spectral data stream.

[0093] In this embodiment of the invention, obtaining the emission image sequence in the optical signal data stream and the spectral data sequence in the spectral data stream based on the emission event timestamp sequence includes:

[0094] Confirm the time extraction range based on the preset time window parameters;

[0095] Construct a data extraction time window based on the time extraction range;

[0096] High-resolution image data from the optical signal data stream is extracted using the data extraction time window and the luminescence event timestamp sequence to obtain a luminescence image sequence.

[0097] High-resolution spectral data is extracted from the spectral data stream using the data extraction time window and the luminescence event timestamp sequence to obtain a spectral data sequence.

[0098] In detail, confirming the time extraction range based on preset time window parameters is a preparatory step for determining the data capture boundary. Its input is the user-preset time window parameters, which typically include the time offset before the event and the time offset after the event. The processing involves calculating a theoretical time interval centered on the luminescence event based on these two parameters.

[0099] In detail, constructing a data extraction time window based on the time extraction range means generating a corresponding, concrete data retrieval instruction for each timestamp in the "luminous event timestamp sequence". This instruction is a "data extraction time window" with the start and end times as boundaries.

[0100] S4. Perform denoising correction and spatiotemporal synchronization processing on the luminescent image sequence and the spectral data sequence to obtain an aligned image block sequence and an aligned spectral curve.

[0101] In this embodiment of the invention, the step of performing denoising correction and spatiotemporal synchronization processing on the luminescent image sequence and the spectral data sequence to obtain an aligned image block sequence and an aligned spectral curve includes:

[0102] The luminescent image sequence is subjected to mean filtering to obtain a preliminary denoised image sequence;

[0103] The preliminary denoised image sequence is subjected to image flat-field correction to obtain the corrected image sequence;

[0104] The baseline of the spectral data sequence is corrected using the asymmetric least squares method to obtain the corrected spectral sequence.

[0105] The corrected image sequence and the corrected spectral sequence are time-synchronized and aligned to obtain an aligned image block sequence and an aligned spectral curve.

[0106] In detail, the mean filtering process performed on the luminescent image sequence to obtain a preliminary denoised image sequence is achieved by applying a "mean filtering" algorithm. This algorithm creates a sliding window (e.g., 3x3 pixels) for each pixel in the image and calculates the arithmetic mean of all pixel values ​​within the window, which is then used to replace the original value of the center pixel. This operation effectively smooths high-frequency noise.

[0107] In detail, the step of performing image flat field correction on the preliminary denoised image sequence to obtain the corrected image sequence is to use pre-acquired "dark field" and "flat field" calibration images, and process each frame according to the formula: Corrected image = (original image - dark field image) / (flat field image - dark field image), thereby compensating for the uneven brightness of the image center and edge caused by factors such as lens vignetting and inconsistent sensor pixel response.

[0108] In detail, the baseline correction of the spectral data sequence using the asymmetric least squares method to obtain the corrected spectral sequence is achieved by assigning smaller weights to spectral peak points and larger weights to baseline points, performing multiple iterations, and finally fitting and subtracting the changed baseline.

[0109] In detail, the step of time-synchronizing and aligning the corrected image sequence and the corrected spectral sequence to obtain an aligned image patch sequence and an aligned spectral curve is based on a unified time axis. An interpolation algorithm is used to generate a spectral curve corresponding to the precise capture time for each image frame. Simultaneously, based on pre-defined spatial calibration, the region of interest (ROI), i.e., the "image patch," corresponding to the spectrometer's field of view, is extracted from each image frame. The final output is an "aligned image patch sequence" and an "aligned spectral curve" that are perfectly matched in both time and space.

[0110] S5. Construct spectral attention weights based on the aligned spectral curves, and perform multi-scale morphological feature extraction on the aligned image block sequence to obtain a multi-scale morphological feature map.

[0111] In this embodiment of the invention, constructing spectral attention weights based on the aligned spectral curves includes:

[0112] Extract the spectral features of the aligned spectral curve to obtain the spectral feature vector;

[0113] The spectral feature vector is matched with a preset rare biological spectral library to obtain a similarity score;

[0114] An initial spectral response scalar is generated based on the similarity score;

[0115] The initial spectral response scalar is mapped to an initial spectral attention weight map;

[0116] The initial spectral attention weight map is weighted and formatted for output to obtain the spectral attention weights.

[0117] In detail, extracting the spectral features of the aligned spectral curve to obtain a spectral feature vector involves extracting essential characteristics from the curve that can effectively distinguish different bioluminescent organisms. These features typically include the main peak wavelength, full width at half maximum (FWHM), and integral intensity within a specific band. The final output is a "spectral feature vector".

[0118] In detail, the step of matching the spectral feature vector with a preset rare biological spectral library to obtain a similarity score is achieved by using a similarity measurement algorithm (such as cosine similarity or Euclidean distance calculation) to compare the feature vector of the current unknown spectrum with the feature vector of each known organism in the spectral library one by one.

[0119] In detail, generating an initial spectral response scalar based on the similarity scores involves fusing multiple alignment results into a single decision signal. The input is one or more "similarity scores" obtained in the previous step. The process involves formulating a fusion rule, for example, extracting the maximum value from all matching scores with organisms in the library, representing the degree of matching between the current spectrum and the most similar rare organism.

[0120] In detail, mapping the initial spectral response scalar to an initial spectral attention weight map is achieved by creating a two-dimensional matrix with the same resolution as the image to be detected (initially all pixel values ​​are zero), and then, according to a pre-defined spatial calibration, setting the values ​​of all pixels to the scalar value within the image region (region of interest) corresponding to the spectrometer's field of view.

[0121] In detail, the weight scaling and formatting of the initial spectral attention weight map to obtain spectral attention weights optimizes the weight map, making it more suitable for neural network processing. Its input is the "initial spectral attention weight map." The processing mainly includes weight scaling, for example, linearly mapping the original [0,1] range to the [0.5,2.0] range. This processing allows high-weight regions to enhance image features (>1), while low-weight regions suppress them (<1), thus preserving more background information than simple binarization. Finally, the processed matrix is ​​formatted into a tensor with the exact same size as the neural network feature map.

[0122] In this embodiment of the invention, the aligned image patch sequence undergoes multi-scale morphological feature extraction to obtain a multi-scale morphological feature map, including:

[0123] The aligned image block sequence is then normalized to obtain a normalized image block sequence.

[0124] Extract the low-level feature map of the standardized image patch sequence, and perform contour enhancement on the low-level feature map to obtain a contour-enhanced feature map;

[0125] The intermediate feature map of the standardized image patch sequence is extracted, and the intermediate feature map is upsampled to obtain the sampled feature map;

[0126] The sampled feature map and the contour-enhanced feature map are added element-wise to obtain a multi-scale fused feature map;

[0127] A multi-scale morphological feature map is generated based on the multi-scale fusion feature map.

[0128] In detail, the aligned image block sequence is standardized to obtain a standardized image block sequence, which involves scaling the pixel values ​​from the original range (e.g., 0-255) to a fixed range (e.g., 0-1), or further normalizing them to a mean of 0 and a standard deviation of 1.

[0129] In detail, the step of extracting the low-level feature maps of the standardized image patch sequence and performing contour enhancement on the low-level feature maps to obtain contour-enhanced feature maps involves inputting the standardized image patch sequence into the shallow convolutional layers of a convolutional neural network. These layers have small receptive fields and are specifically used to extract general features from the low level, such as edges, corners, and color patches, to obtain "low-level feature maps." Subsequently, "contour enhancement" is performed on these feature maps by introducing biomimetic lateral inhibition mechanisms (e.g., using specific convolutional kernels or local normalization operations) to enhance the contrast between the biological contours and the background, suppress uniform regions, and make the edges, textures, and other details of the target more prominent.

[0130] In detail, the extraction of intermediate feature maps from the standardized image patch sequence, followed by upsampling of these intermediate feature maps to obtain sampled feature maps, involves inputting the standardized image patch sequence into deeper convolutional layers within the neural network. These layers have larger receptive fields, enabling them to combine lower-level pixel information into more complex patterns, such as the shape and combination of components, thus obtaining "intermediate feature maps." However, these feature maps have low resolution due to pooling operations. Therefore, it is necessary to increase their spatial size through "upsampling" operations (such as bilinear interpolation or transposed convolution) to obtain "sampled feature maps."

[0131] In detail, the step of generating a multi-scale morphological feature map based on the multi-scale fused feature map involves organizing and encapsulating the multi-scale fused feature map, along with other possible feature maps of different scales in the neural network (for example, there may also be deeper feature maps involved in the fusion), into a structured data set. This set comprehensively covers morphological information at different scales, from microscopic details to macroscopic structures.

[0132] S6. Based on the spectral attention weights, perform rare biological target detection on the multi-scale morphological feature map to obtain the target detection result.

[0133] In this embodiment of the invention, the step of performing rare biological target detection on the multi-scale morphological feature map based on the spectral attention weights to obtain target detection results includes:

[0134] The spectral attention weights and the multi-scale morphological feature maps are fused using feature weighting to obtain a weighted multi-scale feature map.

[0135] A lightweight region proposal network is used to generate a list of region proposals based on the weighted multi-scale feature map;

[0136] Based on the region proposal list, the weighted multi-scale feature map is aligned and classified with regions of interest to obtain region of interest feature vectors.

[0137] Based on the feature vector of the region of interest, target classification and bounding box regression are performed to obtain the target detection result.

[0138] In detail, a lightweight region proposal network is used to generate a list of region proposals based on the weighted multi-scale feature map. The network predefines a series of anchor boxes of different sizes and proportions by sliding a window on the feature map, and performs binary classification (determining whether it is a foreground target or background) and preliminary bounding box coordinate regression on each anchor box.

[0139] In detail, the target classification and bounding box regression processing based on the feature vectors of the region of interest to obtain the target detection result is performed by a small neural network executing two tasks in parallel: "Target classification" determines the specific species category (such as a specific jellyfish) or background to which each feature vector belongs and outputs the category probability; "Bounding box regression" performs secondary fine-tuning of the coordinates of the proposed boxes to make them fit the boundary of the real target more closely. Finally, redundant and overlapping detection boxes are removed by post-processing algorithms such as non-maximum suppression. The output is a structured "target detection result".

[0140] like Figure 2 The diagram shown is a functional block diagram of a rare aquatic organism detection system based on underwater image analysis provided in an embodiment of the present invention.

[0141] The underwater image analysis-based rare aquatic organism detection system 100 of this invention can be installed in an electronic device. Depending on the functions implemented, the underwater image analysis-based rare aquatic organism detection system 100 may include a data acquisition module 101, a sequence data recognition module 102, a noise reduction and correction module 103, a feature extraction module 104, and a target detection module 105. The module described in this invention can also be referred to as a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and are stored in the memory of the electronic device.

[0142] In this embodiment, the functions of each module / unit are as follows:

[0143] The data acquisition module 101 is used to monitor the photon flux and spectral information of the target water area within a preset time period using a preset low-light imager and a spectrometer, and to obtain optical signal data stream and spectral data stream.

[0144] The sequence data recognition module 102 is used to recognize the emission events of the optical signal data stream based on a preset threshold, obtain the emission event timestamp sequence, and obtain the emission image sequence in the optical signal data stream and the spectral data sequence in the spectral data stream based on the emission event timestamp sequence.

[0145] The denoising correction module 103 is used to perform denoising correction and spatiotemporal synchronization processing on the luminescent image sequence and the spectral data sequence to obtain an aligned image block sequence and an aligned spectral curve.

[0146] The feature extraction module 104 is used to construct spectral attention weights based on the aligned spectral curves, and to perform multi-scale morphological feature extraction on the aligned image block sequence to obtain a multi-scale morphological feature map.

[0147] The target detection module 105 is used to perform rare biological target detection on the multi-scale morphological feature map based on the spectral attention weight, and obtain the target detection result.

[0148] In detail, the modules of the underwater rare organism detection system 100 based on underwater image analysis described in this embodiment of the invention employ the same methods as described above. Figure 1 The method used is the same as the underwater image analysis-based method for detecting rare aquatic organisms described in the previous section, and it can produce the same technical effect, so it will not be repeated here.

[0149] In the embodiments provided by this invention, it should be understood that the disclosed devices, systems, and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0150] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0151] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0152] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0153] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0154] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0155] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or systems stated in a system claim may also be implemented by a single unit or system through software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any specific order.

[0156] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for detecting rare aquatic organisms based on underwater image analysis, characterized in that, The method includes: The photon flux and spectral information of the target water area within a preset time period are monitored using a preset low-light imager and spectrometer to obtain optical signal data stream and spectral data stream; Based on a preset threshold, the emission events of the optical signal data stream are identified to obtain a timestamp sequence of emission events; Based on the timestamp sequence of the emission events, obtain the emission image sequence in the optical signal data stream and the spectral data sequence in the spectral data stream; The luminescent image sequence and the spectral data sequence are subjected to denoising correction and spatiotemporal synchronization processing to obtain an aligned image block sequence and an aligned spectral curve. Based on the aligned spectral curves, spectral attention weights are constructed, and multi-scale morphological features are extracted from the aligned image patch sequence to obtain multi-scale morphological feature maps. Rare biological targets are detected based on the spectral attention weights of the multi-scale morphological feature map to obtain target detection results; The step of constructing spectral attention weights based on the aligned spectral curves includes: Extract the spectral features of the aligned spectral curve to obtain the spectral feature vector; The spectral feature vector is matched with a preset rare biological spectral library to obtain a similarity score; An initial spectral response scalar is generated based on the similarity score; The initial spectral response scalar is mapped to an initial spectral attention weight map; The initial spectral attention weight map is weighted and formatted for output to obtain the spectral attention weights.

2. The method for detecting rare aquatic organisms based on underwater image analysis as described in claim 1, characterized in that, The method of using a preset low-light imager and spectrometer to monitor the photon flux and spectral information of the target water area within a preset time period to obtain optical signal data stream and spectral data stream includes: The low-light imager continuously acquires images at a preset acquisition frequency to obtain an image data stream; The image data stream is converted into a grayscale image to obtain a grayscale image data stream; Based on pixel intensity calculation, the photon flux of each frame in the grayscale image data stream is identified to obtain the optical signal data stream; The spectrometer continuously acquires spectra at a preset acquisition frequency to obtain a spectral sequence; Calculate the signal-to-noise ratio of the spectral sequence, and identify low-quality data in the spectral sequence based on a preset signal-to-noise ratio threshold; The low-quality data in the spectral sequence are averaged to obtain a spectral data stream.

3. The method for detecting rare aquatic organisms based on underwater image analysis as described in claim 1, characterized in that, The step of identifying the emission events of the optical signal data stream based on a preset threshold to obtain the emission event timestamp sequence includes: The optical signal data stream is smoothed and denoised to obtain a smoothed optical signal data stream. Based on the pre-acquired background light data and the smoothed light signal data stream, a dynamic threshold is calculated to obtain a dynamic threshold sequence. The smoothed optical signal data stream is used to identify luminescence events using the dynamic threshold sequence to obtain a luminescence event timestamp sequence.

4. The method for detecting rare aquatic organisms based on underwater image analysis as described in claim 3, characterized in that, The dynamic threshold is calculated based on the pre-acquired background light data and the smoothed light signal data stream to obtain a dynamic threshold sequence, including: Background light estimation is performed based on the smoothed optical signal data stream to obtain a real-time background light estimation sequence; Calculate the standard deviation of the background light data to obtain a background light standard deviation sequence; Each standard deviation in the background light standard deviation sequence is multiplied by a preset sensitivity coefficient to obtain the product result; Each product result is added to each corresponding background light estimate in the real-time background light estimate sequence to obtain the dynamic threshold sequence.

5. The method for detecting rare aquatic organisms based on underwater image analysis as described in claim 1, characterized in that, The step of obtaining the emission image sequence in the optical signal data stream and the spectral data sequence in the spectral data stream based on the emission event timestamp sequence includes: Confirm the time extraction range based on the preset time window parameters; Construct a data extraction time window based on the time extraction range; High-resolution image data from the optical signal data stream is extracted using the data extraction time window and the luminescence event timestamp sequence to obtain a luminescence image sequence. High-resolution spectral data is extracted from the spectral data stream using the data extraction time window and the luminescence event timestamp sequence to obtain a spectral data sequence.

6. The method for detecting rare aquatic organisms based on underwater image analysis as described in claim 1, characterized in that, The step of performing denoising correction and spatiotemporal synchronization processing on the luminescent image sequence and the spectral data sequence to obtain an aligned image block sequence and an aligned spectral curve includes: The luminescent image sequence is subjected to mean filtering to obtain a preliminary denoised image sequence; The preliminary denoised image sequence is subjected to image flat-field correction to obtain the corrected image sequence; The baseline of the spectral data sequence is corrected using the asymmetric least squares method to obtain the corrected spectral sequence. The corrected image sequence and the corrected spectral sequence are time-synchronized and aligned to obtain an aligned image block sequence and an aligned spectral curve.

7. The method for detecting rare aquatic organisms based on underwater image analysis as described in claim 1, characterized in that, The aligned image patch sequence undergoes multi-scale morphological feature extraction to obtain a multi-scale morphological feature map, including: The aligned image block sequence is then normalized to obtain a normalized image block sequence. Extract the low-level feature map of the standardized image patch sequence, and perform contour enhancement on the low-level feature map to obtain a contour-enhanced feature map; The intermediate feature map of the standardized image patch sequence is extracted, and the intermediate feature map is upsampled to obtain the sampled feature map; The sampled feature map and the contour-enhanced feature map are added element-wise to obtain a multi-scale fused feature map; A multi-scale morphological feature map is generated based on the multi-scale fusion feature map.

8. The method for detecting rare aquatic organisms based on underwater image analysis as described in claim 1, characterized in that, The rare biological target detection based on the spectral attention weights on the multi-scale morphological feature map yields the target detection results, including: The spectral attention weights and the multi-scale morphological feature maps are fused using feature weighting to obtain a weighted multi-scale feature map. A lightweight region proposal network is used to generate a list of region proposals based on the weighted multi-scale feature map; Based on the region proposal list, the weighted multi-scale feature map is aligned and classified with regions of interest to obtain region of interest feature vectors. Based on the feature vector of the region of interest, target classification and bounding box regression are performed to obtain the target detection result.

9. A rare aquatic organism detection system based on underwater image analysis, characterized in that, The system includes: The data acquisition module is used to monitor the photon flux and spectral information of the target water area within a preset time period using a preset low-light imager and spectrometer, and to obtain optical signal data stream and spectral data stream. The sequence data recognition module is used to identify the emission events of the optical signal data stream based on a preset threshold, obtain the emission event timestamp sequence, and obtain the emission image sequence in the optical signal data stream and the spectral data sequence in the spectral data stream based on the emission event timestamp sequence. The noise reduction and correction module is used to perform noise reduction and correction and spatiotemporal synchronization processing on the luminescent image sequence and the spectral data sequence to obtain an aligned image block sequence and an aligned spectral curve. The feature extraction module is used to construct spectral attention weights based on the aligned spectral curves, and to perform multi-scale morphological feature extraction on the aligned image patch sequence to obtain a multi-scale morphological feature map. The target detection module is used to perform rare biological target detection on the multi-scale morphological feature map based on the spectral attention weights to obtain target detection results; The step of constructing spectral attention weights based on the aligned spectral curves includes: Extract the spectral features of the aligned spectral curve to obtain the spectral feature vector; The spectral feature vector is matched with a preset rare biological spectral library to obtain a similarity score; An initial spectral response scalar is generated based on the similarity score; The initial spectral response scalar is mapped to an initial spectral attention weight map; The initial spectral attention weight map is weighted and formatted for output to obtain the spectral attention weights.

Images