Method and system for detecting disaster-causing marine organisms and environment

Abstract

The application discloses a disaster-causing marine organism and environment detection method and system, relates to the underwater detection technical field, and is applied to an underwater microscopic imaging instrument in a disaster-causing marine organism and environment detection system. The system comprises a disaster-causing marine organism and environment detection device, a counterweight and a communication buoy arranged above the sea surface. The disaster-causing marine organism and environment detection device comprises an underwater support platform, an underwater microscopic imaging instrument arranged on the underwater support platform and an electronic cabin for supplying power to the underwater microscopic imaging instrument. An armored cable is used for electrically connecting the underwater support platform and the communication buoy. The counterweight is connected with the armored cable. The underwater microscopic imaging instrument determines the number of microalgae according to a real-time acquired target fluorescence image and transmits the number of microalgae to a target device through the communication buoy. The application can realize full-coverage offshore observation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of underwater detection, in particular to a disaster-causing marine organism and environment detection method and system. BACKGROUND

[0002] In recent years, the marine ecological environment along the coast of China has shown a good trend, but there are problems of damage to typical nearshore ecosystems and ecological disasters, such as red tide, green tide, brown cyst algae, sea cucumbers, jellyfish, and explosive growth of brine shrimp, which cause harm to the coastal ecological environment, especially in the safety of the cold source water area of the coastal nuclear power plant. Under this circumstance, the research and development of an online monitoring system for disaster-causing marine organisms and coral reefs is particularly important.

[0003] From the perspective of observation and monitoring of disaster-causing marine organisms, monitoring of typical ecological systems such as coral reefs, mangroves, and seagrass beds, the current methods include satellite remote sensing, unmanned aerial vehicle cruising, manual sampling, and seabed deployment of self-contained systems, but there is a lack of long-term in-situ online observation and monitoring system, and the development of equipment for monitoring disaster-causing organisms and easy deployment and recovery is slow.

[0004] From the perspective of conventional marine early warning and monitoring, the early warning and monitoring capability of space-based systems has basically been realized, but space-based observation can only observe areas with shallow water depth and cannot achieve underwater and seabed ecological early warning and monitoring. In the past two decades, domestic relevant units have developed seabed-based observation systems. From the perspective of monitoring of disaster-causing marine organisms and monitoring of typical ecological systems such as coral reefs, the seabed-based observation system can achieve regular observation, but it cannot achieve real-time transmission of large data volumes. In order to achieve real-time transmission of large data volumes, a tethered seabed observation system is developed, but the tethered seabed observation system needs to transmit data through a shore-based station and needs to be powered by the shore-based station, which cannot achieve full-coverage off-shore observation. SUMMARY

[0005] To solve the above problems, the present disclosure provides a disaster-causing marine organism and environment detection method and system, which can achieve full-coverage off-shore observation and reduce data transmission volume.

[0006] To achieve the above purpose, the present disclosure provides the following solutions:

[0007] In a first aspect, the present disclosure provides a method for detecting disaster-causing marine organisms and environment, which is applied to an underwater microscopic imaging instrument in a disaster-causing marine organism and environment detection system, the system comprising: a disaster-causing marine organism and environment detection device, a weight block, and a communication buoy arranged above the sea surface; the disaster-causing marine organism and environment detection device comprising an underwater support platform, an underwater microscopic imaging instrument arranged on the underwater support platform, and an electronic cabin for powering the underwater microscopic imaging instrument; an armored cable for electrically connecting the underwater support platform and the communication buoy; the weight block is connected with the armored cable; the method comprising:

[0008] acquiring a target fluorescence image;

[0009] inputting the target fluorescence image into an improved YOLOv8 target detection network for microalgae cell recognition, and outputting a detection frame set; the improved YOLOv8 target detection network is used for recognizing microalgae cells;

[0010] inputting the detection frame set into a Mask R-CNN network for microalgae segmentation to obtain a binary segmentation mask;

[0011] obtaining the number of microalgae through the binary segmentation mask;

[0012] transmitting the number of microalgae to a target device through the communication buoy.

[0013] In a second aspect, the present disclosure provides a disaster-causing marine organism and environment detection system, which comprises: a disaster-causing marine organism and environment detection device, a weight block, and a communication buoy arranged above the sea surface;

[0014] The disaster-causing marine organism and environment detection device comprises an underwater support platform, an underwater microscopic imaging instrument arranged on the underwater support platform, and an electronic cabin for powering the underwater microscopic imaging instrument, wherein the underwater microscopic imaging instrument can emit a first waveband light source and a second waveband light source;

[0015] The armored cable is used for electrically connecting the underwater support platform and the communication buoy;

[0016] The weight block is connected with the armored cable;

[0017] The underwater microscopic imaging instrument is used for executing the method steps of the first aspect.

[0018] The present application has the following technical effects relative to the prior art:

[0019] Unlike traditional seabed-based observation systems, the communication buoy does not rely on a shore-based station for data transmission and is equipped with an independent electronic cabin specially for system power supply, which makes the entire system independent of the shore-based station for underwater system power supply. Therefore, the system of the present disclosure is free from the limitation of the shore-based station on the offshore observation range and can achieve offshore observation in a wider area. Further, the combination application of the improved YOLOv8 and Mask R-CNN network realizes accurate identification and segmentation of microalgae cells, only the key data such as the number of microalgae need to be transmitted, avoiding the transmission of a large amount of redundant information of the original image, and reducing the data transmission amount. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the present disclosure or the prior art, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and other drawings can also be obtained by those skilled in the art without creative labor.

[0021] Figure 1 The internal overall structure diagram of the disaster-causing marine organism and environment detection device disclosed in the present application is shown in the figure.

[0022] Figure 2 The external overall structure diagram of the disaster-causing marine organism and environment detection device disclosed in the present disclosure is shown in the figure.

[0023] Figure 3 The overall structure diagram of the disaster-causing marine organism and environment detection system disclosed in the present disclosure is shown in the figure.

[0024] Figure 4 The overall structure diagram of the underwater microscopic imaging instrument in the disaster-causing marine organism and environment detection device disclosed in the present disclosure is shown in the figure.

[0025] Figure 5 The internal structure diagram of the underwater microscopic imaging instrument in the disaster-causing marine organism and environment detection device disclosed in the present disclosure is shown in the figure.

[0026] Figure 6 The internal movement structure diagram of the underwater microscopic imaging instrument in the disaster-causing marine organism and environment detection device disclosed in the present disclosure is shown in the figure.

[0027] Figure 7 The flowchart of a disaster-causing marine organism and environment detection method provided by the embodiment of the present disclosure is shown in the figure.

[0028] Explanation of reference numerals:

[0029] 1, underwater microscopic imaging instrument; 2, armored cable joint; 3, underwater support platform; 4, acoustic releaser; 5, ocean current sensor; 6, electronic cabin; 7, multi-parameter water quality instrument; 8, disaster-causing marine organism and environment detection device; 9, counterweight; 10, armored cable; 11, communication buoy; 12, fixed cover; 13, waterproof shell; 14, microscopic objective lens; 15, camera; 16, first waveband light source; 17, second waveband light source; 18, camera fixing frame; 19, lifting ring; 20, side guard plate; 21, main guard plate. DETAILED DESCRIPTION

[0030] The technical solutions in the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present disclosure.

[0031] The purpose of the present disclosure is to provide a disaster-causing marine organism and environment detection device, system and early warning detection method, to realize convenient launching of the system, not subject to the requirements of the shore-based station, equipped with an underwater microscopic imaging instrument and its early warning algorithm, and to realize real-time ecological early warning monitoring.

[0032] In order to make the above-mentioned purposes, features and advantages of the present disclosure more obvious and easy to understand, the present disclosure will be further described in detail below with reference to the drawings and specific embodiments.

[0033] Reference Figure 1 The disaster-causing marine organism and environment detection device disclosed in the embodiments of the present disclosure at least includes an underwater support platform 3, and the underwater support platform 3 is provided with an underwater microscopic imaging instrument 1 and an electronic cabin 6 for supplying power to the underwater microscopic imaging instrument 1. The underwater microscopic imaging instrument 1 is arranged on the underwater support platform 3, and is used for biological identification, counting and ecological alarm of algae in the field of view. The underwater microscopic imaging instrument performs front-end sensing, internal microcomputer biomass technology and analysis, calculates the number of algae microorganisms in the optical field of view, and provides marine ecological early warning.

[0034] Reference Figures 4-6As an implementation form, the underwater microscopic imaging instrument 1 comprises a waterproof shell 13 and a fixed cover 12 arranged at the end of the waterproof shell 13, the inside of the waterproof shell 13 is provided with a camera 15 comprising a focusing head and a filter, a microscopic objective 14 connected with the camera 15, and a first waveband light source 16 and a second waveband light source 17 providing light sources, and the inside of the waterproof shell 13 is further provided with a camera fixing frame 18 for fixing the camera 15, the fixed cover 12 is provided with an opening for the microscopic objective 14 to collect information, the opening is arranged opposite to the microscopic objective 14, and the light beams of the first waveband light source 16 and the second waveband light source 17 can also be emitted through the opening, so as to facilitate the information collection of the microscopic objective 14. The underwater imaging instrument can further comprise a processor (not shown in the figure), and the processor can be connected with the first waveband light source 16, the second waveband light source 17 and the camera 15.

[0035] With reference to Figure 1 In an embodiment, the disaster-causing marine organism and environment detection device further comprises an acoustic release 4, a current sensor 5 and a multi-parameter water quality instrument 7 arranged on the underwater support platform 3, and the electronic cabin 6 is electrically connected with the acoustic release 4, the current sensor 5 and the multi-parameter water quality instrument 7 to supply power for the acoustic release 4, the current sensor 5 and the multi-parameter water quality instrument 7. The acoustic release 4 is a general-purpose device for recovering the underwater device, the current sensor 5 is a general-purpose sensor for measuring hydrological elements in the laying area, i.e. the profile current, and the multi-parameter water quality instrument 7 is used for online acquisition of PH, turbidity, chlorophyll, dissolved oxygen, temperature, salinity and other elements in the observed sea area.

[0036] It should be noted that the electronic cabin 6 is suitable for underwater sensor interfaces with multiple interface forms, multiple data formats and multiple voltage systems, and has fault isolation and state monitoring functions.

[0037] With reference to Figure 2 As an implementation form, the underwater support platform 3 is further provided with a main guard plate 21 and a side guard plate 20 around the circumference, and the main guard plate 21 and the side guard plate 20 are arranged on the adjacent side walls of the underwater support platform 3, for example, the side walls of the underwater support platform 3 are four-sided structures, two main guard plates 21 are arranged opposite to each other, and two side guard plates 20 are arranged opposite to each other. The side guard plate 20 is provided with a window for the underwater microscopic imaging instrument 1 to collect information, the main guard plate 21 is used for protection and flow guidance of the underwater platform, the side guard plate 20 is used for protection and flow guidance of the underwater platform, and provides a window view for the underwater microscopic imaging instrument 1.

[0038] The underwater support platform 3 is further provided with a lifting ring 19 for lifting the underwater support platform 3, the underwater support platform 3 is a load-bearing structure of the disaster-causing marine organism and environment detection device 8, and is made of 316 stainless steel.

[0039] With reference to Figure 3The present disclosure also discloses a disaster-causing marine organism and environment detection system, which applies the disaster-causing marine organism and environment detection device described above, and comprises a communication buoy 11 electrically connected with the underwater support platform 3 through the armored cable 10, and a counterweight 9 connected with the armored cable 10, wherein a universal joint swivel is arranged at the connection position of the armored cable 10 and the communication buoy 11, the communication buoy 11 is arranged above the sea surface, and the communication buoy 11 is connected with the underwater support platform 3 through the armored cable 10, which can realize the traction of the communication buoy 11 through the armored cable 10 as an anchor chain, and can realize the power supply and communication function between the water surface and the underwater through the armored cable 10, and the counterweight 9 arranged on the armored cable 10 between the underwater support platform 3 and the communication buoy 11 can further limit the communication buoy 11, the counterweight 9 is a concrete weight, and the universal joint swivel is arranged at the connection position of the armored cable 10 and the communication buoy 11, which allows the armored cable 10 and the buoy to rotate freely by 360°, avoids the twisting of the cable caused by the sea waves, ocean currents or buoy swing, and prevents the breakage of the armored layer metal wire or the damage of the internal optical fiber.

[0040] It should be noted that, similar to the general buoy, the communication buoy 11 is internally provided with a conventional battery, an underwater data receiver and a 4G communication instrument, the buoy bottom is provided with a bearing puller with a universal joint swivel, and the bearing puller is mechanically pressure-welded with the inner armored layer of the armored cable 10.

[0041] The electronic cabin 6 is used for power supply and communication with the communication buoy 11.

[0042] Reference Figure 1 As an embodiment, the connection position of the underwater support platform 3 and the armored cable 10 is provided with an armored cable joint 2, which is used for fixing the armored cable 10 on the power supply structure of the underwater platform, and realizes the power supply and communication transmission between the water surface and the underwater.

[0043] Figure 7 A flow chart of a disaster-causing marine organism and environment detection method provided by the embodiment of the present disclosure, which is applied to the underwater microscopic imaging instrument in the disaster-causing marine organism and environment detection system in the above embodiment, the system comprising: an underwater support platform, an underwater microscopic imaging instrument arranged on the underwater support platform, an electronic cabin for power supply of the underwater microscopic imaging instrument, a communication buoy arranged above the sea surface, an armored cable for electrically connecting the underwater support platform and the communication buoy, and a counterweight connected with the armored cable, wherein a universal joint swivel is arranged at the connection position of the armored cable and the communication buoy, and the method comprises:

[0044] S101, acquiring a target fluorescence image.

[0045] The underwater microscopic imaging instrument can emit a first waveband light source and a second waveband light source; step S101 includes the following sub-steps S1011-S1014:

[0046] S1011, acquire a first fluorescent image, a second fluorescent image and a background fluorescent image respectively in the first waveband light source irradiation, the second waveband light source irradiation and the light source off state.

[0047] The processor controls the first waveband light source irradiation, the second waveband light source irradiation and the first waveband light source and the second waveband light source off respectively, in the first waveband light source irradiation, the camera shoots the first fluorescent image and sends it to the processor, in the second waveband light source irradiation, the camera shoots the second fluorescent image and sends it to the processor, in the light source off state, the camera shoots the second fluorescent image and sends it to the processor, in this case, the processor acquires the first fluorescent image, the second fluorescent image and the background fluorescent image.

[0048] The first fluorescent image is the image collected by the underwater microscopic imaging instrument in the first waveband light source irradiation; the second fluorescent image is the image collected by the underwater microscopic imaging instrument in the second waveband light source irradiation; the background fluorescent image is the image collected by the underwater microscopic imaging instrument in the light source off state.

[0049] Under the action of the first waveband light source and the second waveband light source, the fluorescence emitted by the underwater organism after passing through the objective lens of the underwater microscopic imaging instrument is collected and transmitted by the underwater microscopic imaging instrument as a fluorescent image, which corresponds to the spatial distribution of the fluorescence signal of the underwater organism and the superposition of the underwater background signal and noise; under the condition that the light source is off, that is, without the action of the light source, the fluorescent image collected and transmitted by the underwater microscopic imaging instrument corresponds to the underwater background fluorescence signal and noise, which is called the background fluorescent image.

[0050] S1012, difference operation is performed on the original fluorescent image and the background fluorescent image by using the time sequence difference method to obtain a net fluorescent image; the original fluorescent image is the first fluorescent image or the second fluorescent image.

[0051] Based on the fluorescent image obtained in step S1011, the time sequence difference method is used to remove the background interference by using the change of the fluorescence intensity in time to improve the accuracy of the fluorescence signal. Specifically, the first waveband light source and the second waveband light source are turned off, and the background fluorescent image is collected, which is represented as:

[0052] ;

[0053] Wherein, The background fluorescent image is represented as ， The environmental autofluorescence signal is represented as The system noise is represented as

[0054] The wavelength of the light emitted by the first waveband light source is The wavelength of the light emitted by the second waveband light source is After the fluorescence of the underwater organism is excited under the action of the first waveband light source, a first fluorescence image is collected using an underwater microscopic imaging instrument

[0055]

[0056] wherein, represents the first fluorescence image obtained under the action of the first waveband light source, represents the fluorescence image of the underwater organism obtained under the action of the first waveband light source, represents a background fluorescence image.

[0057] After the fluorescence of the underwater organism is excited under the action of the second waveband light source, a second fluorescence image is collected using an underwater microscopic imaging instrument

[0058]

[0059] wherein, represents the second fluorescence image obtained under the action of the second waveband light source, represents the fluorescence image of the underwater organism obtained under the action of the second waveband light source, represents a background fluorescence image.

[0060] A net fluorescence image is obtained through difference operation of the two images:

[0061]

[0062] wherein, represents the net fluorescence image, represents or , represents a background fluorescence image.

[0063] S1013, difference operation is performed on the first fluorescence image and the second fluorescence image by using a spectral difference method to obtain a spectral difference image.

[0064] Since there are various organisms under water that can produce fluorescence, for disaster-causing marine organisms, the spectral difference method is used to distinguish the fluorescence signals of the disaster-causing marine organisms from the fluorescence signals of non-disaster-causing marine organisms by using excitation light of different wavelengths, thereby improving fluorescence specificity.

[0065] A spectral difference image is obtained through difference operation of two fluorescence images, and non-specific fluorescence is removed, and the formula is as follows:

[0066] ;​​​​​

[0067] wherein, The spectral difference image eliminates the background interference in the spectrum and improves the specificity of the fluorescence signal of the disaster-causing marine organism.

[0068] S1014, obtaining a final difference fluorescence image according to the net fluorescence image and the spectral difference image, the final difference fluorescence image being the target fluorescence image.

[0069] In order to simultaneously utilize the advantages of the time series difference method and the spectral difference method, the two can be combined to obtain a final difference fluorescence image:

[0070] ;

[0071] wherein, the final difference fluorescence image, the net fluorescence image, the spectral difference image, is a balance coefficient of the time series difference method and the spectral difference method.

[0072] The final difference fluorescence image can minimize background noise, improve the contrast of the target fluorescence image, and make the microalgae fluorescence microscopic image clearer, which is helpful for subsequent microalgae detection and counting.

[0073] S102, inputting the target fluorescence image into an improved YOLOv8 target detection network for microalgae cell recognition, and outputting a detection frame set; the improved YOLOv8 target detection network is used for recognizing microalgae cells.

[0074] Before using the improved YOLOv8 target detection network to recognize microalgae cells, the original improved YOLOv8 target detection network also needs to be trained, and the specific training method further includes the following substeps:

[0075] A1, obtaining a first training sample, the first training sample including: sample images containing microalgae cells, and annotation information corresponding to each sample image, the annotation information including: real boundary box coordinates of the microalgae cells, a class real label, and a confidence real value of target existence.

[0076] The real boundary box coordinates of the microalgae cells are used to measure the accuracy of the predicted frame, the class real label indicates the real class (such as whether it is a target class) of the microalgae in each boundary box, and the confidence real value of target existence reflects the real probability of the existence of the microalgae cells in the boundary box.

[0077] A2, input the sample image into the backbone network of the original improved YOLOv8 target detection network for multi-scale feature extraction. The backbone network of the original improved YOLOv8 target detection network adopts CSPDarknet.

[0078] The sample image is input into the original improved YOLOv8 network, and the backbone network thereof adopts the CSPDarknet architecture, the core function of which is to realize multi-scale feature extraction.

[0079] The sample image is first enhanced in contrast through histogram equalization and normalized to the range, and then scaled to resolution, and multi-scale feature extraction is performed through the backbone network CSPDarknet, wherein the generation process of the feature map of the first layer can be represented as the generation formula of the feature map of the first layer is:

[0080] ;

[0081] The CBS module is a basic feature extraction unit, which is composed of a convolution layer with a 3x3 convolution kernel , batch normalization (BN) and SiLU activation function, and is used for convolution operation, standardization and nonlinear transformation of the feature map of the previous layer ; the C2f module fuses the detailed information (such as the edge and texture of microalgae) of the shallow layer features and the semantic information (such as the overall morphology of microalgae) of the deep layer features through the “cross-stage partial connection” mechanism, improves the feature expression ability, and provides multi-dimensional feature support for subsequent detection.

[0082] A3, input the multi-scale features into the decoupled detection head of the original improved YOLOv8 target detection network to predict the bounding box coordinates of the microalgae cells, calculate the class probability and confidence.

[0083] After the multi-scale features are extracted by the backbone network, they are input into the decoupled detection head of the network, which is specially designed to separate the prediction of different targets for the detection task, and outputs three core results:

[0084] The predicted bounding box coordinates of the microalgae cells (denoted as ): representing the position and size of the microalgae cells predicted by the model;

[0085] The predicted class probability (denoted as ): the probability that the microalgae in the bounding box belongs to a certain class predicted by the model;

[0086] The predicted confidence (denoted as ): the probability that the model predicts that there is a microalgae cell in the bounding box. ​

[0087] The decoupled detection head reduces interference between tasks and improves the prediction accuracy of each task by separating bounding box regression, category classification, and confidence prediction tasks.

[0088] A4. Calculate the total loss based on the predicted bounding box coordinates of the microalgal cells, the predicted class probability, the confidence level, the annotation information, and the total loss function of the detection network.

[0089] Based on the difference between the predicted results and the labeled information, the total loss is calculated using the total loss function of the detection network, and this total loss is used as the objective for model optimization. The total loss function of the detection network is expressed as: ;

[0090] The meanings and calculation methods for each type of loss are as follows:

[0091] The bounding box loss function, also known as the bounding box regression loss, is... The bounding boxes of predicted microalgal cells were calculated using IoU loss. The true bounding box of microalgal cells The difference is expressed by the formula: The Intersection over Union (IoU) is a measure of the magnitude of the two values; the smaller the IoU, the greater the loss.

[0092] The classification loss function, also known as the class classification loss ( ): Calculated using the binary cross-entropy function, the formula is: Measure the predicted class probability With category real labels The differences; among them, ∈{0,1} represents the true label of the category.

[0093] Confidence loss function ( Similarly, the binary cross-entropy function is used, and the formula is:

[0094] ;

[0095] Measure the confidence level of the prediction True confidence level of the existence of the target The differences, among which, .

[0096] Weighting coefficient and This is used to balance the importance of different loss terms and prevent any one loss from dominating the optimization process.

[0097] The test results were filtered using non-maximum suppression (NMS), and results exceeding the threshold were removed. The redundant bounding boxes are output as the final set of detection boxes. K is the number of selected bounding boxes, and the position of the bounding box is the target algae.

[0098] A5, based on the total loss calculated, the gradient of each layer parameter of the original improved YOLOv8 target detection network is calculated by the back propagation algorithm.

[0099] A6, using the optimizer to update the network parameters of the original improved YOLOv8 target detection network according to the gradient to minimize the total loss, wherein the updated original improved YOLOv8 target detection network corresponding to the minimized total loss is the improved YOLOv8 target detection network.

[0100] To get the final available "improved YOLOv8 target detection network", the original network parameters need to be optimized by the following steps:

[0101] Gradient calculation: based on the total loss L, the gradient of each layer parameter (such as convolution kernel weight, BN parameter, etc.) of the network is calculated by the back propagation algorithm, which reflects the influence degree of the parameter on the loss;

[0102] Parameter update: using the optimizer (such as SGD, Adam, etc.) to adjust the network parameters according to the gradient direction, the goal is to minimize the total loss L, that is, to make the predicted bounding box closer to the real box, the class probability more matched to the real class, and the confidence more consistent with the existence of the target;

[0103] Convergence determination: when the total loss decreases to a stable value or reaches the preset training round, stop optimization, and the network at this time is the "improved YOLOv8 target detection network", which can be used for subsequent accurate identification of microalgae cells.

[0104] Before performing A2, the sample image can also be preprocessed, and then the preprocessed sample image is input into the backbone network of the original improved YOLOv8 target detection network for multi-scale feature extraction. The specific preprocessing steps are B1-B4:

[0105] B1, the sample image is enhanced by using the histogram equalization method.

[0106] B2, the image pixel value of the sample image after enhancement is normalized.

[0107] B3, the normalized sample image is scaled according to the preset resolution to obtain a scaled sample image.

[0108] B4, the scaled sample image is input into the backbone network of the original improved YOLOv8 target detection network for multi-scale feature extraction.

[0109] After the improved YOLOv8 target detection network is trained, during use, only the target fluorescence image obtained in step S104 needs to be input into the improved YOLOv8 target detection network for microalgae cell recognition, so that the improved YOLOv8 target detection network can output the detection frame set.

[0110] Notably, before using the improved YOLOv8 target detection network to recognize microalgae cells, the target fluorescence image can also be preprocessed, including:

[0111] The target fluorescence image is enhanced by using a histogram equalization method;

[0112] The image pixel value of the enhanced target fluorescence image is normalized;

[0113] The normalized target fluorescence image is scaled according to a preset resolution to obtain a scaled target fluorescence image;

[0114] The scaled target fluorescence image is input into the improved YOLOv8 target detection network for microalgae cell recognition, and a detection frame set is output.

[0115] S103, input the detection frame set into the Mask R-CNN network for microalgae segmentation to obtain a binary segmentation mask.

[0116] Before using the Mask R-CNN network to segment microalgae, the original Mask R-CNN network also needs to be trained, and the specific training method further includes the following sub-steps:

[0117] C1, obtain a second training sample, the second training sample including a real boundary frame and a real mask of a microalgae cell in a sample image containing the microalgae cell.

[0118] C2, input the real boundary frame into the ResNet50-FPN backbone network of the original Mask R-CNN network to extract multi-scale features to obtain a feature pyramid.

[0119] C3, based on the region proposal network of the original Mask R-CNN network, generate a candidate region on the feature pyramid, for each candidate region, adaptively select a matching feature map level from the feature pyramid according to the size, and through the RoIAlign operation, extract feature information from the corresponding feature map of the selected feature map level to generate a fixed-size region feature map;

[0120] Multi-scale feature extraction is realized by the ResNet50-FPN backbone network of the original Mask R-CNN. The network calculates through multi-layer convolution, and extracts numerical information capable of describing the image content (such as edges, textures and object parts, etc.) from the original image pixels layer by layer, and finally generates a set of multi-scale feature maps {P2, P3, P4, P5}. This set of feature maps is called multi-scale features in the scheme. Based on the anchor mechanism, the region proposal network (RPN) generates candidate regions on these feature maps . Among them, P2 is relatively large in size (such as 256x256), but each value of it may represent relatively basic texture or edge information; P5 is relatively small in size (such as 32x32), but each value of it contains very rich semantic information, which may represent "this is a part of an object".

[0121] C4, the segmentation head with the full convolution structure of the original Mask R-CNN network performs mask prediction on the region feature map to obtain a predicted binary segmentation mask.

[0122] C5, the Dice loss function is used to calculate the Dice loss value between the pixel value of the predicted binary segmentation mask and the pixel value of the real mask, the network parameters of the original Mask R-CNN network are optimized based on the Dice loss value, and finally the optimized original Mask R-CNN network is the Mask R-CNN network.

[0123] The calculation formula of the Dice loss function is:

[0124] ;

[0125] Among them, represents the Dice loss function, N represents the total number of pixels in the predicted mask, represents the i-th pixel value in the predicted mask, represents the i-th pixel value in the real mask.

[0126] It should be noted that the aforementioned "multi-scale features" are global and high-dimensional basic information extracted from the input image by the present disclosure. The subsequent segmentation task is based on the specific region features extracted from these multi-scale features, and a full convolutional segmentation head network is used to generate a "predicted mask" corresponding to the region. In the model training stage, in order to evaluate the accuracy of the "predicted mask", the system compares it with the "real mask" preset in the data set. This comparison is done through a loss function, which is achieved by mathematical operation on the pixel values of the corresponding positions of the two masks. Therefore, the "multi-scale features" are the information source for generating the "predicted mask", and the "pixel values of the predicted mask" and the "pixel values of the real mask" are specific numerical values used to quantify and optimize the prediction results in the training process. The features of each candidate region are aligned to a fixed size through RolAlign operation:

[0127]

[0128] wherein the formula defines a feature extraction method for a detection box : the system first selects the most suitable layer of feature map from a feature pyramid composed of multiple layers with different resolutions according to the size of the detection box; then, through the Region of Interest Alignment (RoIAlign) operation, the feature information of the corresponding region is accurately extracted from the selected feature map in a bilinear interpolation manner, and finally a fixed-size and standardized region feature map is generated for subsequent target classification and bounding box regression. For each detection box, the system will adaptively select a level from the feature pyramid according to its size , and determine the specific feature map used to extract features.

[0129] The segmentation head uses a full convolutional structure to predict the mask , outputting a 28x28 binary matrix . The predicted mask is a low-resolution binary matrix directly output by the network (usually 28x28 pixels), while the final binary segmentation mask is its full-resolution version after spatial restoration. Both are essentially the same segmentation result expressed at different coordinate scales. The predicted mask is upsampled to the original image coordinates through bilinear interpolation to generate the final binary segmentation mask .

[0130] The real mask is the annotation information in the training data. In addition, during the training process, the Dice loss function optimizes the model parameters by comparing the pixel-level similarity of the predicted mask and the real mask, so that the final binary segmentation mask is as close as possible to the boundary and area of the real mask.

[0131] The segmentation head adopts a full convolutional structure for mask prediction, outputs a binary matrix, and after bilinear interpolation upsampling to the original image coordinates, a thresholding is performed to generate the final binary segmentation mask (pixel value is 0 or 1) as the final result of microalgae instance segmentation. The mask loss calculates the pixel-level similarity of the pixel value of the predicted mask and the pixel value of the real mask

[0132] The segmentation head adopts a full convolutional structure to process the ROI features to generate a predicted mask. The predicted mask is a low-resolution probability matrix (e.g., 28x28), where each element value represents the probability of the corresponding pixel belonging to the target. In the inference stage, the probability matrix is bilinearly interpolated and upsampled to the original image resolution, and then binarized by a pre-set threshold to generate the final segmentation mask.

[0133] In the model training stage, a mask loss function (Mask Loss) is used to optimize the parameters of the segmentation head. The loss function measures the similarity between the predicted mask and the real mask (i.e., the annotation information in the training data) by calculating the Dice coefficient (DiceCoefficient). The calculation formula of the mask loss is as follows:

[0134] ;

[0135] The function aligns the pixel value of the predicted mask with the pixel value of the real mask , N is the total number of pixels in the mask (i.e., image height x image width), which ensures the accuracy of the segmentation boundary, thereby avoiding under-segmentation (missed detection) or over-segmentation (false detection), which directly affects the reliability of the subsequent connected region . The mask here is the direct output generated by the segmentation network based on the aforementioned multi-scale features after region alignment and feature processing, representing the preliminary prediction of the target contour.

[0136] This binary segmentation mask is denoted as M seg , which fully retains the spatial distribution information of the microalgae instances and can directly provide input for the morphological repair and area constraint counting in step S104.

[0137] S104, obtaining the number of microalgae through the binary segmentation mask.

[0138] ​In one embodiment, the obtaining the microalgae quantity by binarizing the segmentation mask comprises the following sub-steps S1041-S1043:

[0139] S1041, performing a morphological closing operation on the input binarized segmentation mask to generate a repaired mask;

[0140] S1042, identifying all independent connected regions in the repaired mask;

[0141] S1043, screening all independent connected regions by a preset minimum area threshold to count the effective microalgae quantity, wherein the minimum area threshold is determined by the minimum projection area of microalgae and the actual area of a single pixel of the imaging system.

[0142] The mathematical expression of the morphological closing operation is:

[0143] ;

[0144] Wherein, represents the binarized segmentation mask, represents a predefined structural element, the shape and size of which are set to be able to effectively fill the typical voids in the microalgae cells and connect the broken boundaries, represents a morphological closing operation, represents the repaired mask.

[0145] This step aims to count the segmentation results generated in the previous step to obtain the accurate microalgae quantity. This process is based on the final binarized segmentation mask (denoted as ) output by the previous step, and ensures the biological reasonableness of the counting result through morphological repair and physical area constraint two stages.

[0146] 1. Mask repair based on morphological closing operation.

[0147] In order to eliminate the noise that may be generated in the segmentation process, such as small holes in the target or boundary fracture, first perform a morphological closing operation on the input segmentation mask to generate a repaired mask The mathematical definition of this operation is as follows:

[0148] ;

[0149] Wherein: is the binarized segmentation mask containing all candidate targets generated by step S106 (microalgae segmentation). is a predefined structural element, the shape and size (for example, a circular core) of which are set to be able to effectively fill the typical voids in the microalgae cells and connect the broken boundaries. Representative morphological closing operation (first dilation then erosion). is the output mask after repair. This operation improves the morphological integrity of each connected region in the image , making it more consistent with the physical profile of real microalgae.

[0150] 2. Area threshold-based screening and counting.

[0151] After obtaining the repaired mask , the system identifies all the independent connected regions in it, where is the total number of candidate regions identified. Then, a preset minimum area threshold based on physical size is used to filter out false or non-target regions, and the final count is completed. The final number of microalgae is calculated as follows:

[0152] ;

[0153] where: is the final number of effective microalgae after screening. is the total number of candidate connected regions detected in before area screening. is the area of the jth connected region . For a binary segmented mask, this area is the total number of pixels in the region. is the preset minimum area threshold (unit: pixels). is an indicator function, δ(condition) = {1, 0}, when the area of the region is greater than or equal to the preset minimum area threshold , the region is counted as an effective target (counted as 1), otherwise discarded (counted as 0).

[0154] The preset minimum area threshold is determined by the physical parameters of the imaging system, ensuring reliable conversion from pixel-level statistics to biological counts. is determined by the spatial resolution of the imaging system (such as pixels) and the minimum physical size of microalgae (such as diameter , corresponding to the projected area ):

[0155] ;

[0156] This formula converts pixel-level statistics to biomass counts, ensuring that only regions meeting the target size characteristics are retained.

[0157] Microalgae counting directly depends on the segmented mask of microalgae in step S106 The accuracy directly affects the counting result. Through fine segmentation, the adhered microalgae cells can be effectively distinguished, and the counting deviation caused by under-segmentation or over-segmentation can be reduced; and the area threshold filtering further excludes non-target noises, and improves the robustness of counting. If there is a small amount of error in the segmentation stage (such as misjudgment of a small area), the error can be corrected by adjusting , to ensure that the final counting result conforms to the actual observation value.

[0158] S105, transmitting the microalgae quantity to a target device through the communication buoy.

[0159] The system in the present disclosure is used for monitoring and early warning of disaster-causing marine organisms in coastal nuclear power plants, ecologically fragile areas and marine protected areas, has the advantages of real-time transmission of marine disaster-causing organism warning and observation monitoring data, lightness, miniaturization, etc., and can be directly put into use and recovered by using small ships. Under normal circumstances, the buoy uses an anchor and a chain as a buoy mooring device, and the present disclosure uses an armored cable as a buoy anchor chain and also serves as a power supply and data transmission function.

[0160] It is worth noting that the method steps in the present disclosure can be executed by a processor in the underwater imager.

[0161] In related technologies, there are two forms of data acquisition by underwater observation and monitoring platforms. One is a self-contained seabed observation system, which is powered by a battery and does not have real-time data transmission function for underwater data collection. The data is extracted by periodic recovery. The second form is a tethered seabed observation station, which requires a shore-based station house. The power is transmitted from the shore-based station house to the underwater observation and monitoring platform through the cable. This form can realize real-time data acquisition, but cannot get rid of the dependence on the shore-based station house and shore-based power supply. The system proposed in the present disclosure shifts the second form of shore-based station house to the communication buoy, realizes real-time underwater microscopic imaging and data transmission and early warning of water quality. The electronic cabin of the present system has a standard interface access function, and can increase corresponding sensors according to the observation and monitoring needs of coastal nuclear power plants and marine protected areas for scientific early warning and monitoring.

[0162] Further, the underwater microscopic imager in the present disclosure collects images under different light source states, processes high-quality target fluorescence images by combining time sequence difference method and spectral difference method, detects and segments the images through improved YOLOv8 network and Mask R-CNN network, and finally obtains the quantity of microalgae through segmentation mask. The dual-band light source of the underwater microscopic imager in the system provides multi-dimensional data basis for fluorescence image collection, the armored cable serves as power supply and data transmission, the universal joint ring avoids cable twisting to ensure stability, and the counterweight ensures stable device posture. The cooperation of hardware and algorithm realizes accurate identification, segmentation and counting of microalgae cells, provides reliable biomass data for marine ecological early warning, and meets the needs of real-time monitoring of disaster-causing marine organisms in coastal nuclear power plants and other areas.

[0163] Specifically, the system sets a communication buoy above the sea surface, which can serve as an independent data transmission node. Unlike traditional seabed-based observation systems, the communication buoy does not rely on shore-based stations for data transmission, breaking free from the shore-based station's limitations on off-shore observation range and enabling more extensive off-shore observation. The system is equipped with an independent electronic cabin, which is specifically powered for the underwater microscopic imaging instrument. This independent power supply method enables the entire detection device to operate independently in areas far from the shore without relying on shore-based power supply for the underwater system, overcoming the dependence on shore-based power supply for cable-based seabed observation and achieving truly full-coverage off-shore observation. Further, by acquiring fluorescence images under different light source illumination and off states, and using time difference method and spectral difference method to operate the images, net fluorescence images and spectral difference images are obtained, and finally the final difference fluorescence images are synthesized. This method processes and simplifies the original fluorescence images, removing a large amount of redundant information, greatly reducing the data transmission volume compared to directly transmitting large amounts of original data. Using the improved YOLOv8 target detection network and Mask R-CNN network, the target fluorescence images are processed locally by the underwater microscopic imaging instrument to recognize, segment, and count microalgae cells, and only the key result data such as the number of microalgae is transmitted through the communication buoy. This greatly reduces the amount of data that needs to be transmitted, avoiding the transmission of large amounts of raw data as in traditional observation systems, thereby achieving effective processing and transmission of large amounts of data under limited transmission conditions.

[0164] Furthermore, the counterweight is connected to the armored cable, and a universal joint is provided at the connection between the armored cable and the communication buoy. The counterweight can maintain a stable tension of the armored cable in seawater, avoiding cable sway or displacement due to factors such as water flow; the universal joint allows the armored cable to rotate freely within a certain range, preventing cable entanglement caused by tides, ocean currents, etc., improving the stability of the entire system in complex marine environments and ensuring the reliability of data transmission and equipment operation.

[0165] It should be noted that for those skilled in the art, it is obvious that the present disclosure is not limited to the details of the above exemplary embodiments, and the present disclosure can be implemented in other specific forms without departing from the spirit or essential characteristics of the present disclosure. Therefore, the embodiments should be considered as exemplary and non-limiting, and the scope of the present disclosure is defined by the appended claims rather than the above description, and all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the disclosure. Any reference signs in the claims should not be considered as limiting the claims involved.

Claims

1. A method for detecting marine organisms and their environment that cause disasters, characterized in that, The method is applied to an underwater microscopic imager in a system for detecting marine organisms and their environment that cause damage. The system includes: a device for detecting marine organisms and their environment that cause damage, a counterweight, and a communication buoy positioned above the sea surface. The device includes an underwater support platform, an underwater microscopic imager mounted on the support platform, and an electronics compartment for powering the imager. An armored cable is used to electrically connect the support platform and the communication buoy. The counterweight is connected to the armored cable. The method includes: Acquire target fluorescence image; The target fluorescence image is input into the improved YOLOv8 target detection network for microalgae cell identification, and a set of detection boxes is output; the improved YOLOv8 target detection network is used to identify microalgae cells. The set of detection boxes is input into the Mask R-CNN network for microalgae segmentation to obtain a binary segmentation mask; Microalgae counts were obtained using a binarized segmentation mask. The number of microalgae is transmitted to the target device via a communication buoy; The underwater microscopic imager emits a first-band light source and a second-band light source; acquiring the target fluorescence image includes: The first fluorescence image, the second fluorescence image, and the background fluorescence image were acquired under illumination by the first band light source, under illumination by the second band light source, and under the light source off state, respectively. A net fluorescence image is obtained by performing a difference operation on the original fluorescence image and the background fluorescence image using the time-series difference method; the original fluorescence image is either a first fluorescence image or a second fluorescence image. The spectral difference method is used to perform a difference operation on the first fluorescence image and the second fluorescence image to obtain a spectral difference image; Based on the net fluorescence image and the spectral difference image, the final differential fluorescence image is obtained, and the final differential fluorescence image is the target fluorescence image.

2. The method according to claim 1, characterized in that, The method further includes: Obtain the first training sample, which includes: sample images containing microalgae cells, and annotation information corresponding to each sample image. The annotation information includes: the true bounding box coordinates of the microalgae cells, the true label of the category, and the true confidence value of the existence of the target. The sample images are input into the backbone of the original improved YOLOv8 object detection network for multi-scale feature extraction. The backbone of the original improved YOLOv8 object detection network adopts CSPDarknet. Multi-scale features are input into the decoupled detection head of the original improved YOLOv8 object detection network to predict the bounding box coordinates of microalgal cells, calculate class probabilities and confidence scores; The total loss is calculated based on the predicted bounding box coordinates of the microalgal cells, the predicted class probability, the confidence level, the annotation information, and the total loss function of the detection network. Based on the calculated total loss, the gradients of the parameters of each layer of the original improved YOLOv8 object detection network are calculated using the backpropagation algorithm; The optimizer updates the network parameters of the original improved YOLOv8 object detection network according to the gradient to minimize the total loss. The updated original improved YOLOv8 object detection network corresponding to minimizing the total loss is called the improved YOLOv8 object detection network.

3. The method according to claim 2, characterized in that, The total loss function of the detection network is expressed as: ； ； ； ； Where L represents the total loss function of the detection network, Represents the bounding box loss function. This represents the predicted bounding box coordinates of the microalgal cells. This represents the coordinates of the true bounding box of the microalgal cell. Represents the classification loss function. Indicates the true label of the category. Represents the predicted class probability. Represents the confidence loss function. The true value representing the confidence level of the existence of the target. Indicates the confidence level of the prediction. and This represents the weighting coefficient.

4. The method according to claim 3, characterized in that, The method further includes: Obtain a second training sample, which includes: the true bounding box and true mask of microalgae cells in a sample image containing microalgae cells; The ground truth bounding boxes are fed into the ResNet50-FPN backbone network of the original Mask R-CNN network for multi-scale feature extraction, resulting in a feature pyramid. The region proposal network based on the original Mask R-CNN network generates candidate regions on the feature pyramid. For each candidate region, a matching feature layer level is adaptively selected from the feature pyramid according to the size. The RoIAlign operation is used to extract feature information from the feature map corresponding to the selected feature layer level to generate a fixed-size region feature map. The segmentation head, which uses the fully convolutional structure of the original Mask R-CNN network, is used to predict the mask of the region feature map, resulting in a predicted binarized segmentation mask. The Dice loss function is used to calculate the Dice loss value by comparing the pixel values ​​of the predicted binarized segmentation mask with the pixel values ​​of the real mask. Based on the Dice loss value, the network parameters of the original Mask R-CNN network are optimized, and the optimized original Mask R-CNN network is finally obtained as the Mask R-CNN network.

5. The method according to claim 4, characterized in that, The formula for calculating the Dice loss function is as follows: ； in, This represents the Dice loss function, where N represents the total number of pixels in the predicted mask. This represents the value of the i-th pixel in the predicted mask. This represents the i-th pixel value of the actual mask.

6. The method according to claim 5, characterized in that, The method of obtaining the number of microalgae through binarized segmentation masking includes: Perform morphological closing operations on the input binarized segmentation mask to generate the repaired mask; Identify all independent connected regions in the repaired mask; All independent connected regions are filtered by a preset minimum area threshold to count the number of effective microalgae. The minimum area threshold is determined by the minimum projected area of ​​the microalgae and the actual area of ​​a single pixel in the imaging system.

7. The method according to claim 6, characterized in that, The mathematical expression for the morphological closing operation is: ； in, This represents a binary segmentation mask. This represents a predefined structural element. The shape and size were designed to effectively fill the typical voids within microalgal cells and connect broken boundaries. This represents the morphological closing operation. This indicates the repaired mask.

8. The method according to claim 1, characterized in that, The process involves inputting the target fluorescence image into an improved YOLOv8 target detection network for microalgae cell recognition, outputting a set of detection boxes, including: Histogram equalization was used to enhance the target fluorescence image. The pixel values ​​of the enhanced target fluorescence image are normalized. The normalized target fluorescence image is scaled according to a preset resolution to obtain a scaled target fluorescence image; The scaled target fluorescence image is input into the improved YOLOv8 target detection network for microalgae cell recognition, and a set of detection boxes is output.

9. A system for detecting marine organisms and the environment that cause disasters, characterized in that, The system includes: a disaster-causing marine organism and environmental detection device, a counterweight, and a communication buoy set above the sea surface; The disaster-causing marine organism and environmental detection device includes an underwater support platform, an underwater microscopic imager mounted on the underwater support platform, and an electronic compartment for powering the underwater microscopic imager. Armored cables are used for electrical connection between the underwater support platform and the communication buoy; The counterweight is connected to the armored cable; The underwater microscopic imager is used to perform the method as described in any one of claims 1-8.

Images