A method and system for target detection of underwater in-situ images

Abstract

The application relates to a target detection method and system for underwater in-situ images, which comprises the following steps: preprocessing an original image to obtain a gray image; moving a sliding window to traverse the gray image; when the sliding window moves each time, a first quantile of the brightness value of each pixel in the image sub-block corresponding to the current sliding window is obtained, all the first quantiles obtained by traversal are arranged in sequence to obtain a quantile list; a second quantile of the quantile list is calculated, the second quantile is added with a preset offset value as a quantile threshold, then the quantile threshold is compared with the first quantile to judge whether the image sub-block is foreground, and a binary image is constructed; a connected domain detection algorithm is used to calculate the connected domain on the binary image, the coordinates of the circumscribed rectangle frame of the connected domain are obtained, and the coordinates of the circumscribed rectangle frame are mapped back to the gray image to cut and obtain an ROI image.

Description

Technical Field

[0001] This application relates to the field of water environment monitoring, and in particular to a target detection method and system for underwater in-situ images. Background Technology

[0002] Plankton are widely distributed in the ocean and are a key link in marine ecosystems and marine food webs. Their harmful outbreaks can cause huge losses and are of great research value and observation significance.

[0003] In-situ plankton observation technology based on optical imaging typically involves the following steps: first, capturing raw images underwater using an in-situ imager; then, preprocessing, including target detection; and finally, storing, transmitting, identifying, measuring, and analyzing the target images to obtain the observation results. Target detection involves extracting the target regions from the original images, then cropping them to obtain smaller images (ROIs) containing only one target, facilitating subsequent identification, processing, and analysis. The effectiveness and efficiency of target detection significantly impact the accuracy of subsequent identification and analysis, as well as the overall efficiency of the in-situ observation system.

[0004] In related technologies, Campbell R et al. disclosed an in-situ observation method for plankton and microparticles. First, the original image is downsampled four times using nearest-neighbor interpolation to convert it into an 8-bit image to reduce computational load. Then, Canny edge detection is applied to obtain the location of targets in the image. Next, morphological closing operations are used to close broken edges. Then, OpenCV's findcontours function is used to detect the target's contour, obtaining the target's bounding box. For targets with a boundary greater than 300 pixels, the bounding rectangle is expanded outward from the center by a factor of 50%. Finally, the ROI image corresponding to the bounding box is segmented from the original image.

[0005] Yamazaki H et al. disclosed a wired observation system for comprehensive long-term, high-frequency biological, chemical, and physical measurements to understand planktonic ecosystems. The system uses a blob feature detection algorithm to detect continuous regions with significant brightness differences from surrounding pixels. A threshold is set for the number of continuous pixels; if the number exceeds this threshold, the region is identified as a target, and its bounding box is obtained. The size of the bounding box is then expanded according to a pre-set factor, and the region of interest (ROI) sub-image is obtained by cropping.

[0006] Cheng K et al. disclosed an enhanced convolutional neural network for planktonic organism identification and enumeration. First, a brightness threshold is set to binarize the image. Then, a morphological opening operation is performed on the image to eliminate background noise. Next, the target location is obtained through target edges, and finally, a target ROI sub-image is obtained by cropping.

[0007] Geraldes P et al. disclosed an in-situ real-time zooplankton detection and classification method that utilizes a deep neural network model for target detection. First, a training dataset is constructed by manually annotating targets in a large number of images, and then the network is trained. After training, the original images are input into the network, and the target bounding box information in the image can be obtained at the output, which is then used to crop the ROI sub-image.

[0008] However, some existing detection methods suffer from poor detection results due to background noise interference, making them difficult to be effective in high-turbidity seawater; some detection methods require manual parameter setting, which may need to be reset when the image brightness changes; some detection methods tend to ignore relatively transparent targets in the image, resulting in missed detections and relying on a large number of labeled images as training sets. Furthermore, neural network models have long training times and high computational complexity, which is not conducive to deploying algorithms on low-power, low-cost, and low-computing-power platforms. Summary of the Invention

[0009] This application provides a target detection method and system for underwater in-situ images to solve the problems in related technologies, such as susceptibility to background noise interference, poor adaptability, low efficiency, and difficulty in deploying algorithms on low-power, low-cost, and low-computing-power platforms.

[0010] To achieve the above objectives, the present invention employs the following technical solution:

[0011] A target detection method for underwater in-situ images, characterized by the following steps:

[0012] S1. Preprocess the original image to obtain a grayscale image; use a sliding window to move and traverse the grayscale image; each time the sliding window moves, obtain the first quantile of the brightness value of each pixel in the image sub-block corresponding to the current sliding window, and arrange all the first quantiles obtained in the traversal in order to obtain a quantile list;

[0013] S2. Calculate the second quantile of the quantile list, add a preset bias value to the second quantile as the quantile threshold, then compare the quantile threshold with the first quantile to determine whether the image sub-block is foreground, and construct a binary image;

[0014] S3. Using a connected component detection algorithm, calculate the connected components on the binary image, obtain the coordinates of the bounding rectangle of the connected components, map the coordinates of the bounding rectangle back onto the grayscale image, and crop to obtain the ROI image.

[0015] In some embodiments, constructing the binary image includes:

[0016] Each pixel represents a sliding window on a grayscale image. Its vertical axis indicates the row number of the sliding window in the grayscale image, and its horizontal axis indicates which sliding window in which row it is. Depending on whether the image sub-block belongs to the foreground or the background, the pixel value is represented by 1 or 0 respectively.

[0017] In some embodiments, determining whether the image sub-block is foreground in step S2 includes:

[0018] Determine the relationship between the first quantile in the list and the quantile threshold. If the former is greater than the latter, the image sub-block corresponding to the current first quantile is the foreground; otherwise, it is the background.

[0019] In some embodiments, step S3 includes:

[0020] S31. Using a connected component detection algorithm, calculate the connected components on the binary image and obtain the coordinates of the bounding rectangle of the connected components;

[0021] S32. Transform the coordinates of the circumscribed rectangle onto the grayscale image to obtain target positioning information;

[0022] S33. Based on the target positioning information, crop the required rectangular region from the original image to obtain an ROI image containing the planktonic target.

[0023] In some embodiments, the raw image preprocessing includes converting the raw image to a grayscale image and reducing its size.

[0024] In some embodiments, the relationship between the sliding window's movement step size and the sliding window's width is as follows:

[0025] Patch_step=Factor*Patch_size / 2;

[0026] Where Patch_step is the step size of the sliding window, Factor is the scaling factor of the grayscale image, and Factor*Patch_size is the width of the sliding window.

[0027] In some embodiments, when the sliding window is rectangular, the coordinates of the outer rectangle are transformed to a grayscale image using the following formula:

[0028] x1'=x1*Patch_step / Factor; y1'=y1*Patch_step / Factor;

[0029] x2'=x2*Patch_step / Factor; y2'=y2*Patch_step / Factor;

[0030] Where x1, y1, x2, and y2 are the coordinate values ​​of the circumscribed rectangle, and x1', x2', y1', and y2' are the coordinate values ​​of the corresponding rectangle in the grayscale image.

[0031] In some embodiments, the sliding window is square, rectangular, circular, trapezoidal, or triangular in shape.

[0032] In some embodiments, the values ​​of the first quantile and the second quantile range from 25% to 75%.

[0033] A target detection system for underwater in-situ images, characterized in that it comprises: a computer-readable storage medium and a processor;

[0034] The computer-readable storage medium is used to store executable instructions;

[0035] The processor is used to read executable instructions stored in the computer-readable storage medium and execute the target detection method for underwater in-situ images described above.

[0036] The beneficial effects of the technical solution provided in this application include:

[0037] (1) This invention utilizes a sliding window and describes the image features within the sliding window using quantiles. It has stronger adaptive capabilities, strong resistance to interference from background noise and brightness changes, and can meet the accuracy requirements for target detection in different turbidity seawater environments.

[0038] (2) The present invention uses quantiles to adaptively determine the threshold for dividing the foreground and background. Compared with existing non-learning-based techniques, it has better adaptability, can adapt to changes in image brightness, and has high algorithm sensitivity and good detection effect for low-brightness targets. The algorithm does not require training, is easy to implement, and contains fewer manual adjustment parameters, making it easy to use.

[0039] (3) The present invention maps the sliding window on the original image to a binary small image and performs target bounding box detection on it. The image size processed in this step is smaller and the computational efficiency is higher. The computational optimization method used has a smaller computational load. It can meet the real-time processing requirements of the low-power, low-computing-power embedded computing platform built into the plankton in-situ imager, and can also be post-processed on the cloud server. It has a variety of uses and a wide range of application scenarios.

[0040] This application provides a target detection method and system for underwater in-situ images. The invention first downsamples the image by reducing its size, then uses a sliding window to traverse the image. Simultaneously, it calculates the pixel brightness quantiles within the sliding window region, and then uses these quantiles as a threshold to determine whether the image region corresponding to the sliding window is foreground. Next, a binary image is generated, and the bounding box of the foreground is calculated on it. Finally, the coordinates of the bounding box are mapped back to the original image, and the area of ​​interest (ROI) image is cropped. Therefore, this invention is easy to implement, has stronger adaptability and high sensitivity, requires less computation, and has a wider range of applications. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a flowchart of the underwater in-situ image target detection method in an embodiment of the present invention;

[0043] Figure 2 This is an example of in-situ target detection of planktonic organisms in an embodiment of the present invention. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0045] This application provides a target detection method and system for underwater in-situ images. Working in conjunction with a plankton imager, it solves the problem of locating and extracting the foreground image of the target from the raw images acquired by the imager, which can prepare for subsequent identification, measurement and analysis of the data and improve its accuracy.

[0046] Reference Figure 1 A target detection method for underwater in-situ images includes the following specific steps:

[0047] Step 1: Initialize the original image by converting the color image to grayscale for easier processing and reducing the image size to speed up calculations;

[0048] S11. Average the RGB three channels of the original image I_raw pixel by pixel to obtain the grayscale image I_raw_gray;

[0049] S12. Scale the width and height of the grayscale image I_raw_gray by a factor of Factor to obtain the scaled image I_gray.

[0050] Step 2: Sliding window quantile statistics. The image is traversed using a sliding window. In this embodiment, the 50th percentile of the pixel value in the image sub-block corresponding to each sliding window is calculated, so that this method has better resistance to background noise interference.

[0051] S21. In this embodiment, a sliding window with a window width of Factor*Patch_size is used to move row by row from left to right with a step size of Patch_step, traversing the scaled image I_gray, where Patch_step = Factor*Patch_size / 2.

[0052] S22. Each time the sliding window moves, calculate the 50th percentile Q_0.5 of the brightness value of each pixel in the image sub-block corresponding to the current sliding window, sorted from smallest to largest, and record it in the list Qs_0.5.

[0053] In some embodiments, the quantile of the brightness value of each pixel within the image sub-block corresponding to each sliding window ranges from 25% to 75%.

[0054] In some embodiments, the shape of the sliding window can be a closed shape of any two-dimensional plane, such as a rectangle, circle, trapezoid, or triangle.

[0055] In some embodiments, the traversal order of the sliding windows can be column-by-column from top to bottom, out-of-order traversal, or a parallel computing program can be written to compute all sliding windows simultaneously.

[0056] Step 3: Sliding window foreground judgment. Determine whether the sub-image corresponding to the sliding window is a foreground region containing the target. This makes the method more adaptive and can adapt to changes in image brightness.

[0057] S31. In this embodiment, the 75th percentile Q_0.75 of Qs_0.5 sorted from smallest to largest is calculated, and Q_bias (empirically taken as 2) is added to obtain the quantile threshold Q_thresh that distinguishes between foreground and background;

[0058] S32. Determine the relationship between each element Q_0.5 and Q_thresh in Qs_0.5. If Q_0.5 > Q_thresh, then mark the image sub-block within the sliding window range corresponding to the current Q_0.5 as the foreground region; otherwise, mark the sub-block as the background region.

[0059] S33. Construct a binary image I_bin. Each pixel represents a sliding window on the grayscale image I_gray. Its ordinate indicates the row number of the sliding window in the grayscale image I_gray, and its abscissa indicates which row the sliding window belongs to. Depending on whether the sliding window belongs to the foreground or background, the pixel value is represented by 1 or 0, which accelerates the calculation process of subsequent target localization and ROI extraction steps.

[0060] Step 4: Target localization and ROI extraction. Separate the foreground target from the binary image I_bin, map the coordinates to the original image I_raw, and extract the target image.

[0061] S41. In this embodiment, the connected component detection algorithm is used to calculate the connected components on the binary image I_bin, and the coordinates of the upper left corner (x1, y1) and the lower right corner (x2, y2) of its bounding rectangle are obtained;

[0062] S42. Convert the target bounding box coordinates on the binary image I_bin to the grayscale image I_gray:

[0063] x1'=x1*Patch_step / Factor; y1'=y1*Patch_step / Factor

[0064] x2'=x2*Patch_step / Factor;; y2'=y2*Patch_step / Factor;

[0065] S43. Crop a rectangular region with the top left and bottom right corners (x1', y1') and (x2', y2') respectively from the original image I_raw to obtain an ROI image containing the planktonic target.

[0066] In some embodiments, the traversal order of the sliding windows can be column-by-column from top to bottom, out-of-order traversal, or a parallel computing program can be written to compute all sliding windows simultaneously.

[0067] The results of in-situ target detection of plankton are as follows: Figure 2 As shown, the left image is an original image taken in highly turbid seawater containing two *Phaeocystis* targets. *Phaeocystis* has high transparency, and its brightness is close to the background, making it difficult to detect. The white rectangle in the left image shows the detection effect of this invention on the target location in the image, while the right image shows the extracted ROI.

[0068] Table 1 shows a comparison of the meanings of the variable names involved in a target detection method for underwater in-situ images.

[0069] Table 1

[0070]

[0071]

[0072] This invention also provides an underwater in-situ image target detection system, comprising: a computer-readable storage medium and a processor;

[0073] The computer-readable storage medium is used to store executable instructions;

[0074] The processor is used to read executable instructions stored in the computer-readable storage medium and execute the target detection method for underwater in-situ images described above.

[0075] It should be noted that in this invention, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0076] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A target detection method for underwater in-situ images, characterized in that, The steps include: S1. Preprocess the original image to obtain a grayscale image; use a sliding window to move and traverse the grayscale image; each time the sliding window moves, obtain the first quantile of the brightness value of each pixel in the image sub-block corresponding to the current sliding window, and arrange all the first quantiles obtained in the traversal in order to obtain a quantile list; S2. Calculate the second quantile of the quantile list, add a preset bias value to the second quantile as the quantile threshold; then, compare each first quantile of the quantile list with the quantile threshold to determine whether the image sub-block belongs to the foreground or the background, and construct a binary image based on the determination result; wherein, each pixel in the binary image represents a sliding window on the grayscale image, and the pixel value is represented by 1 or 0 depending on whether the image sub-block belongs to the foreground or the background; S3. Using a connected component detection algorithm, calculate the connected components on the binary image, obtain the coordinates of the bounding rectangle of the connected components, map the coordinates of the bounding rectangle back onto the grayscale image, and crop to obtain the ROI image.

2. The target detection method according to claim 1, characterized in that, The construction of the binary image includes: Each pixel represents a sliding window on a grayscale image. Its vertical axis indicates the row number of the sliding window in the grayscale image, and its horizontal axis indicates which sliding window in which row it is. Depending on whether the image sub-block belongs to the foreground or the background, the pixel value is represented by 1 or 0 respectively.

3. The target detection method according to claim 1, characterized in that, Determining whether the image sub-block is foreground in step S2 includes: Determine the relationship between the first quantile in the list and the quantile threshold. If the former is greater than the latter, the image sub-block corresponding to the current first quantile is the foreground; otherwise, it is the background.

4. The target detection method according to claim 1, characterized in that, Step S3 includes: S31. Using a connected component detection algorithm, calculate the connected components on the binary image and obtain the coordinates of the bounding rectangle of the connected components; S32. Transform the coordinates of the circumscribed rectangle onto the grayscale image to obtain target positioning information; S33. Based on the target positioning information, crop the required rectangular region from the original image to obtain an ROI image containing the planktonic target.

5. The target detection method according to claim 1, characterized in that, The original image preprocessing includes converting the original image into a grayscale image and reducing its size.

6. The target detection method according to claim 5, characterized in that, The relationship between the sliding window's movement step size and its width is as follows: Patch_step=Factor*Patch_size / 2; Where Patch_step is the step size of the sliding window, Factor is the scaling factor of the grayscale image, and Factor*Patch_size is the width of the sliding window.

7. The target detection method according to claim 4, characterized in that, When the sliding window is rectangular, the coordinates of the outer rectangle are transformed to the grayscale image using the following formula: x1'=x1*Patch_step / Factor; y1'=y1*Patch_step / Factor; x2'=x2*Patch_step / Factor; y2'=y2*Patch_step / Factor; Where x1, y1, x2, and y2 are the coordinate values ​​of the circumscribed rectangle, and x1', x2', y1', and y2' are the coordinate values ​​of the corresponding rectangle in the grayscale image.

8. The target detection method according to claim 1, characterized in that, The sliding window can be square, rectangular, circular, trapezoidal, or triangular in shape.

9. The target detection method according to claim 1, characterized in that, The values ​​of the first quantile and the second quantile range from 25% to 75%.

10. A target detection system for underwater in-situ images, characterized in that, include: Computer-readable storage media and processors; The computer-readable storage medium is used to store executable instructions; The processor is used to read executable instructions stored in the computer-readable storage medium and execute the underwater in-situ image target detection method according to any one of claims 1 to 9.

Images