A method and apparatus for detecting the density of seabed products based on a monocular underwater robot

Abstract

This invention provides a method and apparatus for detecting the density of marine products based on a monocular underwater robot, comprising: acquiring images of marine products using a monocular underwater robot; inputting the images into a trained target detection model to obtain the quantity of marine products within the image area, wherein the target detection model is a Yolox target detection network with a self-attention mechanism (SENet) added to the feature extraction backbone; performing HSV transformation on the marine product images, extracting the laser line range from the transformed images, and then performing one-dimensional projection processing on the extracted laser lines; determining the area of ​​the current marine product image based on the pixel length of the projection result and the actual distance between two laser lines; and calculating the marine product density based on the area of ​​the current marine product image and the quantity of marine products within the image area. This invention mainly utilizes the self-attention mechanism and a monocular underwater robot in conjunction with laser positioning technology to achieve rapid detection of marine life and accurate measurement of area.

Description

Technical Field

[0001] This invention relates to the fields of deep learning and computer vision, and in particular to a method and apparatus for detecting the density of seabed products based on a monocular underwater robot. Background Technology

[0002] In recent years, with the in-depth development and utilization of marine resources, advancements in fields such as computer vision have played a crucial role in the exploration of marine resources. Seabed products such as sea cucumbers and sea urchins have become increasingly popular high-value products, and the density of these products is a key factor determining their nutritional content and market value. However, due to the complex and diverse underwater environment, ordinary density detection methods struggle to accurately identify seabed targets. The large amount of suspended matter in seawater causes light intensity attenuation, and the generally uneven seabed topography makes it difficult to calculate area. Therefore, accurately estimating the distribution density of seabed products to determine their economic value has become a significant problem. Thus, using a monocular underwater robot to determine the area of ​​the seabed and employing detection algorithms to determine the density of products in that area, in order to solve the problem of estimating product density in complex seabed topography, is of great research significance.

[0003] Underwater product density detection utilizes knowledge from fields such as deep learning and computer vision to identify the number of targets in captured underwater images. It then calculates the product density of a region by converting the image coordinate system to a world coordinate system using camera intrinsic and extrinsic parameters. Traditional deep learning-based underwater target detection methods typically involve acquiring datasets via cameras, preprocessing and augmenting the images, classifying and labeling them, and then using stratified sampling to divide the datasets into training, validation, and test sets to train a model for underwater target detection.

[0004] Existing methods for detecting seabed density have made relatively slow progress in the underwater field due to the harsh seabed environment, uneven terrain, poor visibility, and the effects of light absorption and scattering. Researchers have made some attempts at seabed density detection technology in recent years, but have yet to overcome the problems of low accuracy in detection models and inaccurate calculations of seabed area. Summary of the Invention

[0005] In view of the shortcomings of existing technologies, this invention provides a method and device for detecting the density of marine products based on a monocular underwater robot. This invention mainly utilizes a self-attention mechanism and a monocular underwater robot in conjunction with laser positioning technology to achieve rapid detection of marine organisms and accurate measurement of area.

[0006] The technical means employed in this invention are as follows:

[0007] A method for detecting the density of seabed products based on a monocular underwater robot includes the following steps:

[0008] The image of the seabed product is acquired by a monocular underwater robot, which is equipped with a laser projection device. The laser projection device is used to project two parallel green laser lines with a fixed distance onto the seabed, and the seabed product image contains the laser lines.

[0009] The seabed product image is input into the trained target detection model to obtain the number of sea products within the image range. The target detection model adopts an improved Yolox target detection network, which is a self-attention mechanism SENet added to the feature extraction backbone part of the Yolox target detection network.

[0010] The seabed product image is converted to HSV, the laser line range is extracted from the converted image, and then the extracted laser lines are subjected to one-dimensional projection processing. Based on the pixel length of the projection result and the actual distance between the two laser lines, the area of ​​the current seabed product image is determined.

[0011] The density of the marine products is calculated based on the area of ​​the current seabed product image and the number of marine products within the image area.

[0012] Furthermore, the training steps of the target detection model include:

[0013] Obtain a dataset of underwater product images, and divide the dataset into a training set, a test set, and a validation set;

[0014] The segmented data is input into the improved Yolox object detection network, and the bias and weights are updated through the backpropagation algorithm to finally obtain an object detection model that predicts the object category and location.

[0015] Furthermore, the biases and weights are updated using the backpropagation algorithm, including:

[0016] The bias and weights are updated based on the following calculations:

[0017]

[0018]

[0019] Among them, b l w represents the bias of layer l. l Represents the weights of the l-th layer, and α is the learning rate. For differentiation, C represents the loss function.

[0020] Furthermore, the underwater product image is converted to HSV, including converting the underwater product image from RGB color space to HSV color space.

[0021] Furthermore, the laser line range is extracted from the converted image, and then the extracted laser lines are subjected to one-dimensional projection processing. Based on the pixel length of the projection result and the actual distance between the two laser lines, the area of ​​the current seabed product image is determined, including:

[0022] Set the HSV value range of the converted image, and create a mask with the same size as the original image. The pixel value in the mask is 0 or 255, where a pixel with a value of 255 indicates that the location may contain a green laser line, while a pixel with a value of 0 indicates that the location cannot contain a green laser line.

[0023] Based on the set HSV value range, the input HSV image is binarized to obtain a binarized image of the same size as the original image. The pixel values ​​outside the HSV value range are set to 0, and the pixel values ​​within the HSV value range are set to 255.

[0024] The laser line region is projected from 2D to 1D. The binarized image is summed column-wise to obtain a one-dimensional array. The array is then divided into two parts, left and right, and the maximum value is calculated for each part. The difference between the indices of the two values ​​is the pixel length of the laser line on the x-axis. This pixel length is then fed into a Yolox detector for processing. The Yolox detector calculates the pixel length of the laser line on the y-axis. The actual distance between laser lines is divided by the number of pixels to obtain the actual distance represented by each pixel. Since the pixel lengths on the x and y axes are known, multiplying them gives the area S of the entire image. i :

[0025]

[0026] Among them, X l Y1 represents the pixel length on the x-axis, Y2 represents the pixel length on the y-axis, and D represents the actual distance between the laser lines.

[0027] This invention also discloses a method for detecting the density of seabed products based on a monocular underwater robot, comprising the following steps:

[0028] By maintaining a constant speed underwater, the monocular underwater robot is used to calculate the quantity and area of ​​products in the entire region using the detection method described in claim 1, and finally obtains the density of products in the seabed area.

[0029] This invention also discloses a seabed product density detection device based on a monocular underwater robot, comprising:

[0030] An image acquisition module is used to acquire images of seabed products using a monocular underwater robot. The monocular underwater robot is equipped with a laser projection device, which projects two parallel green laser lines at a fixed distance onto the seabed. The seabed product images contain these laser lines.

[0031] The target detection module is used to input the seabed product image into the trained target detection model to obtain the number of sea products within the image range. The target detection model adopts an improved Yolox target detection network, which is a self-attention mechanism SENet added to the feature extraction backbone part of the Yolox target detection network.

[0032] The area calculation module is used to perform HSV conversion on the seabed product image, extract the laser line range from the converted image, perform one-dimensional projection processing on the extracted laser lines, and determine the area of ​​the current seabed product image based on the pixel length of the projection result and the actual distance between the two laser lines.

[0033] The density calculation module is used to calculate the density of marine products based on the area of ​​the current seabed product image and the quantity of marine products within the image area.

[0034] Compared with the prior art, the present invention has the following advantages:

[0035] 1. This invention employs the self-attention mechanism SENet, which adds a Squeeze-and-Excitation module to the basic convolutional neural network. This module uses the self-attention mechanism to weight the features of each channel in the image, which can better capture key information inside the target, improve the performance of important features, and thus improve the detection performance of the model.

[0036] 2. This invention's monocular underwater robot can conveniently and accurately measure area by fixing two green laser lines at a certain distance. Furthermore, by accurately controlling its own movement trajectory and field of view, it can achieve non-contact measurement of seabed product density, avoiding the errors caused by contact in traditional measurement methods. Compared to traditional manual measurement or observation, the monocular underwater robot can quickly acquire and process data, greatly accelerating density measurement. It can also be flexibly manipulated, facilitating measurements in different environments. Compared to traditional manual measurement methods, it is more adaptable.

[0037] 3. This invention can effectively solve the problems of traditional density detection methods requiring a large amount of manpower and resources and being easily affected by the external environment; this method uses laser lines as a reference to calculate the area of ​​the seabed region, which can effectively solve the problem of the harsh seabed environment and the difficulty in measuring the area; this method uses a self-attention mechanism to optimize the detection network, which can better help the model learn the key features in the input data; it can enable the detection model to better capture the correlation information between different positions in the input image, thereby improving the accuracy of target detection. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a flowchart of a method for detecting the density of seabed products based on a monocular underwater robot, according to the present invention.

[0040] Figure 2 This is a schematic diagram of the structure of a seabed product density detection device based on a monocular underwater robot according to the present invention. Detailed Implementation

[0041] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0042] like Figure 1The method for detecting seabed product density based on a monocular underwater robot, as shown, firstly, compiles the acquired underwater images into a dataset and trains a detection model. Next, the image with green laser lines acquired by the monocular camera undergoes HSV conversion, defining a detailed HSV range for the laser lines. Based on the defined HSV value range, a mask of the same size as the original image is created. Then, using the defined HSV value range, the input HSV image is binarized to obtain a binarized image of the same size as the original. The laser lines are perpendicular to the horizontal direction of the image. The binarized image is summed column-wise to obtain an array. The number of pixels with consecutive 0 values ​​in the array is calculated. The actual distance between laser lines is divided by the number of pixels to obtain the actual distance represented by each pixel. Knowing the total number of pixels in the image, the area of ​​the entire image can be obtained. Finally, the image acquired by the camera is fed into a target detection network. After performing candidate region extraction, feature extraction, candidate region classification, non-maximum suppression, bounding box regression, and target category determination, the number of products in the current image is obtained. By maintaining a constant speed, the quantity and area of ​​products in the entire region are calculated, ultimately yielding the density of products in the seabed area. The specific steps of this method are as follows:

[0043] S1: Collect underwater images and integrate them into a dataset. After the dataset is integrated, it is fed into the object detection network for training. The specific method is as follows:

[0044] S11: Acquire underwater images and integrate them into a dataset. Then, divide the acquired underwater image dataset into a training dataset and a test dataset in a 7:3 ratio.

[0045] S12: The image processing Yolox object detection network is modified by adding a self-attention mechanism (SENet) to its feature extraction backbone. After dividing the acquired images into training, validation, and test sets, the partitioned data is input into the Yolox detection network, and the bias b is updated using the backpropagation algorithm. l and weight w l Ultimately, a target detection model that can predict object categories and locations is obtained.

[0046]

[0047]

[0048] S2: The monocular underwater robot acquires the current image through a monocular camera and transmits it to the next layer of the network.

[0049] S3: The acquired image undergoes image preprocessing, converting it from RGB color space to HSV color space. The HSV range of the green laser lines is then set, and a mask of the same size as the original image is generated based on this range. Furthermore, the input HSV image is binarized to obtain a binarized image of the same size as the original. The binarized images are then summed column-wise to obtain an array. The longest consecutive sequence of zeros in the array represents the number of pixels between the laser lines, and the image area can be calculated from this sequence. The image acquired by the monocular underwater robot is then fed into a target detection network. After a series of feature extraction processes, the number of products in the image can be detected, and the product density in the image region can be calculated. Specifically, the following method is used:

[0050] S31: Convert the image from RGB color space to HSV color space, and then set the detailed HSV range for the green laser line. Generally, the hue channel value range for the green laser line in HSV image is [50, 70], the saturation channel value range is [200, 255], and the luma channel value range is [180, 255]. Based on the above-set HSV value range, create a mask of the same size as the original image. The pixel value in the mask is 0 or 255, where a pixel value of 255 indicates that the location may contain a green laser line, while a pixel value of 0 indicates that the location cannot contain a green laser line.

[0051] S32: Using the HSV value range set above, binarize the input HSV image to obtain a binarized image of the same size as the original image. During this process, pixel values ​​outside the HSV value range are set to 0 (black), and those within the HSV value range are set to 255 (white). The laser lines are then projected from 2D to 1D, and the actual distance D between the laser lines is known. The binarized image is summed column-wise to obtain a one-dimensional array. The array is then divided into two parts, left and right, and the maximum value is calculated for each part. The difference in indices between the two values ​​is the pixel length X of the laser line on the x-axis. l This value is then passed to the Yolox detector for processing. The Yolox detector calculates the pixel length Y of the laser line on the y-axis. l The actual distance L represented by each pixel can be obtained by dividing the actual distance between laser lines by the number of pixels. a Given the pixel lengths along the x and y axes, multiplying them together gives the area S of the entire image. i This process can be represented by the following formula:

[0052]

[0053] S33: The acquired raw image is fed into the object detection network. The input image is normalized and scaled for processing before being used by the Yolox model for detection. The Yolox network framework is implemented using the PyTorch deep learning framework. During training, the network parameters are updated through backpropagation to obtain the best detection results. The trained model is then used to perform object detection on the test set. In this process, feature extraction, bounding box generation, and category prediction are performed on the samples to count the number of objects present in the image and to define and outline them.

[0054] S4: Current image area S has been obtained. i The number of products N within the image area i Therefore, the product density of the image region can be calculated. The monocular underwater robot is kept moving at a constant speed, scanning all target areas, and the calculated density M is retained. i .

[0055] M i =N i / S i (4)

[0056] Preferably, the present invention also provides a method for region density detection based on the above method, specifically calculating the marine organism density in each image based on the above method, and further comprising:

[0057] S5: After scanning all target areas, calculate the average value of the retained density to obtain the area density.

[0058] Further, the steps to obtain the number of products are as follows: Images acquired by the camera are input into the Yolox object detection network. The network performs candidate region extraction, feature extraction, candidate region classification, non-maximum suppression, bounding box regression, and object category determination to obtain the number of products in the current image. While maintaining a constant robot speed, the number and area of ​​products in the entire region are calculated, ultimately yielding the density of the seabed product area.

[0059] This invention discloses a method for detecting the density of seabed products based on a monocular underwater robot, which effectively solves the problems of traditional density detection methods requiring a large amount of manpower and resources and being easily affected by the external environment. This method uses a laser line as a reference to calculate the area of ​​the seabed region, which can effectively solve the problem of the harsh seabed environment and the difficulty in measuring the area. This method uses a self-attention mechanism to optimize the detection network, which can better help the model learn the key features in the input data. It can enable the detection model to better capture the correlation information between different positions in the input image, thereby improving the accuracy of target detection.

[0060] This invention also discloses a seabed product density detection device based on a monocular underwater robot, such as... Figure 2 As shown, it mainly includes:

[0061] An image acquisition module is used to acquire images of seabed products using a monocular underwater robot. The monocular underwater robot is equipped with a laser projection device, which projects two parallel green laser lines at a fixed distance onto the seabed. The seabed product images contain these laser lines.

[0062] The target detection module is used to input the seabed product image into the trained target detection model to obtain the number of sea products within the image range. The target detection model adopts an improved Yolox target detection network, which is a self-attention mechanism SENet added to the feature extraction backbone part of the Yolox target detection network.

[0063] The area calculation module is used to perform HSV conversion on the seabed product image, extract the laser line range from the converted image, perform one-dimensional projection processing on the extracted laser lines, and determine the area of ​​the current seabed product image based on the pixel length of the projection result and the actual distance between the two laser lines.

[0064] The density calculation module is used to calculate the density of marine products based on the area of ​​the current seabed product image and the quantity of marine products within the image area.

[0065] The description of the seabed product density detection device based on a monocular underwater robot of the present invention is relatively simple because it corresponds to the seabed product density detection method based on a monocular underwater robot in the above embodiment. For related similarities, please refer to the description of the seabed product density detection method based on a monocular underwater robot in the above embodiment, which will not be described in detail here.

[0066] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting the density of seabed products based on a monocular underwater robot, characterized in that, Includes the following steps: The image of the seabed product is acquired by a monocular underwater robot, which is equipped with a laser projection device. The laser projection device is used to project two parallel green laser lines with a fixed distance onto the seabed, and the seabed product image contains the laser lines. The seabed product images are input into a trained target detection model to obtain the number of sea products within the image area. The target detection model employs an improved method. The improved object detection network Object detection networks are used in Adding a self-attention mechanism to the feature extraction backbone part of the object detection network ; The seabed product image is converted to HSV format. The laser line range is extracted from the converted image. The extracted laser lines are then subjected to one-dimensional projection processing. Based on the pixel length of the projection result and the actual distance between the two laser lines, the area of ​​the current seabed product image is determined, including: Set the HSV value range of the converted image, and create a mask with the same size as the original image. The pixel value in the mask is 0 or 255, where a pixel with a value of 255 indicates that the pixel position contains a green laser line, while a pixel with a value of 0 indicates that the pixel position does not contain a green laser line. Based on the set HSV value range, the input HSV image is binarized to obtain a binarized image of the same size as the original image. The pixel values ​​outside the HSV value range are set to 0, and the pixel values ​​within the HSV value range are set to 255. Project the laser line region from 2D to 1D, sum the columns of the binarized image to obtain a one-dimensional array, then divide the array into two parts and calculate the maximum value of each part. The difference between the indices of the two values ​​is the pixel length of the laser line on the x-axis. Then, input the pixel length... The detection head processes the data, and... The detector calculates the pixel length of the laser lines on the y-axis. By dividing the actual distance between the laser lines by the number of pixels, the actual distance represented by each pixel can be obtained. Since the pixel lengths on the x and y axes are known, multiplying them gives the area of ​​the entire image. : in, Represents the pixel length on the x-axis. Represents the pixel length on the y-axis. This represents the actual distance between the laser lines; The density of the marine products is calculated based on the area of ​​the current seabed product image and the number of marine products within the image area.

2. The method for detecting the density of seabed products based on a monocular underwater robot according to claim 1, characterized in that, The training steps of the object detection model include: Obtain a dataset of underwater product images, and divide the dataset into a training set, a test set, and a validation set; Input the segmented data into the improved... In object detection networks, the bias and weights are updated through the backpropagation algorithm, ultimately yielding an object detection model that predicts the object category and location.

3. The method for detecting seabed product density based on a monocular underwater robot according to claim 2, characterized in that, The bias and weights are updated using the backpropagation algorithm, including: The bias and weights are updated based on the following calculations: 。 in, Representing the Layer bias, Representing the Layer weights, It's the learning rate. For the sign of differentiation, This represents the loss function.

4. The method for detecting seabed product density based on a monocular underwater robot according to claim 1, characterized in that, The process of converting the underwater product image to HSV includes converting the underwater product image from the RGB color space to the HSV color space.

5. A method for detecting the density of seabed products based on a monocular underwater robot, characterized in that, Includes the following steps: By maintaining a constant speed underwater, the monocular underwater robot is used to calculate the quantity and area of ​​products in the entire region using the detection method described in claim 1, and finally obtains the density of products in the seabed area.

6. A seabed product density detection device based on a monocular underwater robot, characterized in that, include: An image acquisition module is used to acquire images of seabed products using a monocular underwater robot. The monocular underwater robot is equipped with a laser projection device, which projects two parallel green laser lines at a fixed distance onto the seabed. The seabed product images contain these laser lines. The target detection module is used to input the seabed product image into a trained target detection model to obtain the number of seabed products within the image range. The target detection model employs an improved method. The improved object detection network Object detection networks are used in Adding a self-attention mechanism to the feature extraction backbone part of the object detection network ; The area calculation module is used to perform HSV conversion on the seabed product image, extract the laser line range from the converted image, perform one-dimensional projection processing on the extracted laser lines, and determine the area of ​​the current seabed product image based on the pixel length of the projection result and the actual distance between the two laser lines, including: include: Set the HSV value range of the converted image, and create a mask with the same size as the original image. The pixel value in the mask is 0 or 255, where a pixel with a value of 255 indicates that the pixel position contains a green laser line, while a pixel with a value of 0 indicates that the pixel position does not contain a green laser line. Based on the set HSV value range, the input HSV image is binarized to obtain a binarized image of the same size as the original image. The pixel values ​​outside the HSV value range are set to 0, and the pixel values ​​within the HSV value range are set to 255. Project the laser line region from 2D to 1D, sum the columns of the binarized image to obtain a one-dimensional array, then divide the array into two parts and calculate the maximum value of each part. The difference between the indices of the two values ​​is the pixel length of the laser line on the x-axis. Then, input the pixel length... The detection head processes the data, and... The detector calculates the pixel length of the laser lines on the y-axis. By dividing the actual distance between the laser lines by the number of pixels, the actual distance represented by each pixel can be obtained. Since the pixel lengths on the x and y axes are known, multiplying them gives the area of ​​the entire image. : in, Represents the pixel length on the x-axis. Represents the pixel length on the y-axis. This represents the actual distance between the laser lines; The density calculation module is used to calculate the density of marine products based on the area of ​​the current seabed product image and the quantity of marine products within the image area.

Images