A method for underwater target recognition in underwater intelligent robots

Abstract

This invention discloses an underwater target recognition method for underwater intelligent robots. First, image enhancement processing is used to improve the clarity and color reproduction of underwater images. Then, a specially designed underwater target recognition network model based on a feature fusion attention network architecture is used to obtain a new dehazing target detection model, achieving end-to-end output of clear, haze-free images and realizing underwater target recognition. This method lays a solid foundation for underwater intelligent robots to carry out underwater tasks. Deploying the improved underwater target recognition network model based on a feature fusion attention network architecture into the detection module of underwater intelligent robots improves the feasibility of deploying underwater target recognition algorithms on mobile devices such as underwater intelligent robots, enhances the accuracy and real-time performance of underwater target recognition on underwater intelligent robots, and improves the versatility and practicality of neural network-based underwater target recognition algorithms on mobile devices.

Description

Technical Field

[0001] This invention relates to the field of underwater target detection technology, and more specifically to an underwater target recognition method for underwater intelligent robots. Background Technology

[0002] Underwater target recognition is one of the most important tasks for specialized underwater robots. Vision is an effective way to locate targets, but considering the complexity of the underwater environment, including light scattering by water, refraction by the water itself, interference from suspended matter, the converging effect of underwater ripples on sunlight, and surface debris, the underwater target image is affected by factors such as fogging, color distortion, reduced contrast, blurred image details, and backscattering noise. Underwater target recognition needs to address a series of issues, including color distortion, reduced contrast, blurred image details, interference from suspended matter, and light spot effects. Therefore, processing underwater environmental images is a necessary prerequisite for completing various underwater tasks.

[0003] Currently, most object recognition algorithms based on convolutional neural networks are deployed on PCs or servers, with edge devices primarily responsible for image acquisition and preprocessing. On some edge devices, image detection technology applications are mostly based on traditional object detection algorithms, which are far less robust and versatile than convolutional neural network-based object recognition algorithms. Image detection and deep learning technologies based on Linux systems are still under development, and to date, there is no complete solution for cross-x86 platform porting and application.

[0004] Deep learning typically uses a Graphics Processing Unit (GPU) to deploy a network model for learning, achieving good recognition results. This method requires significant computing power and consumes a lot of power. Currently, compared to traditional algorithms, object recognition algorithms based on convolutional neural networks often have higher accuracy and better versatility. However, both the training and inference phases of these algorithms place high demands on the platform's computing power. The high cost of implementing these algorithms in the final application limits the development and innovation of object recognition algorithms in the civilian sector.

[0005] With the continuous development of deep neural networks, network models are becoming deeper and more complex. While improving the accuracy of target detection, this also leads to increased computational load and slower processing speed. Underwater intelligent robots performing underwater target recognition are typically designed for target detection tasks based on lightweight Linux systems. To deploy deep learning-based target detection networks on mobile devices such as underwater intelligent robots and achieve real-time inference, further research on underwater target recognition algorithms is needed. In recent years, most cameras have relied on traditional image detection algorithms for target detection, which suffers from poor robustness and susceptibility to environmental interference. To investigate the feasibility of deploying target recognition algorithms on mobile devices, improve the accuracy and real-time performance of target recognition on mobile devices, and enhance the versatility and practicality of neural network-based target recognition algorithms, a proposed underwater target recognition method based on lightweight systems such as Linux and combining image detection with deep learning technology plays a crucial role in popularizing neural network-based target recognition algorithms and reducing hardware deployment costs.

[0006] Therefore, how to improve the clarity of underwater images, output fog-free images, and deploy underwater target recognition methods based on neural networks, based on image detection and deep learning technologies in lightweight systems, has become a problem that needs further research and solutions. Summary of the Invention

[0007] To address the shortcomings of the existing technology, this invention provides an underwater target recognition method for underwater intelligent robots, which can be deployed in the lightweight system of underwater intelligent robots to improve the accuracy and real-time performance of underwater target recognition tasks.

[0008] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0009] An underwater target identification method for an underwater intelligent robot, the method comprising:

[0010] S1. Obtain underwater sample images labeled with target object categories to form an underwater sample image dataset;

[0011] S2. Based on the feature fusion attention network architecture, construct an underwater target recognition network model for target recognition in underwater images;

[0012] S3. Using underwater sample images from the underwater sample image dataset as input images, after enhancing the input images, input them into the underwater target recognition network model for training, and obtain the trained underwater target recognition network model.

[0013] S4. Deploy the trained underwater target recognition network model to the detection module of the underwater intelligent robot, and establish communication between the image acquisition module of the underwater intelligent robot and the detection module. Use the underwater image acquired by the image acquisition module of the underwater intelligent robot as the input image, and after enhancement processing, input it to the underwater target recognition network model deployed in the detection module for recognition and detection to obtain the underwater target recognition result.

[0014] As a preferred approach, before inputting the input image into the underwater target recognition network model, a multi-scale enhancement algorithm with color restoration is first used to enhance the input image. Each channel of the original input image is enhanced as follows:

[0015] ;

[0016] in, This represents the original image before enhancement. This represents the enhanced output for the i-th color channel of the original image; This represents the color restoration factor for the i-th color channel of the original image; The bilaterally filtered image representing the original image; , These are the gain and offset values ​​for a multi-scale enhancement algorithm with color restoration, respectively; * indicates a convolution operation; This represents the first step in a multi-scale enhancement algorithm for color restoration. Enhancement weights at each scale, This indicates the number of scales included in a multi-scale enhancement algorithm for color restoration.

[0017] Specifically, the color restoration factor of the i-th color channel of the original image Calculate as follows:

[0018] ;

[0019] In the formula, This represents the pixel value of the i-th color channel in the original image. This shows the sum of the pixel values ​​at each pixel location for the three color channels of the original image; , These are color restoration adjustment parameters used to control the intensity and degree of color restoration.

[0020] Specifically, the bilateral filtered image of the original image Calculate as follows:

[0021] ;

[0022] In the formula, This represents the filtered pixel value; , These represent the pixel coordinates in the original image. and pixel values, Represents the pixel coordinates of the original image; Represents the spatial kernel function, used to measure , The pixel spatial distance between two pixel coordinates; Represents the pixel value kernel function, used to measure... , The difference in pixel values ​​between pixel values; This represents the normalized weighting coefficients of the bilateral filter.

[0023] Specifically, the normalized weight coefficients of the bilateral filter Calculate as follows:

[0024] ;

[0025] ;

[0026] ;

[0027] In the formula, express , The spatial distance between two pixel coordinates; express , The absolute value of the difference between pixel values; A parameter representing the range of weight distribution in the control spatial domain; This parameter represents the range of weight distribution in the grayscale domain.

[0028] As a preferred embodiment, the underwater target recognition network model includes a pre-convolutional layer, a local residual attention feature group structure, a feature attention module, and two post-convolutional layers connected in sequence. The pre-convolutional layer is used to preprocess the input image of the underwater target recognition network model and extract its shallow features. The local residual attention feature group structure is used to extract deep noise features in the input image, retain shallow information, and fuse them. The feature attention module is used to further extract channel features and pixel features in the input image. The two post-convolutional layers are used to perform the final feature fusion and reconstruction processing to obtain the output of the underwater target recognition network model.

[0029] Specifically, the local residual attention feature group structure includes three cascaded fully connected local residual attention feature groups, and the processing procedure of the local residual attention feature group structure is expressed as follows:

[0030] ;

[0031] in, The input image is the local residual attention feature cluster structure. This represents the cluster structure of local residual attention features; Indicates a fully connected state; , , This represents the operation of three local residual attention feature sets; each local residual attention feature set contains cascaded... The processing of a basic block unit and a convolutional unit is represented as follows:

[0032] ;

[0033] in, Indicates the first Operations on a local residual attention feature set; This represents the convolution operation; The first element contained in the local residual attention feature set. Operations on each basic block unit; This is an image pixel overlay operation; each basic block unit includes a cascaded local residual module, a convolution module, and a feature attention module, and its processing is represented as follows:

[0034] ;

[0035] The input image for the basic block unit. Indicates the first Operations on each basic block unit; Represents feature attention operations; This indicates local residual operations, and ; This represents the convolution operation; It is the ReLU activation function; This is a pixel overlay operation for an image.

[0036] Specifically, the feature attention module includes cascaded channel attention units and pixel attention units;

[0037] The processing procedure of the channel attention unit is represented as follows:

[0038] ;

[0039] The input image for the feature attention module, This represents channel attention operations; This represents the convolution operation; This represents the average pooling operation; It is the Sigmoid activation function. The ReLU activation function is used; the output image of the channel attention unit is then compared with the input image. The multiplied image is used as the input image for the feature attention module. ,Right now:

[0040] ;

[0041] The processing procedure of a pixel attention unit is represented as follows:

[0042] ;

[0043] This represents pixel attention operations; the output image of a pixel attention unit is then compared with its own input image. The product is then used as the output of the feature attention module. Therefore, the processing procedure of the feature attention module can be represented as follows:

[0044] ;

[0045] The input image for the feature attention module, Represents feature attention operations; This is a pixel multiplication operation for the image.

[0046] Specifically, the processing procedure of the underwater target recognition network model is as follows:

[0047] ;

[0048] in, The input image is used for the underwater target recognition network model. This represents the processing and computation of the underwater target recognition network model; This represents the convolution operation; This is a pixel overlay operation for an image.

[0049] As a preferred embodiment, the loss function for training the underwater target recognition network model is specifically as follows:

[0050] ;

[0051] In the formula: Represents the loss function; Indicates the training of the first The true object category label of the underwater sample image; The model represents the first Predict the object category of the sample images; Represents L1 norm operations; This indicates the number of underwater sample images used in the training.

[0052] Compared with the prior art, the present invention has the following technical effects:

[0053] 1. This invention proposes an underwater target recognition method for underwater intelligent robots. First, image enhancement processing is used to improve the clarity and color reproduction of underwater images. Then, a new dehazing target detection model is obtained by using a specially designed underwater target recognition network model based on a feature fusion attention network architecture. This model outputs clear, fog-free images in an end-to-end manner and enables the recognition of underwater targets.

[0054] 2. The method of this invention employs a multi-scale enhancement algorithm with color restoration based on bilateral filtering to improve the clarity and robustness of underwater images during training. Then, the trained underwater target recognition network model is deployed to the detection module of an underwater intelligent robot, achieving end-to-end output of clear, fog-free images and performing underwater target recognition. This approach helps to achieve good target recognition accuracy and real-time performance while reducing the network layers of the underwater target recognition network model. It also makes the underwater target recognition network model lightweight, allowing it to be well-suited for deployment on lightweight systems such as Linux. This enables noise reduction, enhancement, defogging, and underwater target recognition of underwater images, laying a solid foundation for underwater intelligent robots to carry out underwater tasks.

[0055] 3. This invention uses an improved underwater target recognition network model based on feature fusion attention network architecture to deploy in the detection module of an underwater intelligent robot. This can effectively help improve the feasibility of deploying underwater target recognition algorithms on mobile devices such as underwater intelligent robots, improve the accuracy and real-time performance of underwater target recognition on underwater intelligent robots, and enhance the versatility and practicality of neural network-based underwater target recognition algorithms on mobile devices. Attached Figure Description

[0056] To make the objectives, technical solutions, and advantages of the invention clearer, the invention will now be described in further detail with reference to the accompanying drawings, wherein:

[0057] Figure 1 This is a flowchart of the underwater target recognition method for underwater intelligent robots disclosed in this invention.

[0058] Figure 2 This is a schematic diagram of the network structure of the underwater target recognition network model constructed in the method of the present invention;

[0059] Figure 3 Example diagram of the testing module equipment;

[0060] Figure 4 A schematic diagram illustrating the communication mechanism between the detection modules of multiple underwater intelligent robots and the surface monitoring computer;

[0061] Figure 5 This is an example diagram illustrating the ROS master communication mechanism;

[0062] Figure 6 This is a schematic diagram of the ROS communication mechanism.

[0063] Figure 7 This is a diagram illustrating the subscriber and publisher process. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but only to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0065] The present invention will now be described in further detail with reference to the accompanying drawings.

[0066] To address the issues of poor feasibility and accuracy of existing underwater target recognition algorithms when deployed on mobile devices, this invention provides an underwater target recognition method for underwater intelligent robots, such as... Figure 1 As shown, the method includes the following steps:

[0067] S1. Obtain underwater sample images labeled with target object categories to form an underwater sample image dataset;

[0068] S2. Based on the feature fusion attention network architecture, construct an underwater target recognition network model for target recognition in underwater images;

[0069] S3. Using underwater sample images from the underwater sample image dataset as input images, after enhancing the input images, input them into the underwater target recognition network model for training, and obtain the trained underwater target recognition network model.

[0070] S4. Deploy the trained underwater target recognition network model to the detection module of the underwater intelligent robot, and establish communication between the image acquisition module of the underwater intelligent robot and the detection module. Use the underwater image acquired by the image acquisition module of the underwater intelligent robot as the input image, and after enhancement processing, input it to the underwater target recognition network model deployed in the detection module for recognition and detection to obtain the underwater target recognition result.

[0071] Currently, underwater intelligent robots face five major challenges in underwater target recognition tasks and the deployment of target recognition algorithms: First, although Gaussian filtering can process images through weighted mean filtering, it only considers the spatial distribution of pixels and fails to account for grayscale differences, resulting in poor image edge preservation. Second, physical model-based dehazing algorithms have advantages in noise handling, but their effectiveness is significantly limited by the estimation of atmospheric light and transmittance. Third, convolutional network learning methods require additional estimation of transmittance and global atmospheric light in the scattering model during dehazing, and the uncertainty of these two parameters may amplify the error. Fourth, image detection and deep learning technologies based on Linux systems are still under development, and there is currently no complete cross-platform porting and application solution. Finally, deploying network models using deep learning methods requires high computing power, and power consumption is also significant. Overall, compared to traditional algorithms, the implementation of convolutional neural network-based target recognition algorithms usually incurs higher costs, restricting their deployment, application, and innovative development in lightweight underwater intelligent robot systems.

[0072] To address this issue, this invention proposes an underwater target recognition method for underwater intelligent robots. First, image enhancement processing is used to improve the clarity and color reproduction of underwater images. Then, a specially designed underwater target recognition network model based on a feature fusion attention network architecture is used to obtain a new dehazing target detection model, achieving end-to-end output of clear, haze-free images and underwater target recognition. This method employs a multi-scale enhancement algorithm with color restoration based on bilateral filtering to improve the clarity of underwater images and robustness during training. The trained underwater target recognition network model is then deployed to the detection module of the underwater intelligent robot, achieving end-to-end output of clear, haze-free images and performing underwater target recognition. This approach allows for good target recognition accuracy and real-time performance while reducing the network layers of the underwater target recognition network model. It also makes the underwater target recognition network model lightweight, enabling it to be deployed on lightweight systems such as Linux, achieving underwater image denoising, enhancement, dehazing, and underwater target recognition, laying a solid foundation for underwater intelligent robots to carry out underwater tasks. This invention deploys an improved underwater target recognition network model based on a feature fusion attention network architecture into the detection module of an underwater intelligent robot. This effectively helps improve the feasibility of deploying underwater target recognition algorithms on mobile devices such as underwater intelligent robots, improves the accuracy and real-time performance of underwater target recognition on underwater intelligent robots, and enhances the versatility and practicality of neural network-based underwater target recognition algorithms on mobile devices.

[0073] The underwater target recognition method for underwater intelligent robots according to the present invention will be described in more detail below.

[0074] 1. Image preprocessing

[0075] After obtaining the target object image dataset, frame extraction, deduplication, and filtering are necessary to improve the dataset quality. By employing target object image preprocessing techniques and data augmentation methods using geometric and pixel transformations, the existing samples can be effectively expanded, increasing both the amount and diversity of the data. The processed image samples will be randomly divided into training, validation, and test sets in an 8:1:1 ratio.

[0076] The frame extraction stage processes the long videos captured on-site. Due to the similarity of scenes and target locations in the video, an equidistant sampling strategy can be adopted, such as extracting one image every 25 frames. Furthermore, after video cropping, frame extraction is necessary to extract temporal features from consecutive frames, facilitating subsequent network model training. Keyframes are extracted at a rate of one frame per second. In the image deduplication stage, the image data obtained from shooting and frame extraction are processed to reduce duplicate images. To reduce manual workload, software can be used for rapid deduplication first, followed by manual review using structural similarity algorithms for similarity analysis. The screening stage involves manually removing invalid data. Before data labeling, low-quality or invalid data is manually removed to ensure subsequent image quality. Removed images include those with low relevance to the task, blurry images with too low a pixel count, and images with poor shooting angles that are difficult to label. Finally, data augmentation is performed through geometric and pixel transformations to expand the sample and increase data diversity. This method not only increases the amount of training data but also avoids sample imbalance, improves the model's convergence and generalization abilities, thereby improving the overall performance of the model.

[0077] 2. Image annotation

[0078] Image data was labeled using the LabelImg tool, and a preliminary dataset was created after review. The annotation interface is shown below. Figure 2 As shown, this is mainly used to label the target categories and their regions in images. During the labeling process, omissions and errors may negatively impact the training results. Target category labels use English characters, and the labeling of target regions should be as accurate as possible to ensure complete enclosing of the target object. This is especially important for convex polygons, where the entire object should be enclosed within the bounding box. Each labeled image generates a corresponding XML file containing the image name and the specific coordinates of each labeled bounding box.

[0079] 3. Preparatory work for training the underwater target recognition network model

[0080] In the practical application experiment, a total of 5000 image samples were collected, which were expanded to a total of 6000 images after data augmentation techniques. To avoid data overlap, the dataset was divided into training, validation, and test sets in an 8:1:1 ratio, with 4800, 600, and 600 samples respectively. The deep learning environment configuration for this practical application experiment is as follows: CPU: Intel(R) Xeon(R) Bronze 3204 (1.9GHz), GPU: NVIDIA GeForce RTX 2060 (6144MiB), compiler: Python 3.8, deep learning framework: PyTorch 1.6, acceleration module: CUDA 11.8.

[0081] 4. A multi-scale enhancement algorithm with color restoration based on bilateral filtering.

[0082] Due to the complex underwater environment and poor image quality, image enhancement algorithms are needed to process camera-captured images, improving image clarity, contrast, and other useful information while removing noise and other useless information to increase target detection accuracy. In recent years, with the continuous development of deep learning, underwater image enhancement has also advanced further. Deep convolutional neural networks possess powerful supervised learning models, offering significant advantages in handling low-level visual tasks. They can better improve image quality and accuracy while reducing reliance on specialized underwater imaging expertise, enabling more scalable and automated image analysis.

[0083] Therefore, this invention proposes a multi-scale Retinex enhancement algorithm (improved MSRCR algorithm) based on bilateral filtering and color restoration for underwater image enhancement, aiming to improve the clarity and robustness of underwater images during training. Furthermore, by modifying the network structure based on a feature fusion attention network architecture, a new underwater target recognition network model is obtained, achieving end-to-end output of clear, fog-free images and target recognition results. The multi-scale enhancement algorithm based on bilateral filtering and color restoration proposed in this invention will be described in detail below.

[0084] The Retinex algorithm has several implementations. Retinex theory is a model of how the human visual system perceives color, aiming to elucidate the mechanisms by which people perceive color and brightness. The mathematical model of this theory was proposed by Land et al., and its formula is shown below.

[0085]

[0086] In this formula, an image is composed of arranged pixels, where x and y represent the horizontal and vertical coordinates of a pixel, respectively. Retinex theory posits that the image S(x,y) perceived by the human eye can be decomposed into the incident image L(x,y) under illumination and the reflectivity R(x,y) of the reflected image. Here, R(x,y) represents the intrinsic properties of the pixel, independent of the incident light.

[0087] The SSR (Single Scale Retinex) algorithm obtains the reflected image component by subtracting the incident image component from the image component perceived by the human eye by taking the logarithm of both sides of the formula. The formula is shown below.

[0088]

[0089] In this formula, since the incident image component is difficult to obtain directly, a Gaussian filter is used to perform a convolution operation on the original image to replace the incident image component. The corresponding formula is shown below.

[0090]

[0091] In the formula, * represents the convolution operation; This represents Gaussian filtering, and the Gaussian filtering formula is shown below.

[0092]

[0093] In the formula, x and y represent the deviation from the center pixel; σ represents the standard deviation of the Gaussian kernel. When the value of σ is too small, the range of the filter is very small, resulting in insufficient smoothing effect. At this time, more of the original details and noise of the image are preserved. When the value of σ is too large, the smoothing effect on the image is better, but excessive smoothing will also lead to a lack of edge details in the image.

[0094] Building upon the SSR algorithm, the MSR (Multi-Scale Retinex) algorithm introduces multiple Gaussian kernels of different scales to perform Retinex processing on the input image. Finally, the MSR algorithm generates the final image by weighted fusion of these processing results from different scales. The corresponding MSR algorithm formula is shown below.

[0095]

[0096] In the formula, This indicates the number of scales included in a multi-scale enhancement algorithm for color restoration, typically set to 3, representing the high, medium, and low scales respectively. This represents the first step in a multi-scale enhancement algorithm for color restoration. The enhancement weights at each scale are taken as follows: .

[0097] The MSRCR (Multi-Scale Retinex with Color Restoration) algorithm is a further optimization of the MSR algorithm, aiming to address the problem that the MSR algorithm may introduce noise and cause local detail color distortion during image processing. To this end, the MSRCR algorithm introduces a color restoration function to adjust the weights between different color channels in the original image. This process can adjust the color of information in darker areas, thereby effectively eliminating color distortion in the image.

[0098] The formula for the existing MSRCR algorithm is shown below:

[0099]

[0100]

[0101] In the formula, Represents the color restoration factor of the i-th color channel of the original image; , For color restoration adjustment parameters, Used to control the intensity of color restoration. Used to control the extent of the impact of color restoration; This represents the pixel value of the i-th color channel in the original image. The formula represents the sum of the pixel values ​​at each pixel location for the three color channels of the original image. Finally, a gain value G and an offset value b are added to the formula to obtain the final MSRCR algorithm, as shown below.

[0102]

[0103] The Gaussian filter used in the MSRCR algorithm is a linear smoothing filter, which, while capable of blurring images, can lead to loss of detail, especially at edges and in textured areas. To address this issue, this invention proposes replacing the Gaussian filter with a bilateral filter in the improved MSRCR algorithm to enhance the quality of underwater images.

[0104] A bilateral filter is a nonlinear filter that considers not only the spatial relationship of pixels in an image but also the differences between their grayscale or color values. By applying a weighted average to the pixels, bilateral filtering can effectively remove noise and smooth the image while preserving edge sharpness, thus achieving high-quality filtering. The formula for bilateral filtering is shown below.

[0105]

[0106] In the formula, This represents the pixel values ​​of the image after bilateral filtering; , These represent the pixel coordinates in the original image. and pixel values, Represents the pixel coordinates of the original image; Represents the spatial kernel function, used to measure , The pixel spatial distance between two pixel coordinates is usually represented by a Gaussian function, and the weight of the spatial distance can be controlled by adjusting the parameters. Represents the pixel value kernel function, used to measure... , The difference between pixel values ​​is usually represented by a Gaussian function, and the weight of the difference can be controlled by adjusting the parameters. These represent the normalized weight coefficients of the bilateral filter, used to ensure that the filtered pixel values ​​are within a certain range. The formula is shown below.

[0107]

[0108]

[0109]

[0110] In the formula, express , The spatial distance between two pixel coordinates; express , The absolute value of the difference between pixel values; A parameter representing the range of weight distribution in the control spatial domain; This represents the parameter controlling the range of grayscale weight distribution. This invention replaces the Gaussian filter function in the existing MSRCR algorithm with a bilateral filter function, resulting in an improved MSRCR underwater image enhancement algorithm based on the bilateral filter function. The algorithm formula is shown below:

[0111]

[0112] in, This represents the original image before enhancement. This represents the enhanced output for the i-th color channel of the original image; This represents the color restoration factor for the i-th color channel of the original image; The bilaterally filtered image representing the original image; , These are the gain and offset values ​​for a multi-scale enhancement algorithm with color restoration, respectively; * indicates a convolution operation; This represents the first step in a multi-scale enhancement algorithm for color restoration. Enhancement weights at each scale, This indicates the number of scales included in a multi-scale enhancement algorithm for color restoration. Typically... Let 3 represent the high, medium, and low scales respectively. .

[0113] In the multi-scale enhancement algorithm with color restoration employed in this invention, bilateral filtering is used instead of traditional Gaussian filtering to enhance underwater images. During the filtering process, not only are the spatial relationships of pixels in the image considered, but also the grayscale or color differences between them are taken into account. Specifically, by performing a weighted average of the pixels, noise is effectively removed and the image is smoothed, while the clarity of edge information in the image is preserved.

[0114] 5. An improved underwater target recognition network model based on a feature fusion attention network architecture

[0115] The underwater target recognition network model structure proposed in this invention, based on a feature fusion attention network architecture, is as follows: Figure 2 As shown, the underwater target recognition network model comprises a pre-convolutional layer, a local residual attention feature group structure, a feature attention module, and two post-convolutional layers connected in sequence. The pre-convolutional layer is used to preprocess the input image of the underwater target recognition network model and extract its shallow features. The local residual attention feature group structure is used to extract deep noise features in the input image, retain shallow information, and fuse them. The feature attention module is used to further extract channel features and pixel features in the input image. The two post-convolutional layers are used for the final feature fusion and reconstruction processing to obtain the output of the underwater target recognition network model.

[0116] This network architecture design helps to more effectively capture pixel features at different levels in underwater images. Finally, the feature attention module fuses all extracted features to obtain a clear, fog-free image in an end-to-end manner, which helps to achieve more accurate underwater target recognition. Simultaneously, a multi-scale enhancement algorithm with color restoration based on bilateral filtering is used to improve the clarity of underwater images and the robustness during training. The trained underwater target recognition network model is then deployed to the detection module of an underwater intelligent robot, achieving end-to-end output of clear, fog-free images and performing underwater target recognition. This approach allows for good target recognition accuracy and real-time performance while reducing the network layers of the underwater target recognition network model. It also makes the underwater target recognition network model lightweight, enabling it to be deployed on lightweight systems such as Linux, achieving underwater image denoising, enhancement, defogging, and underwater target recognition, laying a solid foundation for underwater intelligent robots to carry out underwater tasks.

[0117] 5.1 Network Architecture of Underwater Target Recognition Network Model

[0118] After the input image of the underwater target recognition network model is processed by the pre-convolutional layer, it is first input into the local residual attention feature group structure for deep noise feature extraction. The local residual attention feature group structure consists of three cascaded fully connected local residual attention feature groups. The processing procedure of the local residual attention feature group structure is represented as follows:

[0119] ;

[0120] in, The input image is the local residual attention feature cluster structure. This represents the cluster structure of local residual attention features; Indicates a fully connected state; , , This represents the operation of three local residual attention feature sets; each local residual attention feature set contains cascaded... The processing of a basic block unit and a convolutional unit is represented as follows:

[0121] ;

[0122] in, Indicates the first Operations on a local residual attention feature set; This represents the convolution operation; The first element contained in the local residual attention feature set. Operations on each basic block unit; This is an image pixel overlay operation; each basic block unit includes a cascaded local residual module, a convolution module, and a feature attention module, and its processing is represented as follows:

[0123] ;

[0124] The input image for the basic block unit. Indicates the first Operations on each basic block unit; Represents feature attention operations; This indicates local residual operations, and ; This represents the convolution operation; It is the ReLU activation function; This is a pixel overlay operation for an image.

[0125] The design of the local residual attention feature group structure enables the underwater target recognition network model to directly learn the mapping relationship required for noise removal from the input underwater image, thereby simplifying the denoising process and breaking the limitation of the physical model-based dehazing method in terms of insufficient accuracy in estimating atmospheric light and transmittance.

[0126] The feature attention module includes cascaded channel attention units and pixel attention units. The processing procedure of the channel attention unit is as follows:

[0127] ;

[0128] The input image for the feature attention module, This represents channel attention operations; This represents the convolution operation; This represents the average pooling operation; It is the Sigmoid activation function. The ReLU activation function is used; the output image of the channel attention unit is then compared with the input image. The multiplied image is used as the input image for the feature attention module. ,Right now:

[0129] ;

[0130] The processing procedure of the pixel attention unit is represented as follows:

[0131] ;

[0132] This represents pixel attention operations; the output image of a pixel attention unit is then compared with its own input image. The product is then used as the output of the feature attention module. Therefore, the processing procedure of the feature attention module can be represented as follows:

[0133] ;

[0134] The input image for the feature attention module, Represents feature attention operations; This is a pixel multiplication operation for the image.

[0135] The feature attention module obtains weights through adaptive learning to fuse features. This process can retain low-level information and pass it to deeper layers. Due to the existence of the weight mechanism, the underwater target recognition network model can more effectively focus on effective information such as thick zero areas, high-frequency textures, and color fidelity, thus obtaining better fog-free images and improving the underwater target recognition effect.

[0136] Therefore, combined Figure 2 The processing procedure of the underwater target recognition network model can be represented as follows:

[0137] ;

[0138] in, The input image is used for the underwater target recognition network model. This represents the processing and computation of the underwater target recognition network model; This represents the convolution operation; This is a pixel overlay operation for an image.

[0139] 5.2 Training of the Underwater Target Recognition Network Model

[0140] Using the training data prepared in advance for training the underwater target recognition network model and the hardware equipment configured for the deep learning environment, the underwater target recognition network model is trained using the training set, the training loss is calculated, and the parameters of the underwater target recognition network model are optimized based on the training loss. Then, the training results are verified and tested using the validation set and the test set. The training is completed until the parameter tuning of the underwater target recognition network model converges.

[0141] The preferred loss function for training the underwater target recognition network model is:

[0142] ;

[0143] In the formula: Represents the loss function; Indicates the training of the first The true object category label of the underwater sample image; The model represents the first Predict the object category of the sample images; Represents L1 norm operations; This represents the number of underwater sample images used in the training. Using this loss function helps optimize the convergence performance of the underwater target recognition network model training.

[0144] 6. Development and runtime environment for configuring an underwater target recognition network model on the detection module.

[0145] Setting up the development environment involves installing the Ubuntu 20.04 system on the underwater intelligent robot's detection module, and then installing the development environment required for developing and running the target detection algorithm. The development environment requirements are shown in Table 1 below. After installing the system and the dependent software environment, the ROS development environment can be installed.

[0146] Table 1. Description of Development Environment Parameters

[0147]

[0148] An example of a testing module device built based on the above development environment configuration is shown below. Figure 3 As shown.

[0149] 7. The underwater target recognition network model is deployed to the detection module of the underwater intelligent robot.

[0150] After model training is completed on a high-performance computer, the object recognition algorithm is converted into a TensorRT model via C++ to accelerate the inference process in the AI ​​module system. TensorRT utilizes an optimized algorithmic architecture, enabling faster and more efficient inference speeds on NVIDIA GPUs. The process involves two parts: build and deployment.

[0151] During the construction phase, TensorRT generates optimization decisions to transform and optimize the network; to avoid extra computation, unused network layers are removed and multiple network layers are merged; through vertical layer fusion and horizontal layer fusion, the network and tensors are fused to reduce the computation process and transmission time. Vertical fusion integrates Conv layers, BN layers or offset layers, and activation layers into a single CBR (Convolution, Bias and ReLU), while horizontal fusion merges several identical inputs to reduce the depth and width of the network, reduce repeated calls to the same functions, and improve inference speed.

[0152] During the deployment phase, when data to be processed is received, the system executes the previously defined optimization strategy and performs inference and identification. This process aims to minimize latency while maximizing system throughput.

[0153] To ensure the target detection algorithm meets the requirements of underwater intelligent robots, the above algorithm is deployed on the detection module of the underwater intelligent robot and then verified. To meet the requirements of lightweight design and real-time performance when deploying the target detection model to the underwater intelligent robot's detection module, this invention employs offline inference for verification. Offline inference involves converting the model trained by the user using a deep learning experimental platform into an offline model supported by the underwater intelligent robot's detection module, and then loading and executing the offline model. This offline inference allows for batch loading of models for inference tasks, resulting in faster model execution. The deployment process involves converting the weight parameter file saved by the deep learning experimental platform into a format supported by the underwater intelligent robot's detection module, and then replacing the weight file of the target detection algorithm in the underwater intelligent robot's detection module.

[0154] After obtaining the identification results of underwater targets, the detection module of the underwater intelligent robot can upload the underwater target identification data to the surface monitoring computer through a wireless communication mechanism. Figure 4 The diagram illustrates the communication mechanism between multiple underwater intelligent robot detection modules (Topics) and surface monitoring computers (Computers). The communication between them may require the use of distributed network communication nodes (Nodes).

[0155] 8. ROS establishes communication between the camera and the detection module.

[0156] The communication mechanism of ROS (Robot Operating System) is as follows: Figure 5 As shown, a loosely coupled distributed communication method is employed to achieve effective communication between various ROS nodes. Based on a unified communication standard, ROS's loosely coupled distributed framework allows each node to operate its functions independently, and even allows them to work collaboratively on different hosts. Various sensors on underwater intelligent robots, such as cameras, radar, and GPS, need to transmit data to achieve proper control of the underwater intelligent robot; therefore, process communication is crucial for data transmission.

[0157] ROS has three basic communication methods:

[0158] 1. Topic Communication (Publish-Subscribe Model)

[0159] 2. Service Communication (Request-Response Pattern)

[0160] 3. Parameter Server (Parameter Sharing Mode)

[0161] roscore is a collection of nodes and programs that constitute the core services of the ROS system, hence the name "ROS core". To enable topic communication, roscore must first be run, which is initiated by entering the `roscore` command in the terminal of an Ubuntu system. The `roscore` command will start the following functions:

[0162] ROS Master: The master node responsible for communication.

[0163] ROS Parameter Server: A server used to store and manage parameters.

[0164] Rosout Logging Node: The node used to output logs.

[0165] A node is the execution unit of a ROS program, either a process or an independent executable file that performs a specific task. A message describes the type of data transmitted in a topic; users can define the interface by writing .msg files, and the corresponding code files are generated during the compilation process.

[0166] The ROS Master acts as the control center for all nodes, such as Figure 6As shown, it provides naming and registration services, tracking publishers and subscribers of topics and services. Its role is to enable ROS nodes to locate each other and to provide a parameter server. Once the nodes are located, they can begin communicating. The ROS Master interacts with ROS client libraries (such as roscpp and rospy) through an XML-RPC-based API to store and retrieve information. An API (Application Programming Interface) is a standard set that defines the exchange of information and commands between an application and a computer or network operating system.

[0167] XML-RPC is a Remote Procedure Call (RPC) technology that uses the HTTP protocol as its transport mechanism and transmits commands and data using XML text format. RPC (Remote Procedure Call) allows a local machine to call a procedure (method) on a remote machine, and is a distributed computing technology designed to improve interoperability between different machines.

[0168] ROS communication primarily uses topic-based communication. Topics employ an asynchronous communication mechanism, with nodes transmitting data through a publish / subscribe model. Data is transmitted from publishers to subscribers, and there can be multiple subscribers or publishers for the same topic.

[0169] In ROS, publishers and subscribers are the core components for inter-node communication. They publish and receive messages through topics, forming a flexible and efficient data transmission mechanism. For example, a camera node publishes image data, allowing other nodes in the system to receive real-time images, while object detection nodes subscribe to this image data for processing. Subscribers process received messages by registering callback functions. The system automatically calls these defined callback functions each time a message is received from the ( / camera) topic, enabling object detection nodes to respond to data in real time. This topic-based publish / subscribe mechanism allows nodes in the ROS system to exchange data efficiently without needing to understand each other's internal implementations. Through this decoupled design, ROS improves system flexibility and scalability, simplifies inter-node communication, and supports developers in building more complex and feature-rich robotic applications.

[0170] The process of creating publishers and subscribers in ROS is as follows: Figure 7 As shown.

[0171] Creating a publisher first requires customizing the topic message, and then the following content is written in the publisher code:

[0172] Initialize the ROS node;

[0173] Register node information with the ROS Master, including the published topic name and the message type in the topic;

[0174] Create message data;

[0175] Set the frequency of message publishing to achieve a loop of information publishing.

[0176] Content creation for subscribers in ROS:

[0177] Initialize the ROS node;

[0178] Subscribe to the topics you need;

[0179] The loop waits for messages related to the topic, and enters the callback function upon receiving a message.

[0180] Message processing is done in the callback function.

[0181] To achieve communication between the camera and the detection module based on the above, the following ROS project needs to be created on the Ubuntu system. The ROS workspace (catkin_workspace) is used to store packages, and it generally contains three subspaces: src (Source Space), build (Build Space), and devel (Development Space). Before creating any program, a workspace must be created first. This is done through commands in the Ubuntu system terminal, including creating the workspace, building the workspace, setting up the environment, and checking environment variables. Then, create the detection and camera packages in the src subdirectory of the workspace, and then compile the packages. After the packages are created, two files will be generated in the package directory: a package.xml manifest file, providing information about the packages and defining dependencies between them; and a CMakeLists.txt script file, used for compilation.

[0182] 9. Overview

[0183] For target monitoring in complex underwater environments, this invention provides an underwater target recognition method for underwater intelligent robots. This method employs a multi-scale enhancement algorithm with color restoration based on bilateral filtering to denoise and enhance underwater images. Compared to other image denoising and enhancement methods, it is a nonlinear filter that considers spatial grayscale distribution and has better edge preservation, which helps improve the clarity of underwater images and robustness during model training. Furthermore, this invention proposes to fuse feature information from different depth layers through a depth feature information fusion layer, achieving end-to-end image sharpening without relying on a model. End-to-end training directly learns the mapping relationship required for noise removal from the input underwater image, thus simplifying the denoising process and overcoming the limitations of physical model-based dehazing methods in estimating atmospheric light and transmittance. The underwater target recognition method for underwater intelligent robots proposed in this invention can be deployed and applied on lightweight systems such as Linux, which is conducive to the popularization of target recognition algorithms based on neural networks on mobile devices such as underwater intelligent robots. This improves the feasibility of deploying target recognition algorithms on mobile devices, enhances the accuracy and real-time performance of target recognition on mobile devices, and improves the versatility and practicality of underwater target recognition algorithms based on neural networks on mobile devices.

[0184] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described with reference to preferred embodiments, those skilled in the art should understand that various changes in form and detail can be made without departing from the spirit and scope of the invention as defined in the appended claims.

Claims

1. A method for underwater target recognition in underwater intelligent robots, characterized in that, The method includes: S1. Obtain underwater sample images labeled with target object categories to form an underwater sample image dataset; S2. Based on the feature fusion attention network architecture, construct an underwater target recognition network model for target recognition in underwater images; S3. Using underwater sample images from the underwater sample image dataset as input images, after enhancing the input images, input them into the underwater target recognition network model for training, and obtain the trained underwater target recognition network model. S4. Deploy the trained underwater target recognition network model to the detection module of the underwater intelligent robot, and establish communication between the image acquisition module of the underwater intelligent robot and the detection module. Use the underwater image acquired by the image acquisition module of the underwater intelligent robot as the input image, and after enhancement processing, input it to the underwater target recognition network model deployed in the detection module for recognition and detection to obtain the underwater target recognition result. The underwater target recognition network model comprises a pre-convolutional layer, a local residual attention feature group structure, a feature attention module, and two post-convolutional layers connected in sequence. The pre-convolutional layer preprocesses the input image of the underwater target recognition network model to extract its shallow features. The local residual attention feature group structure extracts deep noise features from the input image, retains shallow information, and fuses them. The feature attention module further extracts channel features and pixel features from the input image. The two post-convolutional layers perform the final feature fusion and reconstruction process to obtain the output of the underwater target recognition network model. The local residual attention feature group structure includes three cascaded fully connected local residual attention feature groups. The processing procedure of the local residual attention feature group structure is expressed as follows: ； in, The input image is the local residual attention feature cluster structure. This represents the cluster structure of local residual attention features; Indicates a fully connected state; , , This represents the operation of three local residual attention feature sets; each local residual attention feature set contains cascaded... The processing of a basic block unit and a convolutional unit is represented as follows: ； in, Indicates the first Operations on a local residual attention feature set; This represents the convolution operation; The first element contained in the local residual attention feature set. Operations on each basic block unit; This is an image pixel overlay operation; each basic block unit includes a cascaded local residual module, a convolution module, and a feature attention module, and its processing is represented as follows: ； The input image for the basic block unit, Indicates the first Operations on each basic block unit; Represents feature attention operations; This indicates local residual operations, and ; This represents the convolution operation; It is the ReLU activation function; This is an image pixel overlay operation; The feature attention module includes cascaded channel attention units and pixel attention units.

2. The underwater target recognition method for underwater intelligent robots according to claim 1, characterized in that, Before inputting the input image into the underwater target recognition network model, a multi-scale enhancement algorithm with color restoration is used to enhance the input image. Each channel of the original input image is enhanced as follows: ； in, This represents the original image before enhancement. This represents the enhanced output for the i-th color channel of the original image; This represents the color restoration factor for the i-th color channel of the original image; The bilaterally filtered image representing the original image; , These are the gain and offset values ​​for a multi-scale enhancement algorithm with color restoration, respectively; * indicates a convolution operation; This represents the first step in a multi-scale enhancement algorithm for color restoration. Enhancement weights at each scale, This indicates the number of scales included in a multi-scale enhancement algorithm for color restoration.

3. The underwater target recognition method for underwater intelligent robots according to claim 2, characterized in that, Color restoration factor of the i-th color channel of the original image Calculate as follows: ； In the formula, This represents the pixel value of the i-th color channel in the original image. This shows the sum of the pixel values ​​at each pixel location for the three color channels of the original image; , These are color restoration adjustment parameters used to control the intensity and degree of color restoration.

4. The underwater target recognition method for underwater intelligent robots according to claim 2, characterized in that, Bilateral filtered image of the original image Calculate as follows: ； In the formula, This represents the filtered pixel value; , These represent the pixel coordinates in the original image. and pixel values, Represents the pixel coordinates of the original image; Represents the spatial kernel function, used to measure , The pixel spatial distance between two pixel coordinates; Represents the pixel value kernel function, used to measure... , The difference in pixel values ​​between pixel values; This represents the normalized weighting coefficients of the bilateral filter.

5. The underwater target recognition method for underwater intelligent robots according to claim 4, characterized in that, Normalized weighting coefficients of bilateral filtering Calculate as follows: ； ； ； In the formula, express , The spatial distance between two pixel coordinates; express , The absolute value of the difference between pixel values; A parameter representing the range of weight distribution in the control spatial domain; This parameter represents the range of weight distribution in the grayscale domain.

6. The underwater target recognition method for underwater intelligent robots according to claim 1, characterized in that, The processing procedure of the channel attention unit is represented as follows: ； The input image for the feature attention module, This represents channel attention operations; This represents the convolution operation; This represents the average pooling operation; It is the Sigmoid activation function. The ReLU activation function is used; the output image of the channel attention unit is then compared with the input image. The multiplied image is used as the input image for the feature attention module. ,Right now: ； The processing procedure of a pixel attention unit is represented as follows: ； This represents pixel attention operations; the output image of a pixel attention unit is then compared with its own input image. The product is then used as the output of the feature attention module. Therefore, the processing procedure of the feature attention module can be represented as follows: ； The input image for the feature attention module, Represents feature attention operations; This is a pixel multiplication operation for the image.

7. The underwater target recognition method for underwater intelligent robots according to claim 6, characterized in that, The processing procedure of the underwater target recognition network model is expressed as follows: ； in, The input image is used for the underwater target recognition network model. This represents the processing and computation of the underwater target recognition network model; This represents the convolution operation; This is a pixel overlay operation for an image.

8. The underwater target recognition method for underwater intelligent robots according to claim 1, characterized in that, The loss function for training the underwater target recognition network model is specifically as follows: ； In the formula: Represents the loss function; Indicates the training of the first The true object category label of the underwater sample image; The model represents the first Predict the object category of the sample images; Represents L1 norm operations; This indicates the number of underwater sample images used in the training.

Images