Hull structure corrosion detection method and device based on target detection and semantic segmentation

Abstract

The invention discloses a ship body structure corrosion detection method and device based on target detection and semantic segmentation, and relates to the technical field of ship body corrosion detection. The method comprises the following steps: inputting a hull picture into a corrosion detection model to obtain a detection frame for positioning a corrosion area; according to the detection frame, cutting the ship body picture to obtain a corresponding corrosion picture; and inputting the corrosion picture into a corrosion identification model, and performing fine-grained classification on the corrosion picture by adopting a semantic segmentation mode to determine a corrosion position and a corrosion type. The ship body corrosion area can be efficiently positioned, fine classification of corrosion types is achieved through semantic segmentation, the detection precision and efficiency are remarkably improved, the intelligent detection requirement is met, and wide application prospects are achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of hull structure corrosion detection, and in particular to a hull structure corrosion detection method and device based on target detection and semantic segmentation. Background Art

[0002] Corrosion of hull structures significantly impacts the safety and service life of ships. Traditional manual inspection methods suffer from low efficiency, high cost, and susceptibility to human interference, making them difficult to meet the demands of large-scale, complex inspection environments. Although image recognition technology has been gradually introduced into hull corrosion inspection in recent years with the application of new inspection equipment such as drones and robots, existing image recognition technology still faces many challenges. Especially in complex backgrounds, it is difficult to distinguish between corroded areas and the background. Traditional methods are often only able to perform rough identification of corroded areas, and it is difficult to achieve fine pixel-level segmentation and accurate classification of corrosion types.

[0003] In view of this, overcoming the defects of the prior art is an urgent problem to be solved in this technical field. Summary of the Invention

[0004] In order to solve these problems, the present invention proposes a hull structure corrosion detection method based on target detection and semantic segmentation. Combining target detection and semantic segmentation technology, it effectively improves the accuracy and efficiency of hull corrosion detection, and can accurately identify and locate corrosion areas in complex environments, avoiding the problems of background interference and rough classification in traditional methods.

[0005] The present invention adopts the following technical solutions: In a first aspect, the present invention provides a method for detecting corrosion of a hull structure based on target detection and semantic segmentation, comprising: Input the hull image into the corrosion detection model to obtain a detection frame for locating the corrosion area; According to the detection frame, a corresponding corrosion image is cropped from the hull image; The corrosion image is input into a corrosion recognition model, and the corrosion image is fine-grainedly classified using a semantic segmentation method to determine the corrosion location and corrosion type.

[0006] Preferably, inputting the corrosion image into a corrosion recognition model and performing fine-grained classification on the corrosion image using semantic segmentation to determine the corrosion location and corrosion type includes: Extracting a feature map F1 from the eroded image, performing group convolution, dilation, and feature extraction on the feature map F1 to obtain a feature map F2; Perform multi-branch feature extraction on the feature map F2 and fuse complementary feature information to obtain a feature map F3; Performing global and local feature extraction on the feature map F1 to obtain a feature map F4; The feature map F3 and the feature map F4 are spliced ​​and fused to obtain a feature map F5; Perform dimensionality reduction mapping on the feature map F5 to obtain the corrosion position and corrosion type.

[0007] Preferably, extracting the feature map F1 from the corrosion image, performing group convolution, expansion and feature extraction on the feature map F1 to obtain the feature map F2 includes: Performing feature extraction on the feature map F1 to extract original detail feature information of the feature map F1 to obtain a feature map F11; Performing feature extraction on the input feature map F1 according to different expansion rates, extracting multi-scale feature information of different ranges in the feature map F1, and obtaining a plurality of feature maps F12; Performing global pooling on each of the feature maps F1 to extract global semantic background feature information in the feature map F1 to obtain a feature map F13; The feature map F11, the plurality of feature maps F12 and the feature map F13 are spliced ​​and fused to obtain a feature map F2, so as to fuse the original detail feature information, the multi-scale feature information and the global semantic background feature information into the feature map F2.

[0008] Preferably, performing multi-branch feature extraction on the feature map F2 and fusing complementary feature information to obtain the feature map F3 includes: The feature map F2 is evenly divided into a plurality of sub-feature maps F21 along the depth direction; Perform feature extraction on each sub-feature map F21 to obtain a feature map F22; After normalizing each feature map F22 using the Sigmoid activation function, the feature maps F22 are spliced ​​and restored to the size of the feature map F2 to obtain a feature map F3, wherein the feature map F3 contains rich global context information and multi-scale information.

[0009] Preferably, extracting features from each sub-feature graph F21 to obtain the feature graph F22 includes: Perform average pooling along the horizontal and vertical directions of the sub-feature map F21, and then weightedly merge it with the sub-feature map F21 to obtain a feature map F23; Perform convolution channel adjustment on the sub-feature map F21 to obtain a feature map F24; Perform pooling and Softmax processing on feature map F24 to obtain feature map F25; After performing group normalization on the feature map F23, it is matrix-multiplied with the feature map F25 to obtain the feature map F26; After performing group normalization, average pooling, and Softmax processing on the feature map F23, it is matrix-multiplied with the feature map F24 to obtain the feature map F27; Feature map F26 and feature map F27 are combined to obtain feature map F22.

[0010] Preferably, the extracting global and local features from the feature map F1 to obtain the feature map F4 comprises: After performing layer normalization processing on the feature map F1, the processed feature map F1 is divided into a feature map F14 and a feature map F15; Performing global feature extraction on the feature map F14 to extract low-frequency global information including the overall shape contour of the corrosion area from the feature map F14; Performing local feature extraction on the feature map F15 to extract high-frequency local information including texture and shape of the corrosion area from the feature map F15; Perform feature fusion on the low-frequency global information and the high-frequency local information to obtain a feature map F4; The feature map F3 and the feature map F4 are subjected to channel adjustment and feature splicing to obtain a feature map F5.

[0011] Preferably, the corrosion detection model includes a backbone network, a neck network and a head network; The backbone network is used to extract feature information of the corrosion area in the hull image; The neck network is used to extract multi-scale corrosion region feature information from the corrosion region feature information; The head network is used to locate the detection frame of the corrosion area from the multi-scale corrosion area feature information.

[0012] Preferably, the method further includes the step of constructing a data set for training the corrosion detection network model and the corrosion identification model: Acquire a hull training image, and perform target detection and annotation on the hull training image to obtain a detection and annotation file, wherein the detection and annotation file includes annotation information of a training detection frame for locating a corrosion area; According to the labeling information of each training detection frame, a corrosion training image is cropped from the hull training image; Performing semantic segmentation and annotation on the corrosion training image to obtain a segmentation and annotation file, wherein the segmentation and annotation file includes a plurality of annotation points and a plurality of corrosion category labels; Constructing the detection annotation file and the hull training image into a corrosion detection dataset; The segmentation annotation file and the corrosion training image are constructed into a corrosion segmentation dataset.

[0013] In a second aspect, the present invention further provides a hull structure corrosion detection device based on target detection and semantic segmentation, which is used to implement the hull structure corrosion detection method based on target detection and semantic segmentation described in the first aspect, and the device includes: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the processor to execute the hull structure corrosion detection method based on target detection and semantic segmentation as described in the first aspect.

[0014] In a third aspect, the present invention further provides a non-volatile computer storage medium, which stores computer-executable instructions, and the computer-executable instructions are executed by one or more processors to execute the hull structure corrosion detection method based on target detection and semantic segmentation described in the first aspect.

[0015] Compared to existing technologies, the present invention has at least the following advantages: By combining target detection and semantic segmentation, the present invention effectively improves the accuracy and efficiency of ship corrosion detection. First, target detection rapidly locates corroded areas on the hull, preventing background interference in subsequent processing and thus improving overall computational efficiency. The rapid response of the target detection phase reduces computational resource overhead, allowing subsequent semantic segmentation to more accurately analyze the corroded areas.

[0016] After locating the corrosion area, semantic segmentation is used to perform fine-grained classification of the corrosion area, accurately identifying the type of corrosion and achieving refined classification of corrosion types. In this way, the present invention can effectively reduce the risk of manual intervention, not only improving the accuracy of corrosion detection, but also effectively reducing misidentification caused by background interference, thereby improving detection efficiency. The technical solution of the present invention can achieve precise identification of corrosion types while ensuring efficient detection, meeting the intelligent requirements of modern ship inspection and having broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive effort.

[0018] Figure 1 This is a flow chart of a method for detecting corrosion of a hull structure based on target detection and semantic segmentation, provided by an embodiment of the present invention; Figure 2This is a structural diagram of a corrosion detection model provided by an embodiment of the present invention; Figure 3a This is a schematic diagram of the structure of a corrosion identification model provided by an embodiment of the present invention; Figure 3b This is a schematic diagram of a process for identifying corrosion areas using a corrosion identification model provided by an embodiment of the present invention; Figure 4 This is a schematic diagram of a feature extraction process of a branch convolutional network in a corrosion identification model provided by an embodiment of the present invention; Figure 5a Schematic diagram of a feature fusion network structure in a corrosion identification model provided by an embodiment of the present invention; Figure 5b This is a schematic diagram of the overall feature extraction process of a feature fusion network in a corrosion identification model provided by an embodiment of the present invention; Figure 6 This is a schematic diagram of a group feature extraction process in a feature fusion network in a corrosion identification model provided by an embodiment of the present invention; Figure 7a Schematic diagram of a global fusion network structure in a corrosion identification model provided by an embodiment of the present invention; Figure 7b This is a schematic diagram of a global fusion network feature extraction process in a corrosion identification model provided by an embodiment of the present invention; Figure 8 This is a schematic diagram of a process for constructing a corrosion detection dataset and a corrosion segmentation dataset provided by an embodiment of the present invention; Figure 9 1 is a schematic structural diagram of a hull structure corrosion detection device based on target detection and semantic segmentation provided by an embodiment of the present invention; The accompanying drawings are numerals as follows: 21: Processor; 22: Memory. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0020] The terms "first," "second," etc., used in this application are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature identified with "first," "second," etc. may explicitly or implicitly include one or more of such features. Throughout this application, unless otherwise specified, "plurality" means two or more.

[0021] In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0022] Existing methods for ship hull corrosion detection primarily rely on manual inspection and traditional image recognition technology. Manual inspection suffers from low efficiency, high labor intensity, and significant influence from operator experience, making it difficult to meet the demands for efficient and accurate inspection of large ship structures in complex environments. Although image recognition technology has been gradually applied to corrosion detection in recent years, existing methods often struggle to accurately distinguish between corroded and non-corroded areas due to factors such as the complex background of the hull surface, the blurred boundaries of the corroded areas, and the irregular morphology. This results in inaccurate corrosion localization and high false detection rates. Furthermore, traditional image recognition methods focus on overall target detection and lack a deep understanding of the internal details of the corroded area. This makes pixel-level corrosion type identification impossible, resulting in coarse-grained detection results that are difficult to adapt to the practical needs of corrosion type classification and assessment. Furthermore, the complex and highly specialized nature of corrosion image annotation and the limited availability of publicly available datasets further hinder model training effectiveness and practical application. Therefore, improving the accuracy and efficiency of corrosion area detection and achieving refined corrosion type identification remain pressing technical challenges in this field.

[0023] To address the problems of insufficient corrosion detection accuracy, rough classification, and low efficiency in existing technologies, this paper proposes a method and device for detecting corrosion in ship structures based on a combination of target detection and semantic segmentation. The method first uses target detection technology to quickly locate the corroded area in a ship hull image. By generating a localization frame for the corroded area, background interference is effectively reduced, improving detection processing efficiency. Subsequently, the image of the corroded area is cropped based on the localization results and input into a corrosion recognition model based on a semantic segmentation network. This allows for pixel-level classification of the various types of corrosion within the corroded area, resulting in a refined identification of the corrosion type.

[0024] This method combines efficient positioning with high-precision classification, significantly improving the accuracy and response speed of ship corrosion detection while also enhancing the model's adaptability in complex environments. By constructing a corrosion detection dataset and a corrosion segmentation dataset, and training the corrosion detection and recognition models separately, it effectively enhances the ability to identify multiple types of corrosion targets.

[0025] The following describes the specific implementation process of the hull structure corrosion detection method and device based on target detection and semantic segmentation with reference to the accompanying drawings and specific examples.

[0026] Embodiment 1: Embodiment 1 of the present invention provides a method for detecting corrosion of a hull structure based on target detection and semantic segmentation, such as Figure 1 As shown, the following steps are included: Step 101: Input the hull image into the corrosion detection model to obtain a detection frame for locating the corrosion area.

[0027] The problem of locating the hull corrosion area is essentially a target detection task, so it is necessary to design a hull corrosion detection neural network structure suitable for this task. The corrosion detection model consists of three parts: the backbone network, the neck network, and the head network. The backbone network is used to extract the feature information of the corrosion area in the hull image, which contains basic feature information from the low to mid-level layers. The neck network is used to extract multi-scale corrosion area feature information from the corrosion area feature information, thereby enhancing the perception of corrosion areas of different scales. Among them, the multi-scale corrosion area feature information contains rich corrosion area information, such as the texture, size, shape, and other information of the corrosion area. The head network is used to locate the detection frame of the corrosion area from the multi-scale corrosion area feature information, make predictions based on the multi-scale feature information, and output the detection frame of the corrosion area.

[0028] The hull image to be inspected is input into the corrosion detection model, which performs feature extraction and target recognition on the hull image and outputs the location of the corroded area in the hull image in the form of one or more detection boxes, each of which contains the coordinate information of the corresponding corroded area.

[0029] Step 102: According to the detection frame, a corresponding corrosion image is cropped from the hull image.

[0030] Based on the detection bounding box of the corrosion area, the corresponding corrosion image is cropped from the hull image according to the coordinate information of the detection bounding box. This cropping operation ensures that the subsequent semantic segmentation task is processed only within the detected corrosion area, effectively reducing background interference, lowering computational complexity, and improving classification accuracy. The cropped corrosion image serves as input to the corrosion recognition model to further identify the corrosion type.

[0031] In one embodiment, the format of the position coordinate information of the detection frame can be (x_center, y_center, width, height). Among them, x_center and y_center are the relative coordinates of the center point of the detection frame, width is the relative value of the detection frame width, and height is the relative value of the bounding box height, which are calculated by the following formulas respectively: x_center = (x_min + x_max) / (2 × image_width) y_center = (y_min + y_max) / (2 × image_height) width = (x_max - x_min) / image_width height = (y_max - y_min) / image_height Where image_width is the width of the hull image, and image_height is the height of the hull image. Based on the complete position coordinate information, the corrosion area can be accurately cut out from the hull image.

[0032] Step 103: Input the corrosion image into a corrosion recognition model, and perform fine-grained classification on the corrosion image using a semantic segmentation method to determine the corrosion location and corrosion type.

[0033] Because there are multiple types of corrosion in the hull corrosion area, and these corrosion types have subtle differences in the image, the corrosion areas need to be classified and identified at the pixel level. To this end, a hull corrosion recognition neural network structure suitable for semantic segmentation tasks is designed, consisting of two parts: an encoder and a decoder.

[0034] The encoder performs preliminary feature extraction on the input corrosion image, obtaining shallow-level feature information. To further enhance information representation, these shallow features are further processed to obtain deep-level feature information, including the texture structure, edge contours, and color variations of the corrosion region. The decoder further extracts features from the shallow features extracted by the encoder, improving the ability to identify corrosion region boundaries and generating a feature map containing detailed corrosion region boundaries. Simultaneously, the deep-level feature information is gradually upsampled to restore the feature map size, enabling high-resolution, fine-grained pixel-level classification. Finally, the deep-level feature information is fused with the processed shallow-level feature information to obtain a feature map that can further identify corrosion regions and corrosion categories.

[0035] The cropped corrosion image from step 102 is fed into the corrosion recognition model, which performs semantic classification on each pixel in the input corrosion image and outputs a corresponding pixel-level label map. This label map reflects the corrosion type at each location in the corrosion area, identifying specific corrosion types such as corrosion at edges or welds, corrosion in areas with coating spalling, and hard scale, enabling fine-grained classification and recognition of corrosion areas.

[0036] In one embodiment, Figure 2 As shown, a schematic diagram of the structure of a corrosion detection model provided by the present invention is shown, which includes three parts: a backbone network, a neck network and a head network. The parts form a synergistic effect through feature extraction, feature fusion and information transmission, thereby realizing accurate detection and positioning of the hull corrosion area.

[0037] In one embodiment, the corrosion detection network identifies corroded areas in a ship hull image. The backbone network input is a 640×640×3 ship hull training image. First, the hull image passes through Feature Extraction Module 1, converting it into a 160×160×64 feature map that captures low-level information such as the image's underlying texture and edges. This feature map is then processed through a series of convolution operations, expanding the number of channels from 64 to 128 and downsampling the spatial resolution to 80×80, resulting in a feature map size of 80×80×128.

[0038] The feature map continues down the corrosion detection network and is input into feature extraction module 4. The output feature map has a size of 20×20×512 and contains rich corrosion-related contextual information. The combination of multiple feature extraction modules effectively increases the receptive field and network capacity. The multi-scale feature fusion module 1 further explores the characteristics of the corrosion area and enhances the model's perception of spatial structure and channel weights. Features at different levels are fused to form a feature map with stronger global expressiveness.

[0039] In the neck network stage, the 20×20×512 feature map is gradually restored to a higher-resolution intermediate feature map through convolution and upsampling, outputting a feature map of 40×40×256. This feature map is aligned with the output of feature extraction module 3 in the backbone network and then undergoes feature concatenation to optimize feature integration and information transfer, enhancing the network's semantic understanding of hull corrosion areas and improving sensitivity to mesoscale corrosion. This process is repeated, and multiple concatenation and fusion with the outputs of other intermediate layers in the backbone network further improve the model's ability to capture fine-grained features for detecting both small- and large-scale corrosion areas.

[0040] After further processing, the 80×80×256 feature map enters the multi-scale feature extraction module 3 again, outputting a feature map of 80×80×128. Downsampling and feature concatenation are then used to further fuse the medium-scale feature map of 40×40×128. Continuing this downsampling process, the model acquires feature representation capabilities at large, medium, and small scales, significantly enhancing its ability to detect corrosion areas of varying sizes.

[0041] Finally, the head network receives three feature maps at different scales (80×80×128, 40×40×256, and 20×20×512) from the neck network and uses them to predict corrosion targets. The detection head generates multiple candidate corroded region bounding boxes and aggregates, filters, and performs non-maximum suppression based on the confidence scores of each region, ultimately outputting optimal corroded region localization boxes. Based on these outputted corroded region localization boxes, the corresponding corroded regions can be cropped from the ship hull training image and used as input for subsequent corrosion type recognition.

[0042] In one embodiment, to address the complex scenarios of ship structure corrosion detection, the corrosion detection network also performs certain processing on the input ship hull images during training, including image data augmentation technology, adaptive anchor box calculation, and adaptive image scaling mechanisms to further enhance the diversity of training samples and the detection capability of small-sized objects. Specifically, image data augmentation technology splices four images using random scaling, random cropping, and random arrangement to enrich the background of the detection target; adaptive anchor box calculation adaptively calculates the optimal anchor box points, calculates the deviation between the anchor box and the annotated box during network training, and reversely updates the network parameters; adaptive image scaling achieves the goal of adding minimal black edges to the scaled image, avoiding excessive redundant information and improving network training speed.

[0043] In one embodiment, Figure 3a Figure 2 shows a schematic diagram of the structure of a corrosion recognition model provided by the present invention. The corrosion recognition model mainly includes two parts: an encoder and a decoder. The encoder is used to extract deep context feature information of the image, and the decoder is used to restore the spatial size of the feature map and output the segmentation result map of the corrosion area.

[0044] Combine Figure 3b , describing the specific implementation process of step 103, step 103 includes: Step 201: extracting a feature map F1 from the eroded image, performing group convolution, dilation, and feature extraction on the feature map F1 to obtain a feature map F2.

[0045] In one embodiment, the encoder includes a coding backbone network, a grouped convolutional network, and a feature fusion network. To accommodate the typically small size of eroded region images and the real-time processing requirements, a lightweight convolutional neural network is preferably used as the coding backbone network to reduce computational resource consumption and improve inference speed. The coding backbone network architecture includes multiple linear bottleneck layers, depthwise separable convolutions, and an inverted residual module. These modules effectively reduce network computational complexity while ensuring efficient feature extraction.

[0046] Specifically, each linear bottleneck layer in the network performs convolution operations by gradually reducing the number of channels, thereby improving computational efficiency. After the depthwise separable convolution operation, the network restores the number of channels through convolution to ensure sufficient feature expression capabilities. The inverted residual structure expands the dimension during the convolution process, then applies depthwise separable convolution and reduces the dimension through convolution. This not only optimizes the computational process but also enables better learning of complex features in high-dimensional space. In addition, when the input and output dimensions are consistent, the residual connection can directly perform a jump connection, thereby accelerating the transfer of feature information.

[0047] The input eroded image is first passed through the encoding backbone network to extract the shallow feature map F1. The shallow feature map F1 is further input into the grouped convolutional network and feature fusion network to extract the deep feature map F2 containing multi-scale information.

[0048] Step 202: Perform multi-branch feature extraction on the feature map F2, fuse complementary feature information, and obtain a feature map F3.

[0049] Specifically, the feature map F2 is further input into a feature fusion network, which adopts a multi-branch structure to fuse context information of different scales, thereby generating a feature map F3.

[0050] Step 203: Perform global and local feature extraction on the feature map F1 to obtain a feature map F4.

[0051] Specifically, the shallow feature map F1 is also fed into the decoder to preserve fine-grained feature information in the image, including fine-grained information such as eroded edges and textures in the eroded image. Furthermore, to further enhance the understanding of the image's spatial structure, the shallow feature map F1 is also fed into the global fusion network for processing, constructing relationships between different spatial locations in the eroded image and outputting a feature map F4 containing contour and edge information.

[0052] Step 204: The feature map F3 and the feature map F4 are concatenated and fused to obtain a feature map F5.

[0053] In one embodiment, in the decoder part, the feature map F3 from the deep network structure of the encoder is first adjusted by 1×1 convolution to adjust the number of channels, and then restored to the shallow feature map by 4 times linear upsampling. Figure 1 At the same time, feature map F4 is also channel-adjusted through 1×1 convolution and concatenated with feature map F3′. It then passes through a 3×3 convolution module and a 4x linear upsampling operation to generate feature map F5 containing rich contextual semantics.

[0054] Step 205: Perform dimensionality reduction mapping on the feature map F5 to obtain the corrosion position and corrosion type.

[0055] In one embodiment, the decoder uses feature map F5 to adjust its channels through a 3×3 convolution, then performs a 4x linear upsampling to reduce the feature map's dimensionality and output a segmentation result map with the same size as the original image. Each pixel in the segmentation result map is assigned a corrosion type label, including but not limited to edge corrosion, weld corrosion, coating spalling corrosion, and hard scale corrosion, enabling automatic identification and classification of corrosion types.

[0056] Combine Figure 4, illustrating the specific implementation process of step 201, step 201 includes: Step 301: performing feature extraction on the feature map F1 to extract original detail feature information of the feature map F1 to obtain a feature map F11; Specifically, in order to enhance the model's perception of corrosion targets of different scales, the grouped convolutional network first uses a 1×1 ordinary convolution block to extract the original detail feature information in the feature map F1 to obtain the feature map F11.

[0057] Step 302: performing feature extraction on the input feature map F1 according to different expansion rates, extracting multi-scale feature information of different ranges in the feature map F1, and obtaining a plurality of feature maps F12; Specifically, in order to more effectively extract multi-resolution features and improve the model's ability to recognize corrosion areas of different sizes, a 3×3 dilation convolution with multiple combinations of different expansion rates is used to extract multi-scale feature information of different ranges in the feature map F1, including information on corrosion areas of different sizes, and obtain multiple feature maps F12.

[0058] Step 303: performing global pooling on each of the feature maps F1 to extract global semantic background feature information in the feature map F1 to obtain a feature map F13; Step 304: The feature map F11, the plurality of feature maps F12 and the feature map F13 are spliced ​​and fused to obtain a feature map F2, so as to fuse the original detail feature information, the multi-scale feature information and the global semantic background feature information into the feature map F2.

[0059] Specifically, the grouped convolutional network concatenates and fuses the feature map F11, multiple feature maps F12, and feature map F13, and merges the original detail features, multi-scale features, and global semantic background feature information into the feature map F2, providing richer feature information for the next feature fusion network, so as to further extract deeper features.

[0060] In one embodiment, in the grouped convolutional network of the encoder, the dilation rate of the 3×3 dilated convolution operation in the combination of multiple dilation rates can be selected from one or more of the following values: 4, 8, 16, and 32. A larger dilation rate enables the model to cover a wide eroded area, while a smaller dilation rate accurately captures local details. This setting is used to extract feature information of different scales in the eroded image, thereby enhancing the model's perception of eroded areas of varying sizes.

[0061] It should be noted that the aforementioned convolution kernel size, sampling size, expansion rate, etc. can be set according to actual conditions and are not specifically limited here.

[0062] In one embodiment, Figure 5aAs shown in FIG, a schematic diagram of the feature fusion network structure in a corrosion recognition model provided by an embodiment of the present invention is shown. The feature fusion network includes a series of convolution blocks, pooling operations, feature weighting and feature splicing operations, combined with Figure 5b , step 202 includes the following steps: Step 401: Evenly divide the feature map F2 into multiple sub-feature maps F21 along the depth direction.

[0063] In one embodiment, a feature map F2 with a length, width, and depth size of H×W×C is input into a feature fusion network, and is divided into 8 sub-feature maps F21 with a length, width, and depth size of H×W×(C / / 8) along the depth direction, so as to achieve uniform distribution of spatial semantic features in each sub-feature map F21.

[0064] Step 402: Perform feature extraction on each sub-feature graph F21 to obtain a feature graph F22.

[0065] Step 403: After normalizing each feature map F22 using a Sigmoid activation function, the feature maps F22 are spliced ​​and restored to the size of the feature map F2 to obtain a feature map F3, wherein the feature map F3 contains rich global context information and multi-scale information.

[0066] Specifically, the eight feature maps F22 are normalized using a Sigmoid activation function and then concatenated to restore the size of the feature map F2. This yields a feature map F3 with dimensions H×W×C for length, width, and depth, which serves as the output of the feature fusion network. Through this multi-branch parallel extraction and fusion mechanism, the feature fusion network effectively achieves dynamic fusion of multi-scale and cross-path feature information, avoiding information loss associated with single-path feature transfer. This effectively captures multi-level spatial features from the macroscopic to the microscopic, enhancing the robustness of the corrosion recognition model for corroded areas in complex backgrounds.

[0067] Combine Figure 6 , describing the specific implementation process of step 402, step 402 includes: Step 501: perform average pooling along the horizontal and vertical directions of the sub-feature map F21, and then weightedly merge it with the sub-feature map F21 to obtain a feature map F23.

[0068] Specifically, average pooling is performed along the horizontal and vertical directions of the sub-feature map F1 to extract directional features, and then a 1×1 convolution adjustment channel is performed and weighted splicing is performed with the sub-feature map F21 to obtain a feature map F23 that fuses directional information and original features to enhance the complementarity between features.

[0069] Step 502: Perform convolution channel adjustment on the sub-feature map F21 to obtain a feature map F24.

[0070] Specifically, a 3×3 convolution channel adjustment is performed on the sub-feature map F21 to cover a wider receptive field and enhance the feature extraction capability of different scales to obtain the feature map F24.

[0071] Step 503: Perform pooling and Softmax processing on the feature map F24 to obtain the feature map F25.

[0072] Specifically, after the feature map F24 is processed by average pooling and Softmax function, the feature map F25 is obtained, which is used to generate the response weight of the salient area.

[0073] Step 504: After performing group normalization processing on the feature map F23, matrix multiplication is performed with the feature map F25 to obtain the feature map F26; Specifically, the feature map F23 is group normalized to stabilize the feature distribution and improve the convergence of the corrosion identification model, and is matrix-multiplied with the feature map F25 to further enrich the feature information and improve the expressive power of the feature map, thereby obtaining the feature map F26.

[0074] Step 505: After performing group normalization, average pooling, and Softmax processing on the feature map F23, it is matrix-multiplied with the feature map F24 to obtain the feature map F27; Step 506: perform feature stitching on the feature map F26 and the feature map F27 to obtain the feature map F22.

[0075] Specifically, the processing from step 501 to step 506 is performed on the eight sub-feature graphs F21 to obtain eight feature graphs F22.

[0076] It should be noted here that the number of the aforementioned sub-feature graphs can be set according to actual conditions and is not specifically limited here.

[0077] In one embodiment, Figure 7a Figure 1 shows a schematic diagram of the global fusion network structure in a corrosion recognition model provided by an embodiment of the present invention. The network aims to fuse the global shape and contour information of the corrosion area with the local detail texture information, thereby improving the model's ability to perceive the contours and boundaries of the corrosion target.

[0078] Combine Figure 7b In step 204, the following steps are specifically included: Step 601: After performing layer normalization processing on the feature map F1, the processed feature map F1 is divided into a feature map F14 and a feature map F15.

[0079] Specifically, the feature map F1 layer is first normalized and then divided into two parts, feature map F14 and feature map F15, which are input into the global branch network and the local branch network respectively to extract the low-frequency global features and high-frequency local features of the image in parallel.

[0080] Step 602: Perform global feature extraction on the feature map F14 to extract low-frequency global information including the overall shape outline of the corrosion area from the feature map F14.

[0081] Specifically, in the global branch network, the input feature map F14 is first fully connected to enhance its global representation. This is then expanded into three branches, generating feature maps F141, F142, and F143, respectively. Feature map F142 compresses spatial information through pooling and then undergoes matrix multiplication with F141 to model long-range spatial dependencies in the image. The result is then processed through a softmax function to generate a weighted feature map F144. Feature map F143 is pooled and matrix multiplied with F144 to produce feature map F145. This feature map contains low-frequency global information, such as the overall outline and shape of the corrosion area, enhancing the model's ability to identify large-scale corrosion targets.

[0082] Step 603: Perform local feature extraction on the feature map F15, and extract high-frequency local information including the texture and shape of the corrosion area from the feature map F15.

[0083] Specifically, in the local branch network, the input feature map F15 is fully connected and expanded into three branches to obtain feature maps F151, F152 and F153 respectively; the feature maps F151 and F152 are both processed by depthwise separable convolution to extract high-frequency local detail texture information and edge features. After the results are fused by dot product, they are sequentially processed by the fully connected layer, the Swish activation function, and then by the full connection and Tanh activation function to generate the feature map F154, thereby constructing a nonlinear expression capability for the changes in corrosion details; after the feature map F153 is processed by depthwise separable convolution, matrix multiplication is performed with the feature map F154 to obtain the feature map F155, which further extracts the representation features of high-frequency local information such as corrosion texture and tiny contours.

[0084] Step 604: Fusing the low-frequency global information and the high-frequency local information to obtain a feature map F4.

[0085] Step 605: performing channel adjustment and feature splicing on the feature map F3 and the feature map F4 to obtain a feature map F5.

[0086] Specifically, feature maps F145 and F155 are concatenated and their channel dimensions are adjusted through a fully connected operation. The adjusted fused feature map is additively fused with the original input feature map F14 to generate the final feature map F5. This provides context for the decoder to generate a precise corrosion segmentation map, thereby improving the efficiency and accuracy of the corrosion recognition model in differentiating corrosion categories.

[0087] In one embodiment, Figure 8 As shown in Figure 2, the steps for constructing the dataset for training the corrosion detection network model and the corrosion recognition model include: Step 701: Acquire a ship hull training image, and perform target detection and annotation on the ship hull training image to obtain a detection and annotation file, wherein the detection and annotation file includes annotation information of a training detection frame for locating a corrosion area.

[0088] Specifically, image data containing corroded areas on the hull can be obtained through drones, robots, or manual photography. To train the model, the hull images must be annotated, including both target detection and semantic segmentation. Image annotation software such as LabelImg, LabelMe, and VIA can be used as annotation tools. During the annotation process, it is important to ensure that the features of the corroded areas are distinct, allowing the human eye to distinguish between intact and corroded areas within the sample. Furthermore, the annotated corroded areas do not contain significant interference factors such as large water stains or silt accumulation.

[0089] Each ship hull training image is annotated for object detection. This involves marking the corrosion area with a rectangular border and generating a corresponding annotation file in JSON, XML, or TXT format. The corresponding corrosion image is then cropped from the ship hull training image based on the location of the annotated border.

[0090] Step 702: Based on the labeling information of each training detection frame, a corrosion training image is cropped from the hull training image.

[0091] Specifically, the object detection annotation file can use the JSON format. Part of the format is as follows: {"images": [{"id": 1, "file_name": "image1.jpg", "height": 720, "width": 1280}], "annotations": [{"id": 1, "image_id": 1, "category_id": 1, "bbox": [640, 360,256, 432], "area": ​​110592, "iscrowd": 0}], "categories": [{"id": 1, "name": "corrosion"}]}. The "bbox": [640, 360, 256, 432]" field represents the rectangular coordinate box of the corrosion area annotated in the ship image, and the "height": 720, "width": 1280" fields represent the length and width of the ship image. Automatic cropping of corrosion images can be achieved based on the annotation information.

[0092] Step 703: semantically segment and annotate the corrosion training image to obtain a segmentation annotation file, wherein the segmentation annotation file includes multiple annotation points and multiple corrosion category labels.

[0093] Specifically, semantic segmentation and annotation are performed on the cropped corrosion image, with the edges of the corrosion being annotated as finely as possible. Multiple points are annotated to form closed polygons, or the corrosion area is circled using the smudge tool. Each type of corrosion is assigned a corresponding label, such as corrosion at edges or welds, corrosion in areas with coating spalling, or hard scale. After annotation is complete, a corresponding semantic segmentation annotation file is generated in either JSON or TXT format.

[0094] In one embodiment, to enrich the training dataset and improve the model's generalization and robustness, seven data augmentation techniques were applied to the image files: 90° and 180° image rotation, brightness adjustment, contrast adjustment, color adjustment, and horizontal and vertical image flipping. The annotation information in the corresponding annotation file was adjusted based on the corresponding data augmentation technique. These techniques were applied to images of ship hulls and corrosion to increase the diversity and quantity of samples.

[0095] Step 704: Construct the detection annotation file and the hull training image into a corrosion detection dataset.

[0096] Specifically, each detection annotation file and its corresponding hull image are stored in designated folders respectively. Then, the dataset is randomly divided in a ratio of 8:1:1 to construct a training set, a validation set, and a test set for the training and performance evaluation of the corrosion detection network model.

[0097] Step 705: Construct the segmentation annotation file and the corrosion training image into a corrosion segmentation dataset.

[0098] Specifically, each segmentation annotation file and its corresponding corrosion image are stored in designated folders respectively; then, the data is randomly divided into training set, validation set and test set in a ratio of 8:1:1 for training and testing of the corrosion recognition network model.

[0099] It should be noted that the above-mentioned corrosion detection dataset and corrosion segmentation dataset are not fixed. Low-quality annotation files will be eliminated during the training process, and more refined annotation information will be added as much as possible to further improve the accuracy of the corrosion detection model and corrosion recognition model.

[0100] Example 2: Based on the hull structure corrosion detection method based on target detection and semantic segmentation provided in the above embodiment, the present invention also provides a hull structure corrosion detection device based on target detection and semantic segmentation that can be used to implement the above method, such as Figure 9 The diagram is a schematic diagram of the device architecture of an embodiment of the present invention. The hull structure corrosion detection device based on target detection and semantic segmentation in this embodiment includes one or more processors 21 and a memory 22. Figure 9 A processor 21 is taken as an example.

[0101] The processor 21 and the memory 22 may be connected via a bus or other means. Figure 9 The bus connection is taken as an example.

[0102] The memory 22 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs and non-volatile computer-executable programs, such as the ship structure corrosion detection method based on target detection and semantic segmentation in Example 1. The processor 21 executes the method by running the non-volatile software programs and instructions stored in the memory 22.

[0103] The memory 22 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state memory device. In some embodiments, the memory 22 may optionally include a memory remotely located relative to the processor 21, and such remote memory may be connected to the processor 21 via a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0104] The program instructions / modules are stored in the memory 22, and when executed by the one or more processors 21, the ship structure corrosion detection method based on target detection and semantic segmentation in the above embodiment 1 is executed, for example, the above described Figures 1-8 The steps shown.

[0105] It is worth noting that the information interaction, execution process, etc. between the modules and units within the above-mentioned devices and systems are based on the same concept as the processing method embodiment of the present invention. The specific content can be found in the description of the method embodiment of the present invention and will not be repeated here.

[0106] Those skilled in the art will understand that all or part of the steps in the various methods of the embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), a disk or an optical disk, etc.

[0107] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A hull structure corrosion detection method based on target detection and semantic segmentation, characterized in that: include: Input the hull image into the corrosion detection model to obtain a detection frame for locating the corrosion area; According to the detection frame, a corresponding corrosion image is cropped from the hull image; The corrosion image is input into a corrosion recognition model, and the corrosion image is fine-grainedly classified using a semantic segmentation method to determine the corrosion location and corrosion type.

2. The method for detecting corrosion of hull structures based on target detection and semantic segmentation according to claim 1, characterized in that: Also includes: Extracting a feature map F1 from the eroded image, performing group convolution, dilation, and feature extraction on the feature map F1 to obtain a feature map F2; Perform multi-branch feature extraction on the feature map F2 and fuse complementary feature information to obtain a feature map F3; Performing global and local feature extraction on the feature map F1 to obtain a feature map F4; The feature map F3 and the feature map F4 are spliced ​​and fused to obtain a feature map F5; Perform dimensionality reduction mapping on the feature map F5 to obtain the corrosion position and corrosion type.

3. The method for detecting hull structure corrosion based on target detection and semantic segmentation according to claim 2, characterized in that: Also includes: Performing feature extraction on the feature map F1 to extract original detail feature information of the feature map F1 to obtain a feature map F11; Performing feature extraction on the input feature map F1 according to different expansion rates, extracting multi-scale feature information of different ranges in the feature map F1, and obtaining a plurality of feature maps F12; Performing global pooling on each of the feature maps F1 to extract global semantic background feature information in the feature map F1 to obtain a feature map F13; The feature map F11, the plurality of feature maps F12 and the feature map F13 are spliced ​​and fused to obtain a feature map F2, so as to fuse the original detail feature information, the multi-scale feature information and the global semantic background feature information into the feature map F2.

4. The method for detecting hull structure corrosion based on target detection and semantic segmentation according to claim 2, characterized in that: Also includes: The feature map F2 is evenly divided into a plurality of sub-feature maps F21 along the depth direction; Perform feature extraction on each sub-feature map F21 to obtain a feature map F22; After normalizing each feature map F22 using the Sigmoid activation function, the feature maps F22 are spliced ​​and restored to the size of the feature map F2 to obtain a feature map F3, wherein the feature map F3 contains rich global context information and multi-scale information.

5. The method for detecting corrosion of hull structures based on target detection and semantic segmentation according to claim 4, characterized in that: Also includes: Perform average pooling along the horizontal and vertical directions of the sub-feature map F21, and then weightedly merge it with the sub-feature map F21 to obtain a feature map F23; Perform convolution channel adjustment on the sub-feature map F21 to obtain a feature map F24; Perform pooling and Softmax processing on feature map F24 to obtain feature map F25; After performing group normalization on the feature map F23, it is matrix-multiplied with the feature map F25 to obtain the feature map F26; After performing group normalization, average pooling, and Softmax processing on the feature map F23, it is matrix-multiplied with the feature map F24 to obtain the feature map F27; Feature map F26 and feature map F27 are combined to obtain feature map F22.

6. The method for detecting corrosion of hull structures based on target detection and semantic segmentation according to claim 2, characterized in that: Also includes: After performing layer normalization processing on the feature map F1, the processed feature map F1 is divided into a feature map F14 and a feature map F15; Performing global feature extraction on the feature map F14 to extract low-frequency global information including the overall shape contour of the corrosion area from the feature map F14; Performing local feature extraction on the feature map F15 to extract high-frequency local information including texture and shape of the corrosion area from the feature map F15; Perform feature fusion on the low-frequency global information and the high-frequency local information to obtain a feature map F4; The feature map F3 and the feature map F4 are subjected to channel adjustment and feature splicing to obtain a feature map F5.

7. The method for detecting hull structure corrosion based on target detection and semantic segmentation according to claim 1, characterized in that: Also includes: The corrosion detection model includes a backbone network, a neck network and a head network; The backbone network is used to extract feature information of the corrosion area in the hull image; The neck network is used to extract multi-scale corrosion region feature information from the corrosion region feature information; The head network is used to locate the detection frame of the corrosion area from the multi-scale corrosion area feature information.

8. The method for detecting hull structure corrosion based on target detection and semantic segmentation according to any one of claims 1 to 7, characterized in that: It also includes the steps for building datasets for training the corrosion detection network model and the corrosion identification model: Acquire a hull training image, and perform target detection and annotation on the hull training image to obtain a detection and annotation file, wherein the detection and annotation file includes annotation information of a training detection frame for locating a corrosion area; According to the labeling information of each training detection frame, a corrosion training image is cropped from the hull training image; Performing semantic segmentation and annotation on the corrosion training image to obtain a segmentation and annotation file, wherein the segmentation and annotation file includes a plurality of annotation points and a plurality of corrosion category labels; Constructing the detection annotation file and the hull training image into a corrosion detection dataset; The segmentation annotation file and the corrosion training image are constructed into a corrosion segmentation dataset.

9. A hull structure corrosion detection device based on target detection and semantic segmentation, characterized in that: include: at least one processor; And, a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the processor to execute the hull structure corrosion detection method based on target detection and semantic segmentation as described in any one of claims 1-8.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the computer executes the hull structure corrosion detection method based on target detection and semantic segmentation as described in any one of claims 1 to 8.

Images