Underwater dark environment target detection method and device based on difference information fusion
By adopting a differential information fusion object detection method in underwater dark environments, using cross-layer iterative structure and multi-scale bar-shaped convolutional subtraction module, the problem of poor object detection performance in the prior art is solved, and higher detection accuracy and accuracy are achieved.
Patent Information
- Application Number
- CN202411825601.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-12
- Publication Date
- 2025-04-29
AI Technical Summary
The prior art has poor target detection performance in underwater dark environments, especially in the case of harsh lighting conditions and severe noise, making it difficult to effectively separate the target and background.
Using a method based on differential information fusion, multi-level features are integrated through cross-layer iterative structures, multi-scale bar-shaped convolution subtraction module extracts the difference information between features of different levels, and decodes the feature map through the guidance attention module to output the target detection results.
It effectively suppresses the interference of redundant background on target detection, improves the accuracy and accuracy of target detection, and is suitable for complex underwater dark environments.
Smart Images
Figure CN119904737A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to an underwater dark environment target detection method and device based on difference information fusion, belonging to the technical field of image processing and computer vision. Background Art
[0002] Due to the absorption and scattering of light by water, the color contrast between the target and the background in the underwater image is low, the boundary is blurred, and the noise is serious, which brings great difficulties to underwater operations. Underwater dark environment target detection is a computer vision task that aims to accurately separate the target from the background in underwater images in response to challenges such as harsh underwater lighting conditions, color distortion, and water ripple interference.
[0003] At present, the commonly used target detection methods mainly include target detection methods based on traditional methods and target detection methods based on deep learning. Target detection methods based on traditional methods usually rely on manually extracted features, such as color, texture, boundary, and brightness. This method often shows insufficient adaptability when facing complex scenes due to its reliance on manual features, especially in dark underwater environments, where its detection performance may be greatly affected. In addition, the manual feature extraction process is time-consuming and requires professional knowledge, which is a significant limitation in practical applications. On the other hand, target detection methods based on deep learning automatically extract image features through neural networks and achieve target detection through end-to-end training. Although this method has greatly improved the adaptability to complex scenes and reduced manual intervention, in dark underwater environments, due to poor lighting conditions and the influence of underwater environmental noise, the feature extraction ability and target detection performance of existing deep learning target detection models will inevitably decline. This limits the application of existing technologies in complex underwater environments. Summary of the invention
[0004] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a method and device for underwater dark environment target detection based on difference information fusion, which integrates multi-level features by adopting a cross-layer iterative structure, better utilizes the complementarity between features of different levels, and uses a multi-scale strip convolution subtraction module to extract the difference information between features of different levels, highlighting the useful information, effectively suppressing the interference of redundant background on target detection, and improving the accuracy of target detection.
[0005] To achieve the above object, the present invention is implemented by adopting the following technical solutions:
[0006] In a first aspect, the present invention provides a method for detecting underwater dark environment targets based on difference information fusion, comprising:
[0007] Acquire an original image in an underwater dark environment; the original image includes a specific target that needs to be detected and identified in the underwater environment, and a background image that constitutes interference;
[0008] Input the acquired original image into the pre-trained PVT-v2 model to extract features of multiple scales of the input image;
[0009] Multi-level features are fused in a cross-layer iterative manner to extract the difference information between fused features at different levels and obtain a difference feature map;
[0010] Fusion of multiple difference feature maps at the same level to obtain an enhanced difference feature map;
[0011] Decoding is performed based on the enhanced difference feature map, and a target detection result is output; the detection result is a pixel-level segmentation mask of the input image, indicating whether each pixel in the image belongs to the target or the background.
[0012] Preferably, the step of inputting the acquired original image into a pre-trained PVT-v2 model to extract features of multiple scales of the input image includes:
[0013] Extract features of four scales, named from low to high , , , .
[0014] Preferably, the method of fusing multi-level features in a cross-layer iterative manner, extracting difference information between fused features at different levels, and obtaining a difference feature map includes:
[0015] The feature map Respectively Fusion to obtain the difference feature map ;
[0016] The feature map Respectively Fusion to obtain the difference feature map ;
[0017] The feature map and Fusion to obtain the difference feature map .
[0018] Preferably, the difference feature map is calculated as follows:
[0019] ;
[0020] in, and Represent high-level fusion features and low-level fusion features respectively. If we calculate ,but refer to , refer to , if calculated ,but refer to , refer to , if calculated ,but refer to , refer to , if calculated ,but refer to , refer to , Refers to the upsampling operation using bilinear interpolation, The convolution kernel size is The strip convolution of |·| refers to the absolute value operation of the obtained result.
[0021] Preferably, the fusion of multiple difference feature maps at the same level to obtain an enhanced difference feature map includes:
[0022] , , At the same level, , At the same level, For one level;
[0023] The difference feature maps at the same level are fused by element-by-element addition operation, and the convolution with a convolution kernel size of 3×3 is used to enhance the feature representation to obtain the enhanced difference feature map. , , among which for Only 3×3 convolution enhancement is performed.
[0024] Preferably, the decoding based on the enhanced difference feature map and outputting the target detection result includes:
[0025] By guiding the attention module to decode the enhanced difference feature map, the decoded feature map of level i+1 And the enhanced difference feature map of level i As input, the feature map Perform layer normalization, then apply 1×5, 5×1, and 3×3 convolution operations and add their results to get the matrix ;
[0026] After layer normalization ;
[0027] For feature maps Perform layer normalization, then apply 1×5, 5×1, and 3×3 convolution operations respectively and add the results to get the matrix ;
[0028] right and Perform splicing to obtain an enhanced matrix ;
[0029] Calculating the Matrix The transposed matrix and augmented matrix of The dot product of is used to get the attention score matrix, and each element of the attention score matrix is divided by the matrix The square root of the dimension is then fed into the Softmax activation function to obtain the weight matrix. Multiply and Add to get the decoding feature map ;
[0030] The obtained decoded feature map The image is upsampled to its original size and a 1×1 convolution is used to output the final target segmentation result.
[0031] In a second aspect, the present invention provides an underwater dark environment target detection device based on difference information fusion, which is used to implement the above-mentioned underwater dark environment target detection method based on difference information fusion, and the device comprises:
[0032] A data acquisition module is used to acquire an original image in an underwater dark environment; the original image includes a specific target that needs to be detected and identified in the underwater environment, as well as a background image that constitutes interference;
[0033] A feature extraction module, used for inputting the acquired original image into a pre-trained PVT-v2 model to extract features of multiple scales of the input image;
[0034] The feature fusion module is used to fuse multi-level features in a cross-layer iterative manner, extract the difference information between fused features at different levels, and obtain a difference feature map;
[0035] A feature enhancement module is used to fuse multiple difference feature maps at the same level to obtain an enhanced difference feature map;
[0036] A decoding output module is used to decode based on the enhanced difference feature map and output a target detection result; the detection result is a pixel-level segmentation mask of the input image, indicating whether each pixel in the image belongs to the target or the background.
[0037] In a third aspect, the present invention provides a computer-readable storage medium storing one or more programs, characterized in that the one or more programs include instructions, which, when executed by a computing device, enable the computing device to execute any one of the above-mentioned underwater dark environment target detection methods based on difference information fusion.
[0038] In a fourth aspect, the present invention provides a computing device comprising one or more processors, a memory and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing any of the above-mentioned underwater dark environment target detection methods based on difference information fusion.
[0039] Compared with the prior art, the beneficial effects achieved by the present invention are:
[0040] (1) The present invention provides an underwater dark environment target detection method based on difference information fusion, which adopts a cross-layer iterative structure to integrate multi-level features, better utilizes the complementarity between features of different levels, and realizes multi-level feature interaction through an iterative method, integrating context information in a wider range;
[0041] (2) The present invention uses a multi-scale strip convolution subtraction module to extract the difference information between features at different levels, highlighting the useful information and effectively suppressing the interference of redundant background on target detection;
[0042] (3) The present invention adopts a guided attention module to mine multi-scale contextual information and capture the relationship between different feature maps, decode the feature maps to obtain the final detection results, and improve the target detection accuracy.
[0043] (4) The underwater dark environment target detection method based on difference information fusion provided by the present invention has important engineering significance in the fields of marine resource development, underwater exploration engineering, etc. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] Figure 1 This is a flow chart of a method for detecting underwater dark environments targets based on difference information fusion provided by an embodiment of the present invention;
[0045] Figure 2 It is a structural diagram of the attention guiding module in an embodiment of the present invention. DETAILED DESCRIPTION
[0046] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.
[0047] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, only structures and / or processing steps closely related to the solutions according to the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.
[0048] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.
[0049] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.
[0050] It should be emphasized here that the step marks mentioned below are not intended to limit the order of the steps, but it should be understood that the steps can be executed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be executed simultaneously.
[0051] An embodiment of the present invention provides a method for detecting underwater dark environment targets based on difference information fusion. Figure 1 As shown, the specific steps include:
[0052] Obtain the original image in the underwater dark environment; the original image includes the target image to be detected and the background image. The target image is the specific object that needs to be detected and identified in the underwater environment, such as coral reefs, submarines, hippocampus, crabs, fish, etc. These targets usually have different shapes, sizes, colors and texture features, and may show low recognizability under insufficient underwater lighting conditions. The background image covers non-target areas in the underwater environment, including water bodies, sand, rocks, aquatic plants, and light spots and shadows caused by uneven lighting. These background images often contain noise and interference information, which poses a challenge to target detection;
[0053] The pre-trained PVT-v2 model is used as the encoder to initially extract features of multiple scales of the input image;
[0054] The highest-level features among the extracted features are cross-layer fused with all corresponding low-level features;
[0055] A multi-scale strip convolution subtraction module is used to extract the difference information between fusion features at different levels to obtain a difference feature map;
[0056] By element-by-element addition, multiple difference feature maps at the same level are fused to obtain an enhanced difference feature map;
[0057] By guiding the attention module to decode the enhanced difference feature map, the final detection result, that is, the target image detection and recognition result, is obtained.
[0058] It should be noted that PVT-v2 is a mature model based on the Vision Transformer architecture. This model can effectively process image features of different scales through its unique multi-scale feature extraction capability, and has been trained and verified on multiple public image recognition and object detection datasets, demonstrating excellent performance.
[0059] In the embodiment of the present invention, the highest level feature among the extracted features is cross-layer fused with all corresponding low-level features, specifically including:
[0060] The four scale features extracted by the encoder are named from low to high , , , ,
[0061] S11, feature map Respectively Fusion, get feature map ;
[0062] S12, feature map Respectively Fusion, get feature map ;
[0063] S13, feature map and Fusion, get feature map .
[0064] Furthermore, a multi-scale strip convolution subtraction module is used to extract the difference information between the fusion features at different levels to obtain the difference feature map, which is calculated as follows:
[0065] (1)
[0066] in, and Represent high-level features and low-level features respectively. For example, if we calculate ,but refer to , refer to , if calculated ,but refer to , refer to , if calculated ,but refer to , refer to , if calculated ,but refer to , refer to , Refers to the upsampling operation using bilinear interpolation, The convolution kernel size is The strip convolution of |·| refers to the absolute value operation of the obtained result.
[0067] In the embodiment of the present invention, multiple difference feature maps at the same level are fused by element-by-element addition to obtain an enhanced difference feature map. The specific implementation process is as follows:
[0068] , , At the same level, , At the same level, For one level.
[0069] The difference feature maps at the same level are fused by element-by-element addition operations, and the feature representation is enhanced using convolution with a convolution kernel size of 3×3, as shown below:
[0070] (2)
[0071] in, For level The enhanced feature difference feature map for Only 3×3 convolution enhancement is performed.
[0072] See also Figure 2 In the embodiment of the present invention, the final detection result is obtained by guiding the attention module to decode the enhanced difference feature map, as follows:
[0073] Decoded feature map at level i+1 And the enhanced difference feature map of level i As input. For the feature map Perform layer normalization, then apply 1×5, 5×1, and 3×3 convolution operations and add their results to get the matrix
[0074] After layer normalization ;
[0075] For feature maps Perform layer normalization, then apply 1×5, 5×1, and 3×3 convolution operations respectively and add the results to get the matrix , expressed as:
[0076] (3)
[0077] (4)
[0078] (5)
[0079] Among them, LN represents the layer normalization operation, They represent the convolution kernel sizes respectively. The convolution operation.
[0080] right and Perform splicing to obtain an enhanced matrix , as shown in formula (6):
[0081] (6)
[0082] Calculating the Matrix The transposed matrix and augmented matrix of The dot product of is used to get the attention score matrix, and each element of the attention score matrix is divided by the matrix The square root of the dimension is then fed into the Softmax activation function to obtain the weight matrix. Multiply and Add to get the decoding feature map , expressed as:
[0083] (7)
[0084] in, represent The transposed matrix of The square root of the dimension of represents the Softmax activation function, is the decoded feature map output by level i.
[0085] Finally, the obtained decoded feature map Upsample to restore the original image size and use 1×1 convolution to output the final target segmentation result. It should be noted that the target segmentation result is essentially a binary image of the same size as the original image, where the value of each pixel indicates whether it is part of the target. Usually, the target area is represented by 1 and the background area is represented by 0.
[0086] Based on the above inventive concept, a second embodiment of the present invention provides an underwater dark environment target detection device based on difference information fusion, the device comprising:
[0087] A data acquisition module, used to acquire original images in underwater dark environments;
[0088] A feature extraction module is used to use the pre-trained PVT-v2 model as an encoder to preliminarily extract features of multiple scales of the input image;
[0089] The feature fusion module is used to cross-layer fuse the highest-level features among the extracted features with all corresponding low-level features to obtain a difference feature map; in the cross-layer fusion process, a multi-scale strip convolution subtraction module is used to extract the difference information between features at different levels;
[0090] A feature enhancement module, used for fusing multiple difference feature maps at the same level by element-by-element addition to obtain an enhanced difference feature map;
[0091] The decoding output module is used to decode the enhanced difference feature map by guiding the attention module to obtain the final detection result.
[0092] It is worth pointing out that the device embodiment corresponds to the above-mentioned method embodiment, and the implementation methods of the above-mentioned method embodiments are all applicable to the device embodiment and can achieve the same or similar technical effects, so they will not be repeated here.
[0093] The third embodiment of the present invention provides a computer-readable storage medium storing one or more programs, wherein the one or more programs include instructions, which, when executed by a computing device, enable the computing device to perform the above-mentioned underwater dark environment target detection method based on difference information fusion.
[0094] A fourth embodiment of the present invention is a computing device, comprising one or more processors, a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include instructions for executing the above-mentioned underwater dark environment target detection method based on difference information fusion.
[0095] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.
[0096] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0097] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0098] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.
Claims
1. A method for underwater dark environment target detection based on difference information fusion, characterized in that: include: Acquire an original image in an underwater dark environment; the original image includes a specific target that needs to be detected and identified in the underwater environment, and a background image that constitutes interference; Input the acquired original image into the pre-trained PVT-v2 model to extract features of multiple scales of the input image; Multi-level features are fused in a cross-layer iterative manner to extract the difference information between fused features at different levels and obtain a difference feature map; Fusion of multiple difference feature maps at the same level to obtain an enhanced difference feature map; Decoding is performed based on the enhanced difference feature map, and a target detection result is output; the detection result is a pixel-level segmentation mask of the input image, indicating whether each pixel in the image belongs to the target or the background.
2. The underwater dark environment target detection method based on difference information fusion according to claim 1 is characterized in that: The step of inputting the acquired original image into the pre-trained PVT-v2 model to extract features of multiple scales of the input image includes: Extract features of four scales, named from low to high , , , .
3. The underwater dark environment target detection method based on difference information fusion according to claim 2 is characterized in that: The method of fusing multi-level features in a cross-layer iterative manner, extracting difference information between fused features at different levels, and obtaining a difference feature map includes: The feature map Respectively Fusion to obtain the difference feature map ; The feature map Respectively Fusion to obtain the difference feature map ; The feature map and Fusion to obtain the difference feature map .
4. The underwater dark environment target detection method based on difference information fusion according to claim 3 is characterized in that: The difference feature map is calculated as follows: ； in, and Represent high-level fusion features and low-level fusion features respectively. If we calculate ,but refer to , refer to , if calculated ,but refer to , refer to , if calculated ,but refer to , refer to , if calculated ,but refer to , refer to , Refers to the upsampling operation using bilinear interpolation, The convolution kernel size is The strip convolution of |·| refers to the absolute value operation of the obtained result.
5. The underwater dark environment target detection method based on difference information fusion according to claim 3 is characterized in that: The fusion of multiple difference feature maps at the same level to obtain an enhanced difference feature map includes: , , At the same level, , At the same level, For one level; The difference feature maps at the same level are fused by element-by-element addition operation, and the convolution with a convolution kernel size of 3×3 is used to enhance the feature representation to obtain the enhanced difference feature map. , , among which for Only 3×3 convolution enhancement is performed.
6. The underwater dark environment target detection method based on difference information fusion according to claim 5 is characterized in that: The decoding based on the enhanced difference feature map and outputting the target detection result includes: By guiding the attention module to decode the enhanced difference feature map, the decoded feature map of level i+1 And the enhanced difference feature map of level i As input, the feature map Perform layer normalization, then apply 1×5, 5×1, and 3×3 convolution operations and add their results to get the matrix ; After layer normalization ; For feature maps Perform layer normalization, then apply 1×5, 5×1, and 3×3 convolution operations respectively and add the results to get the matrix ; right and Perform splicing to obtain an enhanced matrix ; Calculating the Matrix The transposed matrix and augmented matrix of The dot product of is used to get the attention score matrix, and each element of the attention score matrix is divided by the matrix The square root of the dimension is then fed into the Softmax activation function to obtain the weight matrix. Multiply and Add to get the decoding feature map ; The obtained decoded feature map The image is upsampled to its original size and a 1×1 convolution is used to output the final target segmentation result.
7. An underwater dark environment target detection device based on difference information fusion, characterized in that: The device is used to implement the underwater dark environment target detection method based on difference information fusion according to any one of claims 1 to 6, and comprises: A data acquisition module is used to acquire original images in an underwater dark environment; the original images include specific targets that need to be detected and identified in the underwater environment, as well as background images that constitute interference; A feature extraction module, used for inputting the acquired original image into a pre-trained PVT-v2 model to extract features of multiple scales of the input image; The feature fusion module is used to fuse multi-level features in a cross-layer iterative manner, extract the difference information between fused features at different levels, and obtain a difference feature map; A feature enhancement module is used to fuse multiple difference feature maps at the same level to obtain an enhanced difference feature map; A decoding output module is used to decode based on the enhanced difference feature map and output a target detection result; the detection result is a pixel-level segmentation mask of the input image, indicating whether each pixel in the image belongs to the target or the background.
8. A computer-readable storage medium storing one or more programs, characterized in that: The one or more programs include instructions, which, when executed by a computing device, enable the computing device to perform any one of the underwater dark environment target detection methods based on difference information fusion according to claims 1 to 6.
9. A computing device, characterized in that include, One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing any of the underwater dark environment target detection methods based on difference information fusion according to claims 1 to 6.