Target recognition method, system, device and medium in low-contrast scene

Abstract

The application provides a target recognition method, system, device and medium in a low-contrast scene, relating to the field of computer vision of artificial intelligence. The method comprises: using a double-bone encoder to extract multi-level RGB features of an RGB image and multi-level depth features of a depth image respectively; inputting the multi-level RGB features and the multi-level depth features into a cross-modal feature fusion module to generate cross-modal fusion features through double-path fusion and a depth-guided attention mechanism; inputting the cross-modal fusion features into a self-adaptive edge optimization module to generate high-dimensional enhanced features through a dynamic gating mechanism and edge enhancement processing; inputting the high-dimensional enhanced features into a hierarchical feature fusion module to generate globally optimized features through cross-scale fusion and a channel attention mechanism, and generating a saliency prediction map based on the globally optimized features. The application can realize accurate, robust and efficient recognition of targets in a low-contrast scene.

Description

Technical Field

[0001] This application relates to the field of computer vision in artificial intelligence, and more particularly to a method, system, device, and medium for target recognition in low-contrast scenes. Background Technology

[0002] With the rapid development of intelligent monitoring, autonomous driving, and security early warning systems, target recognition technology has become one of the core application areas in computer vision. However, in low-contrast scenes (such as low-light environments at night, foggy, snowy, or dusty weather, backlit scenes, or scenes where the grayscale difference between the target and the background is minimal), image details are easily suppressed, and target features are blurred or weakened. This makes it difficult for traditional target recognition methods to accurately capture effective features, resulting in problems such as low recognition accuracy and high false positive / false negative rates. These scenarios are widespread in practical applications and directly affect the decision-making reliability of intelligent systems.

[0003] Therefore, the main issues currently include the following:

[0004] 1. Weak feature extraction capability in low contrast environments: Traditional target recognition methods rely on manually designed features. In low contrast scenes, due to the small differences in grayscale and texture between the target and the background, it is difficult to extract effective distinguishing features. Although existing deep learning methods have stronger feature learning capabilities, they are still prone to insufficient feature representation when faced with feature blurring caused by weak lighting and noise interference, resulting in confusion between the target and the background and a significant decrease in recognition accuracy.

[0005] 2. The limitations of single-modal data are difficult to overcome: Most existing target recognition methods rely on a single RGB image modality. In low-contrast scenes, the information contained in a single modality is limited and cannot effectively compensate for the lack of details. Some multimodal fusion methods have problems such as simple fusion strategies and insufficient mining of modal information complementarity. They fail to give full play to the synergistic advantages of multi-source data (such as RGB, depth, infrared, etc.) and are difficult to adapt to complex and ever-changing low-contrast scenes.

[0006] 3. Insufficient accuracy in target edge perception and contour extraction: In low-contrast scenes, target edges are often blurry. Existing methods have limited ability to capture edge details and lack dedicated edge enhancement and accurate perception mechanisms, resulting in incomplete target contour extraction and inaccurate boundary localization. This not only affects the accurate identification of targets but also further interferes with the performance of downstream tasks such as target tracking and behavior analysis.

[0007] 4. Insufficient anti-interference capability and robustness: Low-contrast scenes are often accompanied by additional interference factors such as noise, sudden changes in illumination, and partial occlusion. Existing methods lack effective anti-interference mechanisms, and under the combined effect of multiple interferences, false detections and false negatives are prone to occur. In addition, the adaptability to target scale changes in low-contrast scenes is insufficient, and the recognition performance of small or large targets fluctuates greatly, making it difficult to meet the diverse practical application needs. Summary of the Invention

[0008] This application provides a target recognition method, system, device, and medium for low-contrast scenes. By constructing an end-to-end recognition model that integrates cross-modal deep fusion, adaptive edge optimization, and hierarchical feature aggregation, it solves key problems such as blurred feature representation, limited single-modal information, inaccurate target edge perception, and insufficient system anti-interference robustness in low-contrast environments, thereby achieving accurate, robust, and efficient target recognition in low-contrast scenes.

[0009] In a first aspect, this application provides a target recognition method for low-contrast scenes, the method comprising:

[0010] A dual-backbone encoder is used to extract multi-level RGB features from the RGB image and multi-level depth features from the depth image, respectively; wherein, both the RGB image and the depth image are images acquired in low-contrast scenes;

[0011] The multi-level RGB features and multi-level depth features are input into the cross-modal feature fusion module, and cross-modal fused features are generated through dual-path fusion and depth-guided attention mechanism;

[0012] The cross-modal fusion features are input into the adaptive edge optimization module, and high-dimensional enhanced features are generated through dynamic gating mechanism and edge enhancement processing;

[0013] The high-dimensional enhanced features are input into the hierarchical feature fusion module, and global optimized features are generated through cross-scale fusion and channel attention mechanisms. A saliency prediction map is then generated based on the global optimized features.

[0014] In one possible design, the multi-level RGB features and multi-level depth features are input into a cross-modal feature fusion module, and cross-modal fused features are generated through dual-path fusion and a depth-guided attention mechanism, including:

[0015] Based on multi-level RGB features With multi-level deep features The basic information of the saliency target bimodality is calculated using the following formula. :

[0016]

[0017] In the formula, Indicates the kernel size as Depth-separable convolution operations, This represents the nonlinear correction unit operation, which includes depthwise separable convolution (DSConv), batch normalization (BN), and Gaussian error linearization unit (GELU). This represents an element-wise addition operation;

[0018] Based on multi-level RGB features With multi-level deep features The complementary information of the two modes of saliency target can be calculated using the following formula. :

[0019]

[0020]

[0021] In the formula, This represents a feature concatenation operation at the channel dimension. Indicates the kernel size as Depth-separable convolution operations, This represents a bimodal cascaded feature, where dim=1 indicates a transformation in the channel dimension;

[0022] Basic information of the dual-modal saliency target and complementary information The query feature and the deep guidance feature are respectively input into the deep guided attention mechanism to calculate the cross-modal fusion feature. .

[0023] In one possible design, the basic information of the saliency target bimodality is incorporated. and complementary information The query feature and the deep guidance feature are respectively input into the deep guided attention mechanism to calculate the cross-modal fusion feature. ,include:

[0024] The complementary information of the two modes is extracted using depthwise separable convolution operations. and basic information Mapped to the first query feature Second query features First key feature Second key features First-value features Secondary features And obtain the query feature projection results. ;

[0025] The first key feature Second key features First-value features Secondary features Joint bond features are formed by concatenating them along the channel dimension. Joint value characteristics Simultaneously, the projection results of the query features and joint bond features Perform region-level average pooling to obtain the first region-level features. Second regional features and third-region-level features ;

[0026] First-level query features Second region level features Flattened into a sequence, the inter-regional correlation is calculated using matrix multiplication. Furthermore, a depth similarity weighting is introduced to obtain an inter-region correlation matrix for enhancing depth consistency. ;

[0027] Inter-regional correlation matrix Perform a Top K filter to select the K key regions with the highest relevance for each query region, and generate a sparse region graph index. ;

[0028] Query feature projection results For query features, composite key features Joint value characteristics As key values, combined with the region map index Enhanced features are generated through a dual sparse attention mechanism. and Token-level attention matrix ;

[0029] Computing auxiliary-scale dual sparse attention enhancement features and Token-level attention matrix With principal scale enhancement features After concatenation, the features are fused using depthwise separable convolution operations to obtain multi-scale features. ;

[0030] Location enhancement features are extracted through nonlinear correction unit operations, along with multi-scale features. The channels are added together, and then projected back to the original dimensions using depthwise separable convolution operations to obtain cross-modal fusion features. .

[0031] In one possible design, the cross-modal fusion features are input into an adaptive edge optimization module, which generates high-dimensional enhanced features through a dynamic gating mechanism and edge enhancement processing, including:

[0032] Calculate cross-modal fusion features Dynamic gating value Its mean, the calculation process is expressed as follows:

[0033]

[0034]

[0035]

[0036] In the formula, This indicates an adaptive global average pooling operation. and Represents depthwise separable convolution operation DSConv The convolution kernel matrix below, Represents the ReLU function. This represents the Sigmoid function. Indicates the dynamic gating value. This indicates the calculation of the mean. This represents the average value of the dynamic gating value. This represents adaptive mean features;

[0037] when At that time, the edge feature map of the input features is extracted by the learnable edge detection module, and then the line features are extracted by the lightweight Hough transform to obtain the line feature map;

[0038] edge feature map With line feature map The features are then fused together, and then compressed and transformed using a nonlinear correction unit to obtain a dimension-matched feature map. ;

[0039] Feature maps after dimension matching The feature maps are reconstructed from a mesh structure to a sequence structure, sequence modeling is performed using the Mamba module to capture long-range dependencies, and then reconstructed back to a mesh structure to obtain the Mamba-enhanced feature maps. ;

[0040] Feature maps enhanced based on Mamba The high-dimensional enhanced features are calculated using the following formula. :

[0041]

[0042]

[0043] In the formula, Indicates batch normalization, This represents the Hadamard product operation. This represents the Sigmoid function. Indicates the kernel size as Depth-separable convolution operations, Representing multi-scale features, Indicates channel transformation characteristics. This represents the dynamic gating value.

[0044] In one possible design, the high-dimensional enhanced features are input into a hierarchical feature fusion module, which generates globally optimized features through cross-scale fusion and channel attention mechanisms, including:

[0045] Enhanced features in high dimensions Perform an upsampling operation to obtain high-dimensional features with the same size as the target. ;

[0046] By using depthwise separable convolution and unit convolution operations, the upsampled high-dimensional features are... The number of channels was adjusted to match the low-dimensional features. Consistency, resulting in high-dimensional features after dimension matching. ;

[0047] High-dimensional features after dimension matching With low-dimensional features Channel-dimensional splicing and fusion are performed, followed by feature fusion and transformation using a nonlinear correction unit to extract cross-scale complementary information. ;

[0048] Channel attention mechanism for fusing complementary information Weights are assigned, and the residual connections are combined with low-dimensional features to retain the original features, resulting in multi-level fused features. ;

[0049] Multi-level fusion features are achieved through multi-level fusion strategies, channel compression transformation, and super-resolution pixel shuffling operations. The fusion transformation yields the globally optimized features. .

[0050] In one possible design, after generating a saliency prediction map based on the global optimization features, the method includes:

[0051] The dual-backbone encoder, cross-modal feature fusion module, adaptive edge optimization module, and hierarchical feature fusion module are combined to form a recognition model. The recognition model is trained using a multi-scale weighted combination loss function, and target recognition in low-contrast scenes is achieved based on the trained recognition model. The multi-scale weighted combination loss function includes pixel-wise binary cross-entropy loss, joint intersection-union ratio loss, and edge-aware loss.

[0052] In one possible design, the multi-scale weighted combination loss function is expressed as:

[0053]

[0054]

[0055] In the formula, This represents the single-scale combined loss. , and These represent pixel binary cross-entropy loss respectively. Combined loss and edge perception loss The weighting coefficients, v st1 , v st2 and v st3 These represent the first single-scale combined loss, respectively. L 1. Second Single-Scale Combined Loss L 2 and the third single-scale combined loss L The weighting factor is 3.

[0056] Secondly, this application provides a target recognition system for low-contrast scenes, the system comprising:

[0057] The feature extraction unit is configured to extract multi-level RGB features of the RGB image and multi-level depth features of the depth image using a dual-backbone encoder; wherein the RGB image and the depth image are both images acquired in low-contrast scenes;

[0058] The cross-modal fusion unit is configured to input the multi-level RGB features and multi-level depth features into the cross-modal feature fusion module, and generate cross-modal fused features through dual-path fusion and depth-guided attention mechanism;

[0059] The edge optimization unit is configured to input the cross-modal fusion features into the adaptive edge optimization module and generate high-dimensional enhanced features through dynamic gating mechanism and edge enhancement processing.

[0060] The hierarchical fusion unit is configured to input the high-dimensional enhanced features into the hierarchical feature fusion module, generate globally optimized features through cross-scale fusion and channel attention mechanisms, and generate a saliency prediction map based on the globally optimized features.

[0061] Thirdly, embodiments of this application provide an electronic device, including: at least one processor and a memory; the memory stores computer execution instructions; the at least one processor executes the computer execution instructions stored in the memory, causing the at least one processor to perform the target recognition method in low-contrast scenes as described in the first aspect and various possible designs of the first aspect.

[0062] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions. When a processor executes the computer-executable instructions, it implements the target recognition method in low-contrast scenes as described in the first aspect and various possible designs of the first aspect.

[0063] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the target recognition method in low-contrast scenes as described in the first aspect and various possible designs of the first aspect.

[0064] The target recognition method, system, device, and medium for low-contrast scenes provided in this application have at least the following beneficial effects:

[0065] 1) This application uses a dual-backbone network to extract high-quality multi-scale features of RGB and depth modal respectively, and combines them with a depth-guided attention mechanism for deep fusion. This invention can extract more discriminative and robust features from the original data that is blurry and has weak contrast. It effectively overcomes the feature confusion problem caused by the similarity of grayscale and texture between the target and the background, and improves the sufficiency and effectiveness of feature representation.

[0066] 2) The cross-modal feature fusion module designed in this application breaks through the information limitations of a single RGB modality by using a dual-branch (pixel-level association and channel-level complementarity) fusion strategy and combining it with a depth information-guided attention mechanism. It fully explores and utilizes the complementary advantages between RGB and depth and other multi-source data, thereby making up for the lack of detail caused by low contrast at the information level.

[0067] 3) The adaptive edge optimization module introduced in this application enhances the timing of intelligent decision-making through dynamic gating mechanism, and integrates learnable edge detection, lightweight Hough transform and Mamba sequence modeling. It can accurately capture and enhance the blurred target edge and line structure features, effectively solving the problems of incomplete contour extraction and inaccurate boundary positioning, and providing more accurate target structure information for downstream tasks.

[0068] 4) The hierarchical feature fusion module of this application enhances the model's adaptability to targets of different scales through multi-scale feature fusion and channel attention calibration. At the same time, the design of the entire architecture and the multi-scale weighted loss function jointly improve the model's stability and anti-interference ability under complex interference such as noise, sudden changes in illumination, and occlusion, and reduce the false detection and false detection rates, so that it can meet the reliability requirements of diverse practical application scenarios. Attached Figure Description

[0069] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0070] Figure 1 A framework diagram of a target recognition method in a low-contrast scene provided in an embodiment of this application;

[0071] Figure 2 A flowchart illustrating a target recognition method in a low-contrast scene provided in this application embodiment;

[0072] Figure 3 A visualization comparison chart of the PR curves of the method proposed in this application and other existing methods on the SSD dataset is provided for the embodiments of this application.

[0073] Figure 4 A visualization comparison of PR curves of the proposed method and other existing methods on a DUT dataset is provided for embodiments of this application.

[0074] Figure 5 A visualization comparison chart of the PR curves of the method proposed in this application and other existing methods on the NJU2K dataset is provided for embodiments of this application.

[0075] Figure 6 A visualization comparing the PR curves of the proposed method and other existing methods on the NLPR dataset, provided for embodiments of this application;

[0076] Figure 7 A visualization comparison chart of the PR curves of the method proposed in this application and other existing methods on the SIP dataset is provided for embodiments of this application.

[0077] Figure 8 A visualization comparison of the PR curves of the proposed method and other existing methods on the STERE dataset is provided for embodiments of this application.

[0078] Figure 9 A visualization comparison chart of the PR curves of the method proposed in this application and other existing methods on the LFSD dataset is provided for embodiments of this application.

[0079] Figure 10 A comparison chart of the saliency prediction results of the method proposed in this application and other existing methods provided for embodiments of this application;

[0080] Figure 11 This is a structural diagram of a target recognition system for low-contrast scenes provided in an embodiment of this application.

[0081] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0082] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of systems and methods consistent with some aspects of this application as detailed in the appended claims.

[0083] The collection, storage, use, processing, transmission, provision, and disclosure of user data and other information involved in the technical solution of this application all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0084] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0085] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0086] This application provides a target recognition method for low-contrast scenes. The core design revolves around four key concepts: a cross-modal feature fusion module, a deep-guided attention mechanism, an adaptive edge optimization module, and a hierarchical feature fusion module, constructing an end-to-end architecture from feature extraction to prediction. In terms of technical setup, considering the correlation and complementarity of bimodal data, the importance of multi-scale information, and the role of edge details in improving detection accuracy, a multi-module collaborative integrated architecture is designed. Specific advantages are analyzed as follows: First, the bimodal feature extraction module provides high-quality basic features for the model. EdgeNext is used as the RGB feature encoder, balancing lightweight design with spatial feature representation capabilities; MobileNet is selected to extract deep features, adapting to the modal characteristics of deep data. The selection and dimensional adaptation design of the dual-backbone network ensure the consistency and effectiveness of bimodal features, laying the foundation for subsequent fusion. Secondly, the cross-modal feature fusion module achieves deep collaboration of bimodal information. This module uses a dual-branch design of Hadamard product operation and channel concatenation to capture pixel-level correlation information and channel-level complementary information of bimodal features respectively, avoiding information loss in a single fusion method. The integrated deep-guided attention mechanism introduces deep similarity to guide region-level correlation calculation, and combines region-level Top K selection and token-level Top K selection to enhance cross-modal correlation while reducing computational complexity. The design of supplementary location-enhanced features solves the problem of missing local location information in the attention mechanism, realizing cross-modal feature fusion that takes into account both global correlation and local details. Thirdly, the adaptive edge optimization module achieves adaptive enhancement of detailed features. This module employs a dynamic gating mechanism to adaptively decide whether to perform edge enhancement based on feature importance, reducing unnecessary computation while maintaining detail accuracy. The built-in learnable edge detection module and lightweight Hough transform accurately extract edge and line structure features, respectively, enhancing target contour and geometric structure information. Combined with shared Mamba for sequence modeling of structural features, it effectively captures long-distance dependencies between edge and line features, solving the problem of traditional enhancement modules struggling to model global structural consistency and achieving synergistic enhancement of detail features and global dependencies. Fourth, a multi-scale unified representation is constructed through a hierarchical feature fusion module. This module uses upsampling guidance, splicing fusion, and attention calibration to organically combine high-level semantic features with low-level detail features, compensating for the information limitations of single-scale features. The introduction of a channel attention mechanism dynamically adjusts the contribution of different channels, resolving redundancy and imbalance issues between cross-scale features. The residual connection design ensures the preservation of original feature information, avoiding information decay during cross-scale fusion, and achieving cross-scale features that combine semantic consistency and detail accuracy.Finally, this application proposes a multi-scale weighted combined loss function to optimize saliency prediction performance. This function consists of three weighted parts: pixel-wise binary cross-entropy loss to quantify the classification error of each pixel, region-based joint intersection-over-union loss to optimize the global structural consistency of the predicted region, and edge-aware loss to improve the accuracy of the target boundary. These three loss functions work synergistically, mutually reinforcing each other to jointly improve the comprehensive saliency prediction. In terms of results comparison, experimental results on seven public RGB-D datasets show that the proposed method outperforms other existing state-of-the-art methods.

[0087] like Figure 1 The diagram shown is a framework diagram of a target recognition method in low-contrast scenes provided in an embodiment of this application. This method aims to solve problems such as blurred target details and insufficient modal information complementarity in low-contrast scenes. First, to construct high-quality multimodal basic features, this application uses a dual-backbone encoder to extract RGB and depth features respectively: EdgeNext is selected as the RGB feature encoder, outputting five levels of RGB features from low to high level, namely the first RGB feature. Second RGB feature Third RGB feature Fourth RGB feature and the fifth RGB feature MobileNet is used as the deep feature encoder, which simultaneously outputs five levels of deep features, namely the first deep feature. Second depth features Third depth features Fourth depth features and fifth depth features Among them, the extracted RGB and low-level features in the depth features ( , , and It retains rich target details and location information, providing support for subsequent target detail restoration and accurate boundary prediction; the extracted RGB and high-level features in the depth features ( , , , , and It carries abstract semantic information, can accurately characterize the target category attributes, and provides semantic guarantee for target recognition in complex scenarios.

[0088] Secondly, based on the aforementioned multimodal features, this application implements progressive feature optimization through a three-level progressive module:

[0089] The first level is the cross-modal feature fusion module, which integrates all features. , , , , , , , , and As input, on the one hand, a dual-path fusion design is used to fully explore the inherent correlation and complementary value of bimodal data, avoiding information loss caused by a single fusion method; on the other hand, a deep-guided attention mechanism is integrated to optimize regional-level correlation calculations based on deep similarity, and a dual-sparse strategy of regional-level Top K selection and token-level Top K selection is combined to significantly reduce computational complexity while enhancing cross-modal synergy; in addition, location-enhanced features are used to supplement local location information, solving the problem of traditional attention mechanisms neglecting local details, and finally outputting cross-modal fusion features that take into account both global correlation and local details, including the first cross-modal fusion feature. Second cross-modal fusion features Third cross-modal fusion features Fourth cross-modal fusion features Fifth cross-modal fusion features .

[0090] The second level is the adaptive edge optimization module, which uses the first cross-modal fusion features. Second cross-modal fusion features Third cross-modal fusion features Fourth cross-modal fusion features Fifth cross-modal fusion features As input, this approach focuses on the core challenge of blurred target edges in low-contrast scenes: it adaptively determines feature importance through a dynamic gating mechanism, performing edge enhancement only on high-value features to reduce unnecessary computation while maintaining detail accuracy; simultaneously, it integrates edge detection and line feature extraction units, combined with long-distance dependency modeling capabilities, to accurately capture target edge and line structure information, achieving adaptive enhancement of detailed features and outputting high-dimensional enhanced features rich in detail, including the first high-dimensional enhanced feature. Second high-dimensional enhancement features Third-dimensional enhanced features Fourth high-dimensional enhancement features and the fifth high-dimensional enhanced features .

[0091] The third level is the hierarchical feature fusion module, which receives the first high-dimensional enhanced features. Second high-dimensional enhancement features Third-dimensional enhanced features Fourth high-dimensional enhancement features and the fifth high-dimensional enhanced features Furthermore, cross-scale information integration is performed: complementary information between features at different scales is extracted through feature splicing guided by upsampling; at the same time, residual connection design is introduced to effectively avoid information attenuation during cross-scale fusion, and finally, globally optimized features with both semantic consistency and detail accuracy are generated, providing core support for target recognition.

[0092] Finally, to ensure the accuracy of model prediction, this embodiment uses a multi-scale weighted combined loss function to supervise the saliency prediction map output by the model. With the saliency ground truth map as the optimization target, the model parameters are optimized collaboratively through multi-scale supervision signals to ensure the accuracy and robustness of target recognition in low-contrast scenes.

[0093] like Figure 2 The diagram shown is a flowchart of a target recognition method in a low-contrast scene provided by an embodiment of this application. In specific implementation, this method includes the following steps S10-S40.

[0094] S10: Use a dual-backbone encoder to extract multi-level RGB features from the RGB image and multi-level depth features from the depth image, respectively.

[0095] It's important to note that both RGB images and depth images are captured in low-contrast scenes. Specifically, low-contrast scenes refer to nighttime, foggy conditions, backlighting, or scenes where the target and background colors are similar. An RGB image is a color image composed of three color channels: Red, Green, and Blue. Each pixel contains the intensity values ​​of these three channels, providing visual information such as the target's color, texture, lighting, and surface details. For example, RGB images can originate from color photographs taken by ordinary cameras, images captured by surveillance cameras during the day or under normal lighting, or color images of people, landscapes, or objects taken by mobile phones. A depth image, on the other hand, is an image where each pixel value represents the distance (depth) from the camera. It is usually represented as a grayscale image, with closer pixels appearing brighter and farther pixels appearing darker. Its purpose is to provide spatial structure, distance relationships, and contour information of the target, supplementing the geometric information lacking in RGB images in low-contrast scenes. For example, depth images can be obtained from images captured by depth cameras (such as Kinect, iPhone's LiDAR, and structured light cameras), depth maps calculated through binocular vision, depth information maps generated by LiDAR, or depth perception images commonly used in autonomous driving and robot navigation.

[0096] The method described in this application can be applied to autonomous driving, traffic monitoring, and / or security scenarios. When applied to autonomous driving scenarios, the RGB image and depth image can be images captured in low-contrast scenes by ordinary cameras and depth cameras (or LiDAR) equipped on autonomous vehicles. In traffic monitoring and security scenarios, the RGB image and depth image can be images captured by monitoring equipment, which includes monitoring cameras for capturing RGB images and depth cameras or LiDAR for capturing depth images.

[0097] In this embodiment, EdgeNext is selected as the RGB feature encoder to output five levels of RGB features from low to high, and MobileNet is used as the depth feature encoder to output five levels of depth features simultaneously.

[0098] S20: Input multi-level RGB features and multi-level deep features into the cross-modal feature fusion module, and generate cross-modal fused features through dual-path fusion and depth-guided attention mechanism.

[0099] In some embodiments, cross-modal fusion features can be generated through the following steps S201-S203.

[0100] S201: Extract the multi-level RGB features from the encoding end With multi-level deep features Hadamard product operations are performed, and nonlinear unit correction operations and residual connections are combined to enhance the correlation between the two types of features, thereby preserving the basic information of the dual-modal representation of the salient target. The calculation process for this step is expressed as follows:

[0101]

[0102] In the formula, This represents the multi-level RGB features extracted by EdgeNext. This represents the multi-level deep features extracted by MobileNet. Indicates the kernel size as Depth-separable convolution operations, This represents the operation of the nonlinear correction unit (DSConv+BN+GELU). This indicates an element-wise addition operation.

[0103] S202: Extract the multi-level RGB features from the encoding end with depth features The data is concatenated along the channel dimension and compressed using a nonlinear correction unit. A simplified fusion branch is then used to further optimize feature representation, thereby obtaining complementary information from the two modalities of the salient target. The calculation process for this step is expressed as follows:

[0104]

[0105] In the formula, This represents a feature concatenation operation at the channel dimension. Indicates the kernel size as Depth-separable convolution operations, This represents the operation of the nonlinear correction unit (DSConv+BN+GELU). This represents a bimodal cascaded feature, where dim=1 indicates a transformation in the channel dimension.

[0106] S203: Acquired bimodal complementary information and retained bimodal basic information The query features and deep guidance features are respectively input into the deep guidance attention mechanism to achieve attention-weighted fusion, and key region features are highlighted as cross-modal fusion features. It is defined as follows:

[0107]

[0108] In the formula, This indicates a deep attention-guiding mechanism.

[0109] In some embodiments, the specific implementation process of the deep guided attention mechanism is shown in steps S2031-S2037 below.

[0110] S2031: Difference-separable convolution operations are used to separate the complementary information of the two modes. and basic information Mapped to the first query feature Second query features First key feature Second key features First-value features Secondary features And obtain the query feature projection results. The calculation process for this step is expressed as follows:

[0111]

[0112] In the formula, This indicates a channel-dimension splitting operation. Indicates the kernel size as The depthwise separable convolution operation is performed, with subscripts k=1,2,3.

[0113] S2032: The first key feature Second key features First-value features Secondary features Joint bond features are formed by concatenating them along the channel dimension. Joint value characteristics Simultaneously, the projection results of the query features and joint bond features Perform region-level average pooling to obtain the first region-level features. Second regional features and third-region-level features Used for efficient correlation calculation. The calculation process for this step is expressed as follows:

[0114]

[0115] In the formula, This indicates regional average pooling. Indicates the size of the region. Indicates the number of region-level feature windows. and These represent the height and width of the feature map, respectively. This indicates a query projection operation.

[0116] S2033: First-level regional query features Second region level features Flattened into a sequence, the inter-regional correlation is calculated using matrix multiplication. Furthermore, a depth similarity weighting is introduced to obtain an inter-region correlation matrix for enhancing depth consistency. The calculation process for this step is expressed as follows:

[0117]

[0118] In the formula, Represents matrix multiplication. This represents the transpose transformation. This indicates an adaptive global average pooling operation. j Indicates the index of the feature channel. Indicates the number of feature channels. Represents the reshaping of the feature tensor. This represents the outer product operation. Represents the bias constant. and These represent the maximum value function and the minimum value function, respectively. This indicates a feature dimension rearrangement operation. This indicates that the feature dimensions are flattened. This represents the learnable depth-guided weights, with an initial value of 0.5. Indicates the reshaping features of basic information. Indicates the metric characteristics.

[0119] S2034: Inter-regional correlation matrix Perform a Top K filter to select the K key regions with the highest relevance for each query region, thereby generating a sparse region graph index. This is used for key-value aggregation in subsequent dual sparse attention. The computation process for this step is expressed as follows:

[0120]

[0121] In the formula, This indicates a Top K filtering operation, performed before selecting the key region dimension. Find the maximum value and return the value. and the corresponding index , This indicates inserting a single dimension into the attention head dimension. This indicates a dimensional expansion operation. This represents the region map index, used to guide the sparse clustering of subsequent key-value features.

[0122] S2035: Projection results based on RGB query features For query features, composite key features Joint value characteristics As key values, combined with the region map index Enhanced features are generated through a dual sparse attention mechanism. and Token-level attention matrix The calculation process for this step is expressed as follows:

[0123]

[0124] In the formula, Indicates the number of feature channels. Indicates the size of the region. This represents a dual sparse attention function, which uniformly represents the process of calculating region-level sparse key-value aggregation and token-level sparse attention. This represents the attention scaling factor, used to stabilize the numerical range of attention calculations.

[0125] S2036: Computational Auxiliary Scale of Bis-Sparse Attention Enhancement Features and Token-level attention matrix With principal scale enhancement features After concatenation, the data is fused using depthwise separable convolution operations to enhance multi-scale feature representation and obtain multi-scale features. The calculation process for this step is expressed as follows:

[0126]

[0127] In the formula, , and The height, width, and number of channels of the feature map are represented respectively. Indicates the number of region-level feature windows. Indicates the size of the region. This represents a double sparse attention function. Indicates the kernel size as Depth-separable convolution operations, This represents a feature concatenation operation at the channel dimension. This represents the attention scaling factor, used to stabilize the numerical range of attention calculations; This indicates a double sparse attention enhancement feature. This represents the token-level attention matrix.

[0128] S2037: Extracts location enhancement features and multi-scale features through nonlinear correction unit operations. The addition process supplements the positional information, and finally, a depthwise separable convolution operation projects the number of channels back to the original dimension to obtain the final output, which is the cross-modal fusion feature. The calculation process for this step is expressed as follows:

[0129]

[0130] In the formula, This represents the operation of the nonlinear correction unit (DSConv+BN+GELU). Indicates the kernel size as Depth-separable convolution operations, This indicates an element-wise addition operation.

[0131] S30: Input the cross-modal fusion features into the adaptive edge optimization module, and generate high-dimensional enhanced features through dynamic gating mechanism and edge enhancement processing.

[0132] In some embodiments, the adaptive edge optimization module generates high-dimensional enhanced features through the following steps S301-S305.

[0133] S301: Cross-modal fusion features for input Calculate its corresponding dynamic gating value To assess the importance of input features and based on dynamic gating values The mean of the edge enhancement is used to determine whether subsequent edge enhancement procedures need to be performed. The calculation process for this step is expressed as follows:

[0134]

[0135] In the formula, This indicates an adaptive global average pooling operation. and This represents the convolution kernel matrix under depthwise separable convolution operations. Represents the ReLU function. This represents the Sigmoid function. The calculated dynamic gating value is used to evaluate the importance of input features, and its shape is... , This indicates the calculation of the mean. This represents the average value of the dynamic gating value; This represents the adaptive mean feature.

[0136] S302: When At that time, the edge feature map of the input features is extracted through the learnable edge detection module. Then, a lightweight Hough transform is used to extract line features to obtain a line feature map. ;when At this point, no further processing is needed; proceed directly to step S303. This involves calculating the edge feature map. Line feature map The formula is as follows:

[0137]

[0138] In the formula, Indicates the angle Endogenous generation The angle vector is composed of uniformly distributed angle values. , , and These represent the batch size, the width, and the height of the feature tensor, respectively. Indicates the edge detection threshold. This indicates that the nonlinear correction unit (DSConv + BN + GELU) is used twice consecutively. This represents the Sigmoid function. This represents the Hadamard product operation. Indicates the number of angles. Indicates the number of polar diameters. This indicates the index corresponding to the polar radius. This indicates tensor limiting operations. This represents the tensor dimension expansion operation. This represents the (DSConv+BN+ReLU) operation. , and This represents the convolution kernel matrix.

[0139] S303: Transfer edge feature map With line feature map The features are then fused together, and then compressed and transformed using a nonlinear correction unit to obtain a dimension-matched feature map. This prepares the ground for subsequent Mamba modeling. The calculation process for this step is represented as follows:

[0140]

[0141] In the formula, This represents element-wise addition. This represents the operation of the nonlinear correction unit (DSConv + BN + GELU).

[0142] S304: Feature map after dimension matching The feature maps are reconstructed from a mesh structure to a sequence structure, sequence modeling is performed using the Mamba module to capture long-range dependencies, and then reconstructed back to a mesh structure to obtain the Mamba-enhanced feature maps. The calculation process for this step is expressed as follows:

[0143]

[0144] In the formula, , , and These represent batch size, number of channels, height, and width, respectively. This indicates a dimension reshaping operation. This represents the Mamba state dimension, used to capture long-distance dependencies. Represents the dimensional expansion coefficient. Represents linear layer operations. This represents a one-dimensional convolution operation. Represents the SiLU function. This represents a dimension splitting operation, which decomposes the output of a linear layer into time step parameters. State parameters and , This refers to the Softplus function. This represents the Hadamard product operation. Represents the convolution kernel matrix, Represents linear features in dimension. Represents the linear characteristics of the state. Indicates the corrected linear characteristic. Represents a one-dimensional nonlinear characteristic. This represents the time step index feature.

[0145] S305: The Mamba-enhanced feature map is restored to the number of input channels through depthwise separable convolution, and edge weights are generated. Combined with the gating value, the original input features are enhanced in detail to obtain high-dimensional enhanced features. The calculation process for this step is expressed as follows:

[0146]

[0147] In the formula, Indicates batch normalization, This represents the Hadamard product operation. This represents the Sigmoid function. Indicates the kernel size as Depth-separable convolution operations, Indicates channel transformation characteristics. This represents the dynamic gating value.

[0148] S40: Input the high-dimensional enhanced features into the hierarchical feature fusion module, generate global optimized features through cross-scale fusion and channel attention mechanism, and generate a saliency prediction map based on the global optimized features.

[0149] After generating the globally optimized features, the core step of this method lies in decoding them into a final saliency prediction map. This prediction map clearly identifies salient regions in the input image. To achieve this, this embodiment uses a lightweight prediction head module. Specifically, the globally optimized features are input into a depth-separable convolutional layer with a kernel size of 3×3 to integrate feature information and map it to a single-channel space. Subsequently, bilinear interpolation upsampling is applied to the convolution output to restore its spatial dimensions to the resolution of the original input image. Finally, the value of each pixel is normalized using the sigmoid function to generate a saliency prediction map ranging from [0, 1]. The value of each pixel in this prediction map represents the probability that it belongs to a salient target.

[0150] In some embodiments, the hierarchical feature fusion module generates globally optimized features through the following steps S401-S405.

[0151] S401: To improve the spatial resolution of high-dimensional features to the same scale as low-dimensional feature maps, a bilinear interpolation algorithm is used to enhance the input high-dimensional features. Perform an upsampling operation to obtain high-dimensional features with the same size as the target. This prepares the groundwork for the subsequent concatenation of high- and low-dimensional feature maps along the channel dimension. The calculation process for this step is represented as follows:

[0152]

[0153] In the formula, This represents bilinear interpolation. Represents the high-dimensional features of the input. This represents the high-dimensional features after upsampling. This indicates that the upsampling ratio is 2.

[0154] S402: By using depthwise separable convolution and unit convolution operations, the upsampled high-dimensional features are... The number of channels was adjusted to match the low-dimensional features. Consistency, resulting in high-dimensional features after dimension matching. This achieves high-dimensional feature dimension matching and compatibility. The calculation process for this step is expressed as follows:

[0155]

[0156] In the formula, This represents the operation of the nonlinear correction unit (DSConv + BN + GELU). This indicates that feature channel dimensions are transformed through convolution operations, ensuring high-dimensional features with dimension matching. Channel dimension and low-dimensional features The channel dimensions are consistent. This represents the convolution kernel matrix.

[0157] S403: High-dimensional features after dimension matching With low-dimensional features Channel-dimensional splicing and fusion are performed, followed by feature fusion and transformation using a nonlinear correction unit to extract cross-scale complementary information. The calculation process for this step is expressed as follows:

[0158]

[0159] In the formula, This indicates feature concatenation along the channel dimension. This represents the operation of the nonlinear correction unit (DSConv + BN + GELU).

[0160] S404: Using channel attention mechanism to fuse complementary information Weights are assigned to highlight important channel information, and then residual connections are added to low-dimensional features to preserve the original features and enhance the fusion effect. The calculation process for this step is as follows:

[0161]

[0162] In the formula, Indicates the kernel size as Depth-separable convolution operations, This indicates an adaptive global average pooling operation. Represents the ReLU function. This represents the Sigmoid function. The plus sign indicates the Hadamard product operation, and the plus sign indicates element-wise addition. Represents adaptive average complementary features. This indicates a compressible complementary feature.

[0163] S405: Achieves multi-level fusion features through multi-level fusion strategies, channel compression transformation, and super-resolution pixel shuffling operations. The fusion transformation is used to obtain the final globally optimized features. as follows:

[0164]

[0165] In the formula, + indicates element-wise addition. Indicates the kernel size as Depth-separable convolution operations are used for channel compression transformation. This indicates a super-resolution pixel shuffling operation.

[0166] In some embodiments, a recognition model is formed by combining a dual-backbone encoder, a cross-modal feature fusion module, an adaptive edge optimization module, and a hierarchical feature fusion module. The recognition model is trained using a multi-scale weighted combined loss function, and target recognition in low-contrast scenes is achieved based on the trained recognition model. The multi-scale weighted combined loss function includes pixel-wise binary cross-entropy loss, joint intersection-over-union loss, and edge-aware loss.

[0167] Specifically, this embodiment proposes a multi-scale weighted combined loss function, which consists of three key parts: (1) pixel-wise binary cross-entropy loss ( (1) Used to quantify the classification error of each pixel; (2) Region-based joint intersection-union loss ( (3) Edge-aware loss () is used to optimize the global structural consistency of the prediction region; These components are used to improve the accuracy of target boundaries. They work together to improve the overall prediction. Total loss function Defined as follows:

[0168] First, use pixel-wise binary cross-entropy loss. The classification loss, which measures the difference between the original output predicted map and the ground truth (GT), is calculated using the following formula:

[0169]

[0170] In the formula, The model represents the first time. k The original output at each scale Represents the truth graph GT. yes The Middle i The output value of each pixel. Truth graph y The Middle i The ground truth label for each pixel. N This represents the total number of pixels in a single image. This represents the Sigmoid function.

[0171] Secondly, the region-based joint intersection-union ratio loss is utilized. The joint ratio loss, which measures the intersection and joint ratio between the predicted results and their true labels, is calculated using the following formula:

[0172]

[0173] Third, provide edge-aware loss. To measure the difference between saliency prediction edges and true edges. The loss is defined as follows:

[0174]

[0175] In the formula, Indicates the first i The set of neighboring pixels centered at a given pixel. It is the edge loss weighting coefficient.

[0176] Then, the single-scale combined loss is obtained by weighting the sum of the three losses. The definition is as follows:

[0177]

[0178] Where pixel binary cross-entropy loss Combined loss and edge perception loss The weighting coefficients are set to , and .

[0179] Finally, the total loss function It is a loss at three scales The sum of losses. The formula is as follows:

[0180]

[0181] In the formula, the first single-scale combined loss L 1. Second Single-Scale Combined Loss L 2 and the third single-scale combined loss L The weighting factor of 3 is set to , and .

[0182] In this embodiment, 800, 700, and 1485 image pairs were selected from the DUT, NLPR, and NJU2K datasets with additional annotations, respectively, as training sets for model training. The remaining images were reserved for testing. The model was evaluated on seven RGB-D datasets: NLPR, NJU2K, DUT, LFSD, SSD, SIP, and STERE. The information for each training and testing dataset is as follows: NLPR contains 1000 natural scene images with complex backgrounds and multiple objects. NJU2K contains 1985 image pairs collected from indoor, outdoor scenes, and stereoscopic movies. DUT contains 1200 images where objects are difficult to segment in different indoor and outdoor environments. LFSD consists of 100 images with highly complex foreground and background structures. SSD includes 80 images selected from three separate stereoscopic movies. SIP includes 929 image pairs, each containing one or more people in an outdoor scene. STERE contains 797 stereoscopic images downloaded from online resources.

[0183] Table 1. Comparison of quantitative results between the proposed method and state-of-the-art methods in three public benchmark databases.

[0184]

[0185] Table 2 compares the quantitative results of the proposed method with state-of-the-art methods in four public benchmark databases.

[0186]

[0187] It should be noted that the lightweight and heavyweight methods mentioned in Tables 1 and 2 are distinguished by the number of parameters. When M is reached, it is considered a heavyweight method; when the number of parameters is... When M is selected, it is considered a lightweight method. Specifically, the evaluation metrics... E ϕ This indicates an enhanced alignment metric. S αRepresents a measure of structural similarity. F β It represents the overall similarity measure, and MAE represents the mean absolute error measure.

[0188] Tables 1 and 2 provide a detailed comparison of the proposed method with 11 existing state-of-the-art methods, covering seven benchmark RGB-D datasets. Four widely used metrics are employed ( F β Measurement, Mean Absolute Error (MAE) E ϕ Measurement and S α The proposed method is evaluated using metrics. Experimental results show that, in most cases, it consistently outperforms competing techniques, demonstrating its robustness and effectiveness across various scenarios. In particular, the proposed method achieves significant improvements over DFMNet on multiple datasets. F β It achieved a considerable gain in the MAE metric. Specifically, on seven RGB-D benchmark databases, it will... E ϕ ( F β The proposed methods improved the performance metrics by 2.1% (1.8%), 1.6% (0.6%), -1.5% (-0.2%), 1.3% (1.1%), 1.7% (2.2%), 9.0% (16.0%), and -0.2% respectively, while reducing the MAE by 24.0%, 31.4%, 2.9%, 27.8%, 23.8%, 93.5%, and 18.2% respectively. Similarly, compared to the classic MobileSal, the proposed method achieves substantial performance enhancements on seven standard datasets. S α ( F β The accuracy and robustness of the proposed method improved by 4.7% (4.2%), 5.2% (6.8%), 1.7% (2.7%), 1.5% (2.9%), 3.3% (4.7%), 2.0% (0.6%), and 1.3% (2.2%), respectively, while the corresponding MAE decreased by 80.0%, 54.9%, 29.4%, 38.9%, 9.5%, 8.7%, and 27.3%. This highlights its improved accuracy and robustness in low-error prediction. Similarly, the proposed method outperforms MAGNet, achieving consistent progress on all evaluated datasets. F β ( S αThe accuracy of the prediction metrics increased by 3.8% (3.2%), 2.8% (3.5%), 1.2% (1.1%), 3.6% (2.3%), 1.9% (2.1%), 0.8% (1.5%), and 3.2% (3.0%), respectively, while the MAE decreased by 60.0%, 35.3%, 14.7%, 44.4%, -2.4%, 10.9%, and 42.4%, respectively. These results highlight the method's ability to achieve higher accuracy while reducing prediction error, even compared to mature technologies. Compared to the latest HENet, the method proposed in this application also has a good competitive advantage. E ϕ ( F β The performance metrics improved by 0.7% (1.7%), 1.2% (3.1%), -1.6% (0.8%), 0.3% (0.8%), 0.1% (same), 0.2% (0.1%), and -1.5% (0.1%), respectively, while the corresponding MAE decreased by 20.0%, 23.5%, the same, 16.7%, 9.5%, 9.5%, and 5.9%, respectively. Furthermore, the method proposed in this application outperforms the heavyweight CoNet, achieving consistent progress across all evaluated datasets. E ϕ ( S α The metrics increased by 1.5% (2.0%), 2.2% (3.4%), 1.6% (2.3%), 3.3% (2.8%), 2.6% (4.3%), 1.2% (1.9%), and 0.3% (0.9%), respectively, while MAE decreased by 32.0%, 39.2%, 35.3%, 72.2%, 50.0%, 28.3%, and 12.1%, respectively. Although no single method performs exceptionally well across all metrics, the method proposed in this application consistently ranks among the best and often outperforms some heavyweight and outstanding methods such as A2dele, CoNet, and CIRNet.

[0189] Table 3. Comparison of average quantitative results between the proposed method and existing methods in seven benchmark databases.

[0190]

[0191] It should be noted that the average mIOU in Table 3 represents the average intersection-union ratio (IU) across the seven datasets. F β The mean overall similarity measure is represented across the seven datasets, while the mean MAE measures the mean absolute error across the seven datasets.

[0192] To further verify the detection capability of the method proposed in this application, the task performance was systematically evaluated using three key metrics, such as average... F β The average mIOU and average MAE were calculated, and detailed results are summarized in Table 3. Comparative analysis shows that the method proposed in this application is superior to the state-of-the-art method HENet (MoADNet) in all three metrics, and in average... F β The relative improvement rate on the metrics was 0.8% (1.4%), the improvement rate on the average MAE metric was 11.4% (31.4%), and the improvement rate on the average mIOU metric was 0.6% (3.7%). In addition to surpassing the methods mentioned above, the method proposed in this application also compared with several classic SOD methods, all of which showed good performance, including DFMNet, MobileSal, and LSNet. Specifically, for the average mIOU (average... F β In terms of metrics, the proposed method achieves improvements of 5.0% (2.9%), 7.5% (3.7%), and 8.6% (1.8%) on two metrics, respectively, compared to these classical methods. In summary, the proposed method demonstrates state-of-the-art performance on key detection metrics for salient object detection. While it did not reach the theoretical maximum on every metric across all datasets, its performance consistently approaches these best-in-class levels, a result that fully demonstrates the effectiveness and reliability of the method. In addition to the quantitative evaluation of the proposed method, this embodiment further validates its effectiveness by analyzing its visual detection results and PR curves. Figures 3 to 9 As can be clearly seen, the method proposed in this application exhibits strong visual detection performance, sufficient to meet the requirements of most visual tasks that rely on saliency detection. Clearly, in most cases, the proposed method outperforms other existing lightweight methods. In particular, it shows significant performance compared to existing lightweight saliency object detection methods (such as DFMNet, LSNet, and HENet). Furthermore, in some cases, the proposed method even achieves performance comparable to heavyweight saliency detection methods, which is an impressive result considering its lightweight design.

[0193] To qualitatively demonstrate the visual results of the saliency map, this embodiment also compares the saliency maps generated by the proposed method with those generated by other existing methods using various challenging scenarios. These scenarios include large targets, multiple targets, simple scenes, complex scenes, and low-contrast scenes, effectively reflecting the performance of the method under different challenging conditions. Figure 10As shown, the proposed method has significant advantages in accurately identifying, locating, and segmenting the details and boundaries of salient targets. This allows its saliency predictions to more closely align with the ground truth (GT) in various scenarios. Specifically, rows 4 and 5 demonstrate that the proposed method achieves excellent performance in segmenting large targets in simple scenes. As shown in row 4, the image contains a large human statue, and most existing methods struggle to accurately detect, locate, and segment the details of the statue's legs and clothing edges. In contrast, the proposed method clearly identifies these fine target details, reducing the missegmentation rate. Similarly, the image in row 5 shows a red wooden sign in front of a tree. The proposed method excels at more clearly segmenting the hollow structure of the wooden sign, highlighting its competitiveness in resolving the detailed features of large targets. Furthermore, rows 2, 3, and 6 demonstrate the powerful ability of the proposed method to identify and predict single and multiple targets in simple scenes. For example, the image in row 2 shows a tripod on a table. Compared to other methods, the method proposed in this application can more accurately identify and segment detailed information about the legs of the tripod, and its saliency prediction results are highly consistent with the ground truth (GT). Similarly, the image in row 6 shows three swords placed on a table. The method proposed in this application can more accurately locate and identify detailed information about the blade and hilt of each sword, and the resulting saliency map is highly consistent with the GT. Furthermore, the method also performs well in identifying salient objects in complex scenes (rows 8 and 10). For example, there are four Minions playing on a green lawn in row 8. The method proposed in this application outperforms other existing methods by more clearly locating and predicting the overall shape and pose of each Minion, and avoids the false negatives or missed detections that plague other methods. Regarding the identification and prediction of single and multiple targets in low-contrast scenes (rows 1, 7, 9, and 11), the method proposed in this application is able to describe the complex details and shapes of flower branches (row 1) with greater granularity without any false positives. For example, in row 9, characterized by potted flowers placed on a green ground, the method proposed in this application can accurately identify and predict flower stem components with high similarity, with the target location closely aligned with the ground plane (GT). Similarly, in the case of a white bicycle parked on a white ground, the method proposed in this application depicts the bicycle's rim and details more clearly than other methods. Overall, the above qualitative and quantitative comparisons sufficiently verify the effectiveness and reliability of the method proposed in this application.

[0194] To gain a more comprehensive understanding of the performance of the proposed method, its efficiency and flexibility were evaluated by comparing it with 11 existing methods. As shown in Table 1, architectural metrics, including the number of parameters and FLOPs, are provided for the proposed method and existing methods. It should be noted that all evaluations were conducted on an RTX 4070Ti, and the proposed method has 7.4M parameters and 4.5G FLOPs. Table 1 shows that the proposed method differs significantly from the comparative methods. Specifically, the proposed method differs significantly from the previously mentioned comparable methods in terms of the number of parameters (FLOPs). Compared to heavyweight SOD methods such as A2dele, CoNet, CIRNet, HINet, and AISP, the method proposed in this application exhibits relative changes of -51.0% (-88.1%), -83.1% (-80.8%), -92.8% (-89.4%), -95.5% (-72.4%), and -92.9% (-94.0%), respectively. This is further illustrated when the method proposed in this application is compared to lightweight methods such as DFMNet, MoADNet, MobileSal, LSNet, MAGNet, HENet, and BTNet. The number of parameters (FLOPs) of the proposed method varied by -12.9% (+80.0%), +48.0% (+246.1%), +13.8% (-39.2%), +60.9% (+275.0%), +42.3% (+80.00%), +174.1% (+104.5%), and +94.7% (-37.5%), respectively. In summary, the proposed method has proven to be a feasible and effective saliency detection method. While not the most lightweight option in terms of all three key metrics, its performance is still very strong; the trade-off between architectural complexity and detection capability is considered reasonable.

[0195] This application also provides a target recognition system for low-contrast scenes, such as... Figure 11 As shown, the target recognition system for low-contrast scenes includes:

[0196] The feature extraction unit 1101 is configured to use a dual-backbone encoder to extract multi-level RGB features of the RGB image and multi-level depth features of the depth image, respectively; wherein the RGB image and the depth image are both images acquired in low-contrast scenes;

[0197] The cross-modal fusion unit 1102 is configured to input the multi-level RGB features and multi-level depth features into the cross-modal feature fusion module, and generate cross-modal fused features through dual-path fusion and depth-guided attention mechanism;

[0198] The edge optimization unit 1103 is configured to input the cross-modal fusion features into the adaptive edge optimization module and generate high-dimensional enhanced features through dynamic gating mechanism and edge enhancement processing.

[0199] The hierarchical fusion unit 1104 is configured to input the high-dimensional enhanced features into the hierarchical feature fusion module, generate global optimized features through cross-scale fusion and channel attention mechanisms, and generate a saliency prediction map based on the global optimized features.

[0200] This application provides an electronic device. The electronic device may include a processor and a memory, wherein the processor and the memory can communicate; exemplarily, the processor and the memory communicate via a communication bus.

[0201] The processor executes computer execution instructions stored in memory, causing the processor to perform the scheme in the above embodiments. The processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0202] The communication bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus can be divided into address bus, data bus, control bus, etc. Transceivers are used to enable communication between the database access system and other computers (e.g., clients, read-write libraries, and read-only libraries). Memory may include random access memory (RAM) and may also include non-volatile memory.

[0203] The electronic device provided in this application embodiment can be the terminal device described in the above embodiments.

[0204] This application also provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed on a computer, the computer performs the technical solution of the target recognition method in low-contrast scenes described in the above embodiments.

[0205] This application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When the at least one processor executes the computer program, it can implement the technical solution of the target recognition method in the low-contrast scene described in the above embodiments.

[0206] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between systems or modules may be electrical, mechanical, or other forms.

[0207] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to implement the solution of this embodiment according to actual needs.

[0208] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The unit composed of the above modules can be implemented in hardware or in the form of hardware plus software functional units.

[0209] The integrated modules described above, implemented as software functional modules, can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods of the various embodiments of this application.

[0210] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor.

[0211] The memory may include high-speed RAM, and may also include non-volatile storage (NVM), such as at least one disk storage device, and may also be a USB flash drive, external hard drive, read-only memory, disk or optical disc, etc.

[0212] Buses can be Industry Standard Architecture (ISA) buses, Peripheral Component Interconnect (PCI) buses, or Extended Industry Standard Architecture (EISA) buses, etc. Buses can be categorized into address buses, data buses, control buses, etc.

[0213] The aforementioned storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0214] An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor. The processor and storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and storage medium can exist as discrete components in an electronic control unit or main control device.

[0215] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0216] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A target recognition method in low-contrast scenes, characterized in that, The method includes: A dual-backbone encoder is used to extract multi-level RGB features from the RGB image and multi-level depth features from the depth image, respectively; wherein, both the RGB image and the depth image are images acquired in low-contrast scenes; The multi-level RGB features and multi-level depth features are input into the cross-modal feature fusion module, and cross-modal fused features are generated through dual-path fusion and depth-guided attention mechanism; The cross-modal fusion features are input into the adaptive edge optimization module, and high-dimensional enhanced features are generated through dynamic gating mechanism and edge enhancement processing; The high-dimensional enhanced features are input into the hierarchical feature fusion module, and global optimized features are generated through cross-scale fusion and channel attention mechanism. A saliency prediction map is generated based on the global optimized features. The multi-level RGB features and multi-level deep features are input into the cross-modal feature fusion module, and cross-modal fused features are generated through dual-path fusion and depth-guided attention mechanisms, including: Based on multi-level RGB features With multi-level deep features The basic information of the saliency target bimodality is calculated using the following formula. : In the formula, Indicates the kernel size as Depth-separable convolution operations, This represents the nonlinear correction unit operation, which includes depthwise separable convolution (DSConv), batch normalization (BN), and Gaussian error linearization unit (GELU). This represents element-wise addition. Based on multi-level RGB features With multi-level deep features The complementary information of the two modes of saliency target can be calculated using the following formula. : In the formula, This represents a feature concatenation operation at the channel dimension. Indicates the kernel size as Depth-separable convolution operations, This represents a bimodal cascaded feature, where dim=1 indicates a transformation in the channel dimension; Basic information of the dual-modal saliency target and complementary information The query feature and the deep guidance feature are respectively input into the deep guided attention mechanism to calculate the cross-modal fusion feature. ; Basic information of the dual-modal saliency target and complementary information The query feature and the deep guidance feature are respectively input into the deep guided attention mechanism to calculate the cross-modal fusion feature. ,include: The complementary information of the two modes is extracted using depthwise separable convolution operations. and basic information Mapped to the first query feature Second query features First key feature Second key features First-value features Secondary features And obtain the query feature projection results. ; The first key feature Second key features First-value features Secondary features Joint bond features are formed by concatenating them along the channel dimension. Joint value characteristics Simultaneously, the projection results of the query features and joint bond features Perform region-level average pooling to obtain the first region-level features. Second regional features and third-region-level features ; First-level query features Second region level features Flattened into a sequence, the inter-regional correlation is calculated using matrix multiplication. Furthermore, a depth similarity weighting is introduced to obtain an inter-region correlation matrix for enhancing depth consistency. ; Inter-regional correlation matrix Perform a Top K filter to select the K key regions with the highest relevance for each query region, and generate a sparse region graph index. ; Query feature projection results For query features, composite key features Joint value characteristics As key values, combined with the region map index Enhanced features are generated through a dual sparse attention mechanism. and Token-level attention matrix ; Computing auxiliary-scale dual sparse attention enhancement features and Token-level attention matrix With principal scale enhancement features After concatenation, the features are fused using depthwise separable convolution operations to obtain multi-scale features. ; Location enhancement features are extracted through nonlinear correction unit operations, along with multi-scale features. The channels are added together, and then projected back to the original dimensions using depthwise separable convolution operations to obtain cross-modal fusion features. ; The cross-modal fusion features are input into the adaptive edge optimization module, which generates high-dimensional enhanced features through dynamic gating and edge enhancement processing, including: Calculate cross-modal fusion features Dynamic gating value Its mean, the calculation process is expressed as follows: In the formula, This indicates an adaptive global average pooling operation. and Represents depthwise separable convolution operation DSConv The convolution kernel matrix below, Represents the ReLU function. This represents the Sigmoid function. Indicates the dynamic gating value. This indicates the calculation of the mean. This represents the average value of the dynamic gating value. This represents adaptive mean features; when At that time, the edge feature map of the input features is extracted by the learnable edge detection module, and then the line features are extracted by the lightweight Hough transform to obtain the line feature map; edge feature map With line feature map The features are then fused together, and then compressed and transformed using a nonlinear correction unit to obtain a dimension-matched feature map. ; Feature maps after dimension matching The feature maps are reconstructed from a mesh structure to a sequence structure, sequence modeling is performed using the Mamba module to capture long-range dependencies, and then reconstructed back to a mesh structure to obtain the Mamba-enhanced feature maps. ; Feature maps enhanced based on Mamba The high-dimensional enhanced features are calculated using the following formula. : In the formula, Indicates batch normalization, This represents the Hadamard product operation. This represents the Sigmoid function. Indicates the kernel size as Depth-separable convolution operations, Representing multi-scale features, Indicates channel transformation characteristics. This represents the dynamic gating value.

2. The target recognition method in low-contrast scenes according to claim 1, characterized in that, The high-dimensional enhanced features are input into the hierarchical feature fusion module, which generates globally optimized features through cross-scale fusion and channel attention mechanisms, including: Enhanced features in high dimensions Perform an upsampling operation to obtain high-dimensional features with the same size as the target. ; By using depthwise separable convolution and unit convolution operations, the upsampled high-dimensional features are... The number of channels was adjusted to match the low-dimensional features. Consistency, resulting in high-dimensional features after dimension matching. ; High-dimensional features after dimension matching With low-dimensional features Channel-dimensional splicing and fusion are performed, followed by feature fusion and transformation using a nonlinear correction unit to extract cross-scale complementary information. ; Channel attention mechanism for fusing complementary information Weights are assigned, and the residual connections are combined with low-dimensional features to retain the original features, resulting in multi-level fused features. ; Multi-level fusion features are achieved through multi-level fusion strategies, channel compression transformation, and super-resolution pixel shuffling operations. The fusion transformation yields the globally optimized features. .

3. The target recognition method in low-contrast scenes according to any one of claims 1 to 2, characterized in that, After generating a saliency prediction map based on the global optimization features, the method includes: The dual-backbone encoder, cross-modal feature fusion module, adaptive edge optimization module, and hierarchical feature fusion module are combined to form a recognition model. The recognition model is trained using a multi-scale weighted combination loss function, and target recognition in low-contrast scenes is achieved based on the trained recognition model. The multi-scale weighted combination loss function includes pixel-wise binary cross-entropy loss, joint intersection-union ratio loss, and edge-aware loss.

4. The target recognition method in low-contrast scenes according to claim 3, characterized in that, The multi-scale weighted combination loss function is expressed as follows: In the formula, This represents the single-scale combined loss. , and These represent pixel binary cross-entropy loss respectively. Combined loss and edge perception loss The weighting coefficients, v st1 , v st2 and v st3 These represent the first single-scale combined loss, respectively. L 1. Second Single-Scale Combined Loss L 2 and the third single-scale combined loss L The weighting factor is 3.

5. A target recognition system for low-contrast scenes, used to implement the method as described in any one of claims 1 to 4, characterized in that, The system includes: The feature extraction unit is configured to extract multi-level RGB features of the RGB image and multi-level depth features of the depth image using a dual-backbone encoder; wherein the RGB image and the depth image are both images acquired in low-contrast scenes; The cross-modal fusion unit is configured to input the multi-level RGB features and multi-level depth features into the cross-modal feature fusion module, and generate cross-modal fused features through dual-path fusion and depth-guided attention mechanism; The edge optimization unit is configured to input the cross-modal fusion features into the adaptive edge optimization module and generate high-dimensional enhanced features through dynamic gating mechanism and edge enhancement processing. The hierarchical fusion unit is configured to input the high-dimensional enhanced features into the hierarchical feature fusion module, generate globally optimized features through cross-scale fusion and channel attention mechanisms, and generate a saliency prediction map based on the globally optimized features.

6. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the target recognition method in low-contrast scenes as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the target recognition method in a low-contrast scene as described in any one of claims 1-4.

Images