Similarity processing method and device based on fusion attention, equipment and medium

Abstract

The invention relates to the technical field of image detection, can be applied to business scenes such as financial science and technology and medical health, and discloses a similarity processing method and device based on fusion attention, equipment and a medium. Generating a deep attention mask and a semantic attention mask, fusing the mask to obtain a fused attention map, extracting target image features, carrying out weighting to generate a weighted feature map, converting the weighted feature map into a region feature descriptor, calculating a region weight, obtaining background similarity according to the similarity between the region feature descriptor and the corresponding features of the reference image and the region weight, and carrying out image segmentation on the background similarity. And executing the target task based on the background similarity to obtain a target task result. According to the method, through depth and semantic joint attention modeling and region-level weighted similarity calculation, the foreground and the background are accurately distinguished, and the stability and reliability of background similarity measurement and subsequent task execution are improved.

Description

Similarity processing methods, devices, equipment, and media based on fusion attention Technical Field

[0001] This invention relates to the field of image detection technology, and in particular to a similarity processing method, apparatus, device, and medium based on fusion attention. Background Technology

[0002] In the fintech sector, image similarity calculation is widely used in scenarios such as contract image review, remote identity verification, and business data consistency checks. However, current technologies generally rely on single-modal attention allocation methods, typically generating regions of interest based solely on semantic segmentation or depth estimation results, lacking the collaborative utilization of multimodal structural information. When there is complex lighting, occluded objects, or people interfering in the scene, existing methods struggle to simultaneously understand the geometric structure and semantic content of the image, resulting in blurred boundaries between the target background region and the foreground region in feature representation.

[0003] In the healthcare field, many medical auxiliary diagnostic systems and ward monitoring scenarios rely on image comparison or background similarity judgment, such as operating room environment monitoring, patient behavior detection, and medical equipment placement recognition. However, current image representation methods generally face the problem of feature confusion caused by foreground interference, especially in medical scenarios where personnel movement is frequent and instruments partially obscure the background. Traditional feature extraction and comparison methods often encode foreground and background in a mixed manner, making it difficult for the system to accurately determine scene consistency.

[0004] Existing technologies commonly employ fixed-ratio or static strategies to fuse multiple features such as depth, semantics, and texture. However, the stability of background structures, the degree of foreground occlusion, and semantic complexity vary significantly across different business scenarios. Static fusion methods struggle to adapt to real-time changing scene characteristics, preventing attention from dynamically adjusting according to actual occlusion and further impacting the accuracy of background region recognition. Furthermore, in region-level image similarity comparison tasks, traditional methods are often affected by the high response of foreground regions, and the importance of region features cannot be dynamically adjusted based on background credibility, leading to unstable background comparison results. Summary of the Invention

[0005] The main objective of this invention is to provide a similarity processing method, apparatus, device, and storage medium based on fusion attention, aiming to solve the technical problem that existing technologies cannot simultaneously utilize deep structural information and semantic context information to adaptively weight the background region in indoor scenes with significant foreground interference and high semantic complexity, resulting in inaccurate background similarity judgment.

[0006] To achieve the above objectives, this invention provides a similarity processing method based on fused attention, comprising: acquiring a target image and a reference image to be processed; performing depth estimation processing on the target image to generate a depth map, and generating a depth attention mask based on the depth map; performing semantic segmentation processing on the target image to generate a semantic probability map, and generating a semantic attention mask based on the semantic probability map; fusing the depth attention mask and the semantic attention mask to generate a fused attention map; extracting feature maps from the target image, and using the fused attention map to perform weighted processing on the feature maps to generate a weighted feature map; converting the weighted feature map into multiple region feature descriptors, and determining multiple region weights corresponding to the multiple regions according to the fused attention map; determining the background similarity between the target image and the reference image based on the similarity relationship between the multiple region feature descriptors and the corresponding region feature descriptors of the reference image and the region weights; and performing a target task based on the background similarity to obtain the target task result.

[0007] Furthermore, to achieve the above objectives, the present invention provides a similarity processing device based on fused attention, comprising: an image input module for acquiring a target image to be processed and a reference image; a depth attention generation module for performing depth estimation processing on the target image to generate a depth map, and generating a depth attention mask based on the depth map; a semantic attention generation module for performing semantic segmentation processing on the target image to generate a semantic probability map, and generating a semantic attention mask based on the semantic probability map; an attention fusion module for fusing the depth attention mask and the semantic attention mask to generate a fused attention map; and a feature weighted encoding module. The system is used to extract feature maps from the target image and weight the feature maps using the fused attention map to generate a weighted feature map; a region feature modeling module is used to convert the weighted feature map into multiple region feature descriptors and determine multiple region weights corresponding to the multiple regions based on the fused attention map; a similarity calculation module is used to determine the background similarity between the target image and the reference image based on the similarity relationship between the multiple region feature descriptors and the corresponding region feature descriptors of the reference image and the region weights; and a task decision module is used to execute the target task based on the background similarity to obtain the target task result.

[0008] Furthermore, to achieve the above objectives, the present invention also provides a computer device, the computer device including a memory, a processor, and a similarity processing program based on fusion attention stored in the memory and executable on the processor, wherein when the similarity processing program based on fusion attention is executed by the processor, it implements the steps of the similarity processing method based on fusion attention as described above.

[0009] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing a similarity processing program based on fusion attention, wherein when the similarity processing program based on fusion attention is executed by a processor, it implements the steps of the similarity processing method based on fusion attention as described above.

[0010] Beneficial Effects: This invention relates to the field of image detection technology and can be applied to business scenarios such as fintech and healthcare. It discloses a similarity processing method, apparatus, device, and medium based on fused attention, comprising: acquiring a target image and a reference image; performing depth estimation on the target image to generate a depth attention mask; performing semantic segmentation to generate a semantic attention mask; fusing the two attention masks to generate a fused attention map; extracting feature maps from the target image and using the fused attention map for weighted generation to generate a weighted feature map; converting the weighted feature map into region feature descriptors and determining the corresponding region weights; calculating background similarity based on the similarity between the region feature descriptors and corresponding features of the reference image and the region weights; and performing the target task based on the background similarity to obtain the target task result. This invention accurately distinguishes foreground and background through joint attention modeling of depth and semantics and region-level weighted similarity calculation, improving the stability and reliability of background similarity measurement and subsequent task execution. Attached Figure Description

[0011] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the drawings: Figure 1 is a schematic diagram of an application environment of the similarity processing method based on fusion attention in an embodiment of the present invention; Figure 2 is a schematic flowchart of an embodiment of the similarity processing method based on fusion attention in the present invention; Figure 3 is a schematic diagram of the functional modules of a preferred embodiment of the similarity processing device based on fusion attention in the present invention; Figure 4 is a schematic diagram of the structure of a computer device in an embodiment of the present invention; Figure 5 is another schematic diagram of the structure of a computer device in an embodiment of the present invention. Detailed Implementation

[0012] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0013] The similarity processing method based on fusion attention provided in this invention can be applied in the application environment shown in Figure 1, where the client communicates with the server via a network. The server can acquire the target image and a reference image from the client, perform depth estimation on the target image to generate a depth attention mask, perform semantic segmentation to generate a semantic attention mask, fuse the two attention masks to generate a fusion attention map, extract the feature map of the target image, and use the fusion attention map to generate a weighted feature map. The weighted feature map is converted into a region feature descriptor and the corresponding region weight is determined. Background similarity is calculated based on the similarity between the region feature descriptor and the corresponding features of the reference image, as well as the region weights. The target task is then executed based on the background similarity to obtain the target task result. This invention accurately distinguishes between foreground and background through joint attention modeling of depth and semantics and region-level weighted similarity calculation, improving the stability and reliability of background similarity measurement and subsequent task execution. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The invention will be described in detail below through specific embodiments.

[0014] Please refer to Figure 2, which is a flowchart illustrating an embodiment of the similarity processing method based on fusion attention provided by the present invention. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0015] As shown in Figure 2, the similarity processing method based on fusion attention proposed in this invention includes the following steps: S10, acquiring the target image and reference image to be processed; In this embodiment, the process of acquiring the target image and reference image to be processed is used to construct the basic input set for subsequent calculation processes. The two types of images play different roles in the system data link. The target image is used to characterize the real-time or offline input image to be subjected to background similarity analysis, and the reference image is used to provide a comparison benchmark for the background structure. The target image can be acquired by receiving digital image data from a vision sensor through an image acquisition module, or by reading stored image files from an existing data management system. Specific sources include local file systems, network data storage, image stream input interfaces, etc. The image data may be a sequence of pixel values ​​in the form of a two-dimensional matrix, or it may be a tensor structure containing multi-channel information, such as a multi-channel combination containing depth channels, infrared channels, or structured light coding information. During the acquisition of the target image, the input data can be format-parsed and an access interface can be established so that it can be read and manipulated by subsequent processing units in a unified dimension. The acquisition of the reference image usually comes from a pre-constructed image set, which stores background environment images from different spatial locations, different scene structures, or different semantic categories. Reference images may include existing auxiliary information, such as grid division rules, corresponding region feature descriptors, or scene attribute category labels, which are used to assist in the comparison during the similarity calculation stage. When acquiring reference images, it is necessary to ensure that their resolution, number of channels, or color space representation are compatible with the processing flow of the target image. Therefore, after reading the reference image, scaling, color space alignment, or format unification can be performed.

[0016] The acquisition of target and reference images is generally accomplished through the image input module. This module may include an image sensor driver interface, an external data access interface, a communication protocol parsing component, and an input buffer structure. Before the image is formally input into the processing chain, the integrity of the image data is verified to ensure that the pixel matrix is ​​not corrupted and can be successfully used as input for subsequent steps. In practical implementations, target images often have real-time requirements; therefore, the image input module needs to have the ability to cache and synchronize high-frequency input data. Since the set of reference images is usually large, a subset of reference images associated with the target image is retrieved from the indexing system when acquiring reference images, thereby reducing the computational burden.

[0017] This embodiment uses high-quality target and reference images as input, enabling more stable depth estimation, semantic segmentation, and attention weight fusion results in subsequent processing. This improves the accuracy of region feature descriptor construction and enhances the reliability of background similarity calculation. Stable input images reduce biases caused by external factors such as foreground interference, illumination noise, and scale variations, providing a consistent data foundation for overall background similarity analysis and making the final target task results more reliable.

[0018] S20, depth estimation processing is performed on the target image to generate a depth map, and a depth attention mask is generated based on the depth map. In this embodiment, the depth estimation processing of the target image converts the pixel distribution in the image into spatial distance information, enabling the two-dimensional image to obtain an implicit three-dimensional structural representation. The target image is fed into the depth inference model as an input matrix. The model performs feature analysis on edge changes, texture continuity, and region shape, and regresses to obtain the relative depth value of each pixel, forming a depth map. The depth map represents the near and far structures of each region in the scene in a matrix structure consistent with the image, and distance differences can be recorded in floating-point form. When generating the depth attention mask based on the depth map, the depth range needs to be normalized so that different depth values ​​are mapped to a unified interval. Then, the background regions with stable geometric structures are identified by the magnitude of depth changes, depth gradient, or depth clustering, and these regions are assigned higher weights. The final depth attention mask represents the importance of background geometric features in matrix form, providing a stable spatial weight basis for subsequent processing.

[0019] Depth estimation can employ convolutional networks, Transformer structures, or lightweight depth regression models to generate depth maps through single-frame inference. The resolution of the depth map can be adjusted as needed to suit high accuracy or low power consumption requirements in different environments. Depth attention masks can be generated by obtaining the weight matrix through linear mapping or by using exponential mapping to enhance regions with significant depth differences. In noisy scenes, smoothing or filtering operations can be added to stabilize depth variations, making the mask representation more stable. In another implementation, depth intervals can be grouped through depth clustering, and stable structural regions can be assigned higher weights to enhance the overall background structural response.

[0020] This embodiment introduces depth estimation and depth attention masking, which quantifies the geometric structure of the background region in the image and transforms it into a weight matrix, so that regions with stable spatial depth have a higher influence in subsequent processing; at the same time, it reduces the interference caused by abrupt changes in the depth of foreground objects, so that the background structure can still maintain a stable representation under complex lighting or texture conditions.

[0021] S30, perform semantic segmentation on the target image to generate a semantic probability map, and generate a semantic attention mask based on the semantic probability map. In this embodiment, the semantic segmentation of the target image is performed by obtaining the spatial distribution of semantic components in the image through pixel-level category prediction. The target image is input into the semantic segmentation model. The model uses convolutional structures, attention structures, or multi-scale context aggregation structures to learn representations of image texture, edges, and local contextual relationships, and outputs a probability distribution matrix covering all categories. This probability distribution matrix records the probability of each category at each pixel position according to the channel dimension, thus forming a semantic probability map. When generating the semantic attention mask based on the semantic probability map, the probability values ​​of corresponding pixels in these channels are aggregated at the pixel level by filtering category channels related to the background, so that the background category has a unified expression in the mask. Through normalization adjustment, the weight range in the mask can be kept stable, so that the background region can still be consistently expressed under different category proportions and different scene densities. The semantic attention mask represents the importance of the background region at the semantic level in the form of a two-dimensional matrix, which is used to assist subsequent processing in giving differentiated attention to the background region.

[0022] Semantic segmentation models can employ deep networks based on encoder structures. Deep semantic features are extracted through the backbone network, and multi-scale pooling structures are used to aggregate contextual information from different receptive fields, enabling the semantic probability map to simultaneously reflect local details and global structure. The resolution of the semantic probability map can be adjusted as needed to balance computational load and semantic fine-grainedness under different application requirements. Semantic attention masks can be generated by directly accumulating background category probabilities to obtain uniform weights, or by weighted aggregation with different weight ratios for different background categories, ensuring stable responses from background regions in complex scenes. In noisy or densely populated scenes, smoothing processing can be performed on the semantic probability map to reduce the impact of random jitter on the mask.

[0023] This embodiment constructs a semantic probability map and generates a semantic attention mask, so that the background category in the image obtains a clear semantic expression at the pixel level, enabling the background region to have a semantic weight basis independent of the foreground target in subsequent processing; at the same time, it can reduce the interference of foreground semantic changes on background localization, so that the background expression can remain stable and consistent when the scene complexity increases.

[0024] S40, the depth attention mask and the semantic attention mask are fused to generate a fused attention map. In this embodiment, the fusion of the depth attention mask and the semantic attention mask is achieved by introducing a trainable and updateable weight matrix, which allows information from different sources to dynamically adjust at the pixel level. The depth attention mask records the structural differences of the image at the geometric level, while the semantic attention mask reflects the degree of semantic component affiliation of each region of the image. The numerical distributions of the two masks are not consistent, so two learnable weight matrices need to be constructed so that depth information and semantic information can be independently adjusted according to the actual image content. The depth attention mask is multiplied element-wise with its corresponding weight matrix to obtain the depth-gated component; the semantic attention mask is multiplied element-wise with its corresponding weight matrix to obtain the semantic-gated component. Then, the two gated components are added element-wise to fuse the depth structural differences and semantic category distribution into a two-dimensional matrix in a unified space. To avoid scale fluctuations caused by linear superposition, a non-linear saturating activation function is used to map the addition result, so that the fused weights maintain numerical discrimination and a stable gradient between the background and non-background regions.

[0025] During the fusion process, a weight matrix with the same size as the depth mask and semantic mask can be used, allowing for fine-tuning of the weight response at each location with pixel-level precision. The weight matrix can be initialized with a uniform distribution or with random initialization to enhance the exploratory capabilities during the initial training phase. The activation function can be chosen in different forms depending on the task requirements. For example, when dealing with scenes with high-contrast regions, a function with a narrow mapping range can be used to control the upper limit of the weights, while a smooth mapping can be used to enhance buffering capabilities when dealing with scenes with weak textures but complex semantic distributions. The fusion operation can be performed directly in the two-dimensional matrix space, or a high-resolution fusion can be performed after increasing the resolution to improve the stability of local details. In scenarios requiring further robustness enhancement, scale matching of the two masks can be performed before fusion, allowing depth and semantic information to be weighted at the same sampling density.

[0026] This embodiment introduces learnable weights to the depth mask and semantic mask respectively and performs element-wise fusion, which enables geometric structural information and semantic context information to form complementary expressions within the pixel range. This allows the fused attention map to maintain a stable weight distribution even when facing scenes with complex occlusion, uneven semantics, or significant structural changes, thereby providing a more accurate basis for subsequent region analysis.

[0027] S50, extract the feature map of the target image, and use the fused attention map to perform weighted processing on the feature map to generate a weighted feature map. In this embodiment, the processing of extracting the feature map of the target image typically uses a multi-layer convolutional structure to encode spatial texture, edge contours, and local region patterns. The feature map is composed of a multi-channel matrix, with each channel recording the local response within a different receptive field, derived from the results of the network performing convolution, nonlinear transformation, and downsampling on the target image. The fused attention map is a single-channel structure, recording only the weight distribution in the spatial dimension. Therefore, it needs to be copied and expanded in the channel dimension to ensure that its size is consistent with the feature map in terms of height, width, and number of channels. The expanded attention map and the feature map form a one-to-one coefficient relationship at each spatial location, enabling the attention map to achieve positional modulation on the feature map. The element-wise multiplication process amplifies or suppresses the feature response of each channel according to the spatial weights of the fused attention map, thereby concentrating the effective part of the features in structurally stable and semantically reliable regions. Under this modulation, the response of the occluded region in the original feature map is weakened, while the encoding of the background region is preserved and enhanced, making the weighted feature map more suitable for subsequent calculation of background similarity.

[0028] Feature extraction can be accomplished using deep convolutional networks, lightweight convolutional networks, or transformer structures, with different kernel sizes and layer depths selected based on computational resources and image type. Copying and expansion can be implemented along the tensor dimension or via broadcasting, allowing attention weights to automatically expand across the multi-channel structure. Element-wise multiplication can be performed at the same resolution or after upsampling to enhance the spatial fineness of the features. If image resolutions differ, bilinear or convolutional interpolation can be applied to the attention map to ensure its size matches the feature map. After weighting, normalization or linear transformation can be used to stabilize the weighted feature distribution, which is beneficial for subsequent region segmentation and similarity analysis.

[0029] This embodiment achieves simultaneous modulation of spatial saliency and deep semantic representation by expanding and applying the fused attention map to the feature map. This enables the feature map to maintain priority on stable background regions in scenes containing occlusion, structural interference, or complex textures, thereby improving the reliability of subsequent similarity measurements.

[0030] S60, the weighted feature map is converted into multiple region feature descriptors, and multiple region weights corresponding to the multiple regions are determined according to the fused attention map. In this embodiment, the weighted feature map is usually composed of a multi-channel matrix. To extract the local spatial structure, it needs to be divided into multiple regular non-overlapping regions in the spatial dimension. The region division can be based on a fixed grid size or adaptively adjusted according to the input resolution, so that each region corresponds to a specific spatial range. The region feature descriptors are derived from the aggregation of feature vectors of pixel positions within the region. The aggregation method can be generalized average pooling, which enhances large-value responses and preserves the structural differences within the region. The dimension of the pooled vector is usually high, and a more compact representation can be obtained through dimensionality reduction, making the differences between regions easier to compare in the vector space.

[0031] The fused attention map provides weight information for each spatial location. To generate region weights, the average value of the fused attention map needs to be calculated for each region. The average value reflects the importance of a region relative to the global context and is used to characterize the stability and semantic consistency of the background region. After normalization, the region average values ​​form the final region weights, ensuring that the weights of all regions are distributed within a comparable numerical range. A one-to-one correspondence is established between region feature descriptors and region weights, forming an encoding and saliency representation of the local spatial structure.

[0032] Region partitioning can employ a fixed grid, such as dividing the weighted feature map into equally sized grid regions; alternatively, the number of regions can be adaptively selected based on image content, matching the size of local regions to the size of the regions of interest. The exponent parameter of generalized average pooling can be adjusted to control the prominence of high response values, changing the sensitivity of the region descriptor to texture and edges by adjusting the exponent size. Dimensionality reduction can be based on linear transformation, convolutional compression, or principal component compression; different compression methods are suitable for different computational power conditions and storage limitations. Region weight normalization can be achieved through linear or exponential normalization, maintaining a balance of weights across regions and preventing extreme values ​​from causing regional features to lose comparability in subsequent processing.

[0033] This embodiment combines region segmentation, local aggregation, and attention weighting to achieve simultaneous extraction of spatial structure and saliency information. This enables local representation to highlight the stable characteristics of the background region while suppressing distorted regions, thereby improving the reliability of subsequent similarity calculations.

[0034] S70, based on the similarity relationship between the multiple region feature descriptors and the corresponding region feature descriptors of the reference image, and the region weights, the background similarity between the target image and the reference image is determined. In this embodiment, the multiple region feature descriptors originate from the feature convergence results of different spatial regions in the weighted feature map, describing local spatial structure and texture information. The corresponding region feature descriptors of the reference image can be obtained through the same feature extraction and region partitioning methods, making the two sets of descriptors comparable in terms of dimension, spatial layout, and statistical characteristics. To quantify the correlation between the two sets of region feature descriptors, it is necessary to construct a similarity relationship between regions. The similarity relationship can be implemented through vector operations, where each pair of region feature descriptors serves as an input vector, and a cosine similarity value is formed through dot product and normalization. Cosine similarity reflects the closeness of the directional features of the regions and can be used to measure the consistency of the local background structure.

[0035] Region weights indicate the salience of a region in the overall background assessment, forming a weighted local score by combining it with the similarity value of each region. The weighting process multiplies the region weights by their corresponding similarity values, resulting in higher responses for more stable and consistent background regions. The final background similarity is obtained by summing the weighted similarity values ​​of all regions, thus aggregating local structural information on a global scale to form an overall score representing the degree of background similarity between two images.

[0036] Region similarity calculation can employ standard cosine similarity or a variant with a normalization factor to enhance robustness to illumination variations. Dot product operations can be performed in a low-dimensional compressed space to reduce computational cost and preserve directional differences between descriptors. Region weights can be introduced using linear multiplication or exponential modulation; exponential modulation can further amplify the influence of salient regions when there are significant differences in importance between different regions. Weighted local similarity values ​​can be smoothed before summation to reduce the impact of noisy regions. The overall background similarity can be summed using simple summation or, where resources permit, a normalization strategy can be introduced to maintain consistency across different image sizes or the number of regions.

[0037] This embodiment constructs similarity relationships between regional features and combines them with regional weights to form a global score, enabling the background structure to maintain a stable expression even when local differences exist, thereby improving the reliability and anti-interference ability of image background consistency judgment.

[0038] S80, Execute the target task based on the background similarity to obtain the target task result.

[0039] In this embodiment, background similarity is a numerical characterization of the degree of consistency between the target image and the reference image at the static scene level. It is typically output from the previous region-weighted matching process and can be represented as a single real number or a set of real numbers corresponding to different reference objects. A higher value indicates a higher degree of matching between the two images in background elements such as wall texture, floor structure, and fixed furniture outlines; a lower value indicates a greater difference in background structure. To avoid misinterpreting changes in foreground objects as scene changes, the background similarity construction process already incorporates region weights and local feature similarity. Therefore, the current stage can directly regard background similarity as a reliable quantitative indicator of background consistency. Executing the target task means inputting this quantitative indicator into pre-configured decision logic. Through a series of deterministic calculations and comparisons, continuous values ​​are converted into discrete or structured results that can drive subsequent business processing. The target task can be a matching task to determine whether two images belong to the same background scene, or it can be a candidate set selected from multiple candidate reference images based on the closest background, or it can be mapping background similarity to multi-level state labels to drive different levels of processing flows. The target task result is the output of the aforementioned decision logic. It can be a Boolean match / no flag, a list of reference image indices with ranking information, or a status label structure with background similarity levels, used for further processing by the business system. The process of mapping background similarity to the target task result can be broken down into several basic operations in computer implementation, such as comparing floating-point numbers with thresholds, sorting multi-element sets, truncating index sets, and encapsulating the result structure. These operations all revolve around how to use background similarity as an indicator to distinguish different scenarios, filter reference objects, or trigger different types of subsequent processing logic.

[0040] In one implementation, a single threshold can be pre-set for background similarity. The threshold is compared with the background similarity. If the background similarity is not less than the threshold, the target image and the reference image are considered to have successfully matched at the background level. This successful match Boolean state is output as the target task result; otherwise, a match failure flag is output. This implementation is suitable for situations where only the determination of whether they are in the same scene is required, and can be completed using floating-point comparison instructions. Alternatively, when there are many reference images, the background similarity between the target image and a group of reference images can be sorted numerically from high to low. The identifiers of the top-ranked reference objects are selected as the target task result output. In this case, the target task result is represented as an ordered index list. The sorting operation can be implemented using algorithms such as quicksort or heapsort, and an upper limit parameter can be introduced to control the size of the returned result. Another approach is to divide the background similarity interval into multiple level intervals using a segmented mapping method. Each interval is associated with a different task branch. For example, values ​​above the high threshold are mapped to a high consistency level, values ​​between the high and low thresholds are mapped to a medium consistency level, and values ​​below the low threshold are mapped to a low consistency level. The level flags are then packaged into the target task result so that the subsequent system can apply different processing intensities according to the level. For scenarios with time-series frames, a moving average or weighted average within the time window can be performed before background similarity is incorporated into the decision-making logic. This smooths the values ​​from multiple frames before they are compared or sorted, reducing the impact of single-frame noise. Specifically, a circular buffer can be used to store the background similarity of the most recent frames and the average value can be updated online. Regarding parameters, the threshold size, the number of returned reference images, and the width of the level interval can be adjusted through offline statistics or online feedback to adapt to differences in background variation and business tolerance under different scenarios.

[0041] Example Explanation: In a remote health management platform in the healthcare field, a set of reference images can be maintained for each standard examination environment. When a user uploads an image of the current environment via a home terminal, the system first calculates the background similarity between the image and each reference image, and then uses the background similarity as input to execute the target task. If the target task is configured as environmental consistency verification, a combination of threshold decision-making and ranking can be used. When the background similarity of a reference environment exceeds the threshold and ranks highly among all reference environments, the identifier of that reference environment, along with a successful match marker, is output as the target task result, allowing the platform to determine whether the current examination environment meets the preset requirements. Thus, even if family members move around the user or temporary equipment placement changes, as long as the wall structure and fixed arrangement remain stable, the decision result still uses background similarity as the basis for determining environmental consistency.

[0042] In offline financial consulting scenarios, a set of environmental reference images can be saved for different business processing areas to distinguish the background layout of different areas. When the camera terminal captures the current business scene, the system calculates the background similarity between the target image and the reference images of each business area, and executes the target task accordingly. If the target task is configured as business area recognition, a multi-level interval mapping method can be used to map the background similarity to three area matching levels: high, medium, and low. The matching level, along with the corresponding business area identifier, is output as the target task result. The upper-level business system can determine the business area to which the current processing behavior belongs based on this result, ensuring consistency with the spatial division agreed upon in the business rule configuration. Even if there are changes in the foreground such as customers entering or leaving the scene or temporary displays moving, the decision is still mainly based on the background structure to complete the area recognition.

[0043] This embodiment combines background similarity with pre-defined decision logic, transforming continuous numerical indicators into structured target task results. It can stably drive subsequent processing such as scene matching, candidate selection, or level determination even in the presence of foreground occlusion and human activity, making full use of background consistency information. This ensures the controllability and reliability of the decision-making process even when the image content is complex and fluctuating.

[0044] In one embodiment, step S20 includes: S201, inputting the target image into an encoder network and extracting multi-scale coding features of the target image through staged downsampling processing; S202, using a decoder network to perform serialization processing on the multi-scale coding features, including step-by-step upsampling operations, to generate upsampling features of the corresponding scale; S203, fusing the same-scale coding features extracted by the encoder network with the corresponding-scale upsampling features generated by the decoder network through a skip connection mechanism to generate multi-level fusion features with multiple levels; S204, performing regression processing on the final first-level fusion feature in the multi-level fusion features to generate a depth map; S205, determining the difference between the depth value of each pixel position in the depth map and the depth threshold parameter; S206, scaling the difference using a smoothing parameter to obtain a scaled difference; S207, inputting the scaled difference into a non-linear saturating activation function for mapping to generate a depth attention mask.

[0045] In this embodiment, the encoder network can be understood as a forward computation structure formed by the sequential stacking of multiple layers of convolutional units, normalization units, and nonlinear units. By performing multi-level transformations on image data in both spatial and channel dimensions, it maps the original three-channel or multi-channel pixel data into a representation in a high-dimensional feature space. Staged downsampling is one of the key operations in the encoder network. By introducing spatial scale reduction operations between several layers, such as using convolution with a stride greater than one or explicit pooling operations, the width and height dimensions of the feature map gradually decrease at each stage, while the number of channels gradually increases. This allows for the coverage of a larger receptive field at a smaller spatial resolution, enabling the encoder network to simultaneously capture local texture information and large-scale structural information. Multi-scale encoded features refer to a set of feature maps obtained at different downsampling stages, including high-resolution shallow features near the input and low-resolution deep features near the encoder end. These features differ in their degree of spatial detail preservation and semantic abstraction, providing a multi-scale information source for subsequent depth estimation.

[0046] After encoding, a decoder network is introduced to perform serialization processing on the multi-scale encoded features, including progressive upsampling operations. The decoder network typically consists of multiple upsampling units and convolutional units. Progressive upsampling operations can employ deconvolution, interpolation combined with convolution, or subpixel rearrangement, gradually restoring low-resolution deep features to a spatial scale close to the original image size. Serialization processing means that the decoding process sequentially processes features of different scales according to a preset order. Each decoding unit receives the feature map output from the previous layer and generates new upsampled features through upsampling and convolution operations, progressively restoring the features in terms of spatial resolution while continuing to integrate high-level semantic information in the channel dimension. The upsampled features at the corresponding scale refer to the feature maps obtained at each decoding stage, whose spatial dimensions are consistent with the feature maps of the corresponding encoding stage, facilitating subsequent same-scale fusion.

[0047] To compensate for potential loss of detail during pure decoding, a skip connection mechanism is introduced. This mechanism fuses the same-scale encoded features output by the encoder network at different scales with the corresponding upsampled features generated by the decoder network. The skip connection mechanism can be implemented through a simple concatenation operation, i.e., superimposing the same-scale features from the encoder and decoder along the channel dimension. Alternatively, it can be achieved through element-wise addition, or by first mapping the encoded features to channels before combining them with the decoded features. The fused features simultaneously contain fine edge information from shallow layers and global semantic information from deeper layers at each scale, resulting in a more complete spatial structure and semantic representation. By repeating this skip connection and fusion operation across multiple scales, a multi-level fused feature set with multiple layers can be obtained. The fused feature closest to the original image resolution is usually considered the final first-level fused feature, containing the spatial details and global context required for depth estimation.

[0048] After obtaining the final first-level fused features, a regression process is introduced to map the high-dimensional features into a single-channel depth map. This regression process is typically implemented through several convolutional layers and a linear output layer, mapping the multi-channel feature map to a scalar depth value at each pixel location. This scalar can represent absolute depth, relative distance to the camera, or a normalized depth ratio. To simplify network training and inference, the depth values ​​are usually scaled within a preset range, such as mapping physical distances to a fixed numerical interval. Each pixel location in the depth map corresponds to a spatial location in the target image; the depth value reflects the distance of that location from the camera's perspective, with smaller depth values ​​representing closer foreground regions and larger depth values ​​representing farther background regions.

[0049] After obtaining the depth map, a transition region needs to be constructed based on the difference between the depth value and the depth threshold parameter. The depth threshold parameter is used to delineate the boundary between the foreground and background regions in depth space and can be determined through data statistics, empirical settings, or training. For any pixel location in the depth map, the difference between the depth value at that location and the depth threshold parameter can be calculated. A negative difference indicates that the depth is less than the threshold, leaning more towards the foreground; a positive difference indicates that the depth is greater than the threshold, leaning more towards the background. Directly using the raw difference will result in an overly steep transition between the foreground and background, easily producing unstable binary boundaries. Therefore, a smoothing parameter is introduced to scale the difference. The smoothing parameter can be understood as a coefficient controlling the width of the transition band. The larger the value, the smaller the amplitude of the scaled difference, and the smoother the subsequent weight changes; the smaller the value, the larger the amplitude of the scaled difference, and the closer the foreground-background division is to the hard threshold. The scaling process can be achieved through element-wise multiplication, multiplying the difference by the reciprocal of the smoothing parameter or the correlation coefficient to obtain the scaled difference matrix.

[0050] The scaled difference is input into a nonlinear saturating activation function for mapping, which compresses the undefined real-valued range into a stable weight range for constructing a depth attention mask. The nonlinear saturating activation function can be a function with monotonicity and upper and lower bounds, such as a sigmoid function or a hyperbolic tangent function. Through the saturation characteristics at both ends of the numerical range, pixels with depths significantly less than a threshold are mapped to weights close to zero, pixels with depths significantly greater than the threshold are mapped to weights close to one, and pixels with depths close to the threshold are mapped to intermediate weights. This mapping process is performed independently at each pixel location, forming a weight matrix with the same size as the original depth map. This weight matrix is ​​the depth attention mask, which can be used as spatial weighting coefficients in subsequent processing to emphasize distant background regions and suppress nearby foreground regions, thereby highlighting the contribution of background structure to the similarity metric in subsequent calculations.

[0051] For example, the deep attention mask generation function is:

[0052] in, This indicates the depth attention mask at the pixel level. The weight value at each location ranges from zero to one. A value closer to zero indicates a greater bias towards the foreground area, while a value closer to one indicates a greater bias towards the background area further away from the camera. Indicates the depth map at the pixel level The depth value at a given location can be obtained from the continuous depth obtained by the encoder-decoder network regression, and the unit can be distance in the camera coordinate system or normalized depth. This represents the depth threshold parameter, which is used to distinguish the foreground from the background. It can be set based on scene calibration, training statistics, or experience, and is generally taken as a value near the medium depth position. This represents the smoothing parameter, used to control the slope of the threshold transition region. The smaller the absolute value, the steeper the change near the threshold, and the "harder" the distinction between the mask foreground and background. The larger the absolute value, the smoother the transition. () represents a nonlinear saturated activation function, here it is the sigmoid function σ(x)=1 / (1+exp( x)) This function maps the scaled difference to a range of zero to one, which is used to generate a weight mask that can be directly used by subsequent networks.

[0053] This embodiment utilizes a multi-scale structure of encoder and decoder networks, employs a skip connection mechanism to construct multi-level fusion features, and combines differential scaling based on depth threshold and smoothness parameters with nonlinear saturated activation function mapping to obtain a continuous, smooth, and highly correlated depth attention mask on the target image. This depth attention mask numerically suppresses foreground regions close to the camera while enhancing the response of distant background regions, causing subsequent processing to focus more on stable background elements such as walls, ground, and fixed structures. This significantly reduces the interference of foreground occlusion (such as people and moving objects) on background similarity calculation, improving the stability and robustness of background matching results in complex indoor scenes.

[0054] In one embodiment, step S30 includes: S301, inputting the target image into the backbone feature extraction module integrated in the semantic segmentation network to extract deep semantic features, and inputting the deep semantic features into the pyramid pooling module; S302, using the pyramid pooling module to perform multi-scale grid pooling operations on the deep semantic features to generate multi-scale pooled features; S303, performing an upsampling operation on the multi-scale pooled features to make the resolution of the multi-scale pooled features consistent with the resolution of the deep semantic features; S304, combining the upsampled multi-scale pooled features with the deep semantic features. S305. Channel splicing and fusion are performed to generate multi-scale contextual features; S306. The multi-scale contextual features are convolved through a classification mapping layer to generate an initial semantic probability distribution; S307. The initial semantic probability distribution is normalized to generate a semantic probability map; S308. A preset set of background categories is obtained, and the corresponding background category channels are selected based on the set of background categories in the channel dimension of the semantic probability map; S309. Pixel-level summation is performed on the background category channels in the channel dimension to aggregate the probability distribution of multiple channels into a single-channel two-dimensional matrix to generate a semantic attention mask.

[0055] In this embodiment, the target image is input into a network structure integrating semantic segmentation functionality. The network contains a backbone feature extraction module, typically composed of multiple layers of convolutional units, normalization units, and nonlinear units. This module progressively compresses the spatial dimension while simultaneously enhancing feature representation capabilities along the channel dimension. After passing through the backbone module, the target image is mapped to deep semantic features. These deep semantic features correspond to multi-channel feature maps with rich semantic information at lower spatial resolution. Each channel corresponds to a response to a specific semantic pattern, and each spatial location represents an abstract description of a receptive field region in the input image. The backbone module can originate from a convolutional neural network structure, such as a network with multi-stage residual blocks or multi-scale receptive fields, or it can be a hybrid structure combining attention units or transformer units to enhance the representation of long-range dependencies.

[0056] Deep semantic features are fed into the pyramid pooling module to extract contextual information at different spatial scales. The pyramid pooling module divides the deep semantic features into several grid regions in the spatial dimension through multi-scale grid pooling operations. At each scale, pooling operations are used to aggregate the feature vectors within each grid region. For example, a coarser scale can divide the entire feature map into a smaller number of grids to capture the global scene layout; a finer scale divides it into more grids to preserve the semantic differences in local regions. The results of multi-scale grid pooling constitute a set of multi-scale pooled features, where each scale corresponds to a set of spatially aggregated feature vectors. By concatenating or linearly transforming these vectors along the channel dimension, encoded representations representing the context at different scales can be formed. Grid pooling operations can employ average pooling or max pooling, or weighted aggregation can be introduced to use weights learned within the network to emphasize the contribution of certain spatial locations.

[0057] After obtaining multi-scale pooling features, they need to be restored to a spatial resolution consistent with the deep semantic features for position-by-position fusion processing. To this end, an upsampling operation is performed on the multi-scale pooling features. Upsampling can employ bilinear interpolation, nearest-neighbor interpolation combined with convolution, deconvolution, etc., to expand the pooling results at each scale into feature maps with the same width and height as the original deep semantic features. During the upsampling process, feature vectors originally representing a single grid region at the same scale are copied or smoothly diffused to the corresponding spatial region, allowing each spatial location to access contextual information at different scales simultaneously. By upsampling the pooling results at each scale separately, a set of feature maps aligned with the deep semantic features in spatial dimensions can be obtained, providing a foundation for subsequent channel fusion.

[0058] After upsampling the multi-scale pooling features, the upsampling results are concatenated and fused with the deep semantic features along the channel dimension. Channel concatenation refers to linking channels from different feature sources along the channel direction while maintaining the same spatial size, so that the feature vector at each spatial location simultaneously contains local semantics, global context, and pooling statistics at different scales. To improve the quality of the fused representation, one or more convolutional and normalization units can be added after concatenation to perform channel rearrangement and nonlinear transformation on the concatenated multi-scale context features, thereby reducing redundant information and strengthening the feature dimensions related to category discrimination. The multi-scale context features are formed in this stage and serve as the input basis for subsequent classification mapping.

[0059] The classification mapping layer receives multi-scale contextual features and performs convolutional processing, mapping high-dimensional semantic representations to a category space. The classification mapping layer can employ one or more one-dimensional or two-dimensional convolutional layers with small kernel sizes, outputting a vector of length equal to the number of categories at each spatial location. Each component represents the score or log-likelihood of that location belonging to a particular category. The convolutional processing performs isomorphic transformations on local neighborhoods across the entire spatial range by sharing kernel parameters, ensuring that features at different locations are mapped to the category score space with the same parameters. The initial semantic probability distribution obtained through the classification mapping layer is actually an unnormalized category response tensor, which can typically be viewed as the raw score of each pixel for each category.

[0060] To achieve interpretable probabilistic representation, a normalization operation is introduced into the initial semantic probability distribution to generate a semantic probability graph. Normalization can employ an exponentially weighted normalization operation performed independently along the category dimension, ensuring that the sum of all category scores at each pixel location is one, thus allowing each component to be viewed as a category conditional probability. Alternatively, other transformations that maintain non-negativity and controllable sums can be used, as long as the output has probabilistic numerical meaning. The semantic probability graph has the same morphology as the initial semantic probability distribution, remaining a three-dimensional tensor structure. The spatial dimension corresponds to the image plane, and the channel dimension corresponds to the category set. However, the numerical range and statistical meaning have been transformed from raw scores into probabilistic weights, which is beneficial for subsequent channel filtering and aggregation calculations.

[0061] To highlight the semantic information of the background area, a background category set needs to be provided in advance. The background category set refers to several category labels selected from the entire category label space that belong to the static scene components, such as walls, floors, ceilings, large fixed furniture, and long-term stationary equipment. Specific background categories related to specific business operations can also be included, provided that these categories can be considered stable backgrounds in the application scenario. The background category set can be determined through data annotation specifications, business rules, or statistical results from the training set. The number of categories can be one or more, and is not limited to a specific enumeration.

[0062] After obtaining the background category set, a channel filtering operation is performed along the channel dimension of the semantic probability map to select channels belonging to the background category set, forming a subset of background category channels. Channel dimension filtering can be implemented using an index list or mask vector, where channel positions corresponding to the background category are set as valid, and other channel positions are marked as invalid or ignored. In implementation, a binary vector with a length equal to the number of categories can be constructed, assigning a value of 1 to background category positions and a value of zero to non-background category positions. Then, element-wise multiplication is performed along the channel dimension to retain background category channels and suppress foreground category channels. After filtering, the background category channels retain the probability response of each pixel for each background category.

[0063] To obtain a single-channel spatial map that can be directly used for attention weighting, pixel-level summation is performed on the background category channels along the channel dimension. Pixel-level summation means summing the probability values ​​of all background category channels at each spatial location, and treating this sum as the aggregate probability that the location belongs to the background. This operation is performed pixel-by-pixel along the spatial dimension, eliminating the category dimension along the channel dimension, ultimately generating a single-channel two-dimensional matrix. This two-dimensional matrix numerically reflects the background confidence at each pixel location, and structurally aligns with the spatial dimensions of the input image and the depth attention mask, allowing it to be directly used as a semantic attention mask in subsequent weighting operations. Through this construction process, the semantic attention mask aggregates information from multiple background categories at the semantic level, uniformly mapping cross-category background regions into a continuous weight map to highlight semantically background-related regions in the scene.

[0064] This embodiment constructs a multi-scale contextual representation through a backbone feature extraction module and a pyramid pooling module. This is then combined with a classification mapping layer and normalization operations to generate a semantic probability map. Furthermore, by utilizing a set of background categories to filter background category channels along the channel dimension and performing pixel-level summation, a semantic attention mask aligned with the image space can be constructed at the semantic level. This semantic attention mask numerically reflects the aggregation probability of static background categories, increasing the weight of background areas such as walls and floors while suppressing the responses of foreground categories such as people and moving objects. This provides precise semantic weight guidance in subsequent processing stages, making the background similarity assessment process focus more on areas related to the scene structure, reducing the impact of foreground semantic interference on the similarity calculation results, and enhancing the discriminative ability and stability of background matching in complex semantic environments.

[0065] In one embodiment, step S40 includes: S401, initializing a first learnable weight matrix for adjusting the weights of deep information; S402, initializing a second learnable weight matrix for adjusting the weights of semantic information; S403, performing element-wise multiplication of the deep attention mask with the first learnable weight matrix to obtain a depth-gated component; S404, performing element-wise multiplication of the semantic attention mask with the second learnable weight matrix to obtain a semantic-gated component; S405, performing element-wise addition of the depth-gated component and the semantic-gated component, and applying a non-linear saturating activation function to the result of the addition to generate a fused attention map.

[0066] In this embodiment, two input weight maps, a depth attention mask and a semantic attention mask, are introduced and adaptively fused using a set of learnable parameters. The depth attention mask originates from the depth map generated by the depth estimation network. After mapping, it reflects the importance of each pixel at the geometric distance or structural level. The numerical range can be limited to zero to one, used to emphasize regions that are geometrically distant backgrounds or near-static structures. The semantic attention mask is obtained by aggregating the semantic probability map obtained from the semantic segmentation network with background categories. It uses a single channel to represent the probability that each pixel belongs to the background region at the semantic label level. The two attention masks are aligned in spatial resolution, and each pixel location contains a depth weight value and a semantic weight value, describing the salience of the background from both geometric and semantic perspectives, respectively.

[0067] To adaptively adjust the relative contributions of geometric and semantic information in different scenarios, a first learnable weight matrix needs to be constructed to regulate the depth information weights. This first learnable weight matrix can be designed as a two-dimensional parameter matrix of the same size as the depth attention mask, or it can be a multi-dimensional tensor with one channel, storing updatable parameters in a one-to-one correspondence with pixel positions. During initialization, each parameter can be assigned a uniform or Gaussian distribution, or a constant initialization can be used to provide a balanced starting point. During training, this matrix is ​​updated through error backpropagation, enabling the network to automatically learn how to amplify or reduce the influence of depth attention at different spatial locations based on different data distributions. For example, in areas where occlusions are close to the camera, the first learnable weight matrix can gradually reduce the depth weights to weaken the interference caused by occlusion.

[0068] Correspondingly, a second learnable weight matrix is ​​introduced to adjust the semantic information weights. The second learnable weight matrix is ​​consistent with the first learnable weight matrix in size and organization; it can also store a set of trainable parameters at pixel positions, or it can use a lower-dimensional parameter form to expand to the global space via a broadcast mechanism. The initialization strategy can be consistent with the first learnable weight matrix, or a different initial value distribution can be used to introduce prior biases for the depth and semantic channels. During model training, the second learnable weight matrix is ​​updated via gradients in the loss function to spatially adaptively adjust the semantic attention in different regions. For example, it increases semantic weights in regions with complex textures but stable semantic classification, and appropriately suppresses semantic weights near class boundaries to reduce the impact of classification uncertainty on subsequent processing.

[0069] After initializing the learnable weight matrix, the deep attention mask is multiplied element-wise with the first learnable weight matrix. Element-wise multiplication refers to multiplying the scalar values ​​of the two matrices one by one at corresponding spatial locations to form a new two-dimensional matrix output. This operation can be implemented using point-by-point multiplication operations in a matrix multiplication library, or efficiently executed on hardware-accelerated devices using tensor broadcasting. The multiplication result is defined as a depth-gated component, which numerically reflects the combined effect of the deep attention and the first learnable weight matrix. The larger the value at each position in the depth-gated component, the higher the background weight at that position, and the more it is amplified by the parameter matrix. If the deep attention value is high but the weight matrix value is low, the combined result will be suppressed, thereby achieving a spatial redistribution of geometric information.

[0070] Similarly, the semantic attention mask is multiplied element-wise with the second learnable weight matrix to obtain the semantic gating component. The semantic gating component is structurally consistent with the depth gating component, also being a two-dimensional matrix, with each element reflecting the product of the semantic attention and semantic weight parameters at the corresponding position. This multiplication operation allows for fine-grained adjustment of the semantic response in different regions while maintaining the original semantic background probability structure. For example, in regions where the background category is highly reliable, the parameters of the second learnable weight matrix can be updated to values ​​greater than one, thereby amplifying the influence of semantic attention; in regions with high semantic prediction noise, the parameter values ​​can approach zero to reduce the impact of unstable semantic labels on the fusion result.

[0071] After obtaining the depth-gated and semantic-gated components, the two gating results need to be added element-wise at the same spatial location. The element-wise addition operation adds the values ​​of the depth-gated and semantic-gated components at each pixel location, resulting in a hybrid weight matrix before fusion. This matrix numerically integrates the gating results from the depth and semantic channels, achieving a linear superposition of geometric and semantic information. In implementation, this can be done directly using the addition operator to perform point-wise summation on the two matrices, or it can be accomplished in hardware using a vector acceleration unit. When the depth-gated and semantic-gated components participate in the summation, their relative magnitudes are jointly determined by the previously learned weight matrix and the input attention mask, thus automatically forming different weighting ratios in different scenarios.

[0072] To ensure the fused weights remain numerically stable while possessing non-linear discriminative power, the element-wise summation result needs to be mapped using a non-linear saturating activation function. This non-linear saturating activation function can be a family of functions with upper and lower bounds, such as compressing real numbers to the zero-to-one interval through exponential mapping, or using other monotonic functions combining saturation and linear segments. The activation function acts individually on the fused weights at each pixel location, compressing excessively large weights to near their upper bound and increasing excessively small weights to near their lower bound, while preserving subtle differences in the intermediate range. This mapping process avoids gradient explosion or vanishing during training and makes the output more suitable as an attention map for subsequent weighting operations. The mapped result is defined as the fused attention map, which has the same spatial structure as the original attention mask. Each pixel location contains a single-channel weight value, which comprehensively considers geometric depth information, semantic category information, and the adaptive adjustment effect of the learnable weight matrix, and can be directly used for spatial weighting of the feature map.

[0073] This embodiment introduces two sets of learnable weight matrices onto a depth attention mask and a semantic attention mask, forming depth-gated components and semantic-gated components respectively. A fused attention map is then generated through element-wise addition and non-linear saturated activation mapping, enabling adaptive fusion of geometric and semantic information at the pixel level. Compared to fixed-ratio superposition, this fusion mechanism automatically learns appropriate weight combinations for different spatial locations, scene types, and occlusion levels. This strengthens depth attention in geometrically reliable regions and semantic attention in semantically stable regions, resulting in a differentiated weight distribution between complete background regions and complex foreground occlusion regions. The final output fused attention map is numerically more stable and spatially closer to the real background region, providing more accurate region selection capabilities for subsequent feature weighting and similarity calculation. It significantly reduces the impact of foreground interference on the background matching process, improving the robustness and discriminative power of background similarity evaluation under multi-scene and multi-scale conditions.

[0074] In one embodiment, step S50 includes: S501, extracting multi-channel feature data output by the target image during the encoding stage using a feature coding network, and using the multi-channel feature data as a feature map; S502, identifying the number of channel dimensions of the feature map, and copying and extending the fused attention map of the single-channel structure along the channel dimensions to generate an extended attention map whose spatial size and number of channels are consistent with the feature map; S503, performing element-wise multiplication between the extended attention map and the feature map, using the weight values ​​in the extended attention map to perform channel-weighted modulation on the feature map, and using the modulated feature data as a weighted feature map.

[0075] In this embodiment, a feature encoding network is used to extract multi-channel feature data from the target image. The feature encoding network can be a convolutional-based encoding structure, or it can incorporate attention operations or transformer structures. Their common feature is that they use multiple linear or non-linear mappings to map the original three-channel image in pixel space into a feature representation with reduced spatial resolution and increased channel number. At a certain layer in the encoding stage, the tensor output by the network can be represented as a combination of three dimensions: height, width, and number of channels. Each spatial location corresponds to a channel vector, which contains information such as the texture pattern, color distribution, and shape boundary of the neighborhood at that location. Aligning this multi-channel feature tensor spatially with the target image yields a feature map of the target image in the feature space, which is then used for subsequent combination with attention weights.

[0076] After obtaining the feature map, it is necessary to identify the number of channels in the feature map for matching with the single-channel structure of the fused attention map. The number of channels in the feature map can be obtained directly from the shape information of the tensor, or it can be pre-recorded during the network structure definition stage and passed as a fixed parameter to the computation module at runtime. The larger the number of channels, the more subspaces the feature map carries in the directions of semantic category, edge structure, texture mode, etc. Therefore, when copying the fused attention map in the channel dimension, it is necessary to ensure that each channel can receive the attention weight corresponding to its spatial position. To achieve this, the single-channel fused attention map is copied and expanded in the channel dimension to obtain the expanded attention map. Copying and expansion can be achieved through a broadcast mechanism, that is, without changing the spatial size of the original attention map, several copies are made in the channel dimension, so that the expanded attention map is completely consistent with the feature map in the three dimensions of height, width, and channels, thus having a corresponding weight value at each spatial position and each channel.

[0077] After constructing the extended attention map, element-wise multiplication needs to be performed between it and the feature map. Element-wise multiplication refers to multiplying the eigenvalues ​​in the feature map with the weights in the extended attention map at each spatial coordinate and channel index position of the tensor to obtain a new feature response. To meet the requirements of numerical stability and gradient propagation, the values ​​of the attention weights can be restricted to a finite interval through a nonlinear mapping in the previous step, such as zero to one or a finite interval symmetric about zero. This way, irrelevant regions can be suppressed during multiplication without producing excessive amplification effects. To implement this operation, element-wise multiplication can be called within the image processing framework, with the input being a feature map tensor and an extended attention map tensor of the same shape, and the output being an intermediate tensor of the same shape.

[0078] Numerically, the weights in the expanded attention map act as channel-weighted modulation. When a spatial location is identified as a background region by the preceding fusion attention step, the weight at that location approaches the upper weight limit. After multiplying with the feature vector at that location, the responses of most channels are preserved or even slightly amplified, allowing the background region to contribute more in subsequent matching. When a location belongs to a foreground occlusion or semantically unstable region, the weight approaches the lower weight limit. The feature responses of the corresponding channels are significantly compressed by the multiplication operation, approaching zero, thus weakening the influence of that region in the overall feature distribution. Through this pixel-wise and channel-wise multiplication operation, attention information is directly integrated into the feature representation. The resulting modulation result reflects a bias towards the background region in both spatial distribution and channel intensity. This modulated feature tensor is the weighted feature map. The weighted feature map maintains the same tensor shape as the original feature map, but the values ​​at each location have been filtered and rearranged by the fusion attention map, making it more suitable for subsequent region segmentation and similarity calculation.

[0079] This embodiment introduces a fused attention map, expanded by channel dimension replication, onto the multi-channel feature map output by the feature encoding network. Element-wise multiplication is then performed to directly combine the feature response of each spatial location and each channel with its corresponding attention weight. This allows for fine-grained spatial and channel joint modulation of features without altering the network's depth or feature dimension. Compared to methods relying solely on global weighting or single-channel weight control, this channel-weighted modulation consistently enhances background region features across the multi-channel space, improving the stability and discriminability of the background description in subsequent measurements. Furthermore, it significantly attenuates the feature response of foreground occluded regions, reducing the interference of foreground entities such as people and devices on the overall similarity calculation. This provides cleaner, more background-structure-focused information input for subsequent region feature generation and background similarity evaluation.

[0080] In one embodiment, step S60 includes: S601, dividing the weighted feature map into multiple regular grid-like non-overlapping local regions in the spatial dimension; S602, performing generalized average pooling on the feature vectors of all spatial locations within each non-overlapping local region; S603, performing dimensionality reduction and compression on the feature vectors after generalized average pooling to generate a region feature descriptor corresponding to each non-overlapping local region; S604, determining the average value of the fused attention map at all pixel locations within each non-overlapping local region; S605, normalizing the average value to generate a region weight corresponding to each non-overlapping local region.

[0081] In this embodiment, using a weighted feature map and a fused attention map as input, continuous spatial features are converted into discrete region-level representations, and weights corresponding to each region are generated simultaneously to reflect the importance of different spatial regions in subsequent calculations. The weighted feature map can be understood as a feature tensor that has been attention-modulated in both spatial and channel dimensions. It is typically represented as an array with three dimensions: height, width, and number of channels. Each spatial location is associated with a channel vector, recording information such as texture, color, and structure of the neighborhood at that location. The fused attention map can be understood as a single-channel weight map aligned with the spatial dimensions of the original image. Each pixel location stores a real-valued weight to quantify the degree to which that location belongs to the background or has high reference value.

[0082] When performing region-level transformation on a weighted feature map, it is first divided into multiple regular grid-like non-overlapping local regions in the spatial dimension. Regular grid-like means equally spaced along the horizontal and vertical directions; for example, dividing the entire feature map into several rectangular blocks according to a fixed number of rows and columns. Non-overlapping means that any two regions have no intersection in their pixel sets, ensuring that each pixel is assigned to only one region. The number of grids can be selected based on the integer division relationship between the feature map's height and width, or the number and size of regions can be pre-configured, and then the feature map is adapted using integer division and edge clipping. Through this partitioning operation, the originally continuous two-dimensional space is mapped into a finite set of regions, each region containing a set of spatial location indices, facilitating pooling and statistical calculations at the region granularity.

[0083] After obtaining the non-overlapping local regions, generalized average pooling needs to be performed on the feature vectors corresponding to all spatial locations within each region. A feature vector is a multi-dimensional vector obtained by expanding the feature map along the channel dimension at a certain spatial coordinate. Generalized average pooling is an operation that aggregates a set of vectors. It can be represented by a type of mean operation controlled by an exponent or power. When the power parameter takes a specific value, it degenerates into an arithmetic mean or near-extreme selection. In implementation, power transformation, summation, and inverse power transformation combinations can be performed on each channel at all locations within the region. Alternatively, when the parameter is set to one, it is equivalent to performing a simple average on each channel within the region. This operation compresses the feature vectors from multiple locations within the region into a single vector while preserving the overall statistical expression of the region, giving the region a representative point in the feature space.

[0084] Generalized average pooling typically results in region vectors that retain the same dimensionality as the feature map channels. Directly using these vectors for subsequent matching in high-dimensional cases incurs computational and storage costs, necessitating further dimensionality reduction. Dimensionality reduction can be achieved through linear mappings, such as multiplying each region vector by a pre-trained or linked-training compression matrix to map the original high-dimensional vector to a lower-dimensional space. Alternatively, it can be achieved through non-linear mappings, such as small fully connected networks or one-dimensional convolutional structures. Regularization or constraint terms can be incorporated during dimensionality reduction to ensure the compressed vectors maintain discriminative power and stability in the similarity metric space. After dimensionality reduction, each non-overlapping local region is assigned a low-dimensional vector, which serves as the region feature descriptor, used to characterize background texture and structural information at the region level.

[0085] While constructing region feature descriptors, it is also necessary to generate corresponding region weights for each region from the fused attention map to characterize the proportion of influence of different regions in the overall background similarity calculation. Specifically, the fused attention map is first divided into completely corresponding non-overlapping local regions using the same spatial partitioning method as the weighted feature map, ensuring a one-to-one match between the index and the region feature descriptors. For each region, attention values ​​at all pixel locations within that region are collected, and the average of these values ​​is calculated. The averaging operation can use a simple arithmetic average or a weighted average, where the weights can be externally configured or provided by other image attributes. The average value reflects the overall spatial attention intensity of the region; a higher value indicates that the region is given higher importance by multimodal attention.

[0086] To ensure comparability of region weights across different images or scenes, normalization is required for the average values ​​of all regions. Normalization can be performed across the entire image's regions, dividing the average value of each region by the sum of all region averages. This ensures the normalized weights numerically satisfy the constraint that the sum is one. Alternatively, other monotonic mappings can be used to compress the original average values ​​to a fixed range while preserving the relative order between different regions. A lower bound can be added during normalization to prevent gradient vanishing or numerical instability caused by extremely small weights. After normalization, each non-overlapping local region receives a region weight, which is bound to the corresponding region feature descriptor, providing a basis for subsequent weighted metrics at the region level.

[0087] For example, the formula for the region descriptor in GeM pooling:

[0088] in, The region feature descriptor for the r-th local region is usually a fixed-dimensional vector used to represent the comprehensive response of the local region in the channel dimension. This represents the set of pixels in the feature map for the r-th local region, i.e., the set of spatial location indices contained in this region. Indicates the region The number of pixels included is used to perform mean normalization on the accumulated result. The feature vector at position (i,j) of the weighted feature map can be regarded as the response value in the channel dimension. Represents the learnable exponential parameter in GeM pooling, when When α = 1, it degenerates into average pooling, and as α → ∞, it approaches max pooling. By learning α, it can adaptively adjust between average and max pooling.

[0089] This embodiment transforms high-dimensional feature representations in continuous space into a finite number of region feature descriptors by performing regular grid partitioning, generalized average pooling, and dimensionality reduction compression on the weighted feature map. Simultaneously, it utilizes fused attention maps to calculate average values ​​in corresponding regions and normalizes them to generate region weights. This allows for compressed representation and weight allocation of background information at the region granularity. On one hand, the region feature descriptors reduce feature dimensions while maintaining the differences in background texture and structure, reducing the computational cost of subsequent similarity calculations and improving numerical stability. On the other hand, the region weights spatially aggregate the distribution information of multimodal attention, enabling them to automatically increase the contribution of salient background regions and weaken the influence of interfering regions in subsequent weighted measurements. This provides a more compact and robust region-level representation foundation for background similarity evaluation in complex occlusion and semantically mixed scenarios.

[0090] In one embodiment, step S70 includes: S701, pairing each region feature descriptor of the target image with the corresponding region feature descriptor of the reference image according to the spatial location correspondence; S702, performing vector dot product processing on each pair of paired region feature descriptors to determine the cosine similarity value and generating the similarity relationship of the paired regions; S703, multiplying the similarity relationship of each paired region with the corresponding region weight to generate the weighted similarity value of the paired regions; S704, summing the weighted similarity values ​​of all paired regions and using the summed result as the background similarity.

[0091] In this embodiment, the target image's region feature descriptor, the corresponding region feature descriptor of the reference image, and region weights are used as inputs. Vector operations and weighted aggregation are then used to generate a single background similarity value, which quantifies the closeness of the two images at the background level. The region feature descriptor can be understood as a vector representation obtained by aggregating and reducing the dimensionality of each regular grid region in the weighted feature map. Each vector corresponds one-to-one with a spatial grid cell, recording comprehensive information about the texture, structure, color, and other features within that region. The corresponding region feature descriptor of the reference image is obtained by performing the same feature extraction, weighting, and region partitioning process on the reference image. The two sets of descriptors are aligned in spatial indices, ensuring that two vectors at the same grid position reflect corresponding spatial regions. The region weights are derived from the statistical results of the fused attention map within the corresponding grid region, typically obtained by averaging and normalizing the attention values ​​within the region, and are used to express the importance of that region in the global background comparison.

[0092] When establishing region-level correspondences, each region feature descriptor in the target image needs to be paired with the region feature descriptors at the same grid position in the reference image according to spatial location indexes. Spatial location correspondences can be achieved through a unified grid numbering system. For example, after dividing the feature map into several rows and columns of grids, each grid is assigned a row and column index or a one-dimensional sequential number. Both the target and reference images use the same partitioning method and the same numbering rule, so that the region feature descriptor numbered i always corresponds to a region with a consistent position in both images. In implementation, the two sets of region feature descriptors can be stored in sequential arrays and paired using a circular index, or a key-value mapping structure can be used, where the key is the region location code and the value is the corresponding vector. The same key is used to access the vector pairs of the target and reference images. The pairing process ensures that subsequent similarity calculations are performed only between spatially corresponding regions, avoiding noise introduced by cross-region comparisons.

[0093] After obtaining the paired region feature descriptors, a vector dot product is performed on each pair of paired vectors to determine the cosine similarity value. The vector dot product operation is the process of multiplying corresponding elements of two vectors of the same dimension along each dimension and then summing the results. The result is equal to the magnitudes of the two vectors multiplied by the cosine of the angle between them. When the region feature descriptors have already undergone norm normalization in the upstream processing, the magnitudes are constrained to around one unit length. In this case, the dot product value can be directly regarded as an approximation or precise expression of cosine similarity. In implementation, L2 normalization can be performed on each region vector first, making the Euclidean norm of each vector equal to one, and then dot(a,b) can be calculated as the similarity value. Alternatively, without explicit normalization, a strict cosine similarity can be obtained by dividing the dot product value by the product of the magnitudes of the two vectors. Cosine similarity numerically reflects the degree of similarity between two regions in the feature space direction. The closer the value is to one, the more consistent the direction, and the closer the corresponding regions are in texture and structure. Values ​​close to zero or negative indicate greater differences. For each pair of region feature descriptors, a scalar similarity result is generated, forming a similarity sequence arranged by region index, which constitutes a set of similarity relationships between paired regions.

[0094] To highlight high-attention regions and suppress low-attention regions in background similarity calculation, the similarity relationship of each paired region needs to be multiplied by its corresponding region weight to generate a weighted similarity value for the paired regions. Region weights are typically non-negative real numbers. When constraints are applied to all region weights in the preceding normalization process, the sum of all weights can be one or fall within a stable numerical range. In implementation, the region similarity sequence and region weight sequence can be arranged with the same index, and the weighted similarity sequence can be obtained through element-wise multiplication. Each weighted similarity value considers two factors simultaneously: reflecting the similarity of the corresponding region in the feature space, and reflecting the importance of the region in the overall background structure through the weight coefficient. For meshes jointly marked as key background regions by both deep attention and semantic attention, the region weight is usually higher, and the similarity of the corresponding region will contribute more to the final result after weighting. For meshes judged as foreground or noise regions, the region weight is lower, and the impact of their similarity on the overall background similarity is significantly compressed.

[0095] After calculating the weighted similarity for all regions, the weighted similarity values ​​need to be summed, and the sum is used as the background similarity. This summation can be performed on all valid region indices, simply adding the weighted similarity of each region to obtain a single scalar output. Alternatively, it can be adjusted by combining weights and normalization methods to keep the output value within the desired range. In implementation, this can be done by sequentially traversing the region indices, adding the weighted similarity values ​​of each region to the summation variable, and obtaining the sum after the traversal. Alternatively, matrix operations can be used, representing the region similarity sequence and region weight sequence as vectors, and obtaining the summation result all at once through vector dot product. The background similarity, as a global scalar output, is bound to a pair of inputs to the target image and the reference image, representing the overall matching degree of the background portion of the two images after depth and semantic attention filtering.

[0096] For example, the weighted aggregation formula for background similarity regions:

[0097] in, Representing an image With images The background similarity scalar value reflects the overall matching degree of the two images in the background area. This indicates the number of local regions obtained by the division, corresponding to the number of "multiple regular grid-shaped non-overlapping local regions". This represents the region weight of the r-th local region. The "region weight" is derived from the statistical results of the fused attention map within that region and is used to control the contribution ratio of each region to the final similarity. Representing an image With images The region feature descriptor for the r-th local region corresponds to "multiple region feature descriptors" and "the corresponding region feature descriptor of the reference image". The cosine similarity of region r is represented by the dot product of the two normalized region descriptors, taking values ​​in the interval [-1, 1]. The larger the value, the closer the features of the regions are.

[0098] After determining the calculation structure for background similarity, to enhance the discriminative power of the background representation obtained by aggregating multiple region feature descriptors during the training phase, a contrastive loss targeting the background region can be constructed. During training, a background feature vector is first generated for each training image using the aforementioned region feature descriptors and region weights. This vector can be obtained by weighted pooling of features from high-weight regions, and is used to summarize the representation of the static background in the entire image. For a given image, another image from the same background scene but with different shooting angles or occlusion conditions is selected to generate positive sample background features. Meanwhile, several background features were collected from other scenes to form a negative sample set. In each training iteration, using the current background features as anchor points, the contrastive loss is calculated using the following formula:

[0099] in, This represents a contrastive loss scalar constructed for background features, used to measure the relative closeness between the current background representation and positive and negative samples; The background feature vector corresponding to the current target image is a global background representation obtained by aggregating region feature descriptors and region weights. This represents positive sample background features that belong to the same background scene as the current image, and there may be differences in shooting position, lighting or foreground occlusion; This represents a set of negative sample features from other background scenes, used to differentiate between different scenes in the feature space; () represents the feature similarity calculation function. In implementation, the dot product operation of the feature vector after normalization can be used to obtain the cosine similarity value. The temperature parameter is used to scale the similarity. When the value of this parameter is small, the soft maximum operation is more sensitive to differences in similarity; when the value is large, the similarity distribution is smoother. By updating the network parameters through backpropagation using contrastive loss, background features of the same background scene can be made closer in the feature space, while background features of different background scenes can be more separated. This provides a more stable feature foundation for subsequent background similarity calculations based on region weight aggregation.

[0100] This embodiment constructs spatially aligned region pairing relationships based on region feature descriptors and region weights. It uses cosine similarity to measure the feature direction of each region and then weights the similarities of each region before summing them into a single background similarity value. This decomposes the complex image background comparison process into a series of structured linear and nonlinear operations. On one hand, region-level cosine similarity maintains the representation of background detail differences while reducing sensitivity to changes in absolute brightness and scale, ensuring stable reflection of background layout consistency even under varying lighting and viewpoint shifts. On the other hand, region weights further transfer the ability of depth and semantic attention to background regions to the similarity aggregation process, making the weighted accumulation more reliant on matching results from stable background regions while weakening interference from foreground occlusion regions.

[0101] In one embodiment, a similarity processing device based on fusion attention is provided, which corresponds one-to-one with the similarity processing method based on fusion attention in the above embodiments. Referring to Figure 3, Figure 3 is a functional module diagram of a preferred embodiment of the similarity processing device based on fusion attention of the present invention. The device comprises an image input module 10, a depth attention generation module 20, a semantic attention generation module 30, an attention fusion module 40, a feature weighted encoding module 50, a region feature modeling module 60, a similarity calculation module 70, and a task decision module 80. The detailed descriptions of each functional module are as follows: The image input module 10 is used to acquire the target image to be processed and a reference image; the depth attention generation module 20 is used to perform depth estimation processing on the target image, generate a depth map, and generate a depth attention mask based on the depth map; the semantic attention generation module 30 is used to perform semantic segmentation processing on the target image, generate a semantic probability map, and generate a semantic attention mask based on the semantic probability map; the attention fusion module 40 is used to fuse the depth attention mask and the semantic attention mask to generate a fused attention map; the feature weighted encoding module 50 is used to extract the features of the target image... The system generates a weighted feature map by weighting the feature map using the fused attention map. A region feature modeling module 60 converts the weighted feature map into multiple region feature descriptors and determines multiple region weights corresponding to the multiple regions based on the fused attention map. A similarity calculation module 70 determines the background similarity between the target image and the reference image based on the similarity relationship between the multiple region feature descriptors and the corresponding region feature descriptors of the reference image, as well as the region weights. A task decision module 80 executes the target task based on the background similarity to obtain the target task result.

[0102] Specific limitations regarding the similarity processing device based on fusion attention can be found in the aforementioned limitations on the similarity processing method based on fusion attention, and will not be repeated here. Each module in the aforementioned similarity processing device based on fusion attention can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0103] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram is shown in Figure 4. The computer device includes a processor, memory, a network interface, and a database connected via a system bus. The processor of the computer device provides determination and control capabilities. The memory of the computer device includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface of the computer device is used to communicate with external clients via a network connection. When the computer program is executed by the processor, it implements server-side functions or steps of a similarity processing method based on fusion attention.

[0104] In one embodiment, a computer device is provided, which can be a client, and its internal structure can be as shown in Figure 5. The computer device includes a processor, memory, network interface, display screen, and input device connected via a system bus. The processor of the computer device provides determination and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements a client-side function or step of a similarity processing method based on fusion attention.

[0105] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it performs the following steps: acquiring a target image and a reference image; performing depth estimation processing on the target image to generate a depth map, and generating a depth attention mask based on the depth map; performing semantic segmentation processing on the target image to generate a semantic probability map, and generating a semantic attention mask based on the semantic probability map; fusing the depth attention mask and the semantic attention mask to generate a fused attention map; extracting feature maps from the target image, and weighting the feature maps using the fused attention map to generate a weighted feature map; converting the weighted feature map into multiple region feature descriptors, and determining multiple region weights corresponding to the multiple regions based on the fused attention map; determining the background similarity between the target image and the reference image based on the similarity relationship between the multiple region feature descriptors and the corresponding region feature descriptors of the reference image, and the region weights; and performing a target task based on the background similarity to obtain the target task result.

[0106] In one embodiment, a computer-readable storage medium is provided, which may be non-volatile or volatile, and a computer program is stored thereon. When the computer program is executed by a processor, it performs the following steps: acquiring a target image and a reference image to be processed; performing depth estimation processing on the target image to generate a depth map, and generating a depth attention mask based on the depth map; performing semantic segmentation processing on the target image to generate a semantic probability map, and generating a semantic attention mask based on the semantic probability map; fusing the depth attention mask and the semantic attention mask to generate a fused attention map; extracting feature maps from the target image, and using the fused attention map to weight the feature maps to generate a weighted feature map; converting the weighted feature map into multiple region feature descriptors, and determining multiple region weights corresponding to the multiple regions according to the fused attention map; determining the background similarity between the target image and the reference image based on the similarity relationship between the multiple region feature descriptors and the corresponding region feature descriptors of the reference image and the region weights; and performing a target task based on the background similarity to obtain the target task result.

[0107] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0108] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0109] It should be noted that if any software tools or components not belonging to this company appear in the embodiments of this application, they are merely illustrative examples and do not represent actual use. The embodiments described above are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

[0110] The user personal information involved in this application embodiment is all authorized (knowing and consenting) by the relevant parties or fully authorized by all parties, and the executing entity can obtain it through various open, legal and compliant means. The collection, storage, use, processing, transmission, provision and disclosure of the information, data and signals involved all comply with the relevant laws and regulations of the relevant countries and regions, and do not violate public order and good morals.

Claims

1. A similarity processing method based on fusion attention, characterized in that, Includes the following steps: Obtain the target image and reference image to be processed; The target image is subjected to depth estimation processing to generate a depth map, and a depth attention mask is generated based on the depth map; The target image is subjected to semantic segmentation to generate a semantic probability map, and a semantic attention mask is generated based on the semantic probability map; The deep attention mask and the semantic attention mask are fused to generate a fused attention map; The feature map of the target image is extracted, and the feature map is weighted using the fusion attention map to generate a weighted feature map; the weighted feature map is converted into multiple region feature descriptors, and multiple region weights corresponding to the multiple regions are determined according to the fusion attention map; Based on the similarity relationship between the multiple region feature descriptors and the corresponding region feature descriptors of the reference image, as well as the region weights, the background similarity between the target image and the reference image is determined. The target task is performed based on the background similarity to obtain the target task result.

2. The similarity processing method based on fusion attention as described in claim 1, characterized in that, The method involves performing depth estimation processing on the target image to generate a depth map, and generating a depth attention mask based on the depth map. This includes: inputting the target image into an encoder network to extract multi-scale encoded features of the target image through staged downsampling; using a decoder network to perform serialization processing on the multi-scale encoded features, including progressive upsampling operations, to generate upsampled features of corresponding scales; fusing the same-scale encoded features extracted by the encoder network with the corresponding-scale upsampled features generated by the decoder network through a skip connection mechanism to generate multi-level fused features with multiple layers; performing regression processing on the final first-level fused feature in the multi-level fused features to generate a depth map; determining the difference between the depth value at each pixel position in the depth map and a depth threshold parameter; scaling the difference using a smoothing parameter to obtain a scaled difference; and inputting the scaled difference into a nonlinear saturating activation function for mapping to generate a depth attention mask.

3. The similarity processing method based on fusion attention as described in claim 1, characterized in that, The target image is semantically segmented to generate a semantic probability map, and a semantic attention mask is generated based on the semantic probability map. This includes: inputting the target image into a backbone feature extraction module integrated into a semantic segmentation network to extract deep semantic features, and inputting the deep semantic features into a pyramid pooling module; performing multi-scale grid pooling operations on the deep semantic features using the pyramid pooling module to generate multi-scale pooled features; upsampling the multi-scale pooled features to ensure that the resolution of the multi-scale pooled features is consistent with the resolution of the deep semantic features; and then upsampling the upsampled multi-scale pooled features. Degree-pooling features are concatenated and fused with the deep semantic features to generate multi-scale context features; the multi-scale context features are convolved through a classification mapping layer to generate an initial semantic probability distribution; the initial semantic probability distribution is normalized to generate a semantic probability map; a preset set of background categories is obtained, and the corresponding background category channels are selected based on the set of background categories in the channel dimension of the semantic probability map; pixel-level summation is performed on the background category channels in the channel dimension to aggregate the probability distributions of the multi-channel channels into a single-channel two-dimensional matrix to generate a semantic attention mask.

4. The similarity processing method based on fusion attention as described in claim 1, characterized in that, The process of fusing the deep attention mask and the semantic attention mask to generate a fused attention map includes: initializing a first learnable weight matrix for adjusting the weights of deep information; initializing a second learnable weight matrix for adjusting the weights of semantic information; performing element-wise multiplication of the deep attention mask with the first learnable weight matrix to obtain a depth-gated component; performing element-wise multiplication of the semantic attention mask with the second learnable weight matrix to obtain a semantic-gated component; and performing element-wise addition of the depth-gated component and the semantic-gated component, and applying a non-linear saturation activation function to the result of the addition to generate the fused attention map.

5. The similarity processing method based on fusion attention as described in claim 1, characterized in that, Extracting feature maps from the target image and weighting them using the fusion attention map to generate a weighted feature map includes: extracting multi-channel feature data output from the target image during the encoding stage using a feature coding network, and using the multi-channel feature data as the feature map; identifying the number of channel dimensions in the feature map, and copying and extending the single-channel fusion attention map along the channel dimensions to generate an extended attention map whose spatial size and number of channels are consistent with the feature map; performing element-wise multiplication between the extended attention map and the feature map, using the weight values ​​in the extended attention map to perform channel-weighted modulation on the feature map, and using the modulated feature data as the weighted feature map.

6. The similarity processing method based on fusion attention as described in claim 1, characterized in that, The process involves converting the weighted feature map into multiple region feature descriptors and determining multiple region weights corresponding to the multiple regions based on the fused attention map. This includes: dividing the weighted feature map into multiple regular grid-like non-overlapping local regions in a spatial dimension; performing generalized average pooling on the feature vectors of all spatial locations within each non-overlapping local region; performing dimensionality reduction and compression on the feature vectors after generalized average pooling to generate a region feature descriptor corresponding to each non-overlapping local region; determining the average value of the fused attention map at all pixel locations within each non-overlapping local region; and normalizing the average value to generate a region weight corresponding to each non-overlapping local region.

7. The similarity processing method based on fusion attention as described in claim 1, characterized in that, Based on the similarity relationship between the multiple region feature descriptors and the corresponding region feature descriptors of the reference image, and the region weights, the background similarity between the target image and the reference image is determined, including: pairing each region feature descriptor of the target image with the corresponding region feature descriptor of the reference image according to the spatial location correspondence; performing vector dot product processing on each pair of paired region feature descriptors to determine a cosine similarity value, generating a similarity relationship between the paired regions; multiplying the similarity relationship of each paired region with the corresponding region weight to generate a weighted similarity value for the paired regions; summing the weighted similarity values ​​of all paired regions, and using the summed result as the background similarity.

8. A similarity processing device based on fusion attention, characterized in that, The similarity processing device based on fusion attention includes: an image input module for acquiring a target image and a reference image to be processed; a depth attention generation module for performing depth estimation processing on the target image to generate a depth map, and generating a depth attention mask based on the depth map; a semantic attention generation module for performing semantic segmentation processing on the target image to generate a semantic probability map, and generating a semantic attention mask based on the semantic probability map; an attention fusion module for fusing the depth attention mask and the semantic attention mask to generate a fused attention map; a feature weighting encoding module for extracting feature maps from the target image, and using the fused attention map to perform weighting processing on the feature maps to generate a weighted feature map; a region feature modeling module for converting the weighted feature map into multiple region feature descriptors, and determining multiple region weights corresponding to the multiple regions based on the fused attention map; a similarity calculation module for determining the background similarity between the target image and the reference image based on the similarity relationship between the multiple region feature descriptors and the corresponding region feature descriptors of the reference image, as well as the region weights; and a task decision module for executing a target task based on the background similarity to obtain the target task result.

9. A computer device, characterized in that, The computer device includes a memory, a processor, and a similarity processing program based on fusion attention stored in the memory and executable on the processor, wherein when executed by the processor, the similarity processing program based on fusion attention implements the steps of the similarity processing method based on fusion attention as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a similarity processing program based on fusion attention, which, when executed by a processor, implements the steps of the similarity processing method based on fusion attention as described in any one of claims 1-7.

Images