A target recognition method based on multi-scale fusion of occlusion perception relationship

Abstract

The application discloses a target recognition method based on multi-scale fusion of occlusion perception relationship, comprising the following steps: acquiring a to-be-detected image and extracting feature maps of the to-be-detected image with at least two scales and consistent channel numbers; performing global and local feature enhancement operations on the feature maps to obtain an enhanced feature sequence; extracting a physical occlusion relationship feature vector of a target in the enhanced feature sequence based on an intersection-over-union ratio, a depth sequence and a scale difference, and generating a comprehensive attention bias matrix according to the physical occlusion relationship feature vector; superimposing the comprehensive attention bias matrix on a similarity matrix of self-attention to perform self-attention interaction of fusion of occlusion perception relationship; performing multi-scale cross-feature sampling with a reference bounding box of a previous layer as a reference, inputting the multi-scale cross-feature sampling into classification and regression branches after nonlinear mapping to perform multi-task decoupling prediction and coordinate iterative updating, and outputting target categories and bounding box coordinates layer by layer. By using the method, the target identity confusion and missed detection problems in a serious dense occlusion scene are effectively solved, and the recall rate and detection precision are stably improved.

Description

Technical Field

[0001] This invention belongs to the field of target recognition technology, and specifically relates to a target recognition method based on multi-scale fusion occlusion perception relationship. Background Technology

[0002] Object recognition, as one of the core tasks in computer vision, aims to automatically locate and identify predefined categories of object instances from images or videos. In recent years, with the rapid development of artificial intelligence technology, object recognition algorithms have been widely applied in real-world scenarios such as autonomous driving, intelligent security monitoring, and smart cities. However, the real physical world is full of high uncertainty and dynamic changes, among which "object occlusion" and "dense overlap" are the core bottlenecks restricting the robustness and reliability of existing visual perception systems.

[0003] Existing target recognition technologies have mainly evolved from convolutional neural networks to end-to-end detection architectures based on Transformers. However, when dealing with complex occlusion environments, they still have the following significant technical shortcomings:

[0004] 1. Local discriminative features are easily lost and there is a problem of "sampling point drift".

[0005] In traditional CNN-based object detectors, when an object is severely occluded, its exposed visible area is extremely small. Traditional local convolutional operations easily confuse these minute discriminative features, such as edges and textures, with background noise or occlusion features. To overcome the limitations of the local receptive field in CNNs, Transformer-based detection models have introduced deformable attention mechanisms in recent years. However, this mechanism heavily relies on the query vector to predict the offset of sampling points. When the reference point of the object is completely covered by occlusions, the model is prone to "blind prediction," leading to severe "sampling point drift" during feature extraction. The sampling points cannot accurately capture the tiny exposed fragments of the object, resulting in the failure of key feature extraction, which in turn leads to inaccurate localization or direct missed detection.

[0006] 2. "Query conflicts" and identity confusion caused by the lack of explicit physical relationship reasoning.

[0007] In dense scenes with multiple targets occluding each other and having a high degree of overlap, the model not only needs to extract features but also needs to clarify the spatial overlap relationship between targets. Most existing Transformer detection decoders rely on "implicit content similarity" between query vectors for interaction modeling. This implicit interaction, purely based on feature similarity, cannot effectively distinguish between the geometric positional relationship of targets and physical occlusion. The lack of reasoning ability regarding physical occlusion relationships makes the model prone to "query conflicts" when processing highly overlapping bounding boxes. For example, when the features of two targets highly overlap, the model often incorrectly suppresses the response of the occluded target in the background, resulting in severe identity confusion and even causing a large number of valid targets to be lost during the decoding stage, significantly reducing the recall rate in dense scenes.

[0008] In summary, existing target recognition algorithms struggle to balance the accuracy of feature extraction with the recall rate of multi-target recognition when dealing with complex and densely occluded scenes. Summary of the Invention

[0009] To address the challenge of balancing the accuracy of feature extraction with the recall rate of multi-target recognition in existing methods, this invention provides a target recognition method based on multi-scale fusion of occlusion perception relationships.

[0010] The objective of this invention is achieved through the following technical solution:

[0011] This invention provides a target recognition method based on multi-scale occlusion perception relationship fusion, comprising the following steps:

[0012] Acquire the image to be detected and extract feature maps of the image to be detected at least two scales with the same number of channels;

[0013] The feature map is subjected to global and local feature enhancement operations to obtain an enhanced feature sequence;

[0014] Based on the cross-union ratio, depth order, and scale difference, the physical occlusion relationship feature vector of the target in the enhanced feature sequence is extracted, and a comprehensive attention bias matrix is ​​generated based on the physical occlusion relationship feature vector.

[0015] The comprehensive attention bias matrix is ​​superimposed on the self-attention similarity matrix to perform self-attention interaction that fuses occlusion perception relationships; multi-scale cross-feature sampling is performed based on the reference bounding box of the previous layer, and after nonlinear mapping, it is input to the classification and regression branches for multi-task decoupling prediction and coordinate iterative update, and the target category and bounding box coordinates are output layer by layer.

[0016] Compared with the prior art, the present invention has at least the following advantages and beneficial effects:

[0017] The method of this invention extracts multi-scale feature maps from the image to be detected, and explicitly quantizes physical occlusion biases based on the intersection-union ratio, depth order, and scale differences between targets. In the occlusion-aware relationship decoder, these physical occlusion biases are fused with geometric position biases and injected into the self-attention calculation, enabling the model to actively perceive the physical occlusion logic between targets, thereby outputting accurate target bounding boxes and categories. This method effectively solves the problems of target identity confusion and missed detection in severely dense occlusion scenes, achieving a robust improvement in recall and detection accuracy. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0021] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0022] It should be noted that, unless otherwise specified, the embodiments and features described in this invention can be combined with each other.

[0023] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0024] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of this invention is in use, or the orientation or positional relationship commonly understood by those skilled in the art. They are only used for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention. In addition, the terms "first," "second," etc., are only used to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0025] In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0026] This invention discloses a target recognition method based on multi-scale occlusion perception relationships. This method is primarily aimed at practical applications such as autonomous driving and intelligent security monitoring. Addressing the common challenges of mutual occlusion, dense overlap, and environmental noise interference, it provides an end-to-end target recognition scheme with explicit physical occlusion relationship reasoning capabilities and the ability to effectively capture local discriminative features. Through in-depth mining of multi-scale image features and quantitative modeling of the geometric logic between targets, this invention aims to improve the robustness and recognition accuracy of visual perception systems in complex open environments. Specifically, such as... Figure 1 As shown, this method includes steps S1 to S4.

[0027] Step S1: Obtain the image to be detected and extract feature maps of the image to be detected at least two scales with the same number of channels.

[0028] This step aims to extract multi-resolution features with rich semantics and local details from the original input image, providing a high-quality feature base for subsequent attention encoding. Specifically, this step includes steps S11 to S12.

[0029] Step S11: Input the image to be detected into the feature extraction network and extract the original feature maps of at least two levels during the downsampling process of the feature extraction network.

[0030] The image to be detected can come from scenarios such as autonomous driving roads and densely populated security areas. For RGB images, denoted as ,in, This indicates that the values ​​in the tensor belong to the set of real numbers. Indicates the height of the image to be detected. This represents the width of the image to be detected, and 3 represents the number of color channels. After preprocessing such as standardization, the image is input into the feature extraction network. The feature extraction network can be a deep residual network, including but not limited to ResNet-34, ResNet-50, and ResNet-101. This example uses ResNet-50, pre-trained on ImageNet, as the feature map extraction network.

[0031] The original feature map has at least two levels, preferably three levels. This allows for comprehensive coverage of targets of different sizes, from small to very large, ensuring the stability of multi-scale feature fusion while effectively avoiding redundant computation and noise interference from shallow features. It is the optimal solution balancing real-time inference performance and detection accuracy. This approach leverages deep features to extract global contextual semantics to address occlusion, while preserving local details through mid-to-shallow features, thus accurately capturing degradation features at different frequencies.

[0032] Step S12, adopt The convolutional layer performs dimensionality reduction on the original feature map to obtain a feature map with a uniform number of channels. ,in,

[0033] ,

[0034] In the formula, The hierarchy of a deep residual network; For the backbone network The original feature map output by the layer.

[0035] For example, let's take the extraction of feature maps at three scales as an example:

[0036] First, extract the original feature maps output by the ResNet-50 network in stages 3, 4, and 5 of the downsampling process, and denote them as the first original feature maps. Second original feature map and the third original feature map These feature maps have different spatial resolutions and receptive fields, enabling them to cover representations ranging from local details to global semantics. First original feature map. Second original feature map and the third original feature map The spatial dimensions are 1 / 8, 1 / 16, and 1 / 32 of the image to be detected, respectively. C3 captures high-resolution local edges and details, with dimensions of... The number of channels is 512. C4 captures features of a medium receptive field, with a size of [missing information]. The number of channels is 1024. C5 captures high-level features with lower resolution but the richest semantic information, with a size of [size missing]. The number of channels is 2048.

[0037] To adapt to the unified computation dimension of the subsequent Transformer architecture, the following approach is adopted. convolutional layers , , Channel dimensionality reduction and alignment are performed to obtain feature maps with a uniform number of channels. .

[0038] In this embodiment, the number of unified channels after dimensionality reduction is set to . Feature maps of each layer ( The corresponding spatial dimensions are Finally, a sine-cosine two-dimensional spatial location code is applied, which is then flattened and input into a multi-scale local attention encoder.

[0039] Step S2: Perform global and local feature enhancement operations on the feature map to obtain an enhanced feature sequence.

[0040] This step addresses the issues of "sampling point drift" and smoothing of minute features in traditional models under occluded environments. It achieves "content-aware" sampling and discriminative amplification of features through a dual mechanism of local geometric refinement and local pyramid attention. Specifically, this step includes steps S21 to S26.

[0041] Step S21: Use the Local Pyramid Attention Module (MLP) to perform global and local attention enhancement operations on the feature map X to obtain an aggregated feature map.

[0042] This step preferably involves bi-branch computation on the feature map X, a method based on channel attention. and spatial attention Implementation, including channel attention To process the feature map X using a multilayer perceptron (MLP) with the results of average pooling (AvgPool) and max pooling (MaxPool), the activation values ​​are multiplied by the input, i.e.:

[0043] ,

[0044] In the formula, This represents the activation function.

[0045] Spatial attention is equivalent to channel attention The enhanced features are then subjected to average pooling and max pooling again, concatenated, and then subjected to a convolution operation Conv() to calculate spatial weights. Finally, they are combined with channel attention. The enhanced features are multiplied together, that is:

[0046] .

[0047] For the sake of brevity, this set of continuous attention enhancement operations is abstractly defined as a function. :

[0048] .

[0049] Based on this, the specific implementation method of this step includes steps S211 to S213.

[0050] Step S211: Perform attention enhancement operation on the feature map to obtain semantic consistency features.

[0051] This path does not change the feature map size; it directly performs the aforementioned attention enhancement operation on the complete feature map X, resulting in semantically consistent features. ,in:

[0052] ,

[0053] This semantic consistency feature It includes global context and overall category semantics.

[0054] Step S212: Divide the feature map into at least two local blocks, perform attention enhancement operation on each local block, and then stitch them together to obtain detailed features.

[0055] This is a crucial step in anti-occlusion. The system forces the network to focus its attention on tiny, visible, discriminative fragments through fine-grained local perception.

[0056] Specifically, this step first segments the feature map in the spatial dimension to obtain at least two local blocks; then, attention enhancement is performed on each local block to obtain an independently enhanced local block; finally, the independently enhanced local blocks are spliced ​​together to obtain the detail features.

[0057] For example, the feature map X can be divided equally along the H and W spatial dimensions. Taking four local blocks as an example,

[0058] 4 local blocks Independent attention enhancement operations are performed to uncover local high-frequency responses, resulting in independently enhanced local blocks. for:

[0059] .

[0060] The final detailed features for:

[0061] ,

[0062] In the formula, This is for splicing operations.

[0063] Step S213: Add the semantic consistency features and detail features element by element to obtain the aggregated feature map of the feature map.

[0064] This step extracts the semantic consistency features from the global path. Detail features magnified by local path Element-by-element addition is performed to complete the final aggregation of the feature pyramid, resulting in the final aggregated feature map. for:

[0065] .

[0066] Through this extraction and enhancement process, the network not only identifies the overall outline of the target from a macroscopic perspective, but also amplifies the discriminative features that are still exposed under severe occlusion from a microscopic perspective.

[0067] In the example above, after this step, Where L=3, and the dimensions of each layer are: , , ,

[0068] In the formula, Represents aggregated feature maps No. Layer feature map height, Represents aggregated feature maps No. Layer feature map width, For the number of channels, ∈1,2,3.

[0069] Step S22: Perform matrix transformation operation on the aggregated feature map to obtain a feature sequence, which is composed of the feature vector corresponding to each pixel in the feature map.

[0070] Specifically, aggregate feature maps The multi-scale feature maps in the model use matrix transformations to change their shape.

[0071] ,

[0072] ,

[0073] ,

[0074] In the formula, It's a matrix transformation operation that concatenates them along the first dimension of space to obtain a huge feature sequence. ,in,

[0075] ,

[0076] .

[0077] Step S23: Calculate the reference point for each feature vector in the feature sequence.

[0078] For feature sequences Each feature vector in , It is necessary to calculate the reference point. , .

[0079] For aggregated feature maps The size of each feature map is , where each feature vector The absolute coordinates of the corresponding pixel are ,in, .

[0080] Center point offset and normalization. To ensure the coordinates strictly fall within... Between these points, and representing the physical center of the pixel, the coordinates are increased by 0.5, then divided by the width and height respectively, resulting in:

[0081] .

[0082] For the location in The feature vector of a network has its initial reference point as its own location. The network predicts offsets around itself and samples valuable features from its surroundings.

[0083] Step S24: Based on the local spatial geometric prior features and corresponding feature vectors within the reference point's domain, predict the corresponding offset.

[0084] Specifically, this step begins at the reference point. Within the neighborhood, a lightweight depthwise separable convolution operator is used. Extracting local geometric prior features ,in,

[0085] .

[0086] Then, the corresponding feature vectors With this local geometric prior features Fusion, predicting the th... through the first fully connected layer Attention At the scale of, for the first Adaptive offset of each sampling point ,in,

[0087] ,

[0088] In the formula, The output dimension is This represents a tensor for generating corresponding [data] points at L scales, with M attention heads sampling K points at each scale. Offset.

[0089] Step S25: Input the feature vector into the second fully connected layer, and normalize it in terms of scale and sampling point dimensions using the Softmax function to predict the attention weights;

[0090] eigenvectors The input is fed into the second fully connected layer, and the output dimension is... The tensor. Then, through the Softmax function... Normalization is performed on the dimension.

[0091] The normalized attention weights satisfy: .

[0092] in, Let m be a scalar, representing the m-th head in the π / 2 ... The first layer Attention weights for each sampling point.

[0093] Step S26, based on the reference point Offset and attention weight Feature sampling and weighted aggregation are performed on multi-scale feature maps using bilinear interpolation, followed by residual connection and nonlinear transformation to obtain an enhanced feature sequence. Specifically, this step includes steps S261 to S262.

[0094] Step S261, based on the reference point Offset and attention weight Enhanced feature vectors are obtained by performing feature sampling and weighted aggregation on multi-scale feature maps through bilinear interpolation.

[0095] Specifically, enhancing feature vectors for:

[0096]

[0097] In the formula, This represents multi-scale variable attention computation. For characteristic sequences The feature vector in for The corresponding reference point The total number of attention heads is usually M=8. Here, L is the index of the attention head being calculated, and L is the scale layer of the feature map. In this example, L=3. This is the index at the current feature layer level. The number of keypoints sampled by each head in each layer. This is the index of the current sampling point.

[0098] Given a learnable weight matrix of size , Its function is to input features The number of channels is compressed to the dimension of a single head.

[0099] This is the scaling function. Because... yes The normalized coordinates are used, but the resolution of feature maps at different scales is different. Restore it to the first Absolute coordinates on the layer feature map.

[0100] These are the sampled feature values. Since the coordinates after adding the offset are likely not integers, bilinear interpolation is used here to extract the feature values ​​from the first... Layer feature map Obtain the feature vector at the corresponding position.

[0101] Given a learnable weight matrix of size , Its function is to map the output feature dimension of a single head back to the original feature dimension.

[0102] In practice, this is equivalent to splicing the outputs of all attention heads together and then performing a linear projection.

[0103] Step S262: Add the feature vector corresponding to each pixel in the feature map to the enhanced feature vector, and obtain the enhanced feature sequence after residual connection and nonlinear transformation.

[0104] Specifically, the matrix-enhanced feature vector obtained after multi-scale variable attention computation With feature vectors The sums are processed and the residuals are normalized using LayerNorm. Then, the result is transformed nonlinearly through two fully connected layers. The output is used as the input to the next layer of the multi-scale variable attention encoder. After multiple iterations of multi-scale variable attention, the final enhanced feature sequence Y is output.

[0105] The iteration here typically involves 6 layers.

[0106] This step introduces a local geometric refinement mechanism to upgrade feature sampling from "blind prediction" to a "content-aware" mode. Compared to traditional deformable attention, this method can automatically correct the sampling position based on the content features of the reference point's neighborhood, enabling the model to avoid semantically ambiguous occlusion centers and accurately focus on visible discriminative regions such as target edges, effectively solving the feature sampling drift problem under severe occlusion.

[0107] The attention module in this step enhances the model's ability to capture subtle features of incomplete targets. By processing through parallel global and local paths, it forces the network to amplify and utilize those small, visible fragments that are easily smoothed out in global computation, while preserving the overall semantic consistency of the target, thus significantly improving the feature discrimination ability in complex overlapping scenes.

[0108] Step S3: Extract the physical occlusion relationship feature vector of the target in the enhanced feature sequence Y based on the intersection-union ratio, depth order, and scale difference, and generate a comprehensive attention bias matrix based on the physical occlusion relationship feature vector. Specifically, this step includes steps S31 to S32.

[0109] Step S31: Extract the physical occlusion relationship feature vectors of the first query vector and the second query vector in the enhanced feature sequence Y.

[0110] The physical occlusion relationship feature vector is determined based on the intersection-union ratio, the logarithm of the ratio of the front-to-back depth of the target under perspective projection, and the target width. Specifically, this step includes steps S311 to S35.

[0111] Step S311: Obtain the first reference bounding box corresponding to the first query vector and the second reference bounding box corresponding to the second query vector.

[0112] In the decoder network, the query vector Query is used to represent a potential target instance. Given any two query vectors output from the previous layer of the decoder, the first query vector is defined as... With the second query vector , and their respective reference bounding boxes and The prediction target includes the first query vector. Second query vector First query vector The corresponding first reference bounding box Second query vector The corresponding second reference bounding box .

[0113] First reference bounding box Coordinates are defined as:

[0114] The coordinates of the top left corner are:

[0115] The coordinates of the bottom right corner are: .

[0116] Correspondingly, the second reference bounding box Coordinates are defined as:

[0117] The coordinates of the top left corner are:

[0118] The coordinates of the bottom right corner are: .

[0119] During model training, bounding boxes can be obtained from the labels in the dataset. The query vector and bounding box vectors are initialized during model training, and then gradually learn a statistical prediction of the spatial distribution of the dataset and the features of the target objects as the model trains. When the model is used, the same initialization process is performed, and then a deep cross-interaction is calculated with the features of the current input image based on the learned parameter weights. Finally, the precise coordinates and categories of all targets in the current image are output in parallel at once.

[0120] Step S312: Calculate the intersection-union ratio (IUR) of the first reference bounding box and the second reference bounding box. .

[0121] The intersection-union ratio (IUR) is an absolute measure of the degree of overlap, defined as the area of ​​the intersection divided by the area of ​​the union.

[0122]

[0123] In the formula, For area calculation.

[0124] for The width of the overlapping area of ​​the two frames and high Strict boundary checks are required:

[0125]

[0126] ,

[0127] In the formula, To obtain the maximum of the two.

[0128] Then the precise intersection area for:

[0129]

[0130] Union area of ​​the two frames It equals the sum of the areas of the two frames minus their overlapping portion:

[0131]

[0132] The original physical area of ​​a single box is calculated as follows:

[0133] First reference bounding box area :

[0134]

[0135] Area of ​​the second reference bounding box :

[0136]

[0137] Step S313: Extract the ordinate of the center point of the first reference bounding box. The ordinate of the center point of the second reference bounding box Calculate the ordinate of the center point of the first reference bounding box. The ordinate of the center point of the second reference bounding box sign function Through symbolic functions Determine the front and back depth order of the target under perspective projection.

[0138] Step S314: Calculate the logarithm of the ratio of the width of the first reference bounding box and the second reference bounding box. This is to distinguish the scale difference between foreground occluders and background occluded targets.

[0139] In this case, foreground occlusions usually appear larger.

[0140] The width of the first reference bounding box. The width of the second reference bounding box.

[0141] Step S315: Based on the intersection-union ratio , and Determine the physical occlusion relationship feature vector ,in,

[0142]

[0143] For non-zero decimals, To prevent The smoothed minimum value.

[0144] Step S32: Convert the physical occlusion relationship feature vector The first physical occlusion relationship bias matrix is ​​mapped to the second physical occlusion relationship bias matrix. The first physical occlusion relationship bias matrix and the second physical occlusion relationship bias matrix are weighted and fused to obtain the comprehensive attention bias matrix.

[0145] Specifically, this step includes steps S321 to S324.

[0146] Step S321: Convert the physical occlusion relationship feature vector The input to a multilayer perceptron (MLP) is mapped to a high-dimensional first physical occlusion relation bias matrix. .

[0147] Step S322: Quantize the physical location relationship between the first query vector and the second query vector to obtain the quantization result. ,in,

[0148] ,

[0149] in, The x-coordinate of the top left corner of the first reference bounding box. The x-coordinate of the top left corner of the second reference bounding box. The y-coordinate of the top left corner of the first reference bounding box. The ordinate of the top left corner of the second reference bounding box. The height of the first reference bounding box. The width of the first reference bounding box. The height of the second reference bounding box, The width of the second reference bounding box.

[0150] Step S323: Quantify the results The input to the multilayer perceptron (MLP) is mapped to a high-dimensional second physical occlusion relation bias matrix. .

[0151] Second physical occlusion relationship bias matrix This is a traditional spatial geometric position offset.

[0152] Step S324, in the In the layer decoder, the first physical occlusion bias matrix and the second physical occlusion bias matrix are combined. Weighted fusion is performed to obtain the comprehensive attention bias matrix. ,in,

[0153] ,

[0154] In the formula, , For the first The scalar weights that the layer decoder can learn.

[0155] As the number of decoder layers increases, , This enables the model to adaptively transition from focusing on the basic spatial distribution to concentrating on inference with complex occlusion logic, and integrates this attention bias matrix. This is injected into the interaction process of the query vector, thereby guiding the network to output accurate bounding boxes and categories.

[0156] This step quantizes the attention bias based on three physical signals: intersection-over-union ratio, depth order, and scale difference, breaking the limitation of traditional decoders that rely solely on implicit content similarity. This mechanism enables the model to actively distinguish between "foreground occluders" and "background occluders," fundamentally suppressing query conflicts and identity confusion in dense scenes.

[0157] Step S4: The integrated attention bias matrix generated in step S3 is superimposed on the self-attention similarity matrix to perform self-attention interaction for fusing occlusion perception relationship; multi-scale cross-feature sampling is performed based on the reference bounding box of the previous layer, and after nonlinear mapping, it is input to the classification and regression branches for multi-task decoupling prediction and coordinate iterative update, and the target category and bounding box coordinates are output layer by layer.

[0158] After feature interaction and relational inference by the decoder, the final recognition result is output using the task detection head. The decoder contains multiple cascaded refinement layers, given an enhanced feature sequence Y output by the encoder (Y being multi-scale memory features) and an initialized target query feature vector. and the corresponding initial reference bounding box The data is in the first... The specific transfer steps of the refining layer include steps S41 to S44.

[0159] Step S41: Map the query feature vector output from the previous layer into a query matrix, a key matrix, and a value matrix; then map the integrated attention bias matrix generated in step S3. The semantic similarity between the query matrix and the key matrix is ​​calculated by superimposing a similarity matrix on the self-attention matrix. The activated semantic similarity is then multiplied by the value matrix to obtain the query feature vector enhanced by self-attention.

[0160] Step S42: Using the center coordinates of the first and second reference bounding boxes in the previous layer as sampling reference points, the sampling offset and attention weight of the enhanced feature sequence are predicted using the query feature vector after self-attention enhancement. Based on the sampling reference points and sampling offsets, local heterogeneous feature sampling and weighted aggregation are performed in the enhanced feature sequence to obtain the cross-attention feature vector.

[0161] Specifically, local heterogeneous feature sampling can be performed using bilinear interpolation.

[0162] This step enables deep interaction between target-level features and the global multi-scale environmental context.

[0163] Step S43: After inputting the cross-attention feature vector into the feedforward neural network FFN for dimensional expansion and nonlinear activation, the output is the first... The final refined query feature vector of the layer.

[0164] Step S44: Input the refined query feature vector into the classification prediction branch and the regression prediction branch respectively, so as to output the confidence score of the category corresponding to the query vector and the coordinate offset of the bounding box corresponding to the query vector.

[0165] Specifically, the classification prediction branch outputs the confidence scores for each category of the query vector through a fully connected layer.

[0166] The regression prediction branch predicts the coordinate offset relative to the current reference bounding box using a multilayer perceptron (MLP). The reference bounding box of the previous layer With the offset Add them together to generate the updated version. Layer reference bounding box .

[0167] The proposed method significantly reduces the false negative rate for densely overlapping targets while maintaining high accuracy. Experiments demonstrate that this method achieves performance breakthroughs on highly challenging occlusion datasets such as KITTI and CrowdHuman, particularly excelling in high occlusion metrics. This invention possesses strong environmental adaptability and robustness, providing reliable technical support for the safety of downstream perception tasks such as autonomous driving and intelligent security.

[0168] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A target recognition method based on multi-scale occlusion perception relationship fusion, characterized in that, Includes the following steps: Acquire the image to be detected and extract feature maps of the image to be detected at least two scales with the same number of channels; The feature map is subjected to global and local feature enhancement operations to obtain an enhanced feature sequence; Based on the cross-union ratio, depth order, and scale difference, the physical occlusion relationship feature vector of the target in the enhanced feature sequence is extracted, and a comprehensive attention bias matrix is ​​generated based on the physical occlusion relationship feature vector. The comprehensive attention bias matrix is ​​superimposed on the self-attention similarity matrix to perform self-attention interaction that fuses occlusion perception relationships; Based on the reference bounding box of the previous layer, multi-scale cross-feature sampling is performed. After nonlinear mapping, the sample is input to the classification and regression branches for multi-task decoupling prediction and coordinate iterative update. The target category and bounding box coordinates are output layer by layer.

2. The target recognition method based on multi-scale occlusion perception relationship according to claim 1, characterized in that: The extraction of the multi-scale feature map of the image to be detected includes: The image to be detected is input into the feature extraction network, and the original feature maps of at least two levels of the feature extraction network are extracted during the downsampling process; use The convolutional layer performs dimensionality reduction on the original feature map to obtain a feature map with a uniform number of channels. ,in, , In the formula, The hierarchy of a deep residual network; For the backbone network The original feature map output by the layer. Indicates to proceed Convolution processing.

3. The target recognition method based on multi-scale occlusion perception relationship according to claim 1, characterized in that: The process of performing global and local feature enhancement operations on the feature map to obtain an enhanced feature sequence includes: The feature map X is subjected to global and local attention enhancement operations using a Local Pyramid Attention Module (MLP) to obtain an aggregated feature map. A matrix transformation operation is performed on the aggregated feature map to obtain a feature sequence, which is composed of the feature vector corresponding to each pixel in the feature map. Calculate the reference point for each feature vector in the feature sequence; The corresponding offset is predicted based on the local geometric prior features of the reference point and the corresponding feature vector. The feature vector is input into the second fully connected layer and normalized in the scale and sampling point dimensions by the Softmax function to predict the attention weights. Based on the reference point, offset, and attention weight, feature sampling and weighted aggregation are performed on the multi-scale feature map through bilinear interpolation, and the enhanced feature sequence is obtained after residual connection and nonlinear transformation.

4. The target recognition method based on multi-scale occlusion perception relationship according to claim 3, characterized in that: The step of using a Local Pyramid Attention Module (MLP) to perform global and local attention enhancement operations on the feature map X includes: Attention enhancement is performed on the feature map to obtain semantic consistency features. This attention enhancement is based on channel attention and spatial attention. Spatial attention concatenates the channel-attention-enhanced features after average pooling and max pooling, calculates spatial weights through convolution, and finally combines them with the channel attention. Enhanced features are multiplied; The feature map is divided into at least two local blocks, and attention enhancement operations are performed on each local block before they are stitched together to obtain detailed features. The semantic consistency features and detail features are added element by element to obtain the aggregated feature map of the feature map.

5. The target recognition method based on multi-scale occlusion perception relationship according to claim 3, characterized in that: The prediction of the corresponding offset based on the local geometric prior features in the reference point's domain and the corresponding feature vector includes: At the reference point Within the neighborhood, a depthwise separable convolution operator is used to extract local geometric prior features; The local geometric prior features are fused with the corresponding feature vectors, and the corresponding offset is predicted based on the fully connected layer.

6. The target recognition method based on multi-scale occlusion perception relationship according to claim 3, characterized in that: The enhanced feature sequence is obtained by performing feature sampling and weighted aggregation on a multi-scale feature map using bilinear interpolation based on the reference point, offset, and attention weights, followed by residual connection and nonlinear transformation. This includes: Based on the reference point Offset and attention weight Enhanced feature vectors are obtained by performing feature sampling and weighted aggregation on multi-scale feature maps through bilinear interpolation; The feature vector corresponding to each pixel in the feature map is added to the enhanced feature vector, and the enhanced feature sequence is obtained after residual connection and nonlinear transformation.

7. The target recognition method based on multi-scale occlusion perception relationship according to claim 1, characterized in that: The step of extracting physical occlusion relationship feature vectors of targets in the enhanced feature sequence based on intersection-over-union ratio, depth order, and scale difference, and generating a comprehensive attention bias matrix based on the physical occlusion relationship feature vectors, includes: Based on the intersection-union ratio, depth order, and scale difference, the physical occlusion relationship feature vector of the first query vector and the second query vector in the enhanced feature sequence is extracted; The physical occlusion relationship feature vector is mapped to a first physical occlusion relationship bias matrix. The first physical occlusion relationship bias matrix and the second physical occlusion relationship bias matrix are weighted and fused to obtain a comprehensive attention bias matrix.

8. The target recognition method based on multi-scale occlusion perception relationship according to claim 7, characterized in that: The physical occlusion relationship feature vectors extracted from the first and second query vectors in the enhanced feature sequence based on intersection-over-union ratio, depth order, and scale difference include: Obtain the first reference bounding box corresponding to the first query vector and the second reference bounding box corresponding to the second query vector; Calculate the intersection-union ratio (CUI) of the first and second reference bounding boxes. ; Extract the ordinate of the center point of the first reference bounding box The ordinate of the center point of the second reference bounding box Calculate the ordinate of the center point of the first reference bounding box. The ordinate of the center point of the second reference bounding box sign function ; Calculate the logarithm of the ratio of the width of the first reference bounding box to that of the second reference bounding box. ; based on , and The physical occlusion relationship feature vectors are determined.

9. The target recognition method based on multi-scale occlusion perception relationship according to claim 1, characterized in that: The physical occlusion relationship feature vector The mapping is to a first physical occlusion relation bias matrix. The first and second physical occlusion relation bias matrices are then weighted and fused to obtain a comprehensive attention bias matrix, including: physical occlusion relationship feature vector The input to a multilayer perceptron (MLP) is mapped to a high-dimensional first physical occlusion relation bias matrix. ; The physical location relationship between the first query vector and the second query vector is quantized to obtain the quantization result. ; The quantization result The input to the multilayer perceptron (MLP) is mapped to a high-dimensional second physical occlusion relation bias matrix. ; In the In the layer decoder, the first physical occlusion bias matrix and the second physical occlusion bias matrix are combined. Weighted fusion is performed to obtain the comprehensive attention bias matrix. .

10. The target recognition method based on multi-scale occlusion perception relationship according to claim 1, characterized in that: The self-attention interaction that fuses occlusion perception relationship is achieved by superimposing the comprehensive attention bias matrix on the self-attention similarity matrix. Based on the reference bounding box of the previous layer, multi-scale cross-feature sampling is performed. After nonlinear mapping, the sample is input to the classification and regression branches for multi-task decoupled prediction and coordinate iterative update. The target category and bounding box coordinates are output layer by layer, including: The query feature vector output from the previous layer is mapped to a query matrix, a key matrix, and a value matrix; then the comprehensive attention bias matrix is... The semantic similarity between the query matrix and the key matrix is ​​calculated by superimposing the similarity matrix on the self-attention matrix. The activated semantic similarity is then multiplied by the value matrix to obtain the query feature vector enhanced by self-attention. Using the center coordinates of the first and second reference bounding boxes in the previous layer as sampling reference points, the sampling offset and attention weight of the enhanced feature sequence are predicted using the query feature vector enhanced by self-attention. Based on the sampling reference points and sampling offsets, local heterogeneous feature sampling and weighted aggregation are performed in the enhanced feature sequence to obtain the cross-attention feature vector. The cross-attention feature vector is input into the feedforward neural network FFN for dimensional expansion and nonlinear activation, and then a refined query feature vector is output. The refined query feature vector is input into the classification prediction branch and the regression prediction branch respectively, so as to output the confidence score of the category corresponding to the query vector and the coordinate offset of the bounding box corresponding to the query vector.

Images