A target recognition method based on fused semantic features

Abstract

This application discloses a target recognition method based on fused semantic features. The method obtains a target infrared image and its corresponding maneuver category, where the target infrared image contains the target. The target infrared image and maneuver category are input into a target recognition network to obtain first maneuver semantic features, shallow image features, and deep image features. The first maneuver semantic features and shallow image features are fused to obtain a first image fusion feature. The first maneuver semantic features and deep image features are then fused to obtain a second image fusion feature. The first and second image fusion features are upsampled and concatenated to obtain a multi-scale feature representation. This multi-scale feature representation is input into the output head for convolution to obtain the target category and anti-interference recognition result. This improves the accuracy of target recognition.

Description

Technical Field

[0001] This application relates to the field of target recognition technology, and in particular to a target recognition method based on fused semantic features. Background Technology

[0002] In infrared target recognition tasks, aerial infrared targets often experience short-term occlusion, either partially or over a large area, due to maneuvering and interference. This results in incomplete target features in the image, making it difficult for algorithms to reliably identify targets. Furthermore, in the presence of interference, the morphology and some features of the target exhibit high similarity, increasing the likelihood of false positives and false negatives in the aforementioned recognition algorithms and networks. Therefore, there is an urgent need for a target recognition method that can effectively address the problems of partial target occlusion, high similarity between the target and interference leading to recognition instability, and decreased accuracy in complex area-source interference scenarios. Summary of the Invention

[0003] In view of this, this application provides a target recognition method based on fused semantic features to solve the problems of target partial occlusion, target-interference similarity and decreased accuracy caused by the similarity between the target and the interference in complex area source interference scenarios in the prior art.

[0004] The objective of this application can be achieved through the following technical solutions:

[0005] The first aspect of this application is to provide a target recognition method based on fused semantic features, including:

[0006] Acquire the target infrared image and the corresponding maneuver category; the target infrared image contains the target.

[0007] The target infrared image and maneuver category are input into the target recognition network to obtain the first maneuver semantic features, shallow image features, and deep image features;

[0008] The first image fusion feature is obtained by fusing the first motion semantic features and the shallow image features;

[0009] The first semantic features and the deep image features are fused to obtain the second image fusion features;

[0010] Upsampling and stitching operations are performed on the first image fusion feature and the second image fusion feature to obtain a multi-scale feature representation;

[0011] Multi-scale feature representations are input into the output header for convolution operations to obtain target category and anti-interference recognition results.

[0012] In one optional embodiment, the target recognition network includes a learnable maneuver semantic feature space, an ERBlock module, and a SimSPFF module. The target infrared image and maneuver category are input into the target recognition network to obtain first maneuver semantic features, shallow image features, and deep image features, including:

[0013] Based on the maneuver category, determine the first maneuver semantic feature corresponding to the maneuver category in the learnable maneuver semantic feature space;

[0014] Shallow image features are obtained using the ERBlock module;

[0015] Deep image features are obtained using the SimSPFF module.

[0016] In one optional embodiment, determining a first maneuver semantic feature corresponding to a maneuver category in a learnable maneuver semantic feature space, based on the maneuver category, includes:

[0017] Based on the maneuver category, the first maneuver semantic feature corresponding to the maneuver category is determined in the learnable maneuver semantic feature space using the following formula:

[0018]

[0019] in, The first semantic feature of the maneuver category. For motor vehicle category, For a learnable, mobile semantic feature space, , Represents the real number field. For the number of motor categories, Let be the row number of the feature matrix corresponding to the target infrared image. Let be the number of columns in the feature matrix corresponding to the infrared image of the target. This represents the operation of extracting the first maneuver semantic feature from the learnable maneuver semantic feature space based on the maneuver category.

[0020] In an optional embodiment, the target recognition network further includes UAFFM, which fuses the first maneuver semantic features and shallow image features to obtain a first image fusion feature, including:

[0021] The first image fusion feature is obtained by fusing the first maneuver semantic features and shallow image features using UAFFM.

[0022] In one optional embodiment, UAFFM includes convolutional layers, spatial attention units, channel attention units, and Fourier units. UAFFM fuses first motion semantic features and shallow image features to obtain first image fusion features, including:

[0023] The number of channels of the first motion semantic feature is adjusted to be the same as the number of channels of the shallow image feature by using a convolutional layer to obtain the second motion semantic feature;

[0024] Upsampling is performed on the second maneuver semantic feature to adjust its size to be the same as the size of the shallow image feature, thus obtaining the third maneuver semantic feature;

[0025] The third motion semantic features and shallow image features are input into the spatial attention unit to obtain the first spatial attention weight and the second spatial attention weight. The sum of the first spatial attention weight and the second spatial attention weight is 1.

[0026] The third dynamic semantic feature and shallow image feature are input into the channel attention unit to obtain the first channel attention weight and the second channel attention weight. The sum of the first channel attention weight and the second channel attention weight is 1.

[0027] The attention-weighted features are obtained by adding the products of the third maneuver semantic features and the first spatial attention weight, the third maneuver semantic features and the first channel attention weight, the shallow image features and the second spatial attention weight, and the shallow image features and the second channel attention weight.

[0028] The attention-weighted features are input into the Fourier unit to obtain the first image fusion features.

[0029] In one optional embodiment, attention-weighted features are input into a Fourier unit to obtain first image fusion features, including:

[0030] The attention-weighted features are input into the Fourier unit, and a Fourier transform operation is performed on the attention-weighted features to obtain the first frequency domain attention-weighted features.

[0031] The first frequency domain attention-weighted features are processed in the frequency domain to obtain the second frequency domain attention-weighted features;

[0032] Perform an inverse Fourier transform operation on the second frequency domain attention-weighted features to obtain the first image fusion features.

[0033] In one alternative embodiment, frequency domain processing includes at least one of the following operations:

[0034] Convolution operations, batch normalization operations, and operations that apply activation functions.

[0035] In an optional embodiment, the target recognition network further includes CASFM, which fuses the first maneuver semantic features and deep image features to obtain a second image fusion feature, including:

[0036] The first motor semantic features and the deep image features are fused using CASFM to obtain the second image fusion features.

[0037] In one optional embodiment, CASFM includes a convolutional layer, a first cross-attention unit, and a second cross-attention unit. The first cross-attention unit includes a first key weight matrix, a first query weight matrix, and a first value weight matrix. The second cross-attention unit includes a second key weight matrix, a second query weight matrix, and a second value weight matrix. CASFM is used to fuse first dynamic semantic features and deep image features to obtain second image fusion features, including:

[0038] The number of channels of the first motion semantic feature is adjusted to be the same as the number of channels of the deep image feature by using a convolutional layer, thus obtaining the fourth motion semantic feature;

[0039] The fourth mobile semantic features are input into the first key weight matrix, the first query weight matrix, and the first value weight matrix, respectively, to obtain the first key vector, the first query vector, and the first value vector.

[0040] The deep image features are input into the second key weight matrix, the second query weight matrix, and the second value weight matrix, respectively, to obtain the second key vector, the second query vector, and the second value vector.

[0041] Using deep image features as query features, cross-attention calculation is performed based on the second query vector, the first key vector, and the first value vector using the scaling dot product attention mechanism formula to obtain the first attention output feature;

[0042] The fourth mobile semantic feature is used as the query feature. Based on the first query vector, the second key vector and the second value vector, cross attention is calculated using the scaling dot product attention mechanism formula to obtain the second attention output feature.

[0043] The first attention output feature and the fourth maneuver semantic feature are added together to obtain the third image fusion feature;

[0044] The second attention output feature and the deep image feature are added together to obtain the fourth image fusion feature;

[0045] The third and fourth image fusion features are concatenated to obtain the second image fusion feature.

[0046] In an optional embodiment, CASFM further includes convolutional block units, and also includes:

[0047] The second image fusion features are stacked and convolved using convolutional block units.

[0048] A second aspect of this application is to provide a target recognition device based on fused semantic features, comprising:

[0049] The acquisition module is used to acquire the infrared image of the target and the corresponding maneuver category of the infrared image of the target, wherein the infrared image of the target contains the target;

[0050] The first input module is used to input the target infrared image and maneuver category into the target recognition network to obtain the first maneuver semantic features, shallow image features and deep image features;

[0051] The first feature fusion module is used to fuse the first mobile semantic features and shallow image features to obtain the first image fusion features;

[0052] The second feature fusion module is used to fuse the first mobile semantic features and deep image features to obtain the second image fusion features.

[0053] The sampling and stitching module is used to perform upsampling and stitching operations on the first image fusion feature and the second image fusion feature to obtain a multi-scale feature representation;

[0054] The second input module is used to input multi-scale feature representations into the output head for convolution operations to obtain target category and anti-interference recognition results.

[0055] A third aspect of this application is to provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the method as described in the first aspect.

[0056] A fourth aspect of this application is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the method as described in the first aspect.

[0057] Compared with existing technologies, the target recognition method based on fused semantic features provided in this application obtains a target infrared image and its corresponding maneuver category, wherein the target infrared image contains the target; the target infrared image and maneuver category are input into a target recognition network to obtain first maneuver semantic features, shallow image features, and deep image features; the first maneuver semantic features and shallow image features are fused to obtain a first image fusion feature; the first maneuver semantic features and deep image features are fused to obtain a second image fusion feature; the first image fusion feature and the second image fusion feature are upsampled and stitched together to obtain a multi-scale feature representation; the multi-scale feature representation is input into the output head for convolution operation to obtain the target category and anti-interference recognition result. This improves the accuracy of target recognition. Attached Figure Description

[0058] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1 A flowchart of a target recognition method based on fused semantic features provided in an embodiment of this application;

[0060] Figure 2 A schematic diagram illustrating a process for inputting a maneuver category into a learnable maneuver semantic feature space to obtain a first maneuver semantic feature, provided for an embodiment of this application;

[0061] Figure 3 This is a schematic diagram of a process for fusing first maneuver semantic features and shallow image features using UAFFM to obtain first image fused features, as provided in an embodiment of this application.

[0062] Figure 4 This is a schematic diagram of a process for fusing first motor semantic features and deep image features using CASFM to obtain second image fusion features, as provided in an embodiment of this application.

[0063] Figure 5 A structural block diagram of a target recognition device based on fused semantic features provided in an embodiment of this application;

[0064] Figure 6 This is a structural block diagram of an electronic device for implementing a target recognition method based on fused semantic features, provided in an embodiment of this application. Detailed Implementation

[0065] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0066] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0067] It should be understood that in the embodiments of this application, "at least one" means one or more, and "more than one" means two or more. "And / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. The character " / " generally indicates that the related objects before and after it are in an "or" relationship. "Contains A, B and / or C" means containing any one, two, or three of A, B, and C.

[0068] It should be understood that in the embodiments of this application, "B corresponding to A", "B corresponding to A", "A corresponds to B" or "B corresponds to A" means that B is associated with A, and B can be determined based on A. Determining B based on A does not mean that B is determined solely based on A; B can also be determined based on A and / or other information.

[0069] To address the technical problems existing in related technologies, this application provides a target recognition method based on fused semantic features.

[0070] The target recognition method based on fused semantic features provided in this application can be executed by an electronic device, such as a terminal or a server. The terminal can be a smartphone, tablet, laptop, or other similar device. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. It is understood that this application does not specifically limit the executing entity of the target recognition method based on fused semantic features.

[0071] The technical solution of this application will be described in detail below through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments described below are used to explain the technical solution of this application and are not intended to limit actual use.

[0072] To address the technical problems existing in related technologies, embodiments of this application provide a target recognition method based on fused semantic features, such as... Figure 1 As shown, Figure 1 This is a flowchart illustrating a target recognition method based on fused semantic features, provided in an embodiment of this application. It should be noted that the steps shown may be executed in a different logical order than those shown in the flowchart. The method may include the following steps S101 to S106.

[0073] Step S101: Obtain the target infrared image and the corresponding maneuver category.

[0074] It should be noted that a target infrared image is a thermal radiation image acquired by an infrared imaging device and contains the target. A target infrared image can be represented in matrix form, where the value of each pixel reflects the thermal radiation intensity at the corresponding location.

[0075] Maneuver category is the result of classifying the maneuvering actions performed by the target. The maneuver category can be manually labeled in advance or determined based on a preset classification model; this application does not limit this.

[0076] Step S102: Input the target infrared image and maneuver category into the target recognition network to obtain the first maneuver semantic features, shallow image features and deep image features.

[0077] In one optional embodiment, the target recognition network includes a learnable maneuver semantic feature space, an ERBlock (Enhanced Relationship Block) module, and a SimSPFF (Simplified Spatial Pyramid Pooling-Fast) module. The target infrared image and maneuver category are input into the target recognition network to obtain first maneuver semantic features, shallow image features, and deep image features. Specifically, the network includes the following steps: determining the first maneuver semantic feature corresponding to the maneuver category in the learnable maneuver semantic feature space based on the maneuver category; obtaining shallow image features through the ERBlock module; and obtaining deep image features through the SimSPFF module.

[0078] In one specific embodiment, the learnable maneuver semantic feature space has a spatial dimension of 7, corresponding to 6 maneuver categories of the target and unidentified maneuver categories. The 6 maneuver categories can be level flight, left maneuver, right maneuver, climb, roll, and dive. Of course, the maneuver categories can also be other categories, and this application does not limit them. In a more specific embodiment, such as Figure 2 As shown, Figure 2 This is a schematic diagram illustrating a process for inputting a maneuver category into a learnable maneuver semantic feature space to obtain a first maneuver semantic feature, as provided in an embodiment of this application.

[0079] The learnable maneuver semantic feature space can be continuously optimized to obtain more robust first maneuver semantic features.

[0080] In a more specific embodiment, based on the maneuver category, determining a first maneuver semantic feature corresponding to the maneuver category in a learnable maneuver semantic feature space includes:

[0081] Based on the maneuver category, the first maneuver semantic feature corresponding to the maneuver category is determined in the learnable maneuver semantic feature space using the following formula:

[0082] (1)

[0083] in, The first semantic feature of the maneuver category. For motor vehicle category, For a learnable, mobile semantic feature space, , Represents the real number field. For the number of motor categories, Let be the row number of the feature matrix corresponding to the target infrared image. Let be the number of columns in the feature matrix corresponding to the infrared image of the target. This represents the operation of extracting the first maneuver semantic feature from the learnable maneuver semantic feature space based on the maneuver category.

[0084] In a more specific embodiment, the ERBlock module is suitable for shallow or mid-level image feature extraction, enhancing the modeling of the relationship between the target and the background in target infrared images. The SimSPFF module is suitable for depth image feature extraction, helping to improve the recognition ability of targets of different distances and sizes in target infrared images. It should be noted that shallow, mid-level, and deep image features are features extracted at different levels when the convolutional neural network processes images. Shallow image features are usually located in the initial layers of the convolutional neural network, while deep image features are usually located at the end of the convolutional neural network.

[0085] Step S103: Perform feature fusion on the first motor semantic features and shallow image features to obtain the first image fusion features.

[0086] In an optional embodiment, the target recognition network further includes a UAFFM (Unified Attention Fourier Fusion Module) to fuse the first motion semantic features and shallow image features to obtain the first image fused features, specifically including the following steps:

[0087] The first image fusion feature is obtained by fusing the first maneuver semantic features and shallow image features using UAFFM.

[0088] In one specific embodiment, such as Figure 3 As shown, Figure 3 This is a schematic diagram illustrating a process for fusing first maneuver semantic features and shallow image features using UAFFM to obtain first image fused features, as provided in this embodiment of the application. Figure 3 In this model, UAFFM includes convolutional layers, spatial attention units, channel attention units, and Fourier units. UAFFM fuses the first dynamic semantic features and shallow image features to obtain the first image fusion features. Specifically, it includes the following steps:

[0089] The number of channels of the first motion semantic feature is adjusted to be the same as the number of channels of the shallow image feature using a convolutional layer to obtain the second motion semantic feature. The second motion semantic feature is upsampled to adjust its size to be the same as the size of the shallow image feature to obtain the third motion semantic feature. The third motion semantic feature and the shallow image feature are input into a spatial attention unit to obtain the first spatial attention weight and the second spatial attention weight, and the sum of the first spatial attention weight and the second spatial attention weight is 1. The third motion semantic feature and the shallow image feature are input into a channel attention unit to obtain the first channel attention weight and the second channel attention weight, and the sum of the first channel attention weight and the second channel attention weight is 1. The product of the third motion semantic feature and the first spatial attention weight, the product of the third motion semantic feature and the first channel attention weight, the product of the shallow image feature and the second spatial attention weight, and the product of the shallow image feature and the second channel attention weight are added to obtain the attention-weighted feature. The attention-weighted feature is input into a Fourier unit to obtain the first image fusion feature.

[0090] In a more specific embodiment, the second maneuver semantic feature is upsampled to adjust its size to be the same as the size of the shallow image feature, thereby obtaining the third maneuver semantic feature. This specifically includes the following steps: UAFFM upsamples the second maneuver semantic feature through linear interpolation to adjust its size to be the same as the size of the shallow image feature, thereby obtaining the third maneuver semantic feature.

[0091] In a more specific embodiment, the second maneuver semantic feature is upsampled using the following formula to adjust its size to be the same as the size of the shallow image feature, thus obtaining the third maneuver semantic feature:

[0092] (2)

[0093] in, For the third maneuver semantic features, Indicates an upsampling operation. This represents the semantic features of the second maneuver.

[0094] In a more specific embodiment, the third mobile semantic features and shallow image features are input into the spatial attention unit using the following formula to obtain the first spatial attention weights:

[0095] (3)

[0096] in, For the first spatial attention weight, Indicates spatial attention, For the third maneuver semantic features, These are shallow image features.

[0097] In a more specific embodiment, the third mobile semantic features and shallow image features are input into the channel attention unit using the following formula to obtain the first channel attention weights:

[0098] (4)

[0099] in, The first channel attention weight, Indicates channel attention. For the third maneuver semantic features, These are shallow image features.

[0100] In a more specific embodiment, the attention-weighted features are obtained by adding the products of the third maneuver semantic features and the first spatial attention weight, the third maneuver semantic features and the first channel attention weight, the shallow image features and the second spatial attention weight, and the shallow image features and the second channel attention weight using the following formulas:

[0101] (5)

[0102] in, For attention-weighted features, For the third maneuver semantic features, For the first spatial attention weight, These are shallow image features. This represents the attention weight for the first channel.

[0103] In a more specific embodiment, the attention-weighted features are input into the Fourier unit using the following formula to obtain the first image fusion features:

[0104] (6)

[0105] in, For the first image fusion feature, For attention-weighted features, This represents the Fourier transform.

[0106] In a more specific embodiment, the attention-weighted features are input into a Fourier unit to obtain a first image fusion feature, specifically including the following operations: inputting the attention-weighted features into a Fourier unit, performing a Fourier transform operation on the attention-weighted features to obtain a first frequency domain attention-weighted feature; performing frequency domain processing on the first frequency domain attention-weighted feature to obtain a second frequency domain attention-weighted feature; and performing an inverse Fourier transform operation on the second frequency domain attention-weighted feature to obtain the first image fusion feature.

[0107] In a more specific embodiment, frequency domain processing includes at least one of the following operations: convolution operation, batch normalization operation, and application of activation function operation.

[0108] In a more specific embodiment, the first frequency domain attention-weighted features are processed in the frequency domain to obtain the second frequency domain attention-weighted features, specifically including the following steps:

[0109] Convolution, batch normalization, and activation function operations are sequentially performed on the attention-weighted features in the first frequency domain.

[0110] Step S104: Perform feature fusion on the first motor semantic features and deep image features to obtain the second image fusion features.

[0111] In an optional embodiment, the target recognition network further includes a CASFM (Cross Attention Stacked Fusion Module) to fuse the first mobile semantic features and deep image features to obtain a second image fusion feature, including:

[0112] The first motor semantic features and the deep image features are fused using CASFM to obtain the second image fusion features.

[0113] In one specific embodiment, CASFM includes a convolutional layer, a first cross-attention unit, and a second cross-attention unit. The first cross-attention unit includes a first key weight matrix, a first query weight matrix, and a first value weight matrix. The second cross-attention unit includes a second key weight matrix, a second query weight matrix, and a second value weight matrix. CASFM is used to fuse first motion semantic features and deep image features to obtain a second image fusion feature. Specifically, the following steps are included: adjusting the number of channels of the first motion semantic feature to be the same as the number of channels of the deep image feature using a convolutional layer to obtain a fourth motion semantic feature; inputting the fourth motion semantic feature into the first key weight matrix, the first query weight matrix, and the first value weight matrix to obtain a first key vector, a first query vector, and a first value vector; inputting the deep image features into the first key weight matrix, the first query weight matrix, and the first value weight matrix to obtain a first key vector, a first query vector, and a first value vector; and inputting the deep image features into the first key weight matrix, the first query weight matrix, and the first value weight matrix to obtain a second image fusion feature. The second key vector, second query vector, and second value vector are obtained from the two-key weight matrix, the second query weight matrix, and the second value weight matrix. Using deep image features as query features, cross-attention is calculated based on the second query vector, the first key vector, and the first value vector using the scaling dot product attention mechanism formula, resulting in the first attention output feature. Using the fourth mobile semantic feature as query features, cross-attention is calculated based on the first query vector, the second key vector, and the second value vector using the scaling dot product attention mechanism formula, resulting in the second attention output feature. The first attention output feature and the fourth mobile semantic feature are added together to obtain the third image fusion feature. The second attention output feature and deep image features are added together to obtain the fourth image fusion feature. The third image fusion feature and the fourth image fusion feature are concatenated to obtain the second image fusion feature. Figure 4 As shown, Figure 4 This is a schematic diagram of a process for fusing first motor semantic features and deep image features using CASFM to obtain second image fusion features, as provided in an embodiment of this application.

[0114] In a more specific embodiment, the fourth mobile semantic features are input into the first key weight matrix, the first query weight matrix, and the first value weight matrix, respectively, to obtain the first key vector, the first query vector, and the first value vector; the deep image features are input into the second key weight matrix, the second query weight matrix, and the second value weight matrix, respectively, to obtain the second key vector, the second query vector, and the second value vector, which can be represented by the following expression:

[0115] (7)

[0116] in, This is the first query vector. The first key vector, For the first value vector, For the fourth maneuver semantic features, First query matrix, This is the first bond matrix. This is a first-value matrix. This is the second query vector. This is the second key vector. For the second value vector, For deep image features, Second query matrix, This is the second bond matrix. This is the second-valued matrix.

[0117] In a more specific embodiment, the deep image features are used as query features, and cross-attention is calculated based on the second query vector, the first key vector, and the first value vector using the scaling dot product attention mechanism formula to obtain the first attention output features:

[0118] (8)

[0119] in, The first attention output feature, It is the first key vector transpose, It is the first key vector Dimensions This is the second query vector. This is the first value vector.

[0120] In a more specific embodiment, the fourth mobile semantic feature is used as the query feature, and cross-attention is calculated based on the first query vector, the second key vector, and the second value vector using the scaling dot product attention mechanism formula to obtain the second attention output feature:

[0121] (9)

[0122] in, For the second attention output features, It is the second key vector transpose, It is the second key vector Dimensions This is the first query vector. This is the second value vector.

[0123] In a more specific embodiment, the third image fusion feature is obtained by adding the first attention output feature and the fourth mobile semantic feature using the following formula: the fourth image fusion feature is obtained by adding the second attention output feature and the deep image feature; and the second image fusion feature is obtained by concatenating the third and fourth image fusion features.

[0124] (10)

[0125] in, For the second image fusion feature, For concatenation functions, For standard convolution operations, The first attention output feature, For the fourth maneuver semantic features, For the second attention output features, These are features of deep images.

[0126] In another, more specific embodiment, CASFM also includes convolutional block units, and further includes:

[0127] The second image fusion features are stacked and convolved using convolutional block units.

[0128] In a more specific embodiment, the convolutional block unit comprises a convolutional layer, a dropout layer, a batch normalization layer, and an activation layer. The convolutional layer enables the network to better capture the complex relationships between deep image features and dynamic features, extracting detailed features from the fused features. The dropout layer introduces randomness, reducing excessive dependencies between features and effectively preventing overfitting. Batch normalization improves training stability. The final activation layer uses the LeakyReLU (LeakyRectified Linear Uni) function, which alleviates the "zero gradient problem" of linear rectified units in the negative region, enhancing the network's nonlinear fitting ability.

[0129] In a more specific embodiment, the second image fusion features are subjected to a stacked convolution operation using the following formula:

[0130] (11)

[0131] (12)

[0132] (13)

[0133] in, The second image fusion feature is obtained after stacked convolution operations. and This is an intermediate feature map. This indicates a batch normalization operation. This represents a regularization technique. For standard convolution operations, This is the second image fusion feature.

[0134] Step S105: Upsample and stitch the first image fusion feature and the second image fusion feature to obtain a multi-scale feature representation.

[0135] In one alternative embodiment, the first image fusion feature and the second image fusion feature are upsampled by means of bilinear interpolation or deconvolution.

[0136] In this step, after upsampling and concatenation, the resulting new feature map contains information from different scales, i.e., a multi-scale feature representation. Multi-scale feature representation combines global information at low resolution with local details at high resolution, helping the model to learn complex patterns and structures more effectively.

[0137] Step S106: Input the multi-scale feature representation into the output header for convolution operation to obtain the target category and anti-interference recognition results.

[0138] It should be noted that the output header is designed according to the task requirements and application scenarios.

[0139] In one alternative embodiment, the output head includes at least one of the following components: a convolutional layer, an activation function, batch normalization, global pooling, and a fully connected layer.

[0140] In one alternative embodiment, the anti-interference identification result includes at least one of the following: accuracy, stability, robustness, and generalization ability.

[0141] It should also be noted that the experimental results show that after introducing the UAFFM module, the algorithm's precision improved by 3.9% from 90.4% to 94.3%, indicating that the UAFFM module, by introducing spatial attention, channel attention, and Fourier modules, effectively enhances the fusion of shallow image features and maneuver features, enabling the model to detect target locations more accurately and improve precision. The recall rate improved by 0.7% from 81.7% to 82.4%, demonstrating that the UAFFM module improves the network's recall capability. Adding the CASFM module to the basic network improved the network's precision by 3.1% and recall by 1.6%, indicating that the design of using cross-attention stacking in the CASFM module can more fully capture the complex correlation between maneuver features and deep image features, improving the network's detection accuracy. Adding both the UAFFM and CASFM modules to the basic network simultaneously improved precision to 95.1% and recall to 84.6%. Experiments show that the UAFFM and CASFM modules together effectively improve the network's anti-interference recognition performance for infrared images through the fusion of maneuver semantic features and image features.

[0142] Corresponding to the target recognition method based on fused semantic features provided in the embodiments of this application, the embodiments of this application also provide a target recognition device based on fused semantic features, such as... Figure 5 As shown, the target recognition device based on fused semantic features includes:

[0143] The acquisition module 501 is used to acquire the target infrared image and the maneuver category corresponding to the target infrared image, wherein the target infrared image contains the target;

[0144] The first input module 502 is used to input the target infrared image and maneuver category into the target recognition network to obtain the first maneuver semantic features, shallow image features and deep image features;

[0145] The first feature fusion module 503 is used to fuse the first mobile semantic features and shallow image features to obtain the first image fusion features;

[0146] The second feature fusion module 504 is used to fuse the first mobile semantic features and deep image features to obtain the second image fusion features.

[0147] The sampling and stitching module 505 is used to perform upsampling and stitching operations on the first image fusion feature and the second image fusion feature to obtain a multi-scale feature representation;

[0148] The second input module 506 is used to input the multi-scale feature representation into the output head for convolution operation to obtain the target category and anti-interference recognition results.

[0149] In one optional embodiment, the target recognition network includes a learnable maneuverable semantic feature space, an ERBlock module, and a SimSPFF module. The first input module is specifically used for:

[0150] Based on the maneuver category, determine the first maneuver semantic feature corresponding to the maneuver category in the learnable maneuver semantic feature space;

[0151] Shallow image features are obtained using the ERBlock module;

[0152] Deep image features are obtained using the SimSPFF module.

[0153] In one alternative embodiment, the first input module is specifically used for:

[0154] Based on the maneuver category, the first maneuver semantic feature corresponding to the maneuver category is determined in the learnable maneuver semantic feature space using the following formula:

[0155]

[0156] in, The first semantic feature of the maneuver category. For motor vehicle category, For a learnable, mobile semantic feature space, , Represents the real number field. For the number of motor categories, Let be the row number of the feature matrix corresponding to the target infrared image. Let be the number of columns in the feature matrix corresponding to the infrared image of the target. This represents the operation of extracting the first maneuver semantic feature from the learnable maneuver semantic feature space based on the maneuver category.

[0157] In an optional embodiment, the target recognition network further includes a UAFFM (User-Assisted Feature Fusion Model) first feature fusion module, specifically used for:

[0158] The first image fusion feature is obtained by fusing the first maneuver semantic features and shallow image features using UAFFM.

[0159] In one optional embodiment, UAFFM includes convolutional layers, spatial attention units, channel attention units, and Fourier units. The first feature fusion module is specifically used for:

[0160] The number of channels of the first motion semantic feature is adjusted to be the same as the number of channels of the shallow image feature by using a convolutional layer to obtain the second motion semantic feature;

[0161] Upsampling is performed on the second maneuver semantic feature to adjust its size to be the same as the size of the shallow image feature, thus obtaining the third maneuver semantic feature;

[0162] The third motion semantic features and shallow image features are input into the spatial attention unit to obtain the first spatial attention weight and the second spatial attention weight. The sum of the first spatial attention weight and the second spatial attention weight is 1.

[0163] The third dynamic semantic feature and shallow image feature are input into the channel attention unit to obtain the first channel attention weight and the second channel attention weight. The sum of the first channel attention weight and the second channel attention weight is 1.

[0164] The attention-weighted features are obtained by adding the products of the third maneuver semantic features and the first spatial attention weight, the third maneuver semantic features and the first channel attention weight, the shallow image features and the second spatial attention weight, and the shallow image features and the second channel attention weight.

[0165] The attention-weighted features are input into the Fourier unit to obtain the first image fusion features.

[0166] In one optional embodiment, the first feature fusion module is specifically used for:

[0167] The attention-weighted features are input into the Fourier unit, and a Fourier transform operation is performed on the attention-weighted features to obtain the first frequency domain attention-weighted features.

[0168] The first frequency domain attention-weighted features are processed in the frequency domain to obtain the second frequency domain attention-weighted features;

[0169] Perform an inverse Fourier transform operation on the second frequency domain attention-weighted features to obtain the first image fusion features.

[0170] In one alternative embodiment, frequency domain processing includes at least one of the following operations:

[0171] Convolution operations, batch normalization operations, and operations that apply activation functions.

[0172] In an optional embodiment, the target recognition network further includes CASFM, a second feature fusion module, specifically used for:

[0173] The first motor semantic features and the deep image features are fused using CASFM to obtain the second image fusion features.

[0174] In one optional embodiment, CASFM includes a convolutional layer, a first cross-attention unit, and a second cross-attention unit. The first cross-attention unit includes a first key weight matrix, a first query weight matrix, and a first value weight matrix. The second cross-attention unit includes a second key weight matrix, a second query weight matrix, and a second value weight matrix. The second feature fusion module is specifically used for:

[0175] The number of channels of the first motion semantic feature is adjusted to be the same as the number of channels of the deep image feature by using a convolutional layer, thus obtaining the fourth motion semantic feature;

[0176] The fourth mobile semantic features are input into the first key weight matrix, the first query weight matrix, and the first value weight matrix, respectively, to obtain the first key vector, the first query vector, and the first value vector.

[0177] The deep image features are input into the second key weight matrix, the second query weight matrix, and the second value weight matrix, respectively, to obtain the second key vector, the second query vector, and the second value vector.

[0178] Using deep image features as query features, cross-attention calculation is performed based on the second query vector, the first key vector, and the first value vector using the scaling dot product attention mechanism formula to obtain the first attention output feature;

[0179] The fourth mobile semantic feature is used as the query feature. Based on the first query vector, the second key vector and the second value vector, cross attention is calculated using the scaling dot product attention mechanism formula to obtain the second attention output feature.

[0180] The first attention output feature and the fourth maneuver semantic feature are added together to obtain the third image fusion feature;

[0181] The second attention output feature and the deep image feature are added together to obtain the fourth image fusion feature;

[0182] The third and fourth image fusion features are concatenated to obtain the second image fusion feature.

[0183] In an optional embodiment, CASFM further includes convolutional block units, and also includes:

[0184] The second image fusion features are stacked and convolved using convolutional block units.

[0185] Corresponding to the target recognition method based on fused semantic features provided in the embodiments of this application, the embodiments of this application also provide an electronic device for performing the target recognition method based on fused semantic features, such as... Figure 6As shown, the electronic device includes: a processor 601; and a memory 602 for storing a program for a target recognition method based on fused semantic features. After the device is powered on and the program for the target recognition method based on fused semantic features is run by the processor, the following steps are performed:

[0186] Acquire the target infrared image and the corresponding maneuver category; the target infrared image contains the target.

[0187] The target infrared image and maneuver category are input into the target recognition network to obtain the first maneuver semantic features, shallow image features, and deep image features;

[0188] The first image fusion feature is obtained by fusing the first motion semantic features and the shallow image features;

[0189] The first semantic features and the deep image features are fused to obtain the second image fusion features;

[0190] Upsampling and stitching operations are performed on the first image fusion feature and the second image fusion feature to obtain a multi-scale feature representation;

[0191] Multi-scale feature representations are input into the output header for convolution operations to obtain target category and anti-interference recognition results.

[0192] Corresponding to the target recognition method based on fused semantic features provided in the embodiments of this application, the embodiments of this application also provide a computer-readable storage medium storing a program for a target recognition method based on fused semantic features. This program is executed by a processor to perform the following steps:

[0193] Acquire the target infrared image and the corresponding maneuver category; the target infrared image contains the target.

[0194] The target infrared image and maneuver category are input into the target recognition network to obtain the first maneuver semantic features, shallow image features, and deep image features;

[0195] The first image fusion feature is obtained by fusing the first motion semantic features and the shallow image features;

[0196] The first semantic features and the deep image features are fused to obtain the second image fusion features;

[0197] Upsampling and stitching operations are performed on the first image fusion feature and the second image fusion feature to obtain a multi-scale feature representation;

[0198] Multi-scale feature representations are input into the output header for convolution operations to obtain target category and anti-interference recognition results.

[0199] Corresponding to the target recognition method based on fused semantic features provided in the embodiments of this application, the embodiments of this application also provide a computer program containing instructions, which, when executed by a computer, cause the computer to perform the following steps:

[0200] Acquire the target infrared image and the corresponding maneuver category; the target infrared image contains the target.

[0201] The target infrared image and maneuver category are input into the target recognition network to obtain the first maneuver semantic features, shallow image features, and deep image features;

[0202] The first image fusion feature is obtained by fusing the first motion semantic features and the shallow image features;

[0203] The first semantic features and the deep image features are fused to obtain the second image fusion features;

[0204] Upsampling and stitching operations are performed on the first image fusion feature and the second image fusion feature to obtain a multi-scale feature representation;

[0205] Multi-scale feature representations are input into the output header for convolution operations to obtain target category and anti-interference recognition results.

[0206] It should be noted that for a detailed description of the target recognition device, electronic device, computer-readable storage medium and computer program based on fused semantic features provided in the embodiments of this application, please refer to the relevant description of the target recognition method based on fused semantic features provided in the embodiments of this application, which will not be repeated here.

[0207] Although this application discloses preferred embodiments as described above, it is not intended to limit this application. Any person skilled in the art can make possible changes and modifications without departing from the spirit and scope of this application. Therefore, the scope of protection of this application should be determined by the scope defined in the claims of this application.

[0208] In a typical configuration, an electronic device includes one or more processors (Central Processing Units), input / output interfaces, network interfaces, and memory.

[0209] Memory may include non-persistent storage in computer-readable media, such as random access memory and / or non-volatile memory, like read-only memory or flash memory. Memory is an example of computer-readable media.

[0210] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable operations, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PCM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital video disc (DMCD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include non-transitory computer-readable media, such as modulated data signals and carrier waves.

[0211] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0212] Although this application discloses preferred embodiments as described above, it is not intended to limit this application. Any person skilled in the art can make possible changes and modifications without departing from the spirit and scope of this application. Therefore, the scope of protection of this application should be determined by the scope defined in the claims of this application.

Claims

1. A target recognition method based on fused semantic features, characterized in that, include: Acquire a target infrared image and the corresponding maneuver category, wherein the target infrared image contains the target; The target infrared image and the maneuver category are input into the target recognition network to obtain the first maneuver semantic features, shallow image features, and deep image features; The first motion semantic features and the shallow image features are fused to obtain the first image fusion features; The first motion semantic features and the deep image features are fused to obtain the second image fusion features; Upsampling and stitching operations are performed on the first image fusion feature and the second image fusion feature to obtain a multi-scale feature representation; The multi-scale feature representation is input into the output header for convolution operation to obtain the target category and anti-interference recognition result; The target recognition network includes a learnable mobile semantic feature space, an ERBlock module, and a SimSPFF module. The ERBlock module is used to extract shallow image features, and the SimSPFF module is used to extract deep image features. The shallow image features and the deep image features are features extracted from different levels when the convolutional neural network processes images. The shallow image features are located in the initial few layers of the convolutional neural network, and the deep image features are located at the end of the convolutional neural network. The step of inputting the target infrared image and the maneuver category into the target recognition network to obtain first maneuver semantic features, shallow image features, and deep image features includes: Based on the maneuver category, determine the first maneuver semantic feature corresponding to the maneuver category in the learnable maneuver semantic feature space; The shallow image features are obtained through the ERBlock module; The deep image features are obtained through the SimSPFF module; The target recognition network further includes a UAFFM (Unified Alternating Current Filtering Model), which comprises convolutional layers, spatial attention units, channel attention units, and Fourier units. The feature fusion of the first mobile semantic features and the shallow image features to obtain the first image fusion features includes: The first image fusion feature is obtained by fusing the first maneuver semantic feature and the shallow image feature using the UAFFM. The target recognition network further includes CASFM, which comprises a convolutional layer, a first cross-attention unit, and a second cross-attention unit. The first cross-attention unit includes a first key weight matrix, a first query weight matrix, and a first value weight matrix. The second cross-attention unit includes a second key weight matrix, a second query weight matrix, and a second value weight matrix. The feature fusion of the first mobile semantic features and the deep image features to obtain the second image fusion features includes: The first motor semantic feature and the deep image feature are fused using the CASFM to obtain the second image fusion feature.

2. The target recognition method based on fused semantic features according to claim 1, characterized in that, The step of determining the first maneuver semantic feature corresponding to the maneuver category in the learnable maneuver semantic feature space based on the maneuver category includes: Based on the maneuver category, the first maneuver semantic feature corresponding to the maneuver category is determined in the learnable maneuver semantic feature space using the following formula: Among them, the The first maneuver semantic feature corresponding to the maneuver category, the For the aforementioned motor category, the For a learnable, mobile semantic feature space, The Representing the real number field, the For the number of motor categories, the The number of rows in the feature matrix corresponding to the infrared image of the target is the number of rows in the feature matrix. The number of columns in the feature matrix corresponding to the infrared image of the target is [the number of columns in the feature matrix]. This refers to the operation of extracting the first maneuver semantic feature from the learnable maneuver semantic feature space based on the maneuver category.

3. The target recognition method based on fused semantic features according to claim 1, characterized in that, The step of fusing the first maneuver semantic features and the shallow image features using the UAFFM to obtain the first image fusion features includes: The number of channels of the first motion semantic feature is adjusted to be the same as the number of channels of the shallow image feature using the convolutional layer to obtain the second motion semantic feature; Upsample the second maneuver semantic feature to adjust its size to be the same as the size of the shallow image feature, and obtain the third maneuver semantic feature; The third maneuver semantic feature and the shallow image feature are input into the spatial attention unit to obtain the first spatial attention weight and the second spatial attention weight, and the sum of the first spatial attention weight and the second spatial attention weight is 1. The third motion semantic feature and the shallow image feature are input into the channel attention unit to obtain the first channel attention weight and the second channel attention weight, and the sum of the first channel attention weight and the second channel attention weight is 1. The attention-weighted features are obtained by adding the product of the third maneuver semantic feature and the first spatial attention weight, the product of the third maneuver semantic feature and the first channel attention weight, the product of the shallow image feature and the second spatial attention weight, and the product of the shallow image feature and the second channel attention weight. The attention-weighted features are input into the Fourier unit to obtain the first image fusion features.

4. The target recognition method based on fused semantic features according to claim 3, characterized in that, The step of inputting the attention-weighted features into the Fourier unit to obtain the first image fusion features includes: The attention-weighted features are input into the Fourier unit, and a Fourier transform operation is performed on the attention-weighted features to obtain the first frequency domain attention-weighted features; The first frequency domain attention-weighted features are processed in the frequency domain to obtain the second frequency domain attention-weighted features; Perform an inverse Fourier transform operation on the second frequency domain attention-weighted features to obtain the first image fusion features.

5. The target recognition method based on fused semantic features according to claim 4, characterized in that, The frequency domain processing includes at least one of the following operations: Convolution operations, batch normalization operations, and operations that apply activation functions.

6. The target recognition method based on fused semantic features according to claim 1, characterized in that, The step of fusing the first motor semantic features and the deep image features using the CASFM to obtain the second image fusion features includes: The number of channels of the first motion semantic feature is adjusted to be the same as the number of channels of the deep image feature using the convolutional layer to obtain the fourth motion semantic feature; The fourth mobile semantic features are respectively input into the first key weight matrix, the first query weight matrix, and the first value weight matrix to obtain the first key vector, the first query vector, and the first value vector. The deep image features are respectively input into the second key weight matrix, the second query weight matrix, and the second value weight matrix to obtain the second key vector, the second query vector, and the second value vector. Using the deep image features as query features, cross-attention calculation is performed based on the second query vector, the first key vector, and the first value vector using the scaling dot product attention mechanism formula to obtain the first attention output feature; Using the fourth mobile semantic feature as the query feature, cross-attention calculation is performed based on the first query vector, the second key vector, and the second value vector using the scaling dot product attention mechanism formula to obtain the second attention output feature. The first attention output feature and the fourth maneuver semantic feature are added together to obtain the third image fusion feature; The second attention output feature and the deep image feature are added together to obtain the fourth image fusion feature; The third image fusion feature and the fourth image fusion feature are concatenated to obtain the second image fusion feature.

7. The target recognition method based on fused semantic features according to claim 1, characterized in that, The CASFM also includes convolutional block units, and further includes: The second image fusion feature is subjected to stacked convolution operation by convolution block units.

Images