Multi-source photoelectric image intelligent target detection method and system

Abstract

The application relates to the technical field of target detection, in particular to a multi-source photoelectric image intelligent target detection method and system.The method comprises the following steps: acquiring multi-source photoelectric images and performing standardization processing, performing modality perception feature extraction on multi-modal input data, sequentially performing dynamic compensation modeling and deep semantic modeling, performing cross-modal implicit space alignment according to intermediate semantic features, sequentially generating spatial attention weight and channel attention weight according to the aligned cross-modal feature representation, performing element-by-element weighting and channel compression fusion, taking the fused feature representation as backbone input to perform multi-scale feature pyramid construction and target detection, performing coordinate decoding and confidence screening on each scale prediction result, and outputting a final target detection result set.The application enhances the small target discrimination ability by constructing a multi-scale feature pyramid and a target detection network, and realizes unified modeling of multi-scale targets.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and in particular to a multi-source photoelectric image intelligent target detection method and system. Background Technology

[0002] In existing technologies, photoelectric detection is increasingly widely used in fields such as intelligent sensing of photoelectric pods, all-weather monitoring and early warning, and target recognition in complex environments. As detection platforms continue to expand into complex environments and all-weather application scenarios, target detection algorithms with single-spectral image sources can no longer meet the needs of actual detection tasks due to insufficient environmental adaptability and limited scene generalization ability. Multi-source photoelectric payloads such as visible light, near-infrared, mid-wave infrared, and short-wave infrared have natural complementary characteristics in terms of imaging mechanism, information expression dimension, and environmental adaptability, providing an effective way to solve the target detection problem in complex scenarios. Therefore, intelligent target detection of multi-source photoelectric images has become an important development direction in this field.

[0003] Current target detection technologies for multi-source optoelectronic images suffer from the core problem of failing to effectively coordinate the use of multi-source optoelectronic information. Existing technologies struggle to address the technical challenges of accurate cross-modal feature alignment, adaptive fusion of multi-modal features, and long-term stability of system detection performance. This results in the inability to fully leverage the complementary advantages of multi-source optoelectronic payloads, leading to poor fusion detection results. Specifically, in existing technologies, the modal specificity of optoelectronic data easily creates cross-modal semantic gaps, feature alignment is susceptible to environmental interference, and robust alignment schemes are lacking, making it difficult to achieve accurate multi-modal feature matching in complex dynamic scenes. Furthermore, existing fusion strategies cannot adaptively adjust the weights of each modality according to scene characteristics. In complex scenarios such as low light, fog, and smoke obstruction, issues such as weakened effective information and amplified noise arise, resulting in insufficient fusion robustness. Moreover, due to factors such as sensor aging, changes in mechanical posture, and environmental evolution, the registration accuracy between modalities is prone to gradual degradation. Existing technologies lack effective dynamic compensation and adaptive modeling mechanisms, making it difficult to maintain the long-term stability of system detection performance. Therefore, a smart target detection method and system for multi-source optoelectronic images is currently needed. Summary of the Invention

[0004] To address the technical problem that existing multi-source optoelectronic image target detection technologies struggle to achieve accurate target detection in complex environments and all-weather conditions, this invention provides a multi-source optoelectronic image intelligent target detection method and system.

[0005] In a first aspect, the present invention provides a multi-source optoelectronic image intelligent target detection method, which adopts the following technical solution:

[0006] A multi-source photoelectric image intelligent target detection method, including

[0007] Acquire multi-source photoelectric images and perform standardization processing to obtain standardized multimodal input data;

[0008] Modality-aware feature extraction is performed on multimodal input data, followed by dynamic compensation modeling and deep semantic modeling to generate intermediate semantic feature representations;

[0009] Cross-modal implicit spatial alignment is performed based on intermediate semantic features, including spatial flattening of the intermediate semantic feature representation, constructing cross-modal shared proxy semantic nodes, and performing global semantic aggregation and reverse semantic broadcasting of multimodal features through proxy semantic nodes to generate aligned cross-modal feature representations.

[0010] Spatial attention weights and channel attention weights are generated using the aligned cross-modal feature representations, and then a unified fused feature representation is generated by fusing element-wise weighting and channel compression.

[0011] Using fused feature representation as the main input, multi-scale feature pyramid construction and target detection are performed, and the output includes a target location regression branch, a target category prediction branch, and a target confidence branch.

[0012] Coordinate decoding and confidence screening are performed on the prediction results at each scale. The non-maximum suppression algorithm is used to remove redundant overlapping boxes, and the final target detection result set is output.

[0013] Furthermore, the standardization process includes spatial scale unification and geometric registration. The input visible light and infrared images are uniformly scaled to a preset resolution. Bilinear interpolation is used for resampling. Based on the structural consistency alignment principle, edge and gradient features are extracted from different modal images as structural descriptors. The minimum matching error between structurally similar regions is calculated to solve for the two-dimensional geometric transformation matrix. This matrix is ​​then applied to the non-reference modal image to complete geometric registration. The geometric transformation process is expressed as follows:

[0014] ,

[0015] in, Infrared modal image, The image is an infrared image after geometric registration. This is the two-dimensional affine transformation matrix obtained through structural similarity matching. These are the pixel coordinates of the image.

[0016] Furthermore, the standardization process also includes implicit alignment in the feature space. By introducing a deformable convolutional structure in the feature extraction stage, the feature sampling position is adaptively adjusted, and an offset set is introduced to expand the convolutional sampling position from a regular grid to a learnable dynamic position distribution.

[0017] Based on the deformable convolutional structure, a cross-modal structure guidance mechanism is introduced. Intermediate features from the visible and infrared modes are structurally fused and input into the migration prediction branch to generate a joint migration field. The expression for the deformable convolutional structure is:

[0018] ,

[0019] in, To output feature map at location The feature value at point X is the input feature map. The regular sampling grid for the convolution kernel. For the convolution kernel at position The weight of the position, The sampling offset is obtained through adaptive learning by the offset prediction branch network.

[0020] Furthermore, the modality-aware feature extraction and dynamic compensation modeling includes setting an independent feature extraction branch for each input modality, performing low-level semantic modeling of the input tensor through a basic convolutional coding structure, wherein the basic convolutional coding structure is composed of ordinary convolution, batch normalization and SiLU activation function concatenation, and introducing deformable convolutional structures in the modality coding stage to perform dynamic compensation modeling of feature sampling positions.

[0021] Furthermore, the deep semantic modeling includes using a hierarchical semantic aggregation structure to perform deep semantic modeling on the features. This hierarchical semantic aggregation structure consists of a main semantic path and a detail-preserving path. The main semantic path is used to model the overall contour of the target and its semantic context. The detail-preserving path uses a small-scale convolutional structure to retain low-level texture details. Finally, the multi-path features are fused using channel concatenation, and a 1×1 convolution is used to perform channel compression and semantic reshaping on the fused features to generate an intermediate semantic feature tensor with a consistent shape. The expression for the hierarchical semantic aggregation structure is:

[0022] ,

[0023] in, For intermediate semantic feature tensors, For the i-th multipath output feature, Let be the convolutional branch structure with different scales. This is for channel splicing operations. This is a 1×1 convolution operation.

[0024] Furthermore, the global semantic aggregation includes spatially flattening the multimodal features to generate a flattened feature tensor, constructing surrogate semantic nodes through a global semantic convergence operator, and using the surrogate semantic nodes as query vectors to perform global semantic aggregation on the multimodal features. It also involves calculating the attention weights between the surrogate semantic nodes and the multimodal features and completing the semantic aggregation, enabling the surrogate nodes to possess consistent semantic expression capabilities across modalities. The global semantic convergence operator is expressed as follows:

[0025] ,

[0026] in, For proxy semantic nodes, For the number of proxy semantic nodes, , These are the visible light mode characteristics and infrared mode characteristics after flattening, respectively. For the number of spatial locations, For cross-modal feature splicing operations, For global average pooling operation, B is the batch size and C is the number of feature channels.

[0027] Furthermore, the reverse semantic broadcasting includes using multimodal features as queries and proxy semantic nodes after global semantic aggregation as key-value pairs to broadcast the global semantic structure back to the feature space. Then, broadcast weights are calculated, and aligned cross-modal feature representations are obtained through these weights. These representations are then rearranged into a two-dimensional feature map. The cross-modal feature representation is as follows:

[0028] ,

[0029] in, For broadcast weight, These are the proxy semantic features after global semantic aggregation. This is the projection matrix with multimodal features as the query. , This is the projection matrix with proxy semantic nodes as key-value pairs. The result of flattening the multimodal features is the stitching result. , and is a learnable projection weight matrix, d represents the feature dimension scaling factor, and Softmax represents the normalized exponential function.

[0030] Furthermore, the generation of the unified fusion feature representation includes:

[0031] Max pooling and average pooling operations are performed on the aligned cross-modal features to extract the distribution features of local salient regions and robust regions. The two are concatenated and then input into a lightweight convolutional structure, and spatial attention weights are obtained through Sigmoid mapping.

[0032] Global average pooling is used to obtain the overall response statistical features of the aligned cross-modal feature representation after linear layer processing. A nonlinear channel mapping relationship is established through a two-layer fully connected network to generate channel attention weights.

[0033] Having both spatial attention weights and channel attention weights, the modal features are weighted element-wise, the weighted modal features are concatenated along the channel dimension, and channel compression and fusion are performed through 1×1 convolution to generate a unified fused feature representation.

[0034] Furthermore, the construction of the multi-scale feature pyramid and target detection includes using the fused feature representation as the backbone input and constructing a multi-scale semantic feature layer through downsampling convolution. The multi-scale semantic feature layer includes feature layers with different spatial resolutions that respectively represent the semantic responses of small, medium and large targets.

[0035] By constructing a feature pyramid through lateral connectivity and upsampling fusion mechanism, high-level semantic information is fed back to the high-resolution feature layer, enhancing the ability to distinguish small targets;

[0036] A target detection head is constructed for each scale feature layer, and the target detection head includes a bounding box regression network, a category prediction network, and a target confidence prediction network.

[0037] The bounding box regression network outputs the target location regression branch, the category prediction network outputs the target category prediction branch, and the target confidence prediction network outputs the target confidence branch.

[0038] Secondly, a multi-source photoelectric image intelligent target detection system includes:

[0039] The preprocessing module is configured to acquire multi-source photoelectric images and perform standardization processing to obtain standardized multimodal input data;

[0040] The feature modeling module is configured to: extract modality-aware features from multimodal input data, and then perform dynamic compensation modeling and deep semantic modeling in sequence to generate intermediate semantic feature representations;

[0041] The alignment module is configured to perform cross-modal implicit space alignment based on intermediate semantic features, and generate aligned cross-modal feature representations.

[0042] The feature fusion module is configured to: sequentially generate spatial attention weights and channel attention weights from the aligned cross-modal feature representations, and then fuse them through element-wise weighted fusion and channel compression to generate a unified fused feature representation;

[0043] The intelligent detection module is configured to: construct a multi-scale feature pyramid and detect targets using fused feature representation as the main input, and output a target location regression branch, a target category prediction branch, and a target confidence branch;

[0044] The output module is configured to perform coordinate decoding and confidence filtering on the prediction results at each scale, use a non-maximum suppression algorithm to remove redundant overlapping boxes, and output the final target detection result set.

[0045] In summary, the present invention has the following beneficial technical effects:

[0046] 1. This invention introduces multi-source image standardization processing to perform spatial calibration, geometric registration, scale unification, and statistical distribution normalization on images from different spectral sensors. This effectively eliminates the cross-modal feature misalignment problem caused by differences in imaging size, field of view, and imaging mechanism among different sensors, and significantly improves the stability and consistency of cross-modal fusion.

[0047] 2. This invention employs modality-aware feature extraction and dynamic compensation modeling, setting an independent feature extraction branch for each input modality and introducing a deformable convolutional structure to adaptively adjust the feature sampling position. This effectively alleviates the nonlinear spatial offset caused by factors such as disparity and heat source distribution changes in local regions of different modalities, achieving semantic consistency modeling of local regions and preserving the inherent structural characteristics of each modality.

[0048] 3. This invention constructs a cross-modal implicit spatial alignment mechanism, using shared proxy semantic nodes as intermediaries to sequentially perform global semantic aggregation and reverse semantic broadcasting. This establishes implicit semantic correspondences between multimodal features in a high-dimensional semantic space, effectively solving the semantic gap problem caused by differences in imaging mechanisms, response modes, and semantic expression methods among different spectral modalities. This achieves efficient semantic alignment and consistent expression of features from different modalities.

[0049] 4. This invention employs an adaptive feature fusion mechanism to sequentially generate spatial attention weights and channel attention weights, and adaptively allocates the weights of each modality according to scene characteristics. This effectively characterizes the effectiveness of different modalities in different spatial locations and channel dimensions, avoids the problem of weakening effective information or amplifying noise, and significantly improves the robustness of fusion.

[0050] 5. This invention constructs a multi-scale feature pyramid and a target detection network, using fused feature representation as the backbone input. Through lateral connection and upsampling fusion mechanisms, high-level semantic information is fed back to the high-resolution feature layer, enhancing the discrimination ability of small targets and realizing unified modeling of multi-scale targets. This effectively solves the problem of decreased detection performance caused by significant changes in target scale in complex scenes. Attached Figure Description

[0051] Figure 1 This is a schematic diagram of the overall process of a multi-source optoelectronic image intelligent target detection method according to an embodiment of the present invention.

[0052] Figure 2 This is a detection result diagram of a smoke-obscured scene in Embodiment 3 of the present invention.

[0053] Figure 3 This is a detection result diagram of a strong light interference scene in Embodiment 3 of the present invention.

[0054] Figure 4 This is a night scene detection result image of Embodiment 3 of the present invention.

[0055] Figure 5 This is a diagram showing the small target detection results according to an embodiment of the present invention. Detailed Implementation

[0056] The present invention will be further described in detail below with reference to the accompanying drawings.

[0057] Example 1

[0058] Reference Figure 1 This embodiment of a multi-source photoelectric image intelligent target detection method includes:

[0059] Acquire multi-source photoelectric images and perform standardization processing to obtain standardized multimodal input data;

[0060] Modality-aware feature extraction is performed on multimodal input data, followed by dynamic compensation modeling and deep semantic modeling to generate intermediate semantic feature representations.

[0061] Cross-modal implicit spatial alignment is performed based on intermediate semantic features, including spatial flattening of the intermediate semantic feature representation, constructing cross-modal shared proxy semantic nodes, and performing global semantic aggregation and reverse semantic broadcasting of multimodal features through proxy semantic nodes to generate aligned cross-modal feature representations.

[0062] The aligned cross-modal feature representations are sequentially used to generate spatial attention weights and channel attention weights. These weights are then fused through element-wise weighting and channel compression to generate a unified fused feature representation.

[0063] Using fused feature representation as the main input, multi-scale feature pyramid construction and target detection are performed, and the output includes a target location regression branch, a target category prediction branch, and a target confidence branch.

[0064] Coordinate decoding and confidence screening are performed on the prediction results at each scale. The non-maximum suppression algorithm is used to remove redundant overlapping boxes, and the final target detection result set is output.

[0065] Specifically, a multi-source photoelectric image intelligent target detection method includes the following:

[0066] like Figure 1 As shown, step S1 is first executed, which involves acquiring multi-source photoelectric images and performing standardization processing to obtain standardized multimodal input data. In this embodiment, the multi-source photoelectric images include visible light images and infrared images, wherein the visible light image is denoted as... Infrared images are denoted as H represents the image height, W represents the image width, and "3" indicates that the visible light image is in three-channel RGB format, while "1" indicates that the infrared image is in single-channel thermal imaging format. Because images from different spectral bands have significant differences in imaging mechanisms, they exhibit inconsistencies in spatial location, scale, field of view coverage, brightness distribution, and detail retention. Directly inputting unaligned multimodal images into subsequent fusion modules will lead to mismatches in the position and scale of the same target in different modalities, and even semantic conflicts, thereby disrupting the correspondence between cross-modal features. Therefore, it is essential to standardize multi-source optoelectronic images.

[0067] The standardization process first performs spatial scale unification, scaling the input visible light and infrared images to a preset resolution to eliminate the impact of differences in imaging size and field of view between different sensors on subsequent feature modeling. In this embodiment, the scaling process uses bilinear interpolation to resample the images, which maintains image structural continuity and grayscale smoothness while ensuring computational efficiency. After scaling, the modal images are rearranged into a standard tensor format acceptable to the network. Where B represents the batch size, C represents the number of channels, H represents the unified image height, and W represents the unified image width.

[0068] After achieving spatial scale unification, geometric registration is further performed. Due to differences in installation location, imaging angle, and imaging mechanism among different spectral sensors, even after spatial scaling, overall spatial misalignment may still exist between different modal images caused by parallax effects, local heat source changes, and imaging distortion. Therefore, this embodiment, based on the principle of structural consistency alignment, extracts edge and gradient features from different modal images as structural descriptors. By calculating the minimum matching error between structurally similar regions, the two-dimensional geometric transformation matrix K is solved, and this two-dimensional geometric transformation matrix K is applied to the non-reference modal image to achieve global spatial geometric registration. The geometric transformation process is expressed as follows: ,in, Infrared modal image, The image is an infrared image after geometric registration. This is the two-dimensional affine transformation matrix obtained through structural similarity matching. These are the image pixel coordinates. Through this process, a basic correspondence is established between different modal images in coordinate space, thereby eliminating the overall spatial misalignment problem caused by installation position, imaging angle, and imaging mechanism.

[0069] After completing global spatial geometric registration, considering the slight nonlinear shifts caused by changes in local heat source distribution, non-rigid deformation, and parallax effects, this embodiment further performs implicit feature space alignment. Specifically, this embodiment uses a hierarchical feature extraction backbone network to perform feature space alignment on the two inputs: visible light images. With infrared images Encoding is performed separately to obtain multi-level feature maps X. Mid-to-high-level features are selected as alignment targets to balance local structural details and semantic stability. Deformable convolution introduces a learnable sampling offset based on standard convolution, dynamically adjusting the position of each convolution sampling point. For the input feature map X, the calculation process of standard convolution can be represented as follows: ,in, This represents the position index on the output feature map, and R represents the regular sampling grid of the convolution kernel. This represents the weights of the convolution kernel.

[0070] In deformable convolutional structures, an offset set is introduced. The convolution sampling locations are expanded from a regular grid to a learnable dynamic location distribution, and the computation process is updated as follows: The offset It is obtained through adaptive learning by a parallel offset prediction branch network and is used to characterize the spatial misalignment relationships of different modalities in the local structure. To output feature map at location eigenvalues ​​at that location Let R be the input feature map, and R be the regular sampling grid of the convolution kernel. For the convolution kernel at position Weight at each location.

[0071] To further improve local alignment accuracy, this embodiment introduces a cross-modal structure guidance mechanism based on the deformable convolutional structure. This ensures that offset prediction not only relies on single-modal features but also jointly considers multi-modal structural responses, thereby enabling the sampled offset to more accurately reflect the local correspondence between cross-modal modes. Specifically, intermediate features from the visible and infrared modes are structurally fused and input into the offset prediction branch to generate a joint offset field. .in, Indicates the offset prediction network, This represents the result of multimodal feature concatenation. This indicates a splicing operation.

[0072] The offset prediction branch is implemented using a hierarchical convolutional structure: firstly, a 1×1 convolution is used to compress the input multimodal fusion features to obtain intermediate features. ,in The compressed channel count is used to reduce subsequent computational complexity and extract cross-modal shared semantic information. Two concatenated 3×3 convolutions are then used to extract displacement-related local structural information. These two 3×3 convolutions expand the effective receptive field through stacking, gradually refining the spatial distribution of the offset. Each convolution is followed by batch normalization and a ReLU activation function to stabilize the training process and enhance nonlinear expressive power. Finally, the offset Δp is generated through 3×3 convolutions. If the deformable convolution kernel size is k×k, the offset prediction branch outputs 2×k×k channels, representing the horizontal and vertical displacement components of k×k sampling points. When k=3, the output channel count is 18, corresponding to the x- and y-direction offsets of the 9 sampling points in the 3×3 convolution kernel. Optionally, to further adjust the contribution of different sampling points, the offset prediction branch can simultaneously output modulation coefficients. Its output channel number is k×k, which is 9 when k=3, and is normalized to the [0,1] interval by the Sigmoid function. The modulation coefficient is used to weight the sampled values ​​of deformable convolution point by point, realize the adaptive adjustment of the importance of different sampling points, suppress invalid sampling in the background area, and enhance the key sampling in the target area.

[0073] This structural design enables deformable convolution to comprehensively utilize multimodal structural information during the sampling phase, allowing for more precise alignment compensation of local semantic regions. After dynamic sampling via deformable convolution, multimodal features undergo implicit secondary alignment in the feature space, resulting in a consistent structural distribution of response regions for the same target across different modalities in the high-dimensional semantic space. This leads to the obtained aligned shallow feature representations. Where B represents the batch size. Indicates the number of modal feature channels. and These represent feature map height and width being half the size of the input image, respectively. Through the two-level alignment mechanism of spatial scale unification, geometric registration, and implicit alignment of feature space described above, this embodiment achieves consistent modeling of multimodal features at both the spatial structure and semantic expression levels.

[0074] S2. After completing multi-source image registration and standardization, image data from different spectral modalities such as visible light, near-infrared, mid-wave infrared, and short-wave infrared have been unified to a consistent spatial scale, making each modality comparable and fusionable at the input layer. Based on this, this embodiment designs a multimodal semantic feature modeling module to extract discriminative multi-level semantic feature representations from different imaging modalities, providing a structurally consistent and semantically alignable feature foundation for subsequent cross-modal implicit spatial alignment and fusion modeling. The unit's structural design fully considers the differences in imaging mechanisms of multi-source optoelectronic images, including differences in texture density, semantic integrity, and response patterns. Through modal-independent encoding, geometric compensation modeling, and hierarchical semantic aggregation mechanisms, it achieves targeted modeling and balanced representation of multimodal information.

[0075] Specifically, to avoid premature mixing of different modal information at lower levels, which could damage its inherent structural characteristics, this invention sets up an independent feature extraction branch for each input modality to maintain the structural integrity of different spectral modal information. Let the standardized multimodal input tensors be as follows: , Where B represents the batch size. Let H represent the number of modal feature channels, H represent the unified image height, and W represent the unified image width. Within each modal branch, low-level semantic modeling of the input tensor is first performed using a basic convolutional coding structure. This process consists of a cascade of ordinary convolution, batch normalization, and SiLU activation functions, and its computation can be represented as follows: .in, This represents a two-dimensional convolution operation. This indicates a batch normalization operation. This represents the SiLU activation function. Through this basic encoding structure, the original pixel tensor is mapped to a stable low-level texture semantic feature representation. ,in, This indicates the number of feature channels after encoding.

[0076] Due to the differences in imaging mechanisms across different spectra, even after spatial registration is completed in the input stage, nonlinear spatial offsets may still exist in local regions caused by factors such as parallax and changes in heat source distribution for different modalities. To improve the alignability of cross-modal geometric correspondences, this embodiment introduces a deformable convolutional structure in the modality encoding stage to dynamically compensate for and model the feature sampling positions. After deformable convolutional modeling, the features of each modality are updated to... Where B represents the batch size. H represents the number of feature channels obtained after convolutional encoding, H represents the feature map height, and W represents the feature map width.

[0077] After completing cross-modal geometric compensation, this embodiment further employs a hierarchical semantic aggregation structure to perform deep semantic modeling of the features. This structure consists of multi-path convolutional branches, achieving semantic expression balance between modalities with different information density through multi-scale modeling. Specifically, the hierarchical aggregation structure includes two functional branches: a main semantic path and a detail-preserving path. The main semantic path uses a convolutional structure with a large receptive field to extract high-level semantic features, used to model the overall contour of the target and its semantic context. The detail-preserving path uses a small-scale convolutional structure to retain low-level texture details, used to maintain the fine-grained texture expression capability of the visible light modality. Let the multi-path output features be... .in, Represents convolutional branch structures at different scales. Subsequently, the multi-path features were fused using channel splicing and then... Convolution performs channel compression and semantic remodeling on the fused features, which can be represented as follows:

[0078] ,

[0079] Finally, we obtain intermediate semantic feature tensors with consistent shapes. Where B represents the batch size. This indicates the number of feature channels obtained after convolutional encoding. Indicates the feature map height. Indicates the width of the feature map.

[0080] Through the above multi-stage modeling process, this embodiment effectively captures stable target response patterns in the infrared spectrum while preserving fine-grained texture information of visible light images. Furthermore, it alleviates the problem of uneven information density between different modalities through dynamic convolution and multi-path aggregation mechanisms, enabling multi-source optoelectronic features to possess spatial alignability, semantic balance, and cross-modal matching before entering the cross-modal implicit spatial alignment module.

[0081] S3. In the joint modeling of multi-source optoelectronic images, due to the differences in imaging mechanisms, response modes, and semantic expression methods among different spectral modalities, even after front-end geometric registration and semantic feature modeling, there may still be problems such as inconsistencies in local structures, uneven response densities, and semantic interest region shifts between different modal features. These problems make it difficult for traditional cross-modal fusion based on pixel or rule attention mechanisms to establish stable and reliable semantic correspondences, thus restricting the overall fusion performance. To address this, this invention proposes a cross-modal proxy semantic alignment mechanism. By constructing a set of cross-modal shared proxy semantic nodes, the global semantic structure of multimodal features is explicitly modeled in the feature space. Using the proxy semantic nodes as an intermediary, efficient semantic alignment and consistent expression between different modal features are achieved. This module, as the core semantic alignment sub-module in the multimodal semantic feature modeling unit, is used to establish implicit semantic correspondences between different modalities within the feature space, enabling consistent expression of multimodal features in terms of spatial distribution and semantic response, providing a unified semantic feature input basis for subsequent adaptive fusion and intelligent detection units.

[0082] Let the multimodal feature tensors output from the multimodal semantic feature modeling unit be respectively .in, Represents the characteristics of visible light modes. Representing infrared modal characteristics. The system first spatially flattens the multimodal characteristics, which can be represented as follows: ,in Subsequently, a set of proxy semantic nodes is constructed using a global semantic convergence operator. .in, This indicates a cross-modal feature concatenation operation. This represents a global average pooling operation, which ultimately generates... .in, , representing the number of proxy semantic nodes. Furthermore, the number of proxy semantic nodes M can be adaptively determined based on the scene complexity of the input image to reduce computational overhead while ensuring semantic expressiveness. Specifically, the system first constructs a scene complexity index based on multimodal features. Taking into account both the intensity of cross-modal differences and the richness of spatial structure:

[0083] ,

[0084] in, This is a variance operation used to measure the overall activity level of the feature space. Information entropy calculation is used to characterize the complexity of spatial structures. This is a global mean calculation used to calculate the global response intensity of each mode. , , These are weighting coefficients used to balance the contribution ratios of the three metrics. Then, the scene complexity metric is... Mapped to the number of proxy semantic nodes:

[0085] ,

[0086] in, and These represent the lower and upper limits for the number of proxy nodes, respectively, to ensure basic alignment accuracy and prevent excessive computational overhead. This is a scaling factor used to adjust the mapping ratio between scene complexity and the number of proxy nodes. This represents the truncation function, used to limit the calculation results within a reasonable range. Therefore, the system automatically selects a smaller M when the scene is simple or the targets are sparse, while automatically increasing M in scenes with dense targets, complex occlusion, or significant cross-modal differences, thus achieving an adaptive trade-off between computational load and alignment accuracy.

[0087] The proxy semantic node is used to characterize the global semantic structure and key target distribution in the current scene. During the proxy aggregation phase, the proxy semantic node serves as the query vector to perform global semantic aggregation of multimodal features. .in, Then, the attention weights between the proxy semantic nodes and the multimodal features are calculated. And complete semantic aggregation .in, Softmax represents the activation function, where Softmax is the feature dimension scaling factor. The main function of this stage is to aggregate global semantic information from the multimodal feature space into the proxy semantic nodes, enabling the proxy nodes to possess consistent semantic expression capabilities across modalities. During the proxy broadcasting stage, the system uses multimodal features as queries and proxy semantic nodes as key-value pairs to broadcast the global semantic structure back to the feature space. The broadcast weight can be expressed as: By broadcasting weights, we finally obtain the aligned cross-modal feature representation. Finally, it is rearranged into a two-dimensional feature map form. Through this cross-modal proxy semantic alignment mechanism, this invention achieves dynamic alignment and consistent modeling of multi-source optoelectronic features in a high-dimensional semantic space, providing a high-quality cross-modal semantic feature input foundation for subsequent adaptive fusion and intelligent detection modules.

[0088] S4. After completing cross-modal proxy semantic alignment, features from different spectral modalities have established correspondences in the semantic space. However, since attention-weighted averaging weakens effective information or amplifies noise, relying solely on feature fusion is insufficient to complete the final target detection task. Therefore, this embodiment proposes an adaptive feature fusion and intelligent detection module, enabling fused features to perform target localization and category discrimination in semantic spaces at different scales, forming a complete end-to-end detection chain.

[0089] Specifically, the input to the adaptive feature fusion module is the cross-modal feature tensor aligned by the cross-modal implicit space alignment module. To facilitate model calculations, a unified data feature representation is obtained through a linear layer. ,in This indicates a linear layer.

[0090] To effectively characterize the effectiveness of different modalities at different spatial locations and channel dimensions, the adaptive feature fusion module first processes the features... Spatial attention weights are generated. The system performs max pooling and average pooling operations on the two modal features respectively to extract the distribution features of local salient and robust regions. The concatenated features are then input into a lightweight convolutional structure, and spatial attention weights are obtained through sigmoid mapping.

[0091] ,

[0092] in, This indicates that spatial attention weights are obtained through the Sigmoid mapping. This indicates a convolution operation. This indicates a max pooling operation. This indicates the average pooling operation. This represents the learnable weights. This attention map can highlight the target region and suppress noisy regions, thus making the fusion process more focused on regions that are co-significant across modalities.

[0093] Based on the spatial attention weights, the adaptive feature fusion module further models the reliability differences of different modalities along the channel dimension. The system employs global average pooling to obtain the statistical features of each modality in the overall response, and establishes a nonlinear channel mapping relationship through a two-layer fully connected network (MLP) to generate channel attention weights.

[0094] ,

[0095] in, This represents the channel attention weight. Indicates learnable weights, express Activation function. This channel weight can automatically evaluate the contribution of different modalities to the overall representation. For example, the infrared channel weight will be automatically increased in low-light environments, while the visible light channel weight will be stronger in high-contrast environments, thereby achieving adaptive weight allocation between modalities.

[0096] Simultaneously possesses spatial attention weights With channel attention weights Subsequently, the adaptive feature fusion module performs element-wise weighting on the modal features, simultaneously enhancing them in both spatial and semantic dimensions. The weighted two modal features are then concatenated along the channel dimension as follows:

[0097] ,

[0098] and through Convolution performs channel compression and fusion, ultimately yielding a unified fused output feature representation:

[0099] ,in, The weighted visible light mode features, The weighted infrared modal features, which possess both spatial saliency modeling capability and modal contribution adaptive adjustment capability, are the sole source of input features for subsequent target detection.

[0100] S5. Considering the significant changes in target scale in complex scenes, and the different feature resolution requirements for small targets, distant targets, and occluded targets, this invention introduces a multi-scale feature pyramid structure to perform hierarchical modeling of fused features. Specifically, the system uses fused features... As the main input, a multi-scale semantic feature layer is constructed through downsampling convolution:

[0101] ,

[0102] in, These represent semantic feature layers at different spatial resolutions, used to characterize the semantic responses of small, medium, and large targets, respectively. This indicates a downsampling operation.

[0103] A feature pyramid is constructed between multi-scale feature layers through lateral connections and an upsampling fusion mechanism, enabling high-level semantic information to flow back to the high-resolution feature layer, thereby enhancing the ability to distinguish small targets. The lateral connection and upsampling fusion mechanism is implemented as follows: First, for each scale feature layer... Channel alignment was performed using 1×1 convolutions to obtain... ,in To ensure a consistent number of pyramid passages, , Let represent the height and width of the feature map at layer i, respectively. This channel alignment operation is used to eliminate the difference in the number of channels between feature layers of different scales, providing a unified feature representation space for subsequent lateral fusion. Then, layer-by-layer fusion is performed from top to bottom: high-level features... Through the upsampling operator Magnified to low-level features Mutually Using the same spatial resolution, and performing element-wise addition to complete the horizontal blending:

[0104]

[0105] in , Bilinear interpolation or nearest-neighbor interpolation can be used. Bilinear interpolation is suitable for scenes requiring smooth transitions, while nearest-neighbor interpolation is suitable for scenes requiring the preservation of sharp edges. To suppress the aliasing effect introduced by upsampling and further enhance the expression of local details, the system optimizes the fusion result. Applying a 3×3 convolution smoothing operation yields the final pyramid features:

[0106] ,

[0107] in This represents a 3×3 convolution operation, followed by batch normalization and activation functions. This is used to fuse semantic information from adjacent spatial locations and eliminate the grid effect caused by upsampling. Through the above lateral connection strategy, high-level semantic information can be effectively fed back to the high-resolution feature layer, thereby enhancing the ability to distinguish between small targets and occluded targets, while maintaining the computational efficiency and detection accuracy of the multi-scale feature pyramid.

[0108] After constructing the multi-scale feature pyramid, the system builds a target detection head for each scale feature layer to perform target localization and category prediction. Let each scale feature layer be... The detection head then outputs the target location regression branch for each scale feature layer. Target category prediction branch and target confidence branch .in, , , These represent the bounding box regression network, the category prediction network, and the target confidence prediction network, respectively. The bounding box regression network outputs a 4-dimensional vector, representing the offset of the bounding box center point relative to the preset anchor box and the scaling factor of the bounding box width and height. The category prediction network outputs a C-dimensional vector, which, after passing through the Softmax activation function, yields the probability distribution of each category, where C represents the total number of target categories. The target confidence prediction network outputs a 1-dimensional scalar, which, after passing through the Sigmoid activation function, yields the confidence score that the candidate box contains the target. Each scale detection head shares the same network structure parameters but has independent prediction branches for different scale features, thus achieving unified modeling of multi-scale targets. This multi-scale detection head design can simultaneously detect targets of different scales, with large targets detected at the high-resolution feature layer. The small target was effectively detected in the low-resolution feature layer. This allows for the acquisition of refined feature representations, thereby comprehensively improving target detection performance in complex scenarios.

[0109] S6. Perform coordinate decoding and confidence filtering on the prediction results at each scale, use the non-maximum suppression algorithm to remove redundant overlapping boxes, and output the final target detection result set.

[0110] First, coordinate decoding and confidence filtering are performed. Specifically, the multi-scale prediction results output by the detection head include target location regression branches, target category prediction branches, and target confidence branches on feature layers of different scales. Coordinate decoding is then performed on the prediction results output by the multi-scale target detection head, converting the relative offsets output by the bounding box regression network into absolute image coordinates. The coordinate decoding process includes: assuming the bounding box regression parameters output by the network are... ,in, , This represents the normalized value of the offset relative to the center coordinates of the preset anchor frame. , This represents the logarithmic scaling factor relative to the width and height of the anchor box; the network output is converted to absolute bounding box coordinates using the following decoding formula. :

[0111] ,

[0112] ,

[0113] ,

[0114] ;

[0115] in, The center coordinates, width, and height of the anchor frame are preset, and the confidence score of the network output is predicted in conjunction with the target confidence score. The process involves filtering, setting a confidence threshold, and removing candidate boxes with confidence levels below the threshold, retaining only high-confidence candidate detection results for further processing.

[0116] After completing coordinate decoding and confidence filtering, the system fuses the detection results at different scales. Since the same target may be detected repeatedly at multiple scales, there are a large number of redundant overlapping boxes in the output results. Therefore, the non-maximum suppression algorithm is used to remove the redundant overlapping boxes.

[0117] Specifically, for all candidate boxes within the same category, they are sorted in descending order of confidence score. The intersection-union ratio (IoU) between the highest-confidence candidate box and the remaining candidate boxes is calculated sequentially, and an IoU threshold is set. For the IoU with the current highest confidence bounding box, greater than Other candidate boxes are suppressed and removed; this process is repeated until all candidate boxes have been processed, thereby eliminating the problem of duplicate detection.

[0118] To improve the accuracy and robustness of bounding box regression, an L1-norm-based localization loss function is introduced during the training phase. Specifically, for the bounding box parameters output by the target location regression branch, a smoothed L1 loss is used as the localization loss. Let the bounding box parameter vector predicted by the network be... ,in, , This represents the offset of the bounding box center point relative to the corresponding anchor box center point. , This represents the logarithmic scaling factor of the bounding box width and height relative to the anchor box width and height. Let the corresponding true bounding box parameters be... The smoothed L1 loss function is defined as follows:

[0119] ,

[0120] in, These are the bounding box parameters predicted by the network. These are the corresponding true bounding box parameters. For a smooth L1 function, its piecewise definition is:

[0121] ,

[0122] in, This represents the error between the predicted and actual values ​​of the i-th bounding box parameters. This smoothing L1 loss is effective when the error is small. Using the L2 norm (squared loss) ensures gradient stability and fast convergence; even with large errors, i.e. Using the L1 norm (absolute value loss) reduces the interference of outliers on training, enhances the model's robustness to localization noise, and avoids the gradient explosion problem.

[0123] To further improve the bounding box localization accuracy, an L1 norm regularization term can be introduced to impose sparse constraints on the weight parameters of the bounding box regression network: Let the set of learnable weight parameters of the bounding box regression network be... ,in, Let j represent the j-th weight tensor in the network; the L1 norm regularization term is defined as:

[0124] ,

[0125] in, It is an L1 norm. Let w be the k-th element of the weight tensor w. This represents the total number of elements in the tensor. When both localization loss and sparsity constraints are applied, a combined form of the overall regression loss function is constructed:

[0126] ,

[0127] in, L1 regularization is a regularization coefficient used to control the strength of sparse constraints. It can be determined through validation set tuning. This L1 regularization term prompts the network to learn sparse feature representations, suppresses the interference of redundant features, and makes the bounding box regression network focus on the most discriminative feature dimensions, thereby improving the model's generalization ability and detection stability, and reducing the risk of overfitting.

[0128] The final output set of target detection results is represented as ,in This represents the final set of target detection results. This represents the bounding box position of the k-th target, including the center coordinates, width, and height information of the bounding box. This represents the target category of the k-th target. This represents the target confidence score for the k-th target. This indicates the final number of detected targets. This final detection result set can be directly used as an input interface for optoelectronic pod target recognition systems, intelligent surveillance systems, or complex environment perception systems, for engineering applications such as target tracking, threat assessment, and intelligent decision-making, enabling intelligent perception and application output of multi-source optoelectronic information.

[0129] Example 2

[0130] This embodiment provides a simulation experiment of a multi-source optoelectronic image intelligent target detection method;

[0131] To verify the effectiveness and robustness of the proposed algorithm, experiments were conducted on multiple public datasets and representative complex scene data. Some detection results are shown in the figure. The results show that the proposed method can maintain relatively stable detection performance under various complex interference conditions.

[0132] like Figure 2 As shown, in scenes with smoke obscuring the image, the target outline is blurred and the contrast is significantly reduced in the visible light image. After introducing multi-source optoelectronic information fusion, the infrared image can provide a stable target response, effectively compensating for the information attenuation of visible light under smoke conditions, so that the model can still accurately locate the target area.

[0133] like Figure 3 As shown, in strong light interference scenarios, local areas in the visible light image are overexposed, resulting in severe loss of target details. The fusion model, by introducing infrared information, significantly suppresses the impact of strong light areas on the detection results and reduces the false detection rate.

[0134] like Figure 4 As shown, in low-light or nighttime scenes, visible light images have insufficient overall brightness and increased noise levels, making it difficult to extract target features, while infrared images exhibit stronger target response capabilities in such scenes.

[0135] like Figure 5 As shown, experimental results demonstrate that the proposed algorithm can adaptively adjust the contribution ratio of different modal features according to scene characteristics, maintaining good stability while ensuring detection accuracy. Furthermore, in scenes with complex backgrounds and significant target scale variations, the synergistic effect of multi-scale feature modeling and cross-modal fusion mechanisms enables the model to exhibit superior robustness in detecting small and distant targets.

[0136] Example 3

[0137] The difference between this embodiment and Embodiment 1 is that this embodiment provides a multi-source photoelectric image intelligent target detection system;

[0138] The target detection system in this embodiment consists of a data registration and preprocessing module, a modal-aware feature modeling module, a cross-modal implicit space alignment module, an adaptive feature fusion module, an intelligent detection module, and an output module, which are interconnected according to the input, processing, alignment, fusion, detection, and output links.

[0139] Specifically, the data registration and preprocessing module serves as the system's input foundation. It performs geometric correction, brightness and contrast normalization, and multimodal consistency enhancement on raw images from different spectral sensors, ensuring that the multimodal images achieve a unified spatial scale, grayscale distribution, and statistical structure before entering the deep network. Without this normalization process, differences in scale, texture, and response intensity among the multimodal images will disrupt the common subspace structure for subsequent feature extraction, thus affecting the stability of the overall modeling chain. The normalized tensor output by this module serves as the direct input to the modality-aware feature modeling module. The modality-aware feature modeling module receives the normalized multimodal input, constructs an independent semantic encoding path for each spectral modality, and forms a scale- and channel-consistent intermediate semantic feature representation through basic convolutional coding, structural compensation, and hierarchical semantic aggregation. This stage preserves the fine-grained texture of visible light, stably captures the target response patterns of infrared light, and provides a structurally consistent semantic foundation for cross-modal alignment. Its output serves as the input to the cross-modal implicit spatial alignment module. The cross-modal implicit space alignment module takes the same-scale semantic features output by the modality-aware feature extraction module as input. Through attention interaction in the query, key, and value projection spaces, it establishes a one-to-one soft matching relationship in the high-dimensional semantic space. Furthermore, it introduces cross-modal shared proxy semantic nodes, performs global semantic aggregation first, and then broadcasts the global semantic structure back to the feature space, thereby achieving dynamic alignment and consistent modeling of multimodal features.

[0140] This module's functionality relies on the structural consistency established in the previous stage; its output is a prerequisite for weighted integration by subsequent fusion modules. Without semantic alignment, modal fusion will produce conflicting effects and lead to information bias. After completing modal standardization, structural compensation, and semantic alignment, the adaptive feature fusion module uses spatial attention and channel attention to perform hierarchical weighting of the semantic contributions of each modality, and generates a unified fusion feature tensor through channel compression. This module can only perform effective fusion after receiving aligned features; its output is both a concentrated reflection of the system's intelligent fusion performance and the sole input source for subsequent multi-scale prediction by the detection head, structurally tightly coupled with the cross-modal implicit spatial alignment module. The fused features are then input into the target detection head for multi-scale semantic prediction.

[0141] The system uses fused features as the backbone to construct a multi-scale feature pyramid to adapt to targets of different scales. Parallel prediction branches are set up at each scale to complete: target location regression, target category discrimination, and target confidence assessment. The prediction results at each scale are fused after coordinate decoding and confidence filtering, and redundant overlapping boxes are removed by non-maximum suppression (NMS). Finally, the target detection result set (including bounding box position, target category, and confidence) is output.

[0142] A computer-readable storage medium storing a plurality of instructions adapted for loading and execution by a processor of a terminal device of the multi-source photoelectric image intelligent target detection method.

[0143] A terminal device includes a processor and a computer-readable storage medium, the processor being configured to implement various instructions; the computer-readable storage medium being configured to store multiple instructions adapted to be loaded and executed by the processor as described in the multi-source photoelectric image intelligent target detection method.

[0144] The above are all preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Therefore, all equivalent changes made in accordance with the structure, shape and principle of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A multi-source optoelectronic image intelligent target detection method, characterized in that, include: Acquire multi-source photoelectric images and perform standardization processing to obtain standardized multimodal input data; Modality-aware feature extraction is performed on multimodal input data, followed by dynamic compensation modeling and deep semantic modeling to generate intermediate semantic feature representations; Cross-modal implicit spatial alignment is performed based on intermediate semantic features, including spatial flattening of the intermediate semantic feature representation, constructing cross-modal shared proxy semantic nodes, and performing global semantic aggregation and reverse semantic broadcasting of multimodal features through proxy semantic nodes to generate aligned cross-modal feature representations. The global semantic aggregation includes spatially flattening the multimodal features to generate flattened feature tensors, constructing proxy semantic nodes through global semantic convergence operators, using the proxy semantic nodes as query vectors to perform global semantic aggregation on the multimodal features, calculating the attention weights between the proxy semantic nodes and the multimodal features and completing semantic aggregation, so that the proxy nodes have consistent semantic expression capabilities across modalities. The reverse semantic broadcasting includes using multimodal features as queries and proxy semantic nodes after global semantic aggregation as key-value pairs to broadcast the global semantic structure back to the feature space. Then, the broadcast weight is calculated, and the aligned cross-modal feature representation is obtained through the broadcast weight. Finally, it is rearranged into a two-dimensional feature map. Spatial attention weights and channel attention weights are generated using the aligned cross-modal feature representations, and then a unified fused feature representation is generated by fusing element-wise weighting and channel compression. Using fused feature representation as the main input, multi-scale feature pyramid construction and target detection are performed, and the output includes a target location regression branch, a target category prediction branch, and a target confidence branch. Coordinate decoding and confidence screening are performed on the prediction results at each scale. The non-maximum suppression algorithm is used to remove redundant overlapping boxes, and the final target detection result set is output.

2. The intelligent target detection method for multi-source photoelectric images according to claim 1, characterized in that, The standardization process includes spatial scale unification and geometric registration. The input visible light and infrared images are uniformly scaled to a preset resolution. Bilinear interpolation is used for resampling. Based on the structural consistency alignment principle, edge and gradient features are extracted from different modal images as structural descriptors. The minimum matching error between structurally similar regions is calculated to solve for the two-dimensional geometric transformation matrix. This matrix is ​​then applied to the non-reference modal image to complete geometric registration. The geometric transformation process is represented as follows: ， in, Infrared modal image, The image is an infrared image after geometric registration. This is the two-dimensional affine transformation matrix obtained through structural similarity matching. These are the pixel coordinates of the image.

3. The intelligent target detection method for multi-source photoelectric images according to claim 2, characterized in that, The standardization process also includes implicit alignment in the feature space. By introducing a deformable convolutional structure in the feature extraction stage, the feature sampling position is adaptively adjusted, and an offset set is introduced to expand the convolutional sampling position from a regular grid to a learnable dynamic position distribution. Based on the deformable convolutional structure, a cross-modal structure guidance mechanism is introduced. Intermediate features from the visible and infrared modes are structurally fused and input into the migration prediction branch to generate a joint migration field. The expression for the deformable convolutional structure is: ， in, To output feature map at location eigenvalues ​​at that location For the input feature map, The regular sampling grid for the convolution kernel. For the convolution kernel at position The weight of the position, The sampling offset is obtained through adaptive learning by the offset prediction branch network.

4. The intelligent target detection method for multi-source photoelectric images according to claim 1, characterized in that, The dynamic compensation modeling and deep semantic modeling include: In the modality coding stage, a deformable convolutional structure is introduced to dynamically compensate and model the feature sampling locations. A hierarchical semantic aggregation structure is employed for deep semantic modeling of features. This structure consists of a main semantic path and a detail-preserving path. The main semantic path models the overall contour of the target and its semantic context, while the detail-preserving path uses a small-scale convolutional structure to retain low-level texture details. Finally, the features from multiple paths are fused using channel concatenation, and a 1×1 convolution is used to compress the fused features and perform semantic reshaping, generating an intermediate semantic feature tensor with a consistent shape. The expression for the hierarchical semantic aggregation structure is as follows: ， in, For intermediate semantic feature tensors, For the i-th multipath output feature, Let be the convolutional branch structure with different scales. This is for channel splicing operations. This is a 1×1 convolution operation.

5. The intelligent target detection method for multi-source photoelectric images according to claim 1, characterized in that, The global semantic convergence operator is represented as follows: ， in, For proxy semantic nodes, The number of proxy semantic nodes, and These are the visible light mode characteristics and infrared mode characteristics after flattening, respectively. For the number of spatial locations, For cross-modal feature splicing operations, This is a global average pooling operation. For batch size, The number of feature channels, and These represent the height and width of the feature map, respectively.

6. The intelligent target detection method for multi-source photoelectric images according to claim 5, characterized in that, The cross-modal feature is represented as follows: ， in, For broadcast weight, These are the proxy semantic features after global semantic aggregation. This is the projection matrix with multimodal features as the query. , This is the projection matrix with proxy semantic nodes as key-value pairs. The result of flattening the multimodal features is the stitching result. , and is a learnable projection weight matrix, d represents the feature dimension scaling factor, and Softmax represents the normalized exponential function.

7. The intelligent target detection method for multi-source photoelectric images according to claim 1, characterized in that, The generation of a unified fusion feature representation includes: Max pooling and average pooling operations are performed on the aligned cross-modal features to extract the distribution features of local salient regions and robust regions. The two are concatenated and then input into a lightweight convolutional structure, and spatial attention weights are obtained through Sigmoid mapping. Global average pooling is used to obtain the overall response statistical features of the aligned cross-modal feature representation after linear layer processing. A nonlinear channel mapping relationship is established through a two-layer fully connected network to generate channel attention weights. Having both spatial attention weights and channel attention weights, the modal features are weighted element-wise, the weighted modal features are concatenated along the channel dimension, and channel compression and fusion are performed through 1×1 convolution to generate a unified fused feature representation.

8. The intelligent target detection method for multi-source photoelectric images according to claim 1, characterized in that, The construction of the multi-scale feature pyramid and target detection includes using the fused feature representation as the main input and constructing a multi-scale semantic feature layer through downsampling convolution. The multi-scale semantic feature layer includes feature layers with different spatial resolutions that respectively represent the semantic responses of small, medium and large targets. By constructing a feature pyramid through lateral connectivity and upsampling fusion mechanism, high-level semantic information is fed back to the high-resolution feature layer, enhancing the ability to distinguish small targets; A target detection head is constructed for each scale feature layer, and the target detection head includes a bounding box regression network, a category prediction network, and a target confidence prediction network. The bounding box regression network outputs the target location regression branch, the category prediction network outputs the target category prediction branch, and the target confidence prediction network outputs the target confidence branch.

9. A multi-source photoelectric image intelligent target detection system, executing the method of claim 1, characterized in that, include: The preprocessing module is configured to acquire multi-source photoelectric images and perform standardization processing to obtain standardized multimodal input data; The feature modeling module is configured to: extract modality-aware features from multimodal input data, and then perform dynamic compensation modeling and deep semantic modeling in sequence to generate intermediate semantic feature representations; The alignment module is configured to perform cross-modal implicit space alignment based on intermediate semantic features, and generate aligned cross-modal feature representations. The feature fusion module is configured to: sequentially generate spatial attention weights and channel attention weights from the aligned cross-modal feature representations, and then fuse them through element-wise weighted fusion and channel compression to generate a unified fused feature representation; The intelligent detection module is configured to: construct a multi-scale feature pyramid and detect targets using fused feature representation as the main input, and output a target location regression branch, a target category prediction branch, and a target confidence branch; The output module is configured to perform coordinate decoding and confidence filtering on the prediction results at each scale, use a non-maximum suppression algorithm to remove redundant overlapping boxes, and output the final target detection result set.

10. A computer-readable storage medium storing a plurality of instructions, characterized in that, The instructions are adapted to be loaded and executed by the processor of the terminal device as described in claim 1, which is a multi-source optoelectronic image intelligent target detection method.

11. A terminal device, comprising a processor and a computer-readable storage medium, wherein the processor is configured to implement instructions; and the computer-readable storage medium is configured to store multiple instructions, characterized in that, The instructions are adapted to be loaded by a processor and executed as described in claim 1 for a multi-source optoelectronic image intelligent target detection method.

Images