An Optical-Infrared Target Detection Method Based on Coupled Nonlinear Dynamics

Abstract

This invention belongs to the field of image processing technology, specifically relating to an optical-infrared target detection method based on coupled nonlinear dynamics. This detection method breaks through the limitations of data-driven approaches, eliminating the necessity of traditional convolutional layers, and innovatively models visible light and infrared features as coherent complex wave fields. The invention designs coupled wave evolution units to achieve dynamic interaction between features, establishing bidirectional mutual inductance between modes through cross-phase modulation effects, achieving semantic alignment at the physical level between infrared thermal saliency guidance and visible light texture constraints. Simultaneously, a global frequency domain dispersion operator based on the split-step Fourier method is introduced to achieve low-computational-cost global information coupling and frequency domain edge sharpening, improving detection performance and robustness while maintaining the model's computational efficiency.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to an optical-infrared target detection method based on coupled nonlinear dynamics. Background Technology

[0002] Multimodal image fusion is considered an effective way to improve the reliability of target detection. Its core lies in utilizing the complementarity of different sensors in imaging mechanisms and information representation to compensate for the inherent defects of a single modality. Visible light images contain rich texture and fine-grained structural information, providing important basis for category discrimination and precise localization. However, under conditions such as low light, rain, fog, and smoke, image contrast and detail information degrade significantly, making appearance-based feature extraction and matching unstable. Infrared images are formed by target thermal radiation, making them more robust to changes in illumination and providing more stable saliency cues in low-visibility scenes; however, limited by sensor resolution and imaging characteristics, infrared images often lack texture details and target boundaries are more blurred, thus limiting the accuracy of fine recognition and localization. Therefore, obtaining stable and discriminative detection features in complex environments often requires relying on both visible light detail information and infrared thermal saliency information. Thus, how to efficiently fuse visible light and infrared images to achieve feature complementarity has become a key technical path to improve the robustness of target detection in complex scenes.

[0003] Most existing methods treat feature interactions as an end-to-end black-box transformation process. Their core logic lies in training network parameters with a large amount of data to statistically fit a certain fused feature distribution. However, this statistical mapping ignores the fluctuating nature of multimodal signals during propagation—that is, how energy flows from different modalities diffuse in space and how cross-modal mutual induction occurs—making it difficult to establish a theoretical mapping between the fusion process and the physical dynamics. Secondly, the dynamic coupling of global information is difficult to accommodate. Traditional operators often only extract local features within a limited receptive field, making it difficult to capture the long-range dependencies and nonlinear interactions of cross-modal features during dynamic evolution, resulting in poor performance in edge alignment and contrast enhancement of the fused features. Furthermore, existing high-performance detection frameworks, especially attention-based networks, often rely on stacked deep convolutional layers or highly complex attention layers to capture cross-modal features, leading to a significant increase in parameter and computational costs.

[0004] Therefore, in order to break through the traditional fusion paradigm based on statistical mapping, it is urgent to develop a physical-driven multimodal target detection framework based on coupled nonlinear dynamics for target detection in visible and infrared images. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of the prior art by providing an optical-infrared target detection method based on coupled nonlinear dynamics physics, in order to solve the problems of lack of clear physical prior constraints in multimodal image fusion interaction and difficulty in balancing long-range dependency modeling and computational efficiency when capturing global dynamic coupling information.

[0006] The objective of this invention is achieved through the following technical solution:

[0007] An optical-infrared target detection method based on coupled nonlinear dynamics includes the following steps:

[0008] S1, input the images of visible light and infrared modes into their respective feature extraction backbone networks to obtain multi-scale feature maps for each mode;

[0009] S2, input the bimodal feature maps at various scales into the coupled wave evolution unit, construct the initial complex wave field through the feature evolution modeling module in the unit, map the features to the complex domain, so that the features have physical wave properties, and then simulate the dynamic interaction and evolution of the bimodal features;

[0010] S3, for the nonlinear coupled evolution of visible light features and infrared features, adopts the step-by-step Fourier method to discretize the continuous evolution into differentiable neural operator layers. First, the evolution operator is decomposed into linear operators and nonlinear operators using symmetric decomposition. After going through a complete evolution cycle from solving nonlinear operators to solving linear operators and then to solving nonlinear operators, the interactive evolution of dual-modal features is realized.

[0011] S4. The interference principle of wave physics is used to perform coherent superposition on the evolved complex field to form a fused wave field. The complex modulus extraction method is used to return the complex fused field to the real space that the target detector can process, and the fused features after modal interaction are obtained.

[0012] S5, the final multi-scale fusion features are input to the detection neck and head to decode and output the location and category information of the target in the image.

[0013] Further, step S1 specifically includes the following steps:

[0014] A dual-branch feature extraction architecture is adopted, in which a pair of spatially registered visible light images and infrared images are respectively input into two parallel convolutional neural network backbones for feature extraction; then, feature maps are obtained at different depth stages of the two backbone networks. The spatial resolution of these feature maps is gradually reduced and the semantic information is gradually enhanced, thereby forming a set of discrete multi-scale feature maps for each of the visible light and infrared modes.

[0015] Further, step S2 specifically includes the following steps:

[0016] S21, the visible light features obtained in step S1 With infrared features Mapping to the complex domain, we construct the initial complex wave field:

[0017] ;

[0018] in, For spatial coordinates, ;

[0019] S22, Construct a nonlinear Schrödinger heuristic for feature evolution modeling to simulate the dynamic interaction of bimodal features. The dynamic evolution of the bimodal feature field is controlled by the following set of partial differential equations:

[0020] ;

[0021] ;

[0022] Among them, the linear dispersion term Corresponding to controllable propagation items, It is self-phase modulation. It is cross-phase modulation.

[0023] Further, step S3 specifically includes the following steps:

[0024] S31 uses the step-by-step Fourier method to discretize the continuous evolution into differentiable neural operator layers, and the step-by-step Fourier method decomposes the evolution operator into linear operators. and nonlinear operators For a given evolution step size The solution is approximately in the form of an operator exponential product:

[0025] ;

[0026] S32, the symmetric decomposition has the characteristics of time symmetry and second-order accuracy, and decomposes the complex coupled evolution into three ordered substeps: first, half-step local nonlinear phase modulation is executed, then the full-step global linear dispersion operator is executed, and finally, half-step local nonlinear phase modulation is executed again.

[0027] Furthermore, local nonlinear phase modulation is used to process the characteristic wave field in the spatial domain. The instantaneous interaction at the point of origin, while maintaining constant feature energy, achieves deep semantic alignment by changing the phase structure; in the half-step evolution in the real domain, the phase shift of the bimodal feature field is driven by both self-phase modulation and cross-phase modulation:

[0028] ;

[0029] in, It is a visible light characteristic of self-phase modulation. It is an infrared feature with self-phase modulation. is the coupling coefficient.

[0030] Furthermore, in the linear dispersion evolution step, a global linear dispersion operator is designed to realize dispersion evolution. Linear propagation essentially corresponds to the Laplacian operator, and its key lies in global information interaction rather than local convolutional fitting. To complete a one-time global propagation with lower complexity, the linear process is mapped to the frequency domain for solution, letting... This is the result of a two-dimensional Fourier transform. Define frequency coordinates. CWEM applies a learnable phase propagation factor to each frequency component in the frequency domain:

[0031] ;

[0032] in, The dispersion intensity is learned per channel.

[0033] Furthermore, in step S4, after physical evolution, the cross-modal consistency of the dual-mode characteristic wave field and the phase relationship has been explicitly encoded. In order to return the evolved complex wave field to the real-valued feature space compatible with the target detector, the interference principle of wave physics is used to modify the evolved complex field. and Perform coherent superposition to form a fused wave field :

[0034] ;

[0035] in, and Learnable modal weights are used to adaptively adjust the contributions of the two modes under different environmental conditions; complex modulus extraction is employed to regress the complex fused field to the real space that the target detector can process.

[0036] ;

[0037] in, Represents taking the real part of a complex number. It represents the imaginary part of a complex number. Attached Figure Description

[0038] Figure 1 A flowchart of an optical-infrared target detection method based on coupled nonlinear dynamics provided by the present invention;

[0039] Figure 2The network structure diagram of an optical-infrared target detection method based on coupled nonlinear dynamics physics provided by this invention is shown. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0041] like Figure 1 As shown, this invention presents an optical-infrared target detection method based on coupled nonlinear dynamics. First, the input visible light and infrared images are converted into multi-scale features containing different levels of information for subsequent processing through a parallel feature extraction network. Then, the features are mapped to the complex domain using coupled wave evolution units, simulating the dynamic interaction and evolution of the dual-modal features. Next, a stepwise Fourier transform is used to discretize the continuous evolution into differentiable neural operator layers. Symmetric decomposition further decomposes the evolution operators into linear and nonlinear operators, and these are alternately solved to achieve the interactive evolution of the dual-modal features. Finally, the evolved dual-modal features are coherently superimposed and complex modulus extracted to obtain fused features, which are then input to detect the neck and head, ultimately decoding and outputting high-precision target location and category information. Specifically, the method includes the following steps:

[0042] S1 inputs the visible light and infrared images into their respective feature extraction backbone networks to obtain multi-scale feature maps for each modality.

[0043] S2 inputs the bimodal feature maps at various scales into the coupled wave evolution unit. The initial complex wave field is constructed through the feature evolution modeling module in this unit, and the features are mapped to the complex domain to give the features physical wave properties. Then, the dynamic interaction and evolution of the bimodal features are simulated.

[0044] S3, for the nonlinear coupled evolution of visible light features and infrared features, uses the step-by-step Fourier method to discretize the continuous evolution into differentiable neural operator layers. First, the evolution operator is decomposed into linear operators and nonlinear operators using symmetric decomposition. After a complete evolution cycle from solving nonlinear operators to solving linear operators and then back to solving nonlinear operators, the interactive evolution of dual-modal features is realized.

[0045] S4 utilizes the interference principle of wave physics to perform coherent superposition on the evolved complex field to form a fused wave field, and uses a complex modulus extraction method to revert the complex fused field to the real space that the target detector can process, thus obtaining the fused features after modal interaction.

[0046] S5, the final multi-scale fusion features are input to the detection neck and head to decode and output the location and category information of the target in the image.

[0047] Step S1 specifically includes the following steps:

[0048] A dual-branch feature extraction architecture is adopted, in which a pair of spatially registered visible light images and infrared images are respectively input into two parallel convolutional neural network backbones for feature extraction; then, feature maps are obtained at different depth stages of the two backbone networks. The spatial resolution of these feature maps is gradually reduced and the semantic information is gradually enhanced, thereby forming a set of discrete multi-scale feature maps for each of the visible light and infrared modes.

[0049] Step S2 specifically includes the following steps:

[0050] To give the features fluctuating physical properties, the visible light features obtained in step S1 will be... With infrared features Mapping to the complex domain. Constructing the initial complex wave field:

[0051] ;

[0052] in, For spatial coordinates, The above construction corresponds to zero-phase, no-prior-initialization: the phase term of the initial wavefield is not explicitly encoded; in subsequent dynamic evolution, the imaginary part adaptively generates and forms a learnable phase distribution, allowing cross-modal differences to be explicitly encoded as phase shifts. Based on this, this method designs a nonlinear Schrödinger-heuristic feature evolution model to simulate the dynamic interaction of bimodal features. The dynamic evolution of the bimodal feature field is controlled by the following set of partial differential equations:

[0053] ;

[0054] ;

[0055] Among them, the linear dispersion term Corresponding to controllable propagation items, It is self-phase modulation. It is cross-phase modulation. This method uses frequency domain solution to make it behave as a global mixing operator, completing context aggregation and expanding the effective receptive field with a small number of evolution steps; self-phase modulation Phase nonlinearity makes it easier to selectively enhance high-intensity feature regions in coherent superposition. (Cross-phase modulation) It is the physical core for realizing multimodal feature fusion, revealing the mutual induction effect generated by two wave fields through a medium, which is the core mechanism of energy traction between modes. After establishing a continuous spatiotemporal evolution model of bimodal features at the theoretical level, it is transformed into a discrete neural operator that can be executed by the network.

[0056] Step S3 specifically includes the following steps:

[0057] The continuous evolution is discretized into differentiable neural operator layers using the step-by-step Fourier method, which then decomposes the evolutionary operators into linear operators. and nonlinear operators For a given evolution step size The solution is approximately in the form of an operator exponential product:

[0058] ;

[0059] Symmetric decomposition, with its time symmetry and second-order accuracy, maintains the stability of complex feature fields during multi-step evolution, avoiding information loss caused by nonlinear interactions and thus preserving the balance of cross-modal contributions. Furthermore, symmetric decomposition effectively suppresses edge blurring caused by numerical dispersion by reducing splitting errors. This high-order accuracy ensures that the fused feature map not only possesses infrared thermal saliency but also retains sharp geometric edges in the visible light region. This symmetric decomposition breaks down the complex coupled evolution into three ordered substeps: first, half-step local nonlinear phase modulation is performed; then, a full-step global linear dispersion operator is executed; and finally, half-step local nonlinear phase modulation is performed again.

[0060] Local nonlinear phase modulation is responsible for processing characteristic wave fields in the spatial domain. The instantaneous interactions at the point of origin, while maintaining constant feature energy, achieve deep semantic alignment by altering the phase structure. In the half-step of real-domain evolution, the phase shift of the bimodal feature field is driven by both self-phase modulation and cross-phase modulation:

[0061] ;

[0062] in, It is a visible light characteristic of self-phase modulation. It is an infrared feature with self-phase modulation. is the coupling coefficient.

[0063] Self-phase modulation enables the wave field intensity to adaptively change with its own intensity, achieving adaptive saliency purification within the mode. In visible light modes, visible light self-modulation... By adjusting the phase based on local intensity, greater phase accumulation is achieved in high-frequency texture regions, effectively enhancing the structural discriminability within the visible light mode. The energy of the infrared mode is typically concentrated on the heat source target, and infrared self-modulation is used to achieve this. A significant phase gradient is generated in the central region of an infrared target, which, during physical evolution, causes energy to further contract towards the target center, suppressing the diffusion artifacts commonly found in infrared sensors. Cross-phase modulation constitutes the core physical mechanism of cross-modal bidirectional traction. The significant intensity field of infrared exerts semantic guidance on the visible light phase evolution. The high-frequency geometric features of visible light impose boundary constraints on the infrared phase evolution. This makes the target area more uniform, eliminating background noise during superposition. Coupling coefficient The cross-modal guidance intensity is limited to avoid the dominant modulation process of noisy modes. After this local phase modulation is completed, the wavefield enters the linear propagation phase to achieve global semantic interaction.

[0064] As a core intermediate step in the stepwise Fourier symmetric decomposition, the linear dispersion evolution step connects the preceding and following nonlinear phase modulation half-steps. This method designs a global linear dispersion operator to realize dispersion evolution; linear propagation essentially corresponds to the Laplacian operator, and its key lies in global information interaction rather than local convolution fitting. To complete a one-time global propagation with lower complexity, the linear process is mapped to the frequency domain for solution. Let... This is the result of a two-dimensional Fourier transform. Define frequency coordinates. CWEM applies a learnable phase propagation factor to each frequency component in the frequency domain:

[0065] ;

[0066] in, This represents the dispersion intensity learned per channel. This frequency-domain operator leverages frequency domain properties to overcome the locality limitation of traditional convolution. Furthermore, unlike global mixing methods that learn arbitrary frequency-domain filter kernels, the propagation form of the global linear dispersion operator is determined by the physical structure. Constrained by the principle of learning only the propagation intensity, this method ensures both global modeling capability and stability. Through frequency domain phase modulation, this method enables local features to achieve linear coupling with global information without stacking deep networks. After completing a full evolution cycle of "nonlinear-linear-nonlinear", the bimodal features simultaneously undergo local coupling and global propagation in the complex domain, laying the foundation for subsequent coherent superposition to obtain a more consistent fused representation.

[0067] Step S4 specifically includes the following steps:

[0068] After physical evolution, the cross-modal consistency of the dual-mode characteristic wavefield and the phase relationship has been explicitly encoded. In order to revert the evolved complex wavefield to a real-valued feature space compatible with the target detector, the interference principle of wave physics is used to modify the evolved complex field. and Perform coherent superposition to form a fused wave field :

[0069] ;

[0070] in, and Learnable modal weights are used to adaptively adjust the contributions of the two modes under different environmental conditions. To regress the complex fusion field to the real space that the target detector can handle, this method employs complex modulus extraction.

[0071] ;

[0072] in, Represents taking the real part of a complex number. It represents the imaginary part of a complex number.

[0073] This process not only achieves coherent superposition of modal intensities but also utilizes the phase interference mechanism of fluctuations. The resulting enhanced features retain effective information from both modalities while suppressing cross-modal inconsistent backgrounds and artifacts, and are then fed into the neck network for multi-scale aggregation. This method first achieves alignment through physical evolution and then completes consistency enhancement through interferometric reconstruction, structurally improving edge and semantic discriminability and providing a higher-quality feature base for subsequent bounding box regression and class determination.

[0074] Example 1

[0075] This embodiment uses an optical-infrared target detection method based on coupled nonlinear dynamics, including the following steps:

[0076] S1 inputs the visible light and infrared images into their respective feature extraction backbone networks to obtain multi-scale feature maps for each modality.

[0077] S2, the bimodal feature maps at each of the above scales are input into the coupled wave evolution unit. The initial complex wave field is constructed through the feature evolution modeling module in the unit, and the features are mapped to the complex domain to give the features physical wave properties. Then, the dynamic interaction and evolution of the bimodal features are simulated.

[0078] S3, as Figure 2 As shown, the continuous evolution is discretized into differentiable neural operator layers using the step-by-step Fourier method. First, the evolution operator is decomposed into linear and nonlinear operators using symmetric decomposition. After a complete evolution cycle from solving the nonlinear operator to solving the linear operator and finally back to solving the nonlinear operator, the interactive evolution of the dual-modal features is realized.

[0079] S4 utilizes the interference principle of wave physics to perform coherent superposition on the evolved complex field to form a fused wave field, and uses a complex modulus extraction method to revert the complex fused field to the real space that the target detector can process, thus obtaining the fused features after modal interaction.

[0080] S5, the final multi-scale fusion features are input to the detection neck and head to decode and output the location and category information of the target in the image.

[0081] To verify the effectiveness of this invention, this embodiment uses registered visible light-thermal infrared image pairs acquired by a vehicle-mounted thermal imaging camera and a visual camera as data foundation to conduct cross-modal fusion detection experiments, and compares them with current newer multimodal fusion detection methods. The quantitative results are shown in Table 1. As can be seen from Table 1, this invention achieves better results than the comparative methods in several key evaluation indicators, indicating that the proposed method can more fully utilize the complementary information between visible light and thermal infrared images, effectively improving target detection performance.

[0082] Specifically, this invention demonstrates strong advantages in recall, detection accuracy, and overall evaluation metrics, indicating superior target recognition and localization capabilities in complex real-world scenarios. Especially in challenging conditions such as low light, long distances, and small targets, the proposed method reduces the risk of missed detections and improves the stability and reliability of detection results. In summary, through multi-level structural and mechanism innovations, this invention achieves overall improvements in recall, localization accuracy, and robustness, fully demonstrating its highly reliable sensing capabilities and practical engineering value for all-weather, multi-scenario applications.

[0083] Table 1. Quantitative evaluation of the fusion experiment between the present invention and the comparison algorithm.

[0084] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the claims.

[0085] It should be understood that the present invention is not limited to the precise structure shown in the above description, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A method for optical-infrared target detection based on coupled nonlinear dynamics, characterized in that, Includes the following steps: S1, input the images of visible light and infrared modes into their respective feature extraction backbone networks to obtain multi-scale feature maps for each mode; S2, input the bimodal feature maps at various scales into the coupled wave evolution unit, construct the initial complex wave field through the feature evolution modeling module in the unit, map the features to the complex domain, so that the features have physical wave properties, and then simulate the dynamic interaction and evolution of the bimodal features; S3, for the nonlinear coupled evolution of visible light features and infrared features, adopts the step-by-step Fourier method to discretize the continuous evolution into differentiable neural operator layers. First, the evolution operator is decomposed into linear operators and nonlinear operators using symmetric decomposition. After going through a complete evolution cycle from solving nonlinear operators to solving linear operators and then to solving nonlinear operators, the interactive evolution of dual-modal features is realized. S4. The interference principle of wave physics is used to perform coherent superposition on the evolved complex field to form a fused wave field. The complex modulus extraction method is used to return the complex fused field to the real space that the target detector can process, and the fused features after modal interaction are obtained. S5 inputs the final multi-scale fused features into the detection neck and head to decode and output the location and category information of the target in the image.

2. The optical-infrared target detection method based on coupled nonlinear dynamics according to claim 1, wherein, Step S1 specifically includes the following steps: A dual-branch feature extraction architecture is adopted, in which a pair of spatially registered visible light images and infrared images are respectively input into two parallel convolutional neural network backbones for feature extraction; then, feature maps are obtained at different depth stages of the two backbone networks. The spatial resolution of these feature maps is gradually reduced and the semantic information is gradually enhanced, thereby forming a set of discrete multi-scale feature maps for each of the visible light and infrared modes.

3. The method according to claim 1, wherein, Step S2 specifically includes the following steps: S21, mapping the visible features acquired from step S1 to the infrared features to the complex domain, constructing an initial complex wavefield: ； wherein for spatial coordinates, ; S22, Construct a nonlinear Schrödinger heuristic for feature evolution modeling to simulate the dynamic interaction of bimodal features. The dynamic evolution of the bimodal feature field is controlled by the following set of partial differential equations: ； ； wherein the linear dispersion term corresponding controllable propagation term, is self-phase modulation, is cross-phase modulation.

4. The method of claim 1, wherein the method is based on coupled nonlinear dynamics of optical-infrared target detection. Step S3 specifically includes the following steps: S31, discretizing the continuous evolution into differentiable neural operator layers using a piecewise Fourier method, the piecewise Fourier method decomposes the evolution operator into linear and nonlinear operators for a given evolution step size the solution is approximated in form by an operator exponential product:​ ； S32, the symmetric decomposition has the characteristics of time symmetry and second-order accuracy, and decomposes the complex coupled evolution into three ordered substeps: first, half-step local nonlinear phase modulation is executed, then the full-step global linear dispersion operator is executed, and finally, half-step local nonlinear phase modulation is executed again.

5. The optical-infrared target detection method based on coupled nonlinear dynamics according to claim 4, characterized in that: The local nonlinear phase modulation process features the instantaneous interaction of the characteristic wave field in the spatial domain The deep semantic alignment is achieved by changing the phase structure while keeping the constant characteristic energy. In the real domain evolution half-step, the phase shift of the bimodal characteristic field is jointly driven by self-phase modulation and cross-phase modulation. ； wherein, is the visible light characteristic self-phase modulation, is the infrared light characteristic self-phase modulation, is the coupling coefficient.

6. The optical-infrared target detection method based on coupled nonlinear dynamics according to claim 5, characterized in that: For linear dispersive evolution, a global linear dispersion operator is designed to implement the dispersive evolution. Linear propagation is essentially corresponding to Laplace operator. The linear process is mapped to frequency domain to solve, which leads to is the two-dimensional Fourier transform result, is the frequency coordinate, and is defined as CWEM applies a learnable phase propagation factor to each frequency component in the frequency domain. ； wherein, is the dispersion strength learned per channel.

7. The optical-infrared target detection method based on coupled nonlinear dynamics according to claim 1, characterized in that, In step S4, the evolved complex wavefield is returned to the real number feature space compatible with the target detector. Using the interference principle of wave physics, the evolved complex field and Performing coherent superposition to form a fused wavefield : ； wherein, and are learnable modality weights for adaptive adjustment of the two modality contributions under different environmental conditions; complex magnitude extraction is used to regress the complex fused field to a real number space processable by the target detector: ； wherein represents taking the real part of a complex number, represents taking the imaginary part of a complex number.

Images