Loading type double-spectrum hundred-million-pixel array calculation camera

Abstract

The invention relates to the technical field of camera shooting, and discloses a loading type double-spectrum hundred-million-pixel array calculation camera, which is characterized in that a visible light module and an infrared module are constructed into a double-spectrum array structure, and the double-spectrum array structure is subjected to partition thermal control and asynchronous sampling; geometric consistency and registration stability between the double-spectrum arrays can be maintained under the long-time high-power operation condition, so that the temperature gradient between imaging modules is balanced, the overall thermal noise of the arrays is remarkably reduced, and the transition smoothness of the splicing boundary is improved. Besides, through prediction compensation and online feedback correction of an optical mapping model, the camera can still keep optical axis consistency and imaging precision of high-resolution output under the conditions of maneuvering carrier vibration, external environment temperature change and power consumption fluctuation, and the application requirements of all-weather, full-vision-field and high-stability monitoring are met.

Description

Technical Field

[0001] This invention relates to the field of camera technology, and more particularly to a loaded dual-spectrum megapixel array computing camera. Background Technology

[0002] With the rapid development of intelligent security, space remote sensing, and ultra-high-definition video surveillance technologies, onboard dual-spectrum array computing cameras, due to their high resolution, wide field of view, and multispectral fusion capabilities, are gradually becoming key sensing components for drones, unmanned surface vessels, and mobile command equipment. These cameras typically employ a miniaturized and lightweight design, arranging multiple visible light modules and infrared thermal imaging modules in an array and fusing the images. This enables panoramic imaging with a horizontal field of view of up to 360° and outputs real-time visible light and infrared video images at the billion-pixel level, thus balancing the needs for overall situational awareness and local detail monitoring. Simultaneously, the system can perform automatic detection, identification, and recording operations in multi-target scenarios, achieving composite monitoring of personnel, vehicles, and small, slow-moving targets.

[0003] To meet the stringent weight and volume constraints of airborne and shipborne applications, cameras must maintain high-resolution output while minimizing structural weight. However, infrared and high-pixel visible light modules consume significant power and generate substantial heat, creating a conflict between structural heat dissipation capacity and overall size design. Insufficient heat dissipation can lead to thermal drift of optical parameters and slight optical axis shifts in the imaging components, causing registration errors and stitching misalignments between the dual spectral channels, making it difficult to guarantee imaging stability and geometric accuracy. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a mounted dual-spectrum megapixel array computing camera to solve the technical problem that the imaging stability and geometric accuracy of existing mounted dual-spectrum megapixel array computing cameras are difficult to guarantee.

[0005] This invention discloses a loaded dual-spectrum megapixel array computing camera, which includes an imaging unit, a power supply unit, and a main control processing unit; wherein,

[0006] The imaging unit is used to install multiple visible light modules and infrared modules, which are used as imaging modules, and the imaging modules are constructed into a dual-spectrum array structure.

[0007] The power supply unit is used to provide power and thermal management support for the imaging unit and the main control processing unit. A directional heat conduction path based on anisotropic thermal conductive material is arranged between the power supply unit and the heat dissipation surface of the camera's outer shell. The power supply unit guides the generated heat to the heat dissipation surface of the outer shell along a predetermined heat conduction direction through the directional heat conduction path for discharge. A thermal insulation air gap is also provided between the power supply unit and the imaging unit to isolate heat flow and form independent temperature distribution zones.

[0008] The main control processing unit is used to predict the changes in optical intrinsic parameters and optical axis deviation of the imaging module based on the thermal control parameters of the partition and through an optical mapping model. The thermal control parameters include the temperature difference of the partition, the direction of heat flow, the power consumption status, and the temperature change rate of the heat dissipation surface of the camera housing.

[0009] Furthermore, the process by which the main control processing unit predicts the changes in the optical intrinsic parameters and the optical axis deviation of the imaging module based on the thermal control parameters of the partition and through an optical mapping model specifically includes:

[0010] Based on the thermal control parameters of the partition, the thermal load status of each module is determined, and the sampling phase, frame rate, and resolution of each imaging module in the dual-spectrum array are adjusted according to the thermal load status to obtain the first adjustment parameter; the first adjustment parameter represents the set of parameters representing the adjustment range and direction of the sampling phase, frame rate, and resolution of the imaging module relative to the reference state.

[0011] The camera's temperature-related data, the imaging module's strain data, the first adjustment parameter, and the inter-frame registration residual calculated during image stitching are used as model input parameters. The optical mapping model outputs the changes in the imaging module's optical intrinsic parameters and the predicted value of the optical axis deviation.

[0012] Furthermore, the reasoning process of the optical mapping model includes:

[0013] The model input parameters are constructed as an input vector that includes features of the thermal domain, structural domain, optical domain, and weight adjustment factor.

[0014] The mapping function calculates the predicted values ​​of the optical intrinsic parameter changes and optical axis deviation of the imaging module based on the input vector. The mapping function includes a first mapping layer representing the relationship between thermal change and structural deformation, a second mapping layer representing the relationship between structural deformation and optical parameter changes, a third mapping layer representing the direct influence of thermal change on optical parameters, and a nonlinear combination layer for integrating the output results of the first, second, and third mapping layers. The adjustment factor is determined based on a first adjustment parameter and is used to dynamically adjust the weights of the mapping function.

[0015] Furthermore, the optical mapping model includes multiple input feature channels, a weight adjustment unit, a cross-domain feature fusion layer, and a dual-output prediction layer; wherein, the multiple input feature channels include a thermal domain input channel, a structural domain input channel, an optical domain input channel, and a control channel for inputting adjustment factor features;

[0016] The weight adjustment unit takes the adjustment factor features of the control channel as input and dynamically adjusts the weight parameters in the multi-input feature channel and the cross-domain feature fusion layer.

[0017] Furthermore, the training process of the optical mapping model adopts a joint loss function including an optical intrinsic parameter prediction error term and an optical axis deviation prediction error term, and introduces a physical constraint term based on the heat conduction equation and a time series consistency constraint term; wherein, the optical axis deviation prediction error term is measured based on the inter-frame registration residual.

[0018] Furthermore, after obtaining the changes in the module's optical intrinsic parameters and the predicted value of the optical axis deviation, the main control processing unit is also used to perform feedforward compensation operations on the optical parameters based on the predicted values, specifically including:

[0019] The electronic focusing parameters, distortion correction coefficients, focal length, and principal point coordinates in the imaging calibration matrix of the imaging module are adjusted based on the predicted values ​​of changes in optical intrinsic parameters.

[0020] The rotation angle and translation parameters of the attitude mapping matrix of the corresponding imaging module are adjusted based on the optical axis deviation prediction value, as well as the rotation and translation parameters in the dual-spectrum stitching matrix used to represent the relative attitude of the infrared module and the visible light module.

[0021] Furthermore, the main control processing unit is also used to perform an online feedback correction operation based on the inter-frame registration residual calculated during the real-time image stitching process after completing the feedforward compensation operation, specifically including:

[0022] The weight parameters and correction coefficients of the optical mapping model are updated based on the inter-frame registration residual using the residual minimization algorithm, and the corresponding parameters of the feedforward compensated imaging calibration matrix, attitude mapping matrix and dual-spectral stitching matrix are iteratively corrected according to the updated optical mapping model.

[0023] Furthermore, the dual-spectrum array structure in the imaging unit is composed of multiple visible light modules and infrared modules arranged alternately, and local thermal equilibrium channels are formed between each module by setting thermally conductive gap filling components; the thermal conductivity of the thermally conductive gap filling component material is lower than that of the directional thermal path material of the power supply unit, and includes a support pad layer and a thermal interface layer. The support pad layer is tightly attached to the module mounting surface, and the thermal interface layer is electrically connected to the thermal sensing node of the main control processing unit for feedback of local thermal status.

[0024] Furthermore, the directional heat conduction path is composed of a longitudinal heat conduction layer, a transverse diffusion layer and a heat flow damping layer in sequence. The longitudinal heat conduction layer is in direct contact with the bottom surface of the power device of the power supply unit to form a main heat channel. The transverse diffusion layer is connected to the heat dissipation surface of the outer shell through an embedded heat dissipation base to form a heat flow expansion surface. The heat flow damping layer is disposed between the two layers and contains a deformable heat conduction medium.

[0025] The thermal orientation of the longitudinal thermal conductive layer is parallel to the normal direction of the heat dissipation surface of the outer shell, so as to conduct heat flow to the heat dissipation surface along the normal direction of the outer shell.

[0026] The direction of heat flow conduction along the directional heat conduction path between the power supply unit and the heat dissipation surface of the casing is determined by the arrangement and orientation of the longitudinal heat conduction layer.

[0027] Furthermore, the deformable thermally conductive medium changes the contact area between the longitudinal thermally conductive layer and the transverse diffusion layer by undergoing micro-expansion when the temperature rises, thus forming an automatically switching heat flow distribution path.

[0028] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0029] This invention constructs a dual-spectral array structure by integrating visible light and infrared modules, and implements zoned thermal control and asynchronous sampling. This maintains geometric consistency and registration stability between the dual-spectral arrays under prolonged high-power operation, balancing the temperature gradient between imaging modules, significantly reducing overall array thermal noise, and improving the smoothness of stitching boundary transitions. Furthermore, through predictive compensation and online feedback correction using an optical mapping model, the camera maintains high-resolution output optical axis consistency and imaging accuracy even under conditions of vehicle vibration, external environmental temperature changes, and power consumption fluctuations, meeting the application requirements for all-weather, full-field-of-view, and highly stable monitoring. Attached Figure Description

[0030] The accompanying drawings, which are included to provide a further understanding of embodiments of the invention and constitute a part of this application, do not limit the scope of the invention. In the drawings:

[0031] Figure 1 This is a schematic diagram of the structure of a loaded dual-spectrum megapixel array computing camera disclosed in Embodiment 1 of the present invention. Detailed Implementation

[0032] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0033] Example 1

[0034] This invention discloses a mounted dual-spectrum megapixel array computing camera. Please refer to [link / reference]. Figure 1 , Figure 1 This is a schematic diagram of the structure of a mounted dual-spectrum megapixel array computing camera disclosed in an embodiment of the present invention. The camera includes an imaging unit, a power supply unit, and a main control processing unit; wherein,

[0035] The imaging unit is used to install multiple visible light modules and infrared modules, which are used as imaging modules, and the imaging modules are constructed into a dual-spectrum array structure.

[0036] The power supply unit is used to provide power and thermal management support for the imaging unit and the main control processing unit. A directional heat conduction path based on anisotropic thermal conductive material is arranged between the power supply unit and the heat dissipation surface of the camera's outer shell. The power supply unit guides the generated heat to the heat dissipation surface of the outer shell along a predetermined heat conduction direction through the directional heat conduction path for discharge. A thermal insulation air gap is also provided between the power supply unit and the imaging unit to isolate heat flow and form independent temperature distribution zones.

[0037] The main control processing unit is used to predict the changes in optical intrinsic parameters and optical axis deviation of the imaging module based on the thermal control parameters of the partition and through an optical mapping model. The thermal control parameters include the temperature difference of the partition, the direction of heat flow, the power consumption status, and the temperature change rate of the heat dissipation surface of the camera housing.

[0038] Furthermore, the dual-spectrum array structure in the imaging unit is composed of multiple visible light modules and infrared modules arranged alternately, and local thermal equilibrium channels are formed between each module by setting thermally conductive gap filling components; the thermal conductivity of the thermally conductive gap filling component material is lower than that of the directional thermal path material of the power supply unit, and includes a support pad layer and a thermal interface layer. The support pad layer is tightly attached to the module mounting surface, and the thermal interface layer is electrically connected to the thermal sensing node of the main control processing unit for feedback of local thermal status.

[0039] Furthermore, the directional heat conduction path is composed of a longitudinal heat conduction layer, a transverse diffusion layer and a heat flow damping layer in sequence. The longitudinal heat conduction layer is in direct contact with the bottom surface of the power device of the power supply unit to form a main heat channel. The transverse diffusion layer is connected to the heat dissipation surface of the outer shell through an embedded heat dissipation base to form a heat flow expansion surface. The heat flow damping layer is disposed between the two layers and contains a deformable heat conduction medium.

[0040] The thermal orientation of the longitudinal thermal conductive layer is parallel to the normal direction of the heat dissipation surface of the outer shell, so as to conduct heat flow to the heat dissipation surface along the normal direction of the outer shell.

[0041] The direction of heat flow conduction along the directional heat conduction path between the power supply unit and the heat dissipation surface of the casing is determined by the arrangement and orientation of the longitudinal heat conduction layer.

[0042] Furthermore, the deformable thermally conductive medium changes the contact area between the longitudinal thermally conductive layer and the transverse diffusion layer by undergoing micro-expansion when the temperature rises, thus forming an automatically switching heat flow distribution path.

[0043] In this embodiment of the invention, the mounted dual-spectrum megapixel array computing camera is mainly composed of an imaging unit, a power supply unit, and a main control processing unit. The imaging unit performs visible light and infrared imaging tasks, the power supply unit provides power and thermal management support, and the main control processing unit is primarily responsible for system-level control and data fusion calculations. The imaging unit adopts a dual-spectrum array structure, consisting of multiple visible light and infrared modules arranged alternately. This arrangement allows adjacent modules to be spatially alternating, forming simultaneous sampling of the visible and infrared spectra. Each module is mounted on an optical axis positioning substrate in the imaging unit area. The optical axis positioning substrate is aligned and fixed to the imaging reference plane of the camera housing via array positioning pins, ensuring that the optical centerlines of all modules are coplanar during installation. This structure enables the dual-spectrum imaging array to maintain geometric consistency under high-resolution stitching conditions, providing a unified spatial reference for subsequent optical intrinsic parameter prediction and thermal drift correction.

[0044] In the structural arrangement between modules, a thermally conductive gap-filling component is installed in the imaging unit area to form a local thermal equilibrium channel. This component is composed of a support pad layer and a thermal interface layer. The support pad layer is tightly fitted to the module mounting surface to stabilize the module position, while the thermal interface layer is electrically connected to the thermal sensing node of the main control processing unit for real-time feedback of the local thermal state between modules. The thermal conductivity of this component is lower than that of the directional thermal path material in the power supply unit, preventing it from forming a dominant thermal path during heat conduction. Instead, it achieves buffered heat diffusion between modules, reducing the local thermal gradient. Through this local thermal equilibrium design, the operating temperature difference between different modules within the imaging unit area is limited to a safe range, thereby avoiding focal length drift or imaging noise accumulation caused by local overheating.

[0045] The power supply unit area is located on the back of the imaging unit area and is electrically connected to the power management interface of the main control processing unit to provide stable power output to both the imaging unit and the main control processing unit. To support thermal management, a directional heat conduction path is arranged between the power supply unit and the camera housing's heat dissipation surface. This directional heat conduction path consists of a longitudinal heat-conducting layer, a transverse diffusion layer, and a thermal damping layer, arranged sequentially. The longitudinal heat-conducting layer directly contacts the bottom surface of the power devices within the power supply unit, forming a dominant thermal channel that allows the heat generated by the power devices during operation to be rapidly transferred along the predetermined heat conduction direction. The transverse diffusion layer is connected to the housing's heat dissipation surface via an embedded heat dissipation base, forming a large-area heat flow expansion surface that further disperses the heat from the longitudinal heat-conducting layer in a planar direction, thereby improving overall heat dissipation efficiency. The thermal damping layer is positioned between the two layers and contains a deformable thermally conductive medium that undergoes micro-expansion when the temperature rises, thereby changing the contact area between the longitudinal heat-conducting layer and the transverse diffusion layer. As the volume of the medium changes, the equivalent thermal resistance of the heat flow path adjusts accordingly, automatically forming heat flow distribution channels within different temperature ranges. This ensures that during high-power operation, heat conduction preferentially flows along the longitudinal thermal conductive layer to the outer casing's heat dissipation surface, while maintaining a higher thermal resistance during low-power operation or when the ambient temperature drops, thus slowing down the heat dissipation rate and stabilizing the internal cavity temperature. Through this adaptive heat flow distribution method, the power supply unit area can maintain thermal balance under different load conditions.

[0046] The thermal orientation of the longitudinal thermally conductive layer is parallel to the normal direction of the heat dissipation surface of the outer shell, allowing heat flow to be conducted to the heat dissipation surface along the normal direction of the outer shell. The arrangement direction of the thermal orientation is determined by the crystal structure of the longitudinal thermally conductive layer. When it is consistent with the normal direction of the heat dissipation surface of the outer shell, the heat conduction efficiency can be maximized and the lateral heat diffusion loss can be reduced. Thus, the heat flow direction of the directional thermal path between the power supply unit area and the heat dissipation surface of the outer shell is determined by the arrangement orientation of the longitudinal thermally conductive layer, realizing a unidirectional thermal flow channel with controllable heat flow direction. At the same time, a thermal insulation air gap is set between the power supply unit area and the imaging unit area. A closed air layer is formed inside the air gap to suppress heat convection and form a heat flow isolation barrier. This thermal insulation air gap makes the temperature distribution of the power supply unit area and the imaging unit area independent of each other, preventing heat from being transferred from the high-power device area to the imaging module area, thereby reducing the impact of thermal stress on optical components and ensuring the stability of the optical axis and the optical alignment accuracy of the imaging module. Through the above structural arrangement, the imaging unit and the power supply unit form a two-layer partition system in terms of spatial layout and heat conduction path, which enables the camera as a whole to maintain a stable balance of thermal-optical system under high power operation conditions, significantly improving the reliability and long-term imaging consistency of mounted dual-spectrum imaging equipment.

[0047] As a further preferred embodiment, in this invention, the thermally insulating air gap disposed between the imaging unit area and the power supply unit area constitutes a closed cavity structure defined by an outer supporting shell, an inner thermal insulation plate, and a sealing frame. A stable mechanical connection is formed between the outer supporting shell and the inner thermal insulation plate through the sealing frame, ensuring that the cavity maintains its geometric shape even under external vibration or changes in ambient air pressure. Inside the cavity, a thermally insulating support structure is provided, formed by a microporous support skeleton and a low thermal conductivity filling medium. The microporous support skeleton is composed of a continuous honeycomb microporous material, whose porosity matches the thermal conductivity of the filling medium, thus suppressing heat conduction rate while ensuring structural strength. The low thermal conductivity filling medium is preferably a mixture of air, argon, or silica powder to form a stable thermal resistance layer. The support skeleton maintains a stable spacing when the air gap is subjected to external pressure or thermal stress, and its microporous elastic recovery characteristics offset mechanical deformation from the power supply unit area side, preventing thermal stress from being transmitted to the mounting structure of the imaging unit area. Thus, the thermal insulation gap provides mechanical support and forms a low thermal conductivity insulation barrier, achieving a dual balance between structural stiffness and thermal insulation performance.

[0048] Based on this, a first temperature sampling node is fixed on the inner insulation plate, and a second temperature sampling node is fixed on the outer support shell. Both temperature sampling nodes are connected to the thermal control input of the main control processing unit via flexible signal cables. The main control processing unit continuously collects real-time temperature data from the first and second temperature sampling nodes and calculates the temperature difference between them. The temperature difference reflects the trend of heat flux change on both sides of the insulation gap. When this temperature difference exceeds a preset threshold, the main control processing unit generates a heat flux regulation control signal. This control signal is transmitted via the main control's power management interface to the directional heat conduction path within the power supply unit area, driving the variable heat flux damping layer to perform a contact surface reconstruction operation. After receiving the control signal, the variable heat flux damping layer redistributes the pressure at the heat conduction interface through fine-tuning the volume of its internal deformable heat-conducting medium. Changes in its contact area and contact pressure cause adjustments to the equivalent thermal conductivity between the longitudinal heat conduction layer and the transverse diffusion layer, thereby altering the heat flow conduction state from the power supply unit area to the outer shell heat dissipation surface. If a continuous rise in the temperature of the power supply unit area is detected, the control signal guides the heat flux damping layer to increase the contact area to improve the longitudinal heat conduction flux. Conversely, when the temperature of the outer casing's heat dissipation surface decreases or the ambient temperature is low, the control signal reduces the contact area to increase thermal resistance, thereby preventing internal overcooling. The entire adjustment process is adaptively scheduled by the main control processing unit based on the rate of temperature difference change, achieving dynamic closed-loop control of the heat flow channel.

[0049] Through the above operations, the present invention can maintain the dynamic balance of the internal temperature field distribution when the external heat load and environmental conditions change, further improve the thermal isolation efficiency between the imaging unit and the power supply unit, make the internal thermal field response of the camera more stable, reduce the stress and thermal drift error of optical components caused by heat conduction, and thus enhance the imaging stability and geometric consistency of the dual-spectral array in long-term operation and complex thermal environment.

[0050] Furthermore, the process by which the main control processing unit predicts the changes in the optical intrinsic parameters and the optical axis deviation of the imaging module based on the thermal control parameters of the partition and through an optical mapping model specifically includes:

[0051] Based on the thermal control parameters of the partition, the thermal load status of each module is determined, and the sampling phase, frame rate, and resolution of each imaging module in the dual-spectrum array are adjusted according to the thermal load status to obtain the first adjustment parameter; the first adjustment parameter represents the set of parameters representing the adjustment range and direction of the sampling phase, frame rate, and resolution of the imaging module relative to the reference state.

[0052] The camera's temperature-related data, the imaging module's strain data, the first adjustment parameter, and the inter-frame registration residual calculated during image stitching are used as model input parameters. The optical mapping model outputs the changes in the imaging module's optical intrinsic parameters and the predicted value of the optical axis deviation.

[0053] Furthermore, the reasoning process of the optical mapping model includes:

[0054] The model input parameters are constructed as an input vector that includes features of the thermal domain, structural domain, optical domain, and weight adjustment factor.

[0055] The mapping function calculates the predicted values ​​of the optical intrinsic parameter changes and optical axis deviation of the imaging module based on the input vector. The mapping function includes a first mapping layer representing the relationship between thermal change and structural deformation, a second mapping layer representing the relationship between structural deformation and optical parameter changes, a third mapping layer representing the direct influence of thermal change on optical parameters, and a nonlinear combination layer for integrating the output results of the first, second, and third mapping layers. The adjustment factor is determined based on a first adjustment parameter and is used to dynamically adjust the weights of the mapping function.

[0056] Furthermore, the optical mapping model includes multiple input feature channels, a weight adjustment unit, a cross-domain feature fusion layer, and a dual-output prediction layer; wherein, the multiple input feature channels include a thermal domain input channel, a structural domain input channel, an optical domain input channel, and a control channel for inputting adjustment factor features;

[0057] The weight adjustment unit takes the adjustment factor features of the control channel as input and dynamically adjusts the weight parameters in the multi-input feature channel and the cross-domain feature fusion layer.

[0058] Furthermore, the training process of the optical mapping model adopts a joint loss function including an optical intrinsic parameter prediction error term and an optical axis deviation prediction error term, and introduces a physical constraint term based on the heat conduction equation and a time series consistency constraint term; wherein, the optical axis deviation prediction error term is measured based on the inter-frame registration residual.

[0059] Specifically, in this embodiment of the invention, the main control processing unit constitutes the core computing and control center of the camera, used to realize multiple functions such as zone thermal control parameter acquisition, optical mapping model inference, and parameter compensation decision-making. The hardware system of the main control processing unit includes, but is not limited to, a high-performance heterogeneous computing chipset, a thermal sensor interface module, a module strain signal acquisition module, and a video processing interface unit. During operation, the main control processing unit receives thermal control parameters from each zone in real time, including but not limited to the zone temperature difference, heat flow direction, power consumption status, and temperature change rate parameters of the outer shell heat dissipation surface. The main control processing unit first calculates the thermal load status of each imaging module based on the thermal control parameters, wherein the thermal load status is comprehensively quantified by three indicators: heat flux density, average power consumption, and temperature rise rate. When the thermal load of a local module is detected to exceed a set threshold, the main control processing unit dynamically adjusts the sampling phase, frame rate, and resolution of the module according to the thermal conduction characteristics of the zone, thereby balancing the instantaneous power consumption distribution between modules and obtaining a first adjustment parameter reflecting the sampling adjustment status of each module. The first adjustment parameter is used to characterize the adjustment magnitude and direction of the imaging module sampling phase, frame rate, and resolution relative to the standard working state, and is a key adjustment factor connecting the thermal control layer and the imaging control layer. Through the above-mentioned partitioned thermal control and sampling parameter adjustment operations, the main control processing unit couples the thermal management process with the optical calibration process, so that the impact of thermal state changes on subsequent imaging parameter predictions can be quantified and input into the model, thereby constructing a thermal-optical closed loop.

[0060] Based on this, the main control processing unit uses the camera's temperature data, the imaging module's strain data, the first adjustment parameter, and the inter-frame registration residual calculated during image stitching as model input parameters, which are then input into the optical mapping model to execute the inference process. This optical mapping model is a deep learning model based on a multi-channel neural network structure, used to describe the nonlinear mapping relationship between temperature changes, structural deformation, and optical parameter changes. Specifically, the main control processing unit first performs feature preprocessing on the model input, constructing the input parameters into a four-dimensional input vector including thermal domain, structural domain, optical domain, and adjustment factor features. The thermal domain features characterize the temperature change trend and heat flow direction of different zones; the structural domain features reflect the micro-deformation of the module and support structure under thermal stress; the optical domain features consist of the inter-frame registration residual, describing the geometric offset in the current image stitching process; and the adjustment factor features originate from the first adjustment parameter, reflecting the dynamic influence of the sampling control strategy on the imaging module's operating state. The above input vector is then normalized and smoothed using a time window before being fed into the model's multi-input feature channels.

[0061] The core of the optical mapping model's inference is achieved through cross-domain information fusion via multi-layer mapping functions. These mapping functions include a first mapping layer representing the relationship between thermal change and structural deformation, a second mapping layer representing the relationship between structural deformation and changes in optical parameters, a third mapping layer representing the direct impact of thermal change on optical parameters, and a nonlinear combination layer for synthesizing the outputs of the first three layers. Specifically, the first mapping layer employs a kernel-based local correlation extraction mechanism, learning the influence pattern of temperature change on structural deformation through correlation analysis of thermal flux gradients and strain gradients. The second mapping layer uses a bidirectional long short-term memory network structure to capture the temporal correlation between structural strain and optical parameter drift. The third mapping layer preferably employs a multi-head attention mechanism, quantifying the direct impact of thermal drift on optical parameters by calculating the attention weight distribution between thermal and optical domain features. The nonlinear combination layer fuses the output features of the first, second, and third mapping layers through a residual connection structure, establishing a coupled representation space encompassing thermal, structural, and optical domains. The weight adjustment unit in the model takes the adjustment factor feature of the control channel input as dynamic input. Based on the changes in sampling phase, frame rate and resolution, it adaptively adjusts the weight parameters of the multi-input feature channels and the cross-domain feature fusion layer, so that the model can dynamically compensate for the feature distribution under different thermal loads and sampling states, thereby maintaining the stability and universality of the model's prediction performance.

[0062] The output layer of the optical mapping model is a dual-output prediction layer, which simultaneously outputs predicted values ​​for the changes in optical intrinsic parameters of the imaging module and predicted values ​​for optical axis deviation during the inference phase. The changes in optical intrinsic parameters include, but are not limited to, focal length changes, principal point coordinate drift, and distortion coefficient shifts, describing the fluctuations in the internal parameters of the optical system under thermal stress and mechanical deformation. The predicted value for optical axis deviation reflects the minute shifts in the attitude direction of the imaging module. After receiving the model output, the main control processing unit uses the predicted values ​​for changes in optical intrinsic parameters to perform feedforward compensation on the electronic focusing parameters, distortion correction coefficients, and focal length and principal point coordinates in the imaging calibration matrix of the imaging module. It also uses the predicted value for optical axis deviation to correct the rotation angle and translation of the attitude mapping matrix and the dual-spectrum stitching matrix of the corresponding module, ensuring the accuracy of field-of-view overlap and geometric consistency between the modules.

[0063] To ensure the accuracy and physical interpretability of the model predictions, the training process of the optical mapping model employs a joint loss function that includes optical intrinsic parameter prediction error terms and optical axis deviation prediction error terms, and introduces physical constraint terms based on the heat conduction equation and time series consistency constraint terms. The optical intrinsic parameter prediction error terms are measured by calculating the deviations of the predicted focal length, principal point coordinates, and calibration reference. The optical axis deviation prediction error term is calculated based on the image frame registration residuals to measure the geometric error of the attitude deviation. The physical constraint terms utilize the functional relationship between temperature gradient and heat flux density in the heat conduction equation to limit the changes in thermal domain characteristics in the model to a range consistent with the laws of heat diffusion. The time series consistency constraint term is used to maintain the smoothness of the prediction trend within adjacent time windows, thereby avoiding the impact of sudden thermal changes on prediction stability. During the training phase, historical working data stored in the main control processing unit is used for offline training. The network weight parameters are iteratively updated using the Adam optimization algorithm until the joint loss function converges. After training, the model parameters are stored as weight files in the non-volatile storage module of the main control processing unit and loaded and executed at runtime.

[0064] Through the above operations, this embodiment of the invention achieves deep coupling between partitioned thermal control information and the optical mapping model. Thermal control parameters not only participate in the model input but also modulate the model's internal weights in real time through adjustment factors, enabling changes in thermal state to be dynamically reflected in optical parameter predictions. This further endows the camera system with adaptive thermal drift compensation capabilities, allowing it to maintain geometric stability and stitching consistency in dual-spectral array imaging under high-power operation or severe environmental temperature fluctuations, thereby significantly improving the imaging reliability and long-term operational accuracy of the mounted 100-megapixel dual-spectral camera.

[0065] Furthermore, after obtaining the changes in the module's optical intrinsic parameters and the predicted value of the optical axis deviation, the main control processing unit is also used to perform feedforward compensation operations on the optical parameters based on the predicted values, specifically including:

[0066] The electronic focusing parameters, distortion correction coefficients, focal length, and principal point coordinates in the imaging calibration matrix of the imaging module are adjusted based on the predicted values ​​of changes in optical intrinsic parameters.

[0067] The rotation angle and translation parameters of the attitude mapping matrix of the corresponding imaging module are adjusted based on the optical axis deviation prediction value, as well as the rotation and translation parameters in the dual-spectrum stitching matrix used to represent the relative attitude of the infrared module and the visible light module.

[0068] Furthermore, the main control processing unit is also used to perform an online feedback correction operation based on the inter-frame registration residual calculated during the real-time image stitching process after completing the feedforward compensation operation, specifically including:

[0069] The weight parameters and correction coefficients of the optical mapping model are updated based on the inter-frame registration residual using the residual minimization algorithm, and the corresponding parameters of the feedforward compensated imaging calibration matrix, attitude mapping matrix and dual-spectral stitching matrix are iteratively corrected according to the updated optical mapping model.

[0070] Specifically, in this embodiment of the invention, after the main control processing unit outputs the predicted values ​​of the optical intrinsic parameter changes and optical axis deviation of the imaging module through the optical mapping model, it immediately enters the optical parameter feedforward compensation stage. The main control processing unit matches and compares the predicted optical intrinsic parameter changes with the calibration parameters of the current imaging module, and performs compensation control based on the difference. The optical intrinsic parameter changes include, but are not limited to, focal length changes, principal point coordinate drift, and distortion coefficient offset. The main control processing unit first adjusts the electronic focusing parameters of the imaging module according to the predicted value of the focal length change, so that the focal plane of the imaging lens group re-aligns with the photosensitive plane of the image sensor. This electronic focusing process is executed by a micro-stepping motor inside the module, and the motor drive displacement accuracy is controlled at the micrometer level to ensure that the imaging blur caused by the focal length change is corrected in real time. Subsequently, the main control processing unit corrects the radial distortion of the image according to the distortion coefficient offset, and achieves automatic compensation of geometric distortion by updating the distortion correction term in the imaging calibration matrix. For the principal point coordinate drift, the main control processing unit recalculates the principal point coordinate parameters in the calibration matrix to realign the projection center with the sensor array center, thereby ensuring the stability of the image plane coordinate system. The updates to the aforementioned electronic focusing parameters, distortion correction coefficients, and calibration matrix parameters are all executed synchronously in the hardware driver layer and the imaging control algorithm layer, forming a high-precision, low-latency feedforward optical parameter compensation channel.

[0071] After completing optical intrinsic parameter compensation, the main control processing unit performs inter-module attitude correction operations using the optical axis deviation prediction value. The optical axis deviation prediction value reflects the rotation and translation errors of the imaging module's optical axis in the three-dimensional coordinate system. The main control processing unit decomposes this into rotation angle components and translation displacement components, corresponding to the rotation matrix and translation vector in the attitude mapping matrix, respectively. The main control processing unit corrects the attitude mapping matrix of the imaging module based on the rotation angle component, so that the optical axis of each module in the spatial coordinate system is realigned with the geometric center of the array. Based on the translation component, the translation parameters in the dual-spectrum stitching matrix are corrected, so that the relative positional relationship between the infrared module and the visible light module is restored to the calibration reference state. At the same time, the main control processing unit recalculates the rotation parameters in the dual-spectrum stitching matrix based on the relative attitude changes of the infrared module and the visible light module, ensuring that the infrared image and the visible light image maintain a coplanar constraint relationship at the pixel level. The entire feedforward compensation process is based on the prediction value, that is, the correction is completed in advance before thermal drift or structural strain causes actual image misalignment, thus achieving active adjustment of optical parameters before physical changes accumulate to an observable level.

[0072] After completing the feedforward compensation operation, the main control processing unit continues into the online feedback correction stage, which is used to adaptively optimize the model prediction error and residual geometric deviation. This stage updates the weight parameters and correction coefficients of the optical mapping model based on the inter-frame registration residuals calculated in real time during image stitching using a residual minimization algorithm. Specifically, the inter-frame registration residuals consist of the feature point matching error and the difference in the transformation matrix between the output images of adjacent modules. The main control processing unit uses the residual vector as input and employs a least-squares optimization method based on stochastic gradient descent to gradually adjust the weight parameters of each layer in the optical mapping model. The optimization objective is to minimize the mean square error between the predicted output and the inter-frame registration residuals, while simultaneously making minor adjustments to the correction coefficients in the model to bring them back to the optimal parameters under the current thermal state. This residual minimization algorithm is essentially an online fine-tuning process of the model weights, and its optimization rate is adaptively determined by the learning rate control parameter to avoid overfitting or oscillations. Through online correction, the optical mapping model can continuously adapt to changes in the temperature field and structural stress during long-term operation, achieving self-evolution of the model parameters.

[0073] After the model weights and correction coefficients are updated, the main control processing unit recalculates the corresponding parameters in the feedforward compensated imaging calibration matrix, attitude mapping matrix, and dual-spectral stitching matrix based on the updated optical mapping model. In subsequent imaging cycles, it applies the new parameter set to perform image stitching and registration calculations. This operation forms a closed-loop calibration system combining feedforward prediction and feedback correction. Feedforward compensation adjusts the optical and attitude parameters in advance based on model prediction results, while feedback correction re-optimizes the model's internal parameters based on actual imaging errors.

[0074] Through the above operations, the embodiments of the present invention can automatically suppress optical system parameter drift under unattended conditions, so that the geometric consistency, field of view overlap and registration accuracy of the dual-spectrum stitched image remain stable under long-term operation and high-load environment, thereby improving imaging stability by more than an order of magnitude, and providing a solid technical guarantee for the long-term reliable operation of the mounted 100-megapixel dual-spectrum camera on complex dynamic platforms.

[0075] Finally, it should be noted that the above-described embodiments include multiple parallel implementations of the present invention. Deleting or otherwise adjusting one or more of these implementations will not affect the implementation of the solution. Furthermore, the on-board dual-spectrum megapixel array computing camera disclosed in the embodiments of the present invention is merely a preferred embodiment of the present invention and is only used to illustrate the technical solutions of the present invention, not to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A mounted dual-spectrum megapixel array computing camera, characterized in that, The camera includes an imaging unit, a power supply unit, and a main control processing unit; wherein... The imaging unit is used to install multiple visible light modules and infrared modules, which are used as imaging modules, and the imaging modules are constructed into a dual-spectrum array structure. The power supply unit is used to provide power and thermal management support for the imaging unit and the main control processing unit. A directional heat conduction path based on anisotropic thermal conductive material is arranged between the power supply unit and the heat dissipation surface of the camera's outer shell. The power supply unit guides the generated heat to the heat dissipation surface of the outer shell along a predetermined heat conduction direction through the directional heat conduction path for discharge. A thermal insulation air gap is also provided between the power supply unit and the imaging unit to isolate heat flow and form independent temperature distribution zones. The main control processing unit is used to predict the changes in optical intrinsic parameters and optical axis deviation of the imaging module based on the thermal control parameters of the partition and through an optical mapping model. The thermal control parameters include the temperature difference of the partition, the direction of heat flow, the power consumption status, and the temperature change rate of the heat dissipation surface of the camera housing.

2. The mounted dual-spectrum megapixel array computing camera according to claim 1, characterized in that, The process by which the main control processing unit predicts the changes in optical intrinsic parameters and optical axis deviation of the imaging module based on the thermal control parameters of the partition and through an optical mapping model specifically includes: Based on the thermal control parameters of the partition, the thermal load status of each module is determined, and the sampling phase, frame rate, and resolution of each imaging module in the dual-spectrum array are adjusted according to the thermal load status to obtain the first adjustment parameter; the first adjustment parameter represents the set of parameters representing the adjustment range and direction of the sampling phase, frame rate, and resolution of the imaging module relative to the reference state. The camera's temperature-related data, the imaging module's strain data, the first adjustment parameter, and the inter-frame registration residual calculated during image stitching are used as model input parameters. The optical mapping model outputs the changes in the imaging module's optical intrinsic parameters and the predicted value of the optical axis deviation.

3. The onboard dual-spectrum megapixel array computing camera according to claim 2, characterized in that, The reasoning process of the optical mapping model includes: The model input parameters are constructed as an input vector that includes features of the thermal domain, structural domain, optical domain, and weight adjustment factor. The mapping function calculates the predicted values ​​of the optical intrinsic parameter changes and optical axis deviation of the imaging module based on the input vector. The mapping function includes a first mapping layer representing the relationship between thermal change and structural deformation, a second mapping layer representing the relationship between structural deformation and optical parameter changes, a third mapping layer representing the direct influence of thermal change on optical parameters, and a nonlinear combination layer for integrating the output results of the first, second, and third mapping layers. The adjustment factor is determined based on a first adjustment parameter and is used to dynamically adjust the weights of the mapping function.

4. The mounted dual-spectrum megapixel array computing camera according to claim 3, characterized in that, The optical mapping model includes multiple input feature channels, a weight adjustment unit, a cross-domain feature fusion layer, and a dual-output prediction layer; wherein, the multiple input feature channels include a thermal domain input channel, a structural domain input channel, an optical domain input channel, and a control channel for inputting adjustment factor features; The weight adjustment unit takes the adjustment factor features of the control channel as input and dynamically adjusts the weight parameters in the multi-input feature channel and the cross-domain feature fusion layer.

5. The mounted dual-spectrum megapixel array computing camera according to claim 4, characterized in that, The training process of the optical mapping model adopts a joint loss function including optical intrinsic parameter prediction error term and optical axis deviation prediction error term, and introduces physical constraint term and time series consistency constraint term based on heat conduction equation; wherein, the optical axis deviation prediction error term is measured based on inter-frame registration residual.

6. The mounted dual-spectrum megapixel array computing camera according to any one of claims 2-5, characterized in that, After obtaining the changes in the module's optical intrinsic parameters and the predicted value of the optical axis deviation, the main control processing unit is further used to perform feedforward compensation operations on the optical parameters based on the predicted values, specifically including: The electronic focusing parameters, distortion correction coefficients, focal length, and principal point coordinates in the imaging calibration matrix of the imaging module are adjusted based on the predicted values ​​of changes in optical intrinsic parameters. The rotation angle and translation parameters of the attitude mapping matrix of the corresponding imaging module are adjusted based on the optical axis deviation prediction value, as well as the rotation and translation parameters in the dual-spectrum stitching matrix used to represent the relative attitude of the infrared module and the visible light module.

7. The onboard dual-spectrum megapixel array computing camera according to claim 6, characterized in that, The main control processing unit is also used to perform an online feedback correction operation based on the inter-frame registration residual calculated during real-time image stitching after completing the feedforward compensation operation, specifically including: The weight parameters and correction coefficients of the optical mapping model are updated based on the inter-frame registration residual using the residual minimization algorithm, and the corresponding parameters of the feedforward compensated imaging calibration matrix, attitude mapping matrix and dual-spectral stitching matrix are iteratively corrected according to the updated optical mapping model.

8. The mounted dual-spectrum megapixel array computing camera according to claim 1, characterized in that, The dual-spectrum array structure in the imaging unit is composed of multiple visible light modules and infrared modules arranged alternately, and each module forms a local thermal equilibrium channel by setting a thermally conductive gap filling component. The thermal conductivity of the thermally conductive gap-filling component material is lower than that of the directional thermal path material of the power supply unit, and includes a support pad layer and a thermal interface layer. The support pad layer is tightly attached to the module mounting surface, and the thermal interface layer is electrically connected to the thermal sensing node of the main control processing unit for feedback of local thermal status.

9. The onboard dual-spectrum gigapixel array computing camera according to claim 1, characterized in that, The directional heat conduction path is composed of a longitudinal heat conduction layer, a transverse diffusion layer and a heat flow damping layer in sequence. The longitudinal heat conduction layer is in direct contact with the bottom surface of the power device of the power supply unit to form a main heat channel. The transverse diffusion layer is connected to the heat dissipation surface of the outer shell through an embedded heat dissipation base to form a heat flow expansion surface. The heat flow damping layer is set between the two layers and contains a deformable heat conduction medium. The thermal orientation of the longitudinal thermal conductive layer is parallel to the normal direction of the heat dissipation surface of the outer shell, so as to conduct heat flow to the heat dissipation surface along the normal direction of the outer shell. The direction of heat flow conduction along the directional heat conduction path between the power supply unit and the heat dissipation surface of the casing is determined by the arrangement and orientation of the longitudinal heat conduction layer.

10. The mounted dual-spectrum megapixel array computing camera according to claim 9, characterized in that, The deformable thermally conductive medium changes the contact area between the longitudinal thermally conductive layer and the transverse diffusion layer by undergoing micro-expansion when the temperature rises, thus forming an automatically switching heat flow distribution path.

Images