Image processing method, electronic device, storage medium and computer program product

Abstract

The application provides an image processing method, an electronic device, a storage medium and a computer program product. The image processing method comprises: in the case of detecting a sensor switching event, mapping a second gain coefficient of a second sensor based on a first gain coefficient of a first sensor and a predetermined sensor gain mapping relationship corresponding to a current color temperature, wherein the sensor switching event indicates switching from a first mode to a second mode, and the sensor gain mapping relationship is a mapping relationship between the first gain coefficient in a steady state and the second gain coefficient in a steady state at the current color temperature; after entering the second mode, processing an image collected by the second sensor based on the second gain coefficient to obtain an output image. Through the application, smooth switching of white balance across sensor switching moments can be realized, and the continuity and smoothness of the image when switching the sensor are improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more particularly to an image processing method, electronic device, storage medium, and computer program product. Background Technology

[0002] In imaging equipment, multiple sensors with different photosensitive physical properties are typically configured to meet the acquisition needs of different fields of view and focal lengths. When switching between different sensors, due to differences in photoelectric conversion and response deviations between heterogeneous sensors, continuous automatic white balance (AWB) adjustment is required to correct the color of the connected sensor. However, in practical applications, conventional white balance adjustment methods are still prone to problems such as abrupt color transitions and insufficient image continuity when dealing with the sudden changes in spectral response during sensor switching. Summary of the Invention

[0003] This application provides an image processing method, electronic device, storage medium, and computer program product that can improve the continuity of images during sensor switching.

[0004] The technical solution of this application embodiment is implemented as follows: Upon detecting a sensor switching event, a second gain coefficient of the second sensor is obtained by mapping based on the first gain coefficient of the first sensor and a pre-determined sensor gain mapping relationship corresponding to the current color temperature. The sensor switching event indicates a switch from a first mode to a second mode. In the first mode, the image acquired by the first sensor is processed based on the first gain coefficient to obtain an output image. The first gain coefficient is calculated by convergence of the statistical data of the first historical frame. The sensor gain mapping relationship is the mapping relationship between the first gain coefficient in steady state and the second gain coefficient in steady state at the current color temperature. After entering the second mode, the image acquired by the second sensor is processed based on the second gain coefficient to obtain the output image, wherein the second gain coefficient is adjusted by the second historical frame statistics, and the second historical frame statistics take the first frame of the output image obtained from the image acquired by the second sensor as the starting frame.

[0005] This application provides an image processing apparatus, the image processing apparatus comprising: The mapping module is used to map a second gain coefficient of a second sensor to a first gain coefficient of a first sensor and a predetermined sensor gain mapping relationship corresponding to the current color temperature when a sensor switching event is detected. The sensor switching event indicates a switch from a first mode to a second mode. In the first mode, the image acquired by the first sensor is processed based on the first gain coefficient to obtain an output image. The first gain coefficient is calculated by convergence of the statistical data of the first historical frame. The sensor gain mapping relationship is the mapping relationship between the first gain coefficient in steady state and the second gain coefficient in steady state at the current color temperature. The output module is used to process the image acquired by the second sensor based on the second gain coefficient after entering the second mode to obtain the output image, wherein the second gain coefficient is adjusted by the second historical frame statistics data, and the second historical frame statistics data takes the first frame of the output image obtained from the image acquired by the second sensor as the starting frame.

[0006] This application provides an electronic device, including: The first sensor is used to acquire images; The second sensor is used to acquire images; Memory is used to store executable instructions or computer programs. The processor is communicatively connected to the first sensor, the second sensor, and the memory. When the processor executes the executable instructions or computer program stored in the memory, it implements the image processing method provided in the embodiments of this application.

[0007] This application provides a computer-readable storage medium storing a computer program or computer-executable instructions for implementing the image processing method provided in this application when executed by a processor.

[0008] This application provides a computer program product, including a computer program or computer executable instructions, which, when executed by a processor, implement the image processing method provided in this application.

[0009] The embodiments of this application have the following beneficial effects: The image processing method provided in this application, upon detecting a sensor switching event, derives a second gain coefficient based on the current color temperature and gain mapping relationship. This pre-adjusts the spectral response differences between heterogeneous sensors, directly providing a precise initial white balance reference for the succeeding sensor and avoiding color abrupt changes during switching. Furthermore, by limiting the second historical frame statistics after entering the second mode to the first output image as the starting frame, it reduces the white balance algorithm's dependence on residual historical data from previous sensors, eliminating the need for the system to spend several seconds performing smooth cross-frame convergence hampered by historical weights. The gain coefficient calculation and historical data reset scheme of this method work in synergy, reducing color cast caused by convergence lag in the white balance algorithm, achieving smooth white balance switching during cross-sensor switching, and improving the continuity and smoothness of the image during sensor switching. Attached Figure Description

[0010] Figure 1 This is a schematic diagram of the architecture of the image processing system provided in the embodiments of this application; Figure 2 This is a schematic diagram of the structure of the image processing device provided in the embodiments of this application; Figure 3 This is a flowchart illustrating the image processing method provided in the embodiments of this application. Figure 1 ; Figure 4 This is a flowchart illustrating the image processing method provided in the embodiments of this application. Figure 2 ; Figure 5 This is a flowchart illustrating the image processing method provided in the embodiments of this application. Figure 3 ; Figure 6 This is a flowchart illustrating the image processing method provided in the embodiments of this application. Figure 4 ; Figure 7 This is a flowchart illustrating the image processing method provided in the embodiments of this application. Figure 5 ; Figure 8 This is a flowchart illustrating the image processing method provided in the embodiments of this application. Figure 6 ; Figure 9 This is a flowchart illustrating the image processing method provided in the embodiments of this application. Figure 7 . Detailed Implementation

[0011] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0012] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0013] In the following description, the terms "first / second / third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0014] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application.

[0015] In related technologies, with the widespread application of imaging systems containing multiple camera modules (such as tri-lens zoom systems with near-focus, mid-focus, and telephoto lenses), in order to meet users' needs for zoom and field of view expansion in complex scenes, terminal devices need to stop the currently operating first sensor and dynamically switch to the second sensor responsible for another focal length to take over image acquisition when the zoom ratio crosses a specific set value. During this cross-sensor image source switching, due to the change in the physical hardware of the connected system, the imaging system must automatically adjust the white balance of the newly taken-over sensor to ensure the correct color and image continuity of the output video stream. To control hardware costs and computing power, multi-camera systems typically reuse the same image signal processor (ISP) and automatic white balance pipeline in a time-sharing manner. Therefore, the current common practice is as follows: at the moment the switching command is triggered, the white balance algorithm directly uses or copies the steady-state gain value left by the previous sensor as the initial white balance parameter for the new sensor; and when the new sensor outputs subsequent image frames, it still calls the conventional time smoothing filtering logic to fuse the newly input pixel statistics with the historical frame statistics. Through multi-frame weighted iteration, the white balance is gradually controlled to converge to a new steady state.

[0016] However, the aforementioned sensor switching steps, which rely on directly inheriting historical parameters and gradual smooth convergence, can lead to image quality fragmentation and color lag issues when dealing with continuous zoom using multiple sensors. On one hand, different focal lengths are typically paired with different heterogeneous sensors, whose Bayer array filters and photosensitive physical characteristics naturally differ in their spectral responses. Directly inheriting the gain value of the previous sensor as the starting parameter for the new sensor completely ignores the inherent systematic color shift between heterogeneous hardware, resulting in a drastic color cast and color abrupt changes visible to the naked eye the moment the new sensor outputs the image. On the other hand, because the unified white balance algorithm does not change its filtering logic after switching and continues to use the anti-jump cross-frame smooth convergence mechanism, the newly calculated gain coefficient is heavily diluted by the weight of the massive historical data left over from the previous sensor. This conservative iterative mechanism causes a huge algorithmic inertial lag in such image source abrupt changes, forcing the new sensor to undergo a slow transition period of several seconds to completely wash away the old parameters and approach the true white balance steady state. The zoom color cast and stuttering caused by the combined effect of incorrect initial values ​​and convergence lag cannot meet the image quality consistency requirements of high frame rate multi-module switching.

[0017] This application provides an image processing method, electronic device, storage medium, and computer program product that can improve the continuity of images during sensor switching. The following describes exemplary applications of the image processing device provided in this application. This image processing device can be implemented as various types of terminals such as smartphones, cameras, camcorders, drones, robots, security equipment, smart homes, augmented reality devices, virtual reality devices, and vehicle terminals. Exemplary applications when the device is implemented as a terminal will be described below.

[0018] See Figure 1 , Figure 1 This is a schematic diagram of the architecture of an image processing system provided in an embodiment of this application. To support an image processing operation, an image processing application can be provided. For example, the image processing application can be an application dedicated to image processing, or it can be a functional module in other applications (such as a zoom module in a camera application). The image processing system 100 in this embodiment of the application includes at least an image processing device 200, and the image processing device 200 is equipped with multiple sensors (only the first sensor 210 and the second sensor 220 are shown in the figure).

[0019] In some embodiments, the electronic device may be Figure 1 Image processing devices in, see Figure 2 , Figure 2 This is a schematic diagram of the structure of the image processing device 200 provided in the embodiments of this application. Figure 2The image processing device 200 shown includes at least one processor 260, a memory 250, at least one network interface 270, and a user interface 230. The various components in the image processing device 200 are coupled together via a bus system 240. It is understood that the bus system 240 is used to implement communication between these components. In addition to a data bus, the bus system 240 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 2 The general labeled all buses as Bus System 240.

[0020] Processor 260 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor.

[0021] User interface 230 includes one or more output devices 231 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 230 also includes one or more input devices 232, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0022] The memory 250 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 250 may optionally include one or more storage devices physically located away from the processor 260.

[0023] The memory 250 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 250 described in this application embodiment is intended to include any suitable type of memory.

[0024] In some embodiments, memory 250 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0025] Operating system 251 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks; The network communication module 252 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 270, exemplary network interfaces 270 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc. Presentation module 253 is configured to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 231 associated with user interface 230 (e.g., a display screen, a speaker, etc.). The input processing module 254 is used to detect and translate one or more user inputs or interactions from one or more input devices 232.

[0026] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2 An image processing apparatus 255 stored in memory 250 is shown. This apparatus can be software in the form of programs and plug-ins, and includes the following software modules: a mapping module 2551 and an output module 2552. These modules are logically linked and can therefore be arbitrarily combined or further separated according to the functions they implement. The functions of each module will be described below.

[0027] See Figure 3 , Figure 3 This is a flowchart illustrating the image processing method provided in the embodiments of this application. Figure 1 , will combine Figure 3 The steps shown are explained as follows: Figure 3 As shown, taking the terminal as the execution subject of the image processing method as an example, the method includes the following steps 101 to 102.

[0028] It should be noted that the sensor mentioned in the embodiments of this application can be understood as an image input element composed of a lens and a sensor body. Taking a tri-lens zoom image processing device as an example, it is equipped with three sensor bodies and three lenses. The focal lengths of the lenses can be 4mm, 8mm, and 12mm, corresponding to three scenes: close-focus, medium-focus, and telephoto. If the current zoom state is close-focus, the output image of the current tri-lens zoom processing device (multi-sensor system) is the image generated by the close-focus sensor body and its lens. If the current zoom state is switched to medium-focus, the output image is the image generated by the medium-focus sensor body and its lens.

[0029] In step 101, when a sensor switching event is detected, the second gain coefficient of the second sensor is obtained by mapping based on the first gain coefficient of the first sensor and the predetermined sensor gain mapping relationship corresponding to the current color temperature.

[0030] The sensor switching event indicates a switch from the first mode to the second mode. In the first mode, the image acquired by the first sensor is processed based on the first gain coefficient to obtain the output image. The first gain coefficient is calculated by the convergence of the statistical data of the first historical frame. The sensor gain mapping relationship is the mapping relationship between the steady-state first gain coefficient and the steady-state second gain coefficient under the current color temperature.

[0031] Here, the sensor switching event refers to the instruction trigger node in a multi-camera imaging system that, due to optical zoom or field-of-view expansion requirements, needs to schedule the underlying hardware pipeline flow between photosensitive modules at different focal lengths. The first mode and the second mode represent the system's underlying operational states, respectively relying on the first sensor (e.g., a close-up lens) and the second sensor (e.g., a medium-telephoto lens) as the main physical data source for image output. The first gain coefficient is the value currently being adjusted by the first sensor, and since it is calculated by converging statistical data from the first historical frame, it signifies that the coefficient has undergone smooth cross-frame iteration to prevent jumps, making it an accurate parameter perfectly adapted to the current objective lighting environment. Steady state indicates that the mathematical derivation of the automatic white balance algorithm has been finalized, and the parameters no longer undergo significant jumps over time. The sensor gain mapping relationship is a reference magnification conversion model or function specifically proposed in this application embodiment, arising from the unique photoelectric response limitations of heterogeneous sensors of different models and hardware materials when facing the same objective environmental color temperature.

[0032] In some embodiments, when the system detects and receives a sensor switching event, before the underlying data link formally hands over the image acquisition source to the second sensor, the system performs cross-isolation zone gain coefficient prediction for white balance parameters. Specifically, the system extracts the estimated color temperature of the current environment and uses the highly accurate first gain coefficient set that the first sensor finally locked before ending the first mode as the input base. Subsequently, the system uses this current color temperature as a matching index to perform addressing or numerical conversion in the sensor gain mapping relationship pre-stored in the storage medium, thereby equivalently converting and converting the gain coefficient specific to the hardware characteristics of the first sensor into a reference parameter expected to adapt to the underlying hardware characteristics of the second sensor, thus mapping to obtain the second gain coefficient.

[0033] It should be noted that in heterogeneous systems such as multi-view zoom monitoring, even when facing the same pure white light source, different sensors will have irreconcilable systematic shifts in their white balance points. If the original parameters are directly inherited during switching without processing, the image will exhibit severe abrupt color casts. This application's embodiment, however, uses a steady-state mapping relationship to perform reverse calculations, predicting and configuring an optimal starting parameter close to the current real-world space for the new sensor within a very short window before the image is generated. This prediction action eliminates the need for the sensor driver black box not open at the underlying level by specific manufacturers, achieving parameter unification of heterogeneous device characteristics at the software application logic layer.

[0034] In some embodiments, a benchmark mathematical model is established to eliminate the inherent optical response deviations between heterogeneous sensors. Since the photosensitive materials, Bayer arrays, and digital conversion processes of different sensor manufacturers vary, if the systematic offset relationships of these physical devices under known spectral environments are not predetermined through objective means, the algorithm will lose the accurate basis for calibration and conversion when facing sudden switching, and the initial gain parameters issued after pre-compensation will have a high probability of blind deviation. Therefore, the embodiments of this application propose the following steps.

[0035] In some embodiments, see Figure 4 Before the second gain coefficient of the second sensor is mapped based on the first gain coefficient of the first sensor and the sensor gain mapping relationship corresponding to the current color temperature in step 101, steps 103 to 105 can be performed.

[0036] In step 103, under an environment containing multiple standard color temperatures, the images acquired by the first sensor and the second sensor are processed respectively to obtain the first reference image and the second reference image.

[0037] Here, standard color temperature refers to the known reference light source parameters (e.g., D65 light source, A light source, or specific discrete spectral anchor points such as 3000K, 5000K, etc.) projected by precision optical lighting equipment (such as professional standard light boxes). The reference image refers to a calibration sample image generated by a sensor through a fixed image signal processing pipeline under strictly controlled external experimental lighting constraints, which uniquely eliminates interference from other stray light, and awaits evaluation and verification.

[0038] In some embodiments, multiple standard color temperature projection environments with extremely wide coverage can be constructed in a separately built optical darkroom and testing equipment. Then, each standard color temperature light source, representing a single unit, is sequentially illuminated. Under this physical reference, the first and second sensors, as two different front-end receivers of the same camera system, are placed in parallel or alternately under identical testing conditions. The internally integrated white balance algorithm will, according to a conventional pipeline strategy, read the optical information collected by the two photosensitive devices at this time, analyze and adjust it according to a basic deduction mechanism, and output a corresponding visualized reference image frame sequence after calculation for subsequent evaluation and value determination.

[0039] In step 104, when multiple continuously output first reference images and second reference images are in a steady state, the first gain coefficient corresponding to the first sensor in the steady state and the second gain coefficient corresponding to the second sensor in the steady state are extracted.

[0040] Here, continuous output refers to the system's regular temporal behavior of continuously sending image files frame by frame over time. Steady state indicates that the feedback loop of the automatic white balance control system has reached its convergence threshold; for example, the colorimetric result of the current image regarding the neutral gray point no longer shows abrupt changes, and the range of white balance coefficient increments for adjacent frames from the same sensor stably approaches zero. The first and second gain coefficients in steady state refer to the optimal coefficient states that most accurately reflect the physical components' ability to adapt and compensate for the known light source offset characteristics after completely filtering out initial jitter and fluctuation resistance damping processes.

[0041] In some embodiments, the white balance statistics module continuously acquires consecutive frames in a standard environment. During computation, it continuously collects the color temperature and gain estimates calculated from the current frame, and then determines whether these continuous output time series fall within a predefined threshold range in the parameter fluctuation range. When it is detected that the reference image maintains convergence and the parameters stop shifting, the set of matrix adjustment coefficients that are currently in effect and have reached a steady state, which can best balance the R / G / B color intensity characteristics, is recorded, thereby obtaining the steady-state first gain coefficient of the first sensor and the steady-state second gain coefficient of the corresponding second sensor at the same time.

[0042] In step 105, a mapping is established between the steady-state first gain coefficient and the steady-state second gain coefficient under the same color temperature to obtain the sensor gain mapping relationship under multiple color temperatures.

[0043] Here, "same color temperature" means that the measurement parameters corresponding to these two different sensors are matched and aligned under the same absolutely constrained lighting input (e.g., both under constant 4000K lighting). The mapping relationship expresses the specific conversion equation or proportional difference array accumulated between the two at a specific temperature anchor point due to hardware characteristics.

[0044] In some embodiments, the device software can establish a reference dictionary. The system groups the recorded steady-state data generated by different hardware in pairs and calculates the ratio product operator or correlation bias distance between the relevant magnitudes of each channel within the steady-state second gain coefficient and the steady-state first gain coefficient under the same specific calibrated color temperature light source environment. Afterward, the system iterates through multiple environmental color temperature data, such as those distributed from low to high color temperature ranges, and calculates multiple sets of mapped numerical points. These multiple color temperature parameters and their corresponding specific mapping association rules are then uniformly compiled and saved. Before mass production, this final compiled array set or model curve is pre-configured (or hardened) in the system's read-only memory or calibration function configuration file, serving as a basic reference for subsequent calculations.

[0045] This application embodiment obtains a basic sample for pattern finding by capturing reference images from two sensors under the same set of standard controlled color temperature environments. Then, the corresponding coefficient values ​​are captured and the corresponding mapping is established only at the time point when the continuous images confirm that they have entered a steady state, eliminating reading fluctuations and false alarm deviations caused by dynamic convergence damping. Finally, a photoelectric offset mapping relationship spanning multiple color temperature thresholds is established, eliminating the uncontrollable characteristics caused by the lack of publicly available physical parameters of a single sensor (the "black box"). This provides a foundation for accurately calculating the applicable gain coefficient (second gain coefficient) of the second sensor using pre-set parameters (first gain coefficient) during actual operation.

[0046] In some embodiments, in the actual use scenarios of multi-camera imaging systems, the spectral distribution of ambient light sources often exhibits characteristics of infinite continuity and complex gradual changes. Limited by device storage space and the time cost constraints of engineering calibration testing, the read-only standard mapping dictionary pre-calculated and burned into the system can typically only set a limited number of coarse discrete color temperature anchor points (e.g., only setting reference points of 3000K, 4000K, and 5000K). When the color temperature estimated by the current system in a real gradual environment (e.g., 3450K) falls within the gaps of these calibrated discrete color temperature anchor points, forming a non-standard cross-boundary color temperature, the second gain coefficient cannot be accurately mapped. Therefore, embodiments of this application propose the following steps.

[0047] In some embodiments, see Figure 5 The second gain coefficient of the second sensor is obtained by mapping based on the first gain coefficient of the first sensor and the sensor gain mapping relationship corresponding to the current color temperature in step 101, which can be achieved through the following steps 1011 to 1014.

[0048] In step 1011, if the current color temperature does not belong to any of the multiple standard color temperatures, the color temperature range in which the current color temperature is located, which is composed of two adjacent standard color temperatures, is determined.

[0049] Here, the statement that the current color temperature does not belong to any of the multiple standard color temperatures indicates that the currently estimated or measured environmental spectral parameters are outside the pre-recorded set of discrete calibration points. A color temperature range refers to the continuous range of values ​​on the numerical axis that encloses the current color temperature and is bounded by the lower and upper boundary standard color temperature values ​​closest to it.

[0050] In some embodiments, after acquiring the current color temperature, the system compares it with the index sequence of the sensor gain mapping table mounted in the memory. If the comparison confirms that no matching existing calibrated color temperature is found, the calculation process can traverse the discrete sequence and search for the nearest standard color temperature point that is only lower than the current color temperature and the nearest standard color temperature point that is only higher than the current color temperature according to the size sorting rules. The two found adjacent preset node values ​​are used to define a working range that includes the current color temperature.

[0051] In step 1012, the mapping between the first gain coefficient and the second gain coefficient established under two adjacent standard color temperatures is extracted from the sensor gain mapping relationship.

[0052] Here, the mapping established under standard color temperature refers to the exact scaling factor or difference data array that has been locked and solidified into non-volatile storage for the corresponding boundary temperature point during the factory calibration stage.

[0053] In some embodiments, after locking the upper and lower boundary reference points of the color temperature range, the processor issues an addressing call execution instruction to the memory structure according to the two boundary reference addresses obtained, retrieves the two corresponding exact standard mapping parameters, and loads them into the current active operation cache as fixed mathematical support endpoints for subsequent estimation of intermediate unknown states.

[0054] In step 1013, based on the position of the current color temperature in the color temperature range, interpolation calculation is performed on the mappings of the two adjacent standard color temperatures to obtain the target mapping corresponding to the current color temperature.

[0055] Here, the current color temperature's position within the color temperature range represents the smooth proportional distribution tendency of the difference between the current color temperature and the upper and lower boundary standard color temperature values ​​(i.e., the mathematical weight of deviation from the center point). Interpolation calculation refers to the mathematical numerical method process of using known scattered data to fit and estimate undetermined coordinate variables located within the scattered data range. Target mapping is an equivalent conversion parameter specifically adapted to this non-standard color temperature environment, output by the system based on the accurate proportion it is in.

[0056] In some embodiments, after extracting two boundary mapping parameters, a subtraction is performed to calculate the temperature difference between the current color temperature and one of the boundary ends, and the ratio of this difference to the entire color temperature range is calculated. This ratio is then used as a balancing weight multiplier to linearly weight and combine the solidified mapping parameters extracted from both ends (or to perform a nonlinear mapping operation based on a preset polynomial fitting curve), thereby smoothly transitioning to a specific target mapping value. In other embodiments, the step of interpolating the mapping to obtain the target mapping can also be achieved by directly using a lookup table to match the nearest second gain coefficient, or by using features such as the current color temperature and ambient light intensity as input, and directly predicting and outputting the target mapping corresponding to the current color temperature using a pre-trained machine learning model.

[0057] In some embodiments, in order to concretize the abstract concept of computation positioning into a rigorous and lightweight underlying machine computation pipeline and accurately allocate the proportions among various stored parameters, the embodiments of this application propose the following steps.

[0058] In some embodiments, such as Figure 6 As shown, two adjacent standard color temperatures include a first standard color temperature and a second standard color temperature. The extracted mappings include a first mapping corresponding to the first standard color temperature and a second mapping corresponding to the second standard color temperature. In this case, step 1013, based on the position of the current color temperature in the color temperature range, interpolates the extracted mappings under the two adjacent standard color temperatures to obtain the target mapping corresponding to the current color temperature. This can be achieved through the following steps 10131 to 10134.

[0059] In step 10131, a first temperature difference between the current color temperature and the first standard color temperature is calculated, and a second temperature difference between the second standard color temperature and the first standard color temperature is calculated.

[0060] Here, the first standard color temperature and the second standard color temperature correspond to the nearest known calibration anchor points (such as the lower limit node and the upper limit node of the color temperature range) distributed on both sides of the current color temperature on the numerical axis.

[0061] In some embodiments, the current color temperature is set to (For example, an estimated 3200K), the first standard color temperature is set. This is denoted as the lower limit of low color temperature (e.g., 3000K in the calibration table), the second standard color temperature. This is denoted as the upper limit high color temperature (e.g., 4000K in the calibration table). First, calculate the first temperature difference between the current color temperature and the first standard color temperature. Furthermore, the full span between these two adjacent standard color temperature anchor points was calculated, which is the second temperature difference calculated based on the second standard color temperature and the first standard color temperature. .

[0062] In step 10132, the ratio of the first temperature difference to the second temperature difference is determined as the interpolation ratio of the current color temperature within the color temperature range.

[0063] In some embodiments, the two temperature offset values ​​mentioned above are extracted, and the interpolation ratio at the corresponding position of the color temperature point is calculated. Let this interpolation ratio be denoted as... The specific calculation formula is as follows: Therefore, the system outputs an interpolation ratio between 0 and 1.

[0064] In step 10133, the gain mapping change between the first mapping and the second mapping is calculated.

[0065] In some embodiments, it is assumed that the color temperature range extracted from the form is consistent with the first standard color temperature. The corresponding first mapping (the number of systems currently called at the underlying level) is denoted as With the second standard color temperature The corresponding second mapping is denoted as The extracted values ​​with strong correlation features are subtracted to obtain the constant of the gain mapping change formed by the two, i.e., the algebraic expression is obtained. The value.

[0066] In step 10134, the first mapping is adjusted based on the interpolation ratio and the change in gain mapping to obtain the target mapping corresponding to the current color temperature.

[0067] In some embodiments, the final target mapping corresponding to the current color temperature is denoted as the current dynamic scaling factor. The previously calculated interpolation ratio can be used. Multiplying this by the maximum change span constant (gain mapping change) of the frequency band yields a customized offset correction increment. This correction increment is then used to weight and adjust the first mapping of the starting base data points, thereby executing the following calculation formula to obtain the final target result: The one-dimensional constant directly output by this calculation formula is the target mapping value that contains the characteristics of ambient light under the current controlled color temperature.

[0068] In some embodiments, after obtaining the target mapping, it can be achieved through... The second gain coefficient was calculated. ,in, This is the first gain coefficient.

[0069] The embodiments of this application determine the interpolation ratio by calculating the temperature distance ratio, and then use the weighted parameter adjustment benchmark to lock the final coefficient, thereby ensuring the response speed when heterogeneous image sensors perform switching tasks.

[0070] In step 1014, the second gain coefficient of the second sensor is obtained by mapping based on the first gain coefficient of the first sensor and the target mapping.

[0071] In some embodiments, after dynamically calculating the target mapping operation factor for precise tuning, a multiplier or offset superposition function can be used to input the existing gain coefficient that is currently in the first mode and is normally performing the anti-color cast function of the first sensor, to complete the mathematical multiplier conversion and unpacking operation, and output the second gain coefficient that will be applied to the initialization of the second sensor.

[0072] This application's embodiments, when faced with continuously varying, unrecorded ambient color temperatures, determine the adjacent standard color temperature ranges and extract the corresponding discrete mapping benchmarks, then use the weight of the current color temperature within the range to perform interpolation. This replaces the simple, coarse nearest-threshold matching, avoiding quantization errors caused by forced merging of values. It utilizes a target mapping synthesized with distributed weights to calculate the first sensor parameters, enabling the output of a second gain coefficient that closely matches light fluctuations. This ensures that the second sensor has a precise optical correction basis when performing sensor switching under varying, continuous, real-world lighting conditions, maintaining smooth and consistent color mapping fidelity.

[0073] In step 102, after entering the second mode, the image acquired by the second sensor is processed based on the second gain coefficient to obtain the output image.

[0074] The second gain coefficient is adjusted by the second historical frame statistical data, which takes the first output image obtained from the image acquired by the second sensor as the starting frame.

[0075] Here, the second gain coefficient is the optimal solution predicted by the aforementioned physical transformation in the initial stage, and is continuously used by the system as a dynamically adjustable variable in subsequent pipelines for color adjustment values ​​of pixel magnification. The second historical frame statistics represent the data accumulator or cache array constructed by the system to maintain the continuous characteristics of the image when running the daily automatic white balance algorithm in the second mode. The starting frame represents the origin of a specific timeline queue at the computer data structure level; that is, the second historical frame statistics are based solely on this starting frame in terms of lifecycle and mathematical weight allocation, expanding unidirectionally backward, and do not include historical statistics before this frame.

[0076] In some embodiments, the actual hardware handover of the image input stream occurs during the process of completing the second gain coefficient prediction and entering the second mode. The system captures real light and then performs white balance correction rendering on the image using the second gain coefficient calculated in the above steps to obtain the output image of the first frame in the second mode. In this output action, in order to prevent the inertia of the large amount of historical data left over from the first mode from diluting and logically binding the current highly accurate new gain coefficient, the overall automatic white balance program sets this first output image belonging to the second mode sequence as the starting reference for establishing new data, thereby building a second historical frame statistical data with a completely new attribute queue in the algorithm state machine.

[0077] In some embodiments, the specific implementation process of outputting images based on the starting frame and adjusting them according to relevant data is essentially a forced state transition and historical data reset. When the control pipeline that manages white balance determines that the current processing is the first frame output image, it will directly block the call response of the temporal smoothing filtering mechanism related to the old lens (first sensor) and its derived data, forcibly ensuring that the real-time pixel statistical features of the first frame itself occupy 100% processing weight and the baseline writing status. Since the starting frame does not inherit values ​​from previous time dimensions in the system logic, the system skips the damped convergence stage that normal white balance requires several seconds and spans dozens of frames to complete. This not only eliminates the lag color cast caused by the algorithm trailing mechanism, but also enters a stable state of color consistency without visual gradient gaps or awkward stuttering.

[0078] This embodiment of the application corrects the steady-state parameter shift caused by differences in the photoelectric characteristics of different sensors by converting a first gain coefficient to a second gain coefficient based on the current color temperature and sensor gain mapping relationship when a sensor switching event is detected. This provides the second sensor with suitable initial environmental white balance parameters and avoids the initial color cast error caused by directly using the first gain coefficient. Simultaneously, by limiting the adjustment of the second historical frame statistical data to use the first frame output image of the second sensor as the starting point, the system cuts off the call to the weight of old historical frame statistical data in the new round of white balance calculation. The automatic white balance algorithm does not require the time and multiple frames of images to remove the inertial influence of old historical parameters, and outputs a stable gain coefficient in the first frame, shortening the frame-level response time during cross-sensor switching (e.g., about 40ms per frame, less than the time perceptible to the naked eye), ensuring color consistency of the image before and after sensor switching.

[0079] Furthermore, since the solution in this embodiment is built purely based on software control flow and parameter mapping prediction, the system does not require the expensive approach of allocating an independent physical ISP channel for each heterogeneous sensor, achieving a smooth switch with zero additional hardware cost. It is particularly suitable for tri-lens or multi-lens hybrid zoom systems with the same model for mid-to-long-range telephoto and heterogeneous for close-range, achieving the best balance between cost and image quality under extremely limited hardware computing resources. Moreover, the gain mapping relationship is established based on measured data and can support linear interpolation to adapt to complex color temperature environments, making it more accurate than a single empirical coefficient.

[0080] In some embodiments, during the white balance correction process of image data processing, if a uniform scalar constant is used to directly magnify all pixels in the entire image, or if mixing operations are performed without stripping the color sub-layers, it can lead to color crosstalk caused by the superposition of non-homologous spectra such as red, green, and blue. This is particularly problematic for heterogeneous multi-sensor systems with varying photosensitive characteristics; different sensor models have independent wavelength response deviations for red, green, and blue light. If differential constraints are not applied to the pixel data captured by different frequency spectra, the second gain coefficient will be unable to correct the original biases in each frequency band, resulting in composite chromaticity shift and color distortion in the output image after switching. Therefore, embodiments of this application propose the following steps.

[0081] It should be noted that in complex scenarios involving multiple sensors, if only the historical statistical data is reset or cleared, the system will lose its color reference due to the instantaneous loss of the smoothing constraint weights in the time dimension. This usually leads to severe transitional abruptness and color jumps in the image output by the new sensor. However, in this embodiment, the mechanism for erasing and clearing historical frame statistical data is a closely coordinated operation with the preceding precise mapping process of sensor gain. Since the first frame image, as the new starting frame, directly applies the second gain coefficient, which has been pre-calculated based on the heterogeneous hardware characteristics through mapping relationships, this precise prediction basis fills the reference gap caused by discarding historical weights. Therefore, this solution, at the moment of forcibly blocking the history of old parameters and resetting the data origin, not only does not cause visual color jumps and abrupt changes, but also eliminates the lag convergence period required by conventional smoothing algorithms, achieving extremely fast and smooth transition of image color during multi-sensor breakpoint switching.

[0082] In some embodiments, see Figure 7 The second gain coefficient contains gain components of multiple color channels. In this case, the output image obtained by processing the image acquired by the second sensor based on the second gain coefficient in step 102 can be achieved through the following steps 1021 to 1022.

[0083] In step 1021, pixel data of each color channel of the image acquired by the second sensor are extracted respectively.

[0084] Here, color channels correspond to the basic three primary color bands (such as red, green, and blue channels) in the image file encapsulation structure, and each channel carries an independently acquired monochromatic signal intensity. Pixel data is the raw digital quantity output after the image photosensitive array completes photoelectric conversion and is quantized analog-to-digital.

[0085] In some embodiments, after receiving a newly acquired raw photosensitive image file from the second sensor, the system performs array parsing on its pixel arrangement format. By locating the data distribution position of its color filters (Bayer array), a channel stripping operation is performed to separate a set of independent and continuous red pixel layer sequences, a set of green pixel layer sequences, and a set of blue pixel layer sequences reflected by a single color light under the same light source environment. Cross-reading of data between channels is stopped so that each becomes its own source data waiting to be independently modulated.

[0086] In step 1022, the pixel data of each color channel is adjusted by the gain components of the corresponding color channels contained in the second gain coefficient to obtain the output image.

[0087] Here, the second gain coefficient in the pre-input condition is not a single value, but a digital array consisting of three or more parameters, whose internal gain components include red compensation coefficients, green compensation coefficients, and blue compensation coefficients. The output image represents the white balance image after the individual channels have been processed and merged.

[0088] In some embodiments, after acquiring the original red, green, and blue pixel layer data extracted from the aforementioned channels, the system schedules the pre-configured component coefficients within the second gain coefficient to form a one-to-one mapping of isomorphic values. Taking multiplication as an example, the system extracts the red gain component and performs dot product amplification correction with all independent pixel data within the red channel, simultaneously extracting the blue and green gain components respectively, and synchronously calculating the pixel data changes within the corresponding other different monochrome channels, while maintaining numerical separation calculations for each color domain. After resetting the multiplier adjustment of each monochrome channel, the system re-aligns and combines the discretized digital channels with modified correction parameters to produce a new output image after color correction and merging.

[0089] This application's embodiments extract monochrome channel pixels by analyzing the basic image structure, replacing the uniform global adjustment of the entire image with independent adjustment actions for different channel pixels. This operation prevents crosstalk when processing differentiated channels such as red, green, and blue, accurately compensates for the non-flat spectral response color shift caused by replacing heterogeneous sensor hardware, and ensures the fidelity of the color structure of the subsequent output image.

[0090] In some embodiments, after switching between multiple sensors and completing initial parameter prediction, the automatic white balance algorithm by default includes a cross-frame weighted filtering function that continues the previous image. If, after calculating the initial output parameters of the new sensor, the inertial dependence of subsequent time series on existing old reference parameters is not severed, the automatic white balance algorithm will continue to retrieve historical data left over from the first sensor to participate in the current calculation when performing time-dimensional weighted iterations on subsequent images. This mixing of old image data will contaminate the newly generated target parameters, causing the system to pull the originally accurate initial parameters back to a color-biased state, resulting in a slow, fading color-biased phenomenon in the output image. In order to sever the correlation of old parameters and restore the anti-flicker function for normal ambient light variations under the premise of eliminating interference, the embodiments of this application propose the following steps.

[0091] In some embodiments, see Figure 8 After processing the image acquired by the second sensor based on the second gain coefficient in step 102 to obtain the output image, steps 106 to 109 can also be performed.

[0092] In step 106, an identifier is added to the first frame output image obtained by processing the image acquired by the second sensor based on the second gain coefficient.

[0093] Here, the first frame refers to the first image in the physical output sequence that has completed white balance processing after the sensor switchover. The identifier represents a status parameter label in the data stream of this first frame image or a separately configured label, used to mark the special node where this frame image is in the switchover process.

[0094] In some embodiments, during the synchronization process of applying the second gain coefficient to the first set of pixel data returned by the second sensor and generating an output image, the system can define and mark the output image of the frame as an identifier frame with exclusive characteristics by rewriting the parameter bits of the frame image attribute information or outputting a state variable with a specific extreme value on the image processing state bus, so as to trigger subsequent interception commands.

[0095] In other embodiments, the aforementioned action of adding a marker to the first frame image and triggering the clearing of historical data can also be implemented independently of the image data stream itself, in the form of a bypass control signal. For example, by setting and raising the value of a specific control register at the underlying level, or by triggering a high-priority hardware interrupt to the automatic white balance module, the first frame switching event can be forcibly reported; this trigger signal can be directly triggered by the switching command at the sensor's underlying level, or it can be actively detected and injected by the image signal processor when it senses a change in image format.

[0096] In step 107, if an identifier is detected, the statistics of the first historical frame before the first output image are cleared.

[0097] Here, the first historical frame statistics refer to the old pixel measurement feature values ​​that the system continuously accumulates and stores for subsequent smoothing calculations when processing images acquired by the first sensor in past time periods. Clearing is manifested as the variable overwriting or erasing operation performed on the storage address space that stores these historical parameters.

[0098] In some embodiments, when the system reads and identifies the identifier containing a specific value in the image processing pipeline, it issues a data blocking and write-zero instruction to the variable range storing the statistical data of the first historical frame according to a preset pointer. The original historical accumulated reference weights and their specific values ​​are assigned to an empty set or given zero weight.

[0099] In step 108, the image with the identifier is used as the starting frame, and the statistical data corresponding to the output images after the starting frame is added to the second historical frame statistical data.

[0100] Here, the starting frame represents the zeroth input base point of a completely new data accumulation cycle. The second historical frame statistics are a newly created or reactivated iterative data set specifically adapted to the second sensor, distinct from the emptied old data pool.

[0101] In some embodiments, since the old data corresponding to the first frame has been cleared and zeroed, the system extracts the objective statistical data calculated from the identified first frame image itself, uses it as a numerical base with a unique reference weight, and writes it into the current historical reference storage space as the starting point. When acquiring the second, third, and other output images subsequently acquired in chronological order from the starting frame image, the system calculates the current statistical data of each subsequent image and arranges them in chronological order or continuously accumulates and merges them into the statistical data of the second historical frame, thereby achieving continuous retention of the features of subsequent images.

[0102] In step 109, the second gain coefficient is adjusted based on the statistical data of the second historical frame, and the subsequent images acquired by the second sensor are processed based on the adjusted second gain coefficient to obtain the output image.

[0103] Here, adjustment refers to updating the existing second gain parameter based on newly imported temporal characteristic data during the calculation process. Subsequent images refer to the normalized output images arranged on the timeline after the first frame identifier image.

[0104] In some embodiments, the system extracts newly generated statistical values ​​from the current single-frame image for subsequent images being acquired. Then, it retrieves the statistical data from the second historical frame, which has been newly accumulated based on previous steps. Following a normal time-smoothing weighting ratio (e.g., allocating a smaller proportion to the current frame and a larger proportion to historical frame data), the system performs a weighted summation calculation on the values ​​of the current new frame and the second historical frame statistical data, outputting an updated gain coefficient that balances current real-time lighting feedback with stable anti-jump requirements. Finally, this updated coefficient is sent to process the pixel data stream from the second sensor to obtain a normal image sequence.

[0105] This embodiment of the application actively adds a specific identifier to the first frame output by the second sensor. During the continuous operation of the algorithm, it triggers and completes the clearing operation of the statistical data of the first historical frame, avoiding the trailing color error caused by the historical parameters of the old sensor during cross-frame iterative calculations. At the same time, it uses the image with the identifier to reset as a new starting frame and integrates the newly added statistical data into the statistical data of the second historical frame, guiding the system to adjust the subsequent second gain coefficient based on the new base data. This process, while avoiding the binding of historical erroneous parameters, restores the smooth function of anti-jump over time by weighted fusion of new historical parameters, ensuring the color stability of the images subsequently produced by the new sensor.

[0106] In some embodiments, after clearing and resetting historical data, the system enters a normalized processing phase for subsequently acquired images. When ambient light levels fluctuate slightly, if white balance is corrected by independently deducing the pixel features of each subsequent frame and directly using it as the final result, subtle lighting interference can cause noticeable color parameter jumps and video flickering between consecutive frames. Therefore, this application provides the following steps.

[0107] In some embodiments, see Figure 9 The step 109, which involves adjusting the second gain coefficient based on the statistical data of the second historical frame and processing the subsequent images acquired by the second sensor based on the adjusted second gain coefficient to obtain the output image, can be achieved through the following steps 1091 to 1093.

[0108] In step 1091, the original gain coefficient of the image acquired by the second sensor is determined based on the pixel statistics of each color channel in the image acquired by the second sensor.

[0109] Here, the raw gain coefficient of the image refers to the gain coefficient calculated solely from the image data of the current single frame, without incorporating historical accumulation information over time. Pixel statistics objectively represent the pixel values ​​of different color components within the current single frame.

[0110] In some embodiments, the system parses and separates the spectral pixel values ​​of red, green, and blue dimensions contained in the latest acquired single-frame image, extracts the overall quantization data of each color dimension, and performs cross-comparison. Based on the proportion of differences between the intensity distributions of different color frequency bands reflected by the specific frame itself, the system derives the monochrome scaling compensation back-calculation values ​​required to bring the proportions of each color towards neutral gray, thereby outputting the original gain coefficients specific to the next objective image slice.

[0111] In step 1092, after outputting the first frame image, the original gain coefficient is weighted and fused with the gain coefficient obtained from the statistical data of the second historical frame, and the fusion result is determined as the updated second gain coefficient.

[0112] Here, the gain coefficient obtained from the statistical data of the second historical frame represents the reference value accumulated from the previous image array at the previous processing time.

[0113] In some embodiments, after the first frame has completed its own identification action, for subsequently acquired image frames, the original gain coefficient of that subsequent frame is extracted, and the gain coefficient calculated from the currently accumulated second historical frame statistical data is retrieved. The system can apply a small reference decimal weight to the original gain coefficient (to capture the impact of currently changing parameters) and a larger reference decimal weight to the gain coefficient obtained from the second historical frame statistical data (to prevent the impact of sudden parameter changes on the main parameters). Subsequently, the two values ​​with different multiplier weights are summed to generate the updated second gain coefficient waiting to take effect.

[0114] In step 1093, the updated second gain coefficient is used to process the subsequent images acquired by the second sensor to obtain the output image.

[0115] Here, the updated second gain coefficient refers to the integrated parameter that is finally sent to the underlying logic to perform white balance physical correction after the single-frame transient data and historical smoothing weighted fusion.

[0116] In some embodiments, after obtaining the calculated updated second gain coefficient, the system converts it into register configuration instructions recognizable by the underlying image signal processor or related image pipeline. These instructions are then written into the image processing pipeline corresponding to the second sensor, sequentially performing corresponding signal scaling and bias correction on each color channel of each subsequent frame of raw image data that is continuously input at high speed. Thus, after overcoming the abrupt change in sensor hardware switching, the system can continuously render and output a video stream with stable color performance and smooth transitions at the output end.

[0117] This application's embodiment calculates the transient original gain coefficient by separating and taking the image values ​​of subsequent frames within the normal time stream, and then executes a pure algebraic instruction to objectively weight and sum this coefficient with historical data benchmarks. This replaces the approach of directly overwriting the adjustment coefficients based solely on single-frame measurements. This numerical fusion step limits the controlled convergence speed range of subsequent sequence parameter updates, prevents image color jumps caused by instantaneous ambient light jitter, and both restores and tracks newly generated spectral conversions while maintaining the color translation transition function between frames across the time dimension.

[0118] The following description continues to illustrate the exemplary structure of the image processing apparatus 255 provided in the embodiments of this application as a software module. In some embodiments, such as... Figure 2 As shown, the software modules stored in the image processing device 255 of the memory 250 may include: The mapping module 2551 is used to map the second gain coefficient of the second sensor based on the first gain coefficient of the first sensor and the sensor gain mapping relationship corresponding to the current color temperature when a sensor switching event is detected. The sensor switching event indicates a switch from the first mode to the second mode. In the first mode, the image acquired by the first sensor is processed based on the first gain coefficient to obtain the output image. The first gain coefficient is calculated by convergence of the statistical data of the first historical frame. The sensor gain mapping relationship is the mapping relationship between the steady-state first gain coefficient and the steady-state second gain coefficient under the current color temperature. The output module 2552 is used to process the image acquired by the second sensor based on the second gain coefficient after entering the second mode to obtain the output image. The second gain coefficient is adjusted by the second historical frame statistics data, and the second historical frame statistics data takes the first output image obtained from the image acquired by the second sensor as the starting frame.

[0119] This application provides a computer program product comprising a computer program or computer-executable instructions stored in a computer-readable storage medium. The processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the image processing method described in this application.

[0120] This application provides a computer-readable storage medium storing computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, the processor will execute the image processing method provided in this application. For example, ... Figure 3 The image processing method shown.

[0121] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0122] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0123] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located in one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0124] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. An image processing method, characterized in that, The method, applied to a multi-sensor system, includes: Upon detecting a sensor switching event, a second gain coefficient of the second sensor is obtained by mapping based on the first gain coefficient of the first sensor and a pre-determined sensor gain mapping relationship corresponding to the current color temperature. The sensor switching event indicates a switch from a first mode to a second mode. In the first mode, the image acquired by the first sensor is processed based on the first gain coefficient to obtain an output image. The first gain coefficient is calculated by convergence of the statistical data of the first historical frame. The sensor gain mapping relationship is the mapping relationship between the first gain coefficient in steady state and the second gain coefficient in steady state at the current color temperature. After entering the second mode, the image acquired by the second sensor is processed based on the second gain coefficient to obtain the output image, wherein the second gain coefficient is adjusted by the second historical frame statistics, and the second historical frame statistics take the first frame of the output image obtained from the image acquired by the second sensor as the starting frame.

2. The image processing method according to claim 1, characterized in that, Before mapping the second gain coefficient of the second sensor based on the first gain coefficient of the first sensor and a predetermined sensor gain mapping relationship corresponding to the current color temperature, the method further includes: In an environment containing multiple standard color temperatures, the images acquired by the first sensor and the images acquired by the second sensor are processed respectively to obtain a first reference image and a second reference image; When multiple first reference images and second reference images are in a steady state after continuous output, the first gain coefficient corresponding to the steady state of the first sensor and the second gain coefficient corresponding to the steady state of the second sensor are extracted. A mapping is established between the first gain coefficient and the second gain coefficient in steady state at the same color temperature to obtain the sensor gain mapping relationship at multiple color temperatures.

3. The image processing method according to claim 2, characterized in that, The process of mapping the first gain coefficient of the first sensor to the second gain coefficient of the second sensor based on the first gain coefficient of the first sensor and the predetermined sensor gain mapping relationship corresponding to the current color temperature includes: If the current color temperature does not belong to any of the multiple standard color temperatures, determine the color temperature range in which the current color temperature is located, which is composed of two adjacent standard color temperatures; From the sensor gain mapping relationship, extract the mapping between the first gain coefficient and the second gain coefficient established under two adjacent standard color temperatures; Based on the position of the current color temperature in the color temperature range, interpolation calculation is performed on the extracted mappings under the two adjacent standard color temperatures to obtain the target mapping corresponding to the current color temperature; Based on the first gain coefficient of the first sensor and the target mapping, the second gain coefficient of the second sensor is obtained by mapping.

4. The image processing method according to claim 3, characterized in that, The two adjacent standard color temperatures include a first standard color temperature and a second standard color temperature, and the extracted mapping includes a first mapping corresponding to the first standard color temperature and a second mapping corresponding to the second standard color temperature; The step of interpolating the extracted mappings under two adjacent standard color temperatures based on the position of the current color temperature within the color temperature range to obtain the target mapping corresponding to the current color temperature includes: Calculate the first temperature difference between the current color temperature and the first standard color temperature, and the second temperature difference between the second standard color temperature and the first standard color temperature; The ratio of the first temperature difference to the second temperature difference is determined as the interpolation ratio of the current color temperature in the color temperature range; Calculate the gain mapping change between the first mapping and the second mapping; The first mapping is adjusted based on the interpolation ratio and the gain mapping change to obtain the target mapping corresponding to the current color temperature.

5. The image processing method according to claim 1, characterized in that, The second gain coefficient includes gain components for multiple color channels; The process of processing the image acquired by the second sensor based on the second gain coefficient to obtain the output image includes: Pixel data for each color channel of the image acquired by the second sensor are extracted respectively; The pixel data of each color channel is adjusted by the gain component corresponding to the color channel included in the second gain coefficient to obtain the output image.

6. The image processing method according to claim 1, characterized in that, After processing the image acquired by the second sensor based on the second gain coefficient to obtain the output image, the method further includes: Add an identifier to the first frame of the output image obtained by processing the image acquired by the second sensor based on the second gain coefficient; If the identifier is detected, the first historical frame statistics data prior to the first output image are cleared; The image with the identifier is used as the starting frame, and the statistical data corresponding to the output images after the starting frame is added to the second historical frame statistical data. The second gain coefficient is adjusted based on the second historical frame statistical data, and the subsequent images acquired by the second sensor are processed based on the adjusted second gain coefficient to obtain the output image.

7. The image processing method according to claim 6, characterized in that, The step of adjusting the second gain coefficient based on the second historical frame statistical data, and processing subsequent images acquired by the second sensor based on the adjusted second gain coefficient to obtain the output image includes: The original gain coefficient of the image acquired by the second sensor is determined based on the pixel statistics of each color channel in the image acquired by the second sensor. After outputting the first frame of the output image, the original gain coefficient is weighted and fused with the gain coefficient obtained from the second historical frame statistics, and the fusion result is determined as the updated second gain coefficient. The updated second gain coefficient is used to process subsequent images acquired by the second sensor to obtain the output image.

8. An electronic device, characterized in that, include: The first sensor is used to acquire images; The second sensor is used to acquire images; Memory is used to store executable instructions or computer programs. A processor, communicatively connected to the first sensor, the second sensor, and the memory, is configured to execute the executable instructions or computer program stored in the memory to implement the image processing method according to any one of claims 1 to 7.

9. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, the image processing method according to any one of claims 1 to 7 is implemented.

10. A computer program product comprising computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, the image processing method according to any one of claims 1 to 7 is implemented.

Images