A method and system for adjusting ISP parameters in scenes with sudden changes in light and shadow.
By using an ISP parameter adjustment method that combines AI prediction and rapid hardware response, the latency problem of traditional ISPs in scenes with sudden changes in light and shadow is solved, enabling rapid emergency response and smooth transition, thereby improving imaging quality and visual perception accuracy.
Patent Information
- Application Number
- CN202611622093.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-10-20
- Publication Date
- 2026-11-17
AI Technical Summary
Traditional ISPs suffer from response delays in parameter adjustment during scenes with sudden changes in light and shadow, resulting in overexposure or underexposure of the image, which affects image quality and visual perception accuracy.
By pre-identifying scene types using an AI module and writing emergency parameters into a shadow register, combined with hardware threshold detection, a frame-level ISP parameter emergency response is achieved, and latency is reduced by smoothly transitioning to optimal parameters.
It enables rapid emergency response of ISP parameters, reduces the risk of image flicker, and ensures the stability of image quality and the accuracy of visual perception.
Smart Images

Figure CN123123718A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image signal processing technology, and in particular to a method and system for adjusting ISP parameters in scenes with sudden changes in light and shadow. Background Technology
[0002] The Image Sensor (ISP) is a core hardware module in imaging equipment. It is responsible for converting the raw image data captured by the CMOS sensor into a full-color image with realistic colors, and controlling shooting parameters through algorithms such as automatic exposure, automatic white balance, and automatic focus. Based on this, ISPs are widely used in fields such as automotive cameras, security monitoring, and action cameras.
[0003] However, in scenarios with sudden changes in lighting, such as vehicles entering and exiting tunnels, switching to backlight, or sudden changes in perspective, the parameter adjustment link of traditional ISPs needs to go through serial links such as statistical reading, software calculation, driver encapsulation, and register writing. There is a significant response delay from the occurrence of sudden changes in light and shadow to the actual effect of parameters on the image.
[0004] Currently, ISP parameter adjustment usually relies on software to complete the calculation and parameter distribution. If a sudden change in light and shadow occurs, the current frame is often overexposed or underexposed before the parameters are ready, which will lead to short-term image flicker and image quality degradation, affecting the recognition accuracy of the subsequent visual perception system. Summary of the Invention
[0005] To overcome the problems existing in related technologies, this application provides a method and system for adjusting ISP parameters in scenes with sudden changes in light and shadow. This method uses an AI module to pre-identify the scene type and issue corresponding emergency parameters, which can achieve one-frame-level hardware-level emergency response of ISP parameters in scenes with sudden changes in light and shadow, and smoothly return to the optimal image quality of software iteration after the emergency.
[0006] According to a first aspect of the embodiments of this application, a method for adjusting ISP parameters in a scene with sudden changes in light and shadow is provided, the method comprising: The scenario type is determined by integrating multi-source scenario information. Emergency parameters corresponding to the scenario type are pre-written into the first shadow register, and mutation detection thresholds corresponding to the scenario type are written into the hardware threshold register. For each frame of input image, the number of pixels exceeding the mutation detection threshold is counted in parallel, and a light and shadow mutation is determined based on the number of pixels. Under normal conditions, the active register takes the value of the second shadow register to drive the ISP pipeline; In response to the determination of a sudden change in light and shadow, an emergency state is entered, and the activity register switches to retrieve the value of the first shadow register; During the emergency state, the software-driven algorithm calculates the optimal parameters in parallel and writes them into the second shadow register. Once the second shadow register is ready, it enters a transition state and transitions from the first shadow register to the second shadow register by weighted interpolation within a preset number of frames.
[0007] In some embodiments, the multi-source scene information includes image recognition information, day / night information, weather information, and location information; The image recognition information, day and night information, weather information and location information are assigned hierarchical priorities, and the fusion weight of each information source is dynamically adjusted according to the confidence level and signal validity of each information source to determine the scene type.
[0008] In some embodiments, if the confidence level of the image recognition information is higher than a confidence level threshold, the scene type is determined based on the recognition result of the image recognition information, and the value of the mutation detection threshold is adjusted based on the day and night information, the weather information, and the location information. If the confidence level of the image recognition information is lower than the confidence threshold, the scene type is determined based on the weighted voting result of the day / night information, the weather information, and the location information. In cases where the weather information has not been updated for a preset period of time, the fusion weight of the weather information is reset to zero. If the signal accuracy of the positioning information is lower than the accuracy threshold, the fusion weight of the positioning information is reset to zero.
[0009] In some embodiments, writing the emergency parameters includes event-triggered writing and periodic writing; The event-triggered writing includes: in response to the location information indicating that it is about to enter the light and shadow change area, writing the emergency parameters corresponding to the light and shadow change area into the first shadow register; The periodic writing includes: re-evaluating the scene type and writing the corresponding emergency parameters into the first shadow register at preset frame intervals.
[0010] In some embodiments, the number of pixels includes the number of overexposed pixels. and number of underexposed pixels The mutation detection threshold includes an upper threshold. and lower threshold ; when and At that time, it was determined that an overexposed light and shadow abrupt change had occurred; when and At that time, it was determined that an underexposed light and shadow change had occurred; in, For pixel values greater than the upper threshold The number of pixels, For pixel values less than the lower threshold The number of pixels, The overexposure count threshold, This is the underexposure count threshold.
[0011] In some embodiments, when and When this occurs, it is determined to be a sudden change in contrast, where: like The system identifies overexposure-type light and shadow abrupt changes and marks them as having increased contrast. like The system identifies underexposure-type light and shadow abrupt changes and marks them as having increased contrast.
[0012] In some embodiments, the upper threshold and the lower threshold are dynamically determined based on the scene brightness level and are updated via a shadow threshold register. The updated upper limit threshold and lower limit threshold are written to the shadow threshold register, and the values in the shadow threshold register are loaded into the hardware threshold register during frame blanking. The upper limit threshold and the lower limit threshold are adjusted in interval according to the standard deviation of the scene's brightness; when When this occurs, increase the distance between the upper limit threshold and the lower limit threshold; when When this occurs, the distance between the upper limit threshold and the lower limit threshold is reduced; in, The standard deviation of brightness, This is the standard deviation threshold.
[0013] In some embodiments, the first shadow register is locked in the emergency state; In the transition state, the first shadow register retains its current value, while the second shadow register receives real-time updates from the software-driven algorithm.
[0014] In some embodiments, the emergency parameters are generated in the following manner: Detection and processing of multiple scenes with sudden changes in light and shadow; Obtain the algorithm output parameters for each scenario, and take the average value of the algorithm output parameters for different scenarios to obtain an emergency parameter library indexed by scenario type; The emergency parameter library is stored in on-chip RAM, and the emergency parameters include exposure gain, exposure time, white balance gain, and noise reduction intensity.
[0015] According to a second aspect of the embodiments of this application, an ISP parameter adjustment system for scenes with sudden changes in light and shadow is provided, the system comprising: The scene determination module is configured to fuse multi-source scene information to determine the scene type, write the emergency parameters corresponding to the scene type into the first shadow register, and write the corresponding mutation detection threshold into the hardware threshold register. The input detection module is configured to count the number of pixels exceeding the mutation detection threshold in parallel for each frame of image, and determine whether a light and shadow mutation has occurred based on the number of pixels. The configuration register module includes a first shadow register, a second shadow register, and an active register. It is configured such that, under normal conditions, the active register takes the value of the second shadow register; when a sudden change in light and shadow is detected and an emergency state is entered, the active register takes the value of the first shadow register; and under transitional conditions, it transitions from the first shadow register to the second shadow register by weighted interpolation. The software-driven module is configured to write the optimal parameters into the second shadow register in parallel computation.
[0016] The technical solutions provided by the embodiments of this application may include the following beneficial effects: This application constructs a dual-shadow register collaborative architecture by setting up a first shadow register and a second shadow register to receive emergency parameters and optimal parameters respectively, and manages the source of parameter values for the active register through three-state control logic of normal state, emergency state and transition state.
[0017] Since the emergency parameters are pre-identified by the AI module and written into the first shadow register before the light and shadow change occurs, the ISP hardware can directly use the ready emergency parameters to drive the pipeline in the next frame after the change is detected in the current frame, without waiting for software calculation and bus transmission. Therefore, the multi-frame delay of the traditional software parameter tuning link can be reduced to a single frame delay, realizing emergency response to the instant of light and shadow change.
[0018] Furthermore, by using ISP hardware to count the number of pixels exceeding the dynamic threshold in parallel in the RAW domain for each frame of image to autonomously detect changes in light and shadow, and dynamically configuring the detection threshold according to the scene type, the risk of false triggering caused by noise or local point light sources can be reduced while ensuring detection sensitivity.
[0019] Furthermore, by using weighted interpolation during the transition state to smoothly transition from emergency parameters to the optimal parameters calculated by the software, screen flicker that may be caused by a direct jump from emergency parameters to optimal parameters can be suppressed, ensuring continuous and stable image quality. Attached Figure Description
[0020] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0021] Figure 1 This is a flowchart illustrating an ISP parameter adjustment method according to an exemplary embodiment.
[0022] Figure 2 This is a schematic diagram of a three-state machine transition according to an exemplary embodiment.
[0023] Figure 3 This is a schematic diagram illustrating a multi-source information fusion decision-making process according to an exemplary embodiment.
[0024] Figure 4 This is a structural block diagram of an ISP parameter adjustment system according to an exemplary embodiment. Detailed Implementation
[0025] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.
[0026] The Image Sensor (ISP) is a crucial module in imaging equipment that converts RAW data captured by a CMOS sensor into a full-color image. In applications such as automotive cameras, security monitoring, and action cameras, ISP parameter adjustments directly impact image quality. Traditional ISPs employ a serial link for automatic exposure: "hardware statistics—software iteration—driver issuance—register activation." This link introduces latency in statistical readout, algorithm convergence, bus transmission, and register writing. Coupled with the inherent multi-frame buffering lag in the ISP pipeline, this results in a significant response delay from the occurrence of sudden changes in light and shadow to the actual application of parameters to the image.
[0027] In scenarios involving sudden changes in lighting, such as vehicles entering and exiting tunnels, switching to backlighting, and rapid changes in perspective, traditional solutions face three main challenges: First, the software parameter tuning link usually has a delay of several frames from parameter calculation to register activation. At the moment when the light and shadow change, the parameters cannot be activated in real time, and the picture will be briefly overexposed or underexposed. Second, relying solely on hardware to generate parameter adjustment instructions based on brightness statistics, without prior scene constraints, makes it impossible to distinguish between real light and shadow changes and temporary brightness fluctuations caused by image noise and local bright light sources, which can easily lead to incorrect adjustments. Third, the configuration parameters for AI real-time inference have inherent processing delays. If calculations are performed only after a sudden change in light and shadow occurs, the inference delay will be added to the response path, making it impossible to participate in instantaneous emergency adjustments.
[0028] To at least partially alleviate or solve the above problems, embodiments of this application provide a method for adjusting ISP parameters in a scene with sudden changes in light and shadow.
[0029] This method can be executed by an ISP chip that integrates an AI acceleration unit. For example, the ISP chip executing the method provided in this application embodiment can be a SoC (System-on-a-Chip) integrating an AI inference module, with the AI inference module directly connected to the ISP pipeline via an on-chip high-speed interface.
[0030] Understandably, the main function of the AI module is to pre-identify the scene type and send the corresponding emergency parameters to the first shadow register before the light and shadow change occurs, rather than performing real-time reasoning after the change occurs, thereby isolating the inherent latency of AI reasoning from the emergency response path.
[0031] Figure 1 This is a flowchart illustrating an ISP parameter adjustment method according to an exemplary embodiment. In the following text, the ISP chip will be used as the execution entity, with reference to... Figure 1 The steps shown illustrate the method provided in this application embodiment. The method mainly includes the following steps: S110: Integrate multi-source scene information to determine the scene type, pre-write the emergency parameters corresponding to the scene type into the first shadow register, and write the mutation detection threshold corresponding to the scene type into the hardware threshold register.
[0032] Understandably, completing scene perception and parameter preloading before the actual occurrence of sudden changes in light and shadow completely isolates the processing latency of scene recognition and parameter decision-making from the subsequent emergency response path.
[0033] In some embodiments, the above-described fusion and determination process can be performed by an AI module integrated into the SoC.
[0034] For example, the AI module can be a dedicated AI acceleration unit integrated inside the SoC, which can be directly connected to the ISP pipeline through an on-chip high-speed interface, bypassing the traditional software-driven bus transmission path, and enabling the rapid writing of emergency parameters.
[0035] It should be noted that the specific hardware form for determining the execution scenario is not limited in the embodiments of this application. Any processing unit that can realize multi-source information fusion and complete parameter pre-writing before mutation can be applied.
[0036] In some embodiments, multi-source scene information includes image recognition information, day / night information, weather information, and location information.
[0037] Among them, image recognition information is obtained by performing lightweight scene classification on YUV frames output by the ISP, and outputting scene labels and corresponding confidence scores; day and night information is obtained by calculating the solar altitude angle based on system time and location; weather information is obtained from the system weather service to obtain the current weather status and light intensity estimate; and location information is obtained by matching latitude and longitude with a map database to infer whether the current location is in an area prone to sudden changes in light and shadow.
[0038] For example, the set of scene labels output by image recognition can include, but is not limited to, types such as: backlight, night scene, tunnel, indoor, outdoor daylight, high contrast, and low light. Day and night information can be divided into three states: daytime, dusk or dawn, and night. Location information can infer whether the vehicle is currently located in areas such as tunnel entrance, underground parking lot entrance, and high-rise canyon.
[0039] In some embodiments, hierarchical priorities are set for the above four types of information sources, and the fusion weight of each information source is dynamically adjusted according to the confidence level and signal validity of each information source. Figure 3 This is a schematic diagram illustrating multi-source information fusion decision-making according to an exemplary embodiment, with reference to... Figure 3 The four information sources are prioritized from highest to lowest as follows: image recognition information (P0), day and night information (P1), weather information (P2), and location information (P3).
[0040] Understandably, image recognition directly observes the current scene, offering the highest real-time performance and serving as the fundamental basis for scene judgment; therefore, it is set as the highest priority P0. Day / night information is deterministic, determining the overall brightness range and significantly impacting detection threshold configuration. Weather information affects the total ambient light, but its update delay is significant, serving only as an aid for coarse scene classification. Location information is indirect inference; GPS may fail, so it is only used for early prediction and auxiliary confirmation.
[0041] In some embodiments, when the confidence level of the image recognition information is higher than the confidence level threshold, the scene type is determined based on the recognition result of the image recognition information, and the value of the mutation detection threshold is adjusted based on day and night information, weather information, and location information.
[0042] For example, the confidence threshold can be set to 0.7. If the confidence of the image recognition information is lower than the confidence threshold, the scene type is determined based on the weighted voting results of day / night information, weather information, and location information.
[0043] For example, multi-source weighted voting can be initiated when the confidence level is below 0.5 or when the recognition result fluctuates between the two scenarios.
[0044] It should be noted that the fusion weight of each information source is not a fixed value, but is dynamically adjusted according to the current credibility of the information source.
[0045] For example, the fusion weights for image recognition can be between 0.4 and 0.7, with an upper limit of 0.7 when the confidence level is higher than 0.8 and a lower limit of 0.4 when the confidence level is lower than 0.5; the fusion weights for day and night information can be between 0.1 and 0.25; the fusion weights for weather information can be between 0.0 and 0.15; and the fusion weights for location information can be between 0.0 and 0.2. The sum of all fusion weights is constrained to 1.0.
[0046] In some embodiments, if weather information is not updated for a preset time period, the fusion weight of the weather information is reset to zero. For example, the preset time period can be 30 minutes. If the signal accuracy of the location information is lower than an accuracy threshold, the fusion weight of the location information is reset to zero. For example, the accuracy threshold can be 100 meters.
[0047] The preset duration and accuracy thresholds mentioned above can be set as needed, and this application embodiment does not limit them.
[0048] It should be noted that location information has significant predictive value in specific scenarios. When location information clearly indicates that a vehicle is about to enter an area prone to sudden changes in light and shadow, such as a tunnel or underground parking lot, the location information can play a decisive role and trigger pre-configuration, even if image recognition and day / night information have not yet reflected the scene changes.
[0049] In this embodiment, by employing a multi-source information fusion mechanism that combines hierarchical priority with dynamic weighting, the system can automatically downgrade the fusion strategy under complex conditions such as low-light image recognition failure, GPS signal loss, and expired weather data. This effectively reduces the risk of misjudgment of scene labels due to the failure of a single information source. Therefore, it can adapt to various extreme conditions while ensuring the reliability of fusion prediction.
[0050] In some embodiments, based on the scene type determined by multi-source fusion, corresponding parameters are selected from the emergency parameter library stored in on-chip RAM and written to the first shadow register via a direct connection interface with the ISP. The emergency parameter library is indexed by scene type, and each scene type corresponds to a set of emergency parameters, including but not limited to exposure gain, exposure time, white balance gain, and noise reduction intensity.
[0051] In some embodiments, emergency parameters are generated as follows: Multiple scenes with sudden changes in light and shadow are detected and processed to obtain the algorithm output parameters for each scene. The average value of the algorithm output parameters for different scenes is taken to obtain an emergency parameter library indexed by scene type.
[0052] Understandably, by averaging the algorithm outputs of a large number of mutation scenarios, a baseline parameter with cross-scenario robustness can be obtained, providing acceptable imaging results at the moment of mutation, without pursuing the optimal for a specific scenario.
[0053] For example, the above detection process can be carried out by using ISP software-driven algorithms to run the 3A algorithm on multiple pre-collected light and shadow change scene samples in an offline environment, record the parameter values after convergence of each scene, and then group them by scene type and take the average.
[0054] It should be noted that updating the first shadow register usually does not affect the operation of the current ISP pipeline. Since the first shadow register is not directly connected to the active register, new values can be safely written to the first shadow register at any time, effectively reducing the risk of changes in the current frame parameters midway.
[0055] In some embodiments, writing emergency parameters includes event-triggered writing and periodic writing.
[0056] The event-triggered write includes: in response to the location information combined with navigation data clearly indicating that the vehicle is about to enter a region of sudden change in light and shadow, writing the emergency parameters corresponding to that region into the first shadow register.
[0057] Periodic writing includes: re-evaluating the scene type and writing the corresponding emergency parameters to the first shadow register at preset frame intervals to correct parameter deviations caused by slow environmental drift. For example, the preset frame number can be configured according to the application scenario, and this application embodiment does not limit this.
[0058] In some embodiments, while writing to the first shadow register, a mutation detection threshold corresponding to the current scene type is also written to the hardware threshold register. The mutation detection threshold includes an upper limit threshold. and lower threshold It updates periodically based on the current scene's brightness level or immediately when switching scenes.
[0059] For example, in extremely bright scenes (such as midday sunlight or snow), It can be set between 245 and 250. It can be set between 30 and 40; in bright scenes (such as ordinary outdoor daytime), It can be set between 235 and 245. It can be set between 20 and 30; in dark scenes (such as dusk, night street scenes), It can be set between 180 and 200. It can be set between 50 and 70.
[0060] It should be noted that the above values are example configurations for 8-bit input data, and this application embodiment does not limit them.
[0061] In some embodiments, the update of the mutation detection threshold is relayed through a shadow threshold register: the updated threshold is then transferred to the shadow threshold register. and The shadow threshold register is written to, and the ISP hardware loads the value of the shadow threshold register into the hardware threshold register during frame blanking.
[0062] Understandably, updating the threshold through the shadow threshold register can effectively reduce the risk of threshold changes during the effective line scan of the same frame, thereby ensuring that the threshold used for pixel statistics within the same frame is consistent and improving the reliability of the statistical results.
[0063] In some embodiments, and The spacing can be further adjusted based on the scene's brightness standard deviation σ to adapt to sub-scenes with different contrast characteristics. σ is obtained by statistically analyzing the effective pixel grayscale values of the current frame or several recent frames.
[0064] Specifically, set If the standard deviation threshold is used, the adjustment rules are as follows: when At that time, increase and The spacing.
[0065] Understandably, when a scene has high contrast, the distribution range of pixel values within a frame is large. Expanding the judgment window for normal pixels can reduce the risk of false triggering caused by the inherent brightness differences in the scene.
[0066] when When, shrink and The spacing.
[0067] Understandably, when scene contrast is low, the pixel values within a frame are concentrated, and even a slight change in brightness should attract attention. Therefore, the decision window is narrowed to improve detection sensitivity.
[0068] For example, In the vehicle ISP, it can be set to 30 (under 8-bit grayscale). This application embodiment... The specific value is not limited.
[0069] It should be noted that the specific values mentioned above are merely examples, and the embodiments of this application do not limit them.
[0070] In this embodiment of the application, due to and The spacing is adaptively adjusted by the brightness standard deviation σ. Therefore, differentiated detection threshold configurations can be provided for sub-scenes with different contrast characteristics within the same broad scene category, reducing the risk of false triggering in high-contrast scenes and improving detection sensitivity in low-contrast scenes, further enhancing the adaptive capability of mutation detection.
[0071] It should be noted that since ISPs typically support inputs at different resolutions, the total number of pixels varies with the resolution. Therefore, in addition to switching the abrupt change detection threshold during frame blanking, resolution-related statistical parameters can be updated according to the software-driven configuration to adapt to the detection requirements at different resolutions.
[0072] In this embodiment, since scene determination, emergency parameter preloading, and mutation detection threshold configuration are completed before the light and shadow abrupt change occurs, the inherent processing delay of scene recognition does not occupy the time of subsequent emergency response paths. Furthermore, because the mutation detection threshold is dynamically set based on scene semantic information, the hardware detection module can adaptively adjust its detection sensitivity for different scenes, balancing sensitivity and anti-interference capability.
[0073] S120. In the RAW domain of the ISP pipeline, count the number of pixels exceeding the mutation detection threshold in parallel for each frame of input image.
[0074] Understandably, this step is performed at the front end of the ISP pipeline, and statistics are calculated in parallel as pixel data flows from the sensor into the pipeline. Typically, no additional frame buffering or waiting for complete frame data is required. The statistical operation is synchronized with the normal pipeline processing and is completed before the frame blanking period begins. The statistical results are used for subsequent abrupt change determination during the frame blanking period.
[0075] In some embodiments, the number of pixels includes the number of overexposed pixels. and number of underexposed pixels Among them, the mutation detection threshold includes an upper threshold. and lower threshold , For pixel values greater than in the current frame The number of pixels, For pixel values less than in the current frame The number of pixels. In addition, it can count the total number of valid pixels in the current frame for ratio calculations, excluding pixels in black borders and occluded areas.
[0076] It should be noted that the above statistics are performed in the RAW domain, that is, the raw pixel values output by the sensor are statistically analyzed, rather than the YUV or RGB data after ISP processing. The advantage of performing statistics in the RAW domain is that RAW data directly reflects the light intensity received by the sensor, is free from interference from subsequent ISP processing, and can more accurately capture real light changes.
[0077] In this embodiment, since pixel statistics and pipeline processing are performed in parallel, the statistical operation does not introduce additional frame delay. Therefore, the ISP hardware can obtain the brightness distribution statistics of the current frame at the end of each frame, providing real-time data support for subsequent abrupt change determination.
[0078] S130. Determine whether a sudden change in light and shadow occurs based on the number of pixels.
[0079] Understandably, this step is performed during frame blanking. It uses Nh and Nl obtained from S120 to make a logical judgment to determine whether a sudden change in light and shadow has occurred in the current frame and the direction of the change (overexposure or underexposure).
[0080] In some embodiments, a The overexposure count threshold, If the underexposure count threshold is used, the determination logic is as follows: when and At that time, it was determined that an overexposed light and shadow abrupt change had occurred; when and At that time, it was determined that an underexposed light and shadow change had occurred; when and At that time, the image was determined to be normal.
[0081] In some embodiments, when and When both conditions are met, it indicates that the mid-tone pixels in the image are polarizing towards the bright and dark extremes, which is determined to be a sudden change in contrast. The processing rule is as follows: like The system identifies overexposure-type light and shadow abrupt changes and marks them as having increased contrast. like The system identifies underexposure-type light and shadow abrupt changes and marks them as having increased contrast.
[0082] The contrast increase marker can be synchronized to the relevant modules in the ISP pipeline to apply the contrast adjustment parameters.
[0083] It should be noted that, and The configuration can be tailored to the application scenario and resolution, but this application does not limit the specific configuration.
[0084] In this embodiment of the application, since the mutation determination is completed autonomously by the ISP hardware during frame blanking without relying on the additional participation of the CPU or AI module, the determination process has deterministic timing and extremely low latency.
[0085] By incorporating the pre-configured dynamic thresholds in the S110, the hardware detection module can automatically adjust its detection sensitivity in different scenarios. Therefore, while ensuring the ability to capture real-time changes in light and shadow, it reduces the risk of false triggering caused by noise, localized point light sources, and moving objects resulting in artificial brightness fluctuations.
[0086] It should be noted that since mutation detection is based on the statistical results of the current frame and is determined during frame blanking, and the determination result affects the register switching of the next frame, there is an inherent delay of one frame from detection to response. That is, after a change in lighting is detected in the current frame, the emergency parameters take effect at the beginning of the next frame.
[0087] In response to the detection of a sudden change in light and shadow, an emergency state is entered, and the active register switches to retrieve the value of the first shadow register.
[0088] Figure 2 This is a schematic diagram illustrating a three-state machine transition according to an exemplary embodiment. (Reference) Figure 2 The ISP parameter adjustment system includes three operating states: normal, emergency, and transition.
[0089] Under normal conditions, the active register always uses the value of the second shadow register to drive the ISP pipeline. That is, the software-driven 3A (auto exposure, auto white balance, auto focus) algorithm runs normally, and after parameter calculation, it is written to the second shadow register via the bus and latched to the active register every frame. During this period, the first shadow register stores pre-configured emergency parameters, which are only used as backup and do not affect the pipeline. Simultaneously, the input detection module continuously performs abrupt change detection.
[0090] In response to the judgment result of S130 indicating a sudden change in light and shadow, the system switches from normal state to emergency state.
[0091] In an emergency, the active register immediately switches to the value of the first shadow register. Since the emergency parameters in the first shadow register have been pre-configured before the light and shadow change occurs, the parameter switch can take effect instantly without software calculation or bus transmission.
[0092] For example, if a vehicle enters a tunnel from the outside and causes the image to darken suddenly, S130 determines that an underexposure-type light and shadow change has occurred. The system immediately enters an emergency state, and the active register switches to retrieve the pre-configured tunnel scene emergency parameters in the first shadow register. The pipeline uses the emergency parameters to process the image in the next frame, reducing the response delay of several frames in the traditional solution to one frame.
[0093] It should be noted that the emergency state is a time-constrained temporary state. Its purpose is to provide an acceptable transitional solution when the software-driven algorithm has not yet completed the calculation of the optimal parameters, rather than to replace the optimal parameters of the software-driven algorithm in the long term.
[0094] In some embodiments, the first shadow register is locked in an emergency. Understandably, this locking mechanism can effectively reduce secondary screen jumps caused by parameter changes during an emergency, ensuring parameter stability during the emergency.
[0095] In an emergency, the software-driven algorithm calculates the optimal parameters in parallel and writes them into the second shadow register.
[0096] Understandably, while the active register switches to the first shadow register, the software-driven 3A algorithm continues to operate, iterating and converging based on the statistical information of the current frame to calculate the optimal parameters suitable for the current actual scenario. After the calculation is completed, the optimal parameters are written to the second shadow register via the bus.
[0097] It should be noted that the computation time of the software-driven algorithm depends on the algorithm complexity and hardware platform performance, and may require several frames to converge to the optimal parameters. This application embodiment does not limit the specific number of convergence frames. During this time, the ISP pipeline continuously uses the emergency parameters in the first shadow register to reduce overexposure or underexposure in the image.
[0098] Once the second shadow register is ready, it enters a transition state and transitions from the first shadow register to the second shadow register by weighted interpolation within a preset number of frames.
[0099] Understandably, once the software-driven algorithm completes the optimal parameter calculation and writes it to the second shadow register, the system transitions from an emergency state to a transitional state. (Reference) Figure 2 During the transition state, the value of the active register is calculated using the weighted interpolation formula: ACTIVE=w×SREG1+(1-w)×SREG2; SREG1 is the value of the first shadow register, SREG2 is the value of the second shadow register, and the weight w is pre-configured by the software driver.
[0100] In some embodiments, the weight w decreases linearly from 1 to 0 over a preset number of frames.
[0101] For example, the preset number of frames can be set to 3 frames. Then, the weight of the first transition frame is w=2 / 3, the weight of the second transition frame is w=1 / 3, and the weight of the third transition frame is w=0. After the transition is completed, the active register takes the value of the second shadow register completely, and the system returns to the normal state. The preset number of frames and the weight change method can be configured according to the application scenario, and this application embodiment does not limit them.
[0102] In some embodiments, during the transition state, the first shadow register maintains its current value (continues to be locked) to ensure that the starting parameters do not change during the transition; the second shadow register receives real-time updates from the software-driven algorithm so that the transition endpoint is always the latest optimal parameters of the software algorithm.
[0103] In this embodiment, the first shadow register is locked in an emergency state and its value remains unchanged in a transition state, while the second shadow register is allowed to continuously receive the latest calculation results driven by the software. Therefore, it is possible to ensure that the parameters do not change midway during the emergency period while ensuring that the transition endpoint always converges to the latest optimal parameters of the software algorithm.
[0104] In this embodiment, the transition state achieves a smooth transition from emergency parameters to optimal parameters through frame-by-frame weighted interpolation, effectively suppressing screen flicker caused by sudden jumps in parameters from emergency values to optimal values. Therefore, a continuous and stable transition in image quality can be achieved while ensuring rapid emergency response.
[0105] After the transition is complete, the system returns to normal. The active register resumes to the value of the second shadow register, and the first shadow register resumes to a writable state, preparing for the next possible sudden change in light and shadow. The three-state machine continues to run in a loop, realizing closed-loop control of "prediction - hardware emergency - software takeover - smooth return".
[0106] Next, the above method will be illustrated with a complete application scenario. Taking a vehicle entering a tunnel as an example: Step 1: The vehicle is driving normally on the highway. The system continuously integrates image recognition, day and night, weather and location information, determines that the current scene is "outdoor daytime", the system is in normal state, and the activity register takes the optimal software parameters from the second shadow register.
[0107] Step 2: The vehicle navigation system indicates that the vehicle is about to enter the tunnel ahead. The positioning information triggers an event to write the information. Emergency parameters corresponding to the "tunnel" scenario are selected from the emergency parameter library and written to the first shadow register. At the same time, the mutation detection threshold suitable for dark scenes is written to the hardware threshold register.
[0108] Step 3: As the vehicle enters the tunnel, the screen brightness drops sharply. The input detection module detects a large number of images below the lower threshold in the RAW domain. Underexposed pixels, and The underexposure criteria are met.
[0109] Step 4: The system immediately enters emergency mode, the active register switches to retrieve the pre-configured tunnel emergency parameters from the first shadow register, the first shadow register is locked, and the next frame will process the image with these parameters to avoid a black screen.
[0110] Step 5: The software-driven 3A algorithm iterates in parallel based on the actual illumination statistics in the tunnel, and converges to obtain the optimal parameters after several frames, which are then written into the second shadow register.
[0111] Step Six: The system enters a transition state, smoothly transitioning from emergency parameters to optimal parameters over 3 frames using weighted interpolation, resulting in a stable improvement in image quality to its best. After the transition is complete, the system returns to normal, and the first shadow register is restored to a writable state.
[0112] Based on this, when the vehicle exits the tunnel, a similar process will be triggered again—the "outdoor daytime" scene parameters will be pre-configured based on the location information, and the input detection module will enter an emergency state after detecting an overexposure-type sudden change, and then smoothly transition to the optimal software parameters.
[0113] It should be noted that the methods described in the embodiments of this application are also applicable to other light and shadow change scenarios in security monitoring, such as sunrise and sunset, and rapid switching of action cameras between light and dark environments, and are not limited to the above description of vehicles entering and exiting tunnels.
[0114] refer to Figure 4 , Figure 4 This is a structural block diagram illustrating an ISP parameter adjustment system according to an exemplary embodiment. The system includes a scene determination module, an input detection module, a configuration register module, and a software driver module. Exemplarily, the scene determination module can be implemented by an AI acceleration unit integrated in a SoC, and the specific hardware form is not limited in this application embodiment.
[0115] The scene determination module is configured to integrate multi-source scene information to determine the scene type, write the emergency parameters corresponding to the scene type into the first shadow register, and write the corresponding mutation detection threshold into the hardware threshold register.
[0116] The input detection module is configured to count the number of pixels exceeding the mutation detection threshold in parallel in the RAW domain of the ISP pipeline for each frame image, and determine whether a light and shadow mutation has occurred based on the number of pixels.
[0117] The configuration register module includes a first shadow register, a second shadow register, and an active register. It is configured such that, under normal conditions, the active register takes the value of the second shadow register; in response to a sudden change in light and shadow indicating an emergency state, the active register takes the value of the first shadow register; and under transitional conditions, it transitions from the first shadow register to the second shadow register using weighted interpolation. Under emergency conditions, the configuration register module locks the first shadow register.
[0118] The software driver module is configured to write the optimal parameters to the second shadow register for parallel computation.
[0119] Regarding the system in the above embodiments, the specific ways in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated here.
[0120] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the claims.
[0121] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.
Claims
1. A method for adjusting ISP parameters in a light-and-shadow mutation scene, characterized in that, include: The scenario type is determined by integrating multi-source scenario information. Emergency parameters corresponding to the scenario type are pre-written into the first shadow register, and mutation detection thresholds corresponding to the scenario type are written into the hardware threshold register. For each frame of input image, the number of pixels exceeding the mutation detection threshold is counted in parallel, and a light and shadow mutation is determined based on the number of pixels. Under normal conditions, the active register takes the value of the second shadow register to drive the ISP pipeline; In response to the determination of a sudden change in light and shadow, an emergency state is entered, and the activity register switches to retrieve the value of the first shadow register; During the emergency state, the software-driven algorithm calculates the optimal parameters in parallel and writes them into the second shadow register. Once the second shadow register is ready, it enters a transition state and transitions from the first shadow register to the second shadow register by weighted interpolation within a preset number of frames.
2. The method according to claim 1, characterized in that, The multi-source scene information includes image recognition information, day and night information, weather information, and location information; The image recognition information, day and night information, weather information and location information are assigned hierarchical priorities, and the fusion weight of each information source is dynamically adjusted according to the confidence level and signal validity of each information source to determine the scene type.
3. The method according to claim 2, characterized in that, If the confidence level of the image recognition information is higher than the confidence level threshold, the scene type is determined based on the recognition result of the image recognition information, and the value of the mutation detection threshold is adjusted based on the day and night information, the weather information, and the location information. If the confidence level of the image recognition information is lower than the confidence threshold, the scene type is determined based on the weighted voting result of the day / night information, the weather information, and the location information. In cases where the weather information has not been updated for a preset period of time, the fusion weight of the weather information is reset to zero. If the signal accuracy of the positioning information is lower than the accuracy threshold, the fusion weight of the positioning information is reset to zero.
4. The method according to claim 3, characterized in that, The writing of the emergency parameters includes event-triggered writing and periodic writing; The event-triggered writing includes: in response to the location information indicating that it is about to enter the light and shadow change area, writing the emergency parameters corresponding to the light and shadow change area into the first shadow register; The periodic writing includes: re-evaluating the scene type and writing the corresponding emergency parameters into the first shadow register at preset frame intervals.
5. The method according to claim 1, characterized in that, The number of pixels includes the number of overexposed pixels. and number of underexposed pixels The mutation detection threshold includes an upper threshold. and lower threshold ; when and At that time, it was determined that an overexposed light and shadow abrupt change had occurred; when and At that time, it was determined that an underexposed light and shadow change had occurred; in, For pixel values greater than the upper threshold The number of pixels, For pixel values less than the lower threshold The number of pixels, The overexposure count threshold, This is the underexposure count threshold.
6. The method according to claim 5, characterized in that, when and When this occurs, it is determined to be a sudden change in contrast, where: like The system identifies overexposure-type light and shadow abrupt changes and marks them as having increased contrast. like The system identifies underexposure-type light and shadow abrupt changes and marks them as having increased contrast.
7. The method according to claim 5, characterized in that, The upper and lower thresholds are dynamically determined based on the scene brightness level and are updated via a shadow threshold register. The updated upper limit threshold and lower limit threshold are written to the shadow threshold register, and the values in the shadow threshold register are loaded into the hardware threshold register during frame blanking. The upper limit threshold and the lower limit threshold are adjusted in interval according to the standard deviation of the scene's brightness; when When this occurs, increase the distance between the upper limit threshold and the lower limit threshold; when When this occurs, the distance between the upper limit threshold and the lower limit threshold is reduced; in, The standard deviation of brightness, This is the standard deviation threshold.
8. The method according to claim 1, characterized in that, In the emergency state, the first shadow register is locked; In the transition state, the first shadow register retains its current value, while the second shadow register receives real-time updates from the software-driven algorithm.
9. The method according to claim 8, characterized in that, The emergency parameters are generated in the following manner: Detection and processing of multiple scenes with sudden changes in light and shadow; Obtain the algorithm output parameters for each scenario, and take the average value of the algorithm output parameters for different scenarios to obtain an emergency parameter library indexed by scenario type; The emergency parameters include exposure gain, exposure time, white balance gain, and noise reduction intensity.
10. An ISP parameter adjustment system for scenes with sudden changes in light and shadow, characterized in that, include: The scene determination module is configured to fuse multi-source scene information to determine the scene type, write the emergency parameters corresponding to the scene type into the first shadow register, and write the corresponding mutation detection threshold into the hardware threshold register. The input detection module is configured to count the number of pixels exceeding the mutation detection threshold in parallel for each frame of image, and determine whether a light and shadow mutation has occurred based on the number of pixels. The configuration register module includes a first shadow register, a second shadow register, and an active register. It is configured such that, under normal conditions, the active register takes the value of the second shadow register; when a sudden change in light and shadow is detected and an emergency state is entered, the active register takes the value of the first shadow register; and under transitional conditions, it transitions from the first shadow register to the second shadow register by weighted interpolation. The software-driven module is configured to write the optimal parameters into the second shadow register in parallel computation.
