A method and apparatus for image dehazing within a sensor

By employing a cross-channel harmonic mean algorithm and memristor memory array within the sensor, the problems of high computational cost and artifacts in high-resolution image dehazing are solved, achieving efficient image dehazing processing suitable for edge imaging systems such as mobile terminals and vehicle cameras.

CN123123749APending Publication Date: 2026-11-17HUNAN UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611178120.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-05
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing image dehazing methods are computationally and storage-intensive under high-resolution conditions. Furthermore, traditional algorithms are incompatible with simulation calculation methods, leading to increased processor load and the need for external data transfer. In addition, the continuity of local features is easily disrupted, resulting in artifacts.

Method used

A cross-channel harmonic mean algorithm is used to replace the traditional dark channel minimum pooling, and it is mapped onto the photoelectric sensing and memristor memory array and two-stage peripheral analog circuitry to realize background light estimation, transmittance estimation and image restoration within the sensor.

Benefits of technology

It reduces external data movement and processor workload, suppresses edge artifacts, maintains the continuity of local features, and is suitable for real-time dehazing of high-resolution images.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123123749A_ABST
    Figure CN123123749A_ABST
Patent Text Reader

Abstract

This invention relates to the field of image dehazing and in-sensor computation technology, specifically disclosing a method and apparatus for in-sensor image dehazing. The apparatus includes a photoelectric sensing and memristor storage array with multiple color channels, a background light estimation circuit, a transmittance estimation and image restoration circuit, and an analog-to-digital conversion module. After color channel separation, the pixel information of each channel in a hazy scene is mapped to a memristor state. The background light estimation circuit extracts dark features and obtains background light information within each channel using a local harmonic mean kernel. The transmittance estimation and image restoration circuit performs cross-channel harmonic mean calculations on the dark features of multiple channels at the same spatial location to obtain transmittance and complete image restoration for each channel. The restored results are then converted to digital and merged to form a dehazed image. This invention integrates image sensing, storage, and dehazing computation within the sensor, reducing external data transmission and processor computational burden during high-resolution image processing, and helping to preserve image structural information. It is suitable for mobile terminals, intelligent monitoring, drones, and vehicle-mounted imaging equipment.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the fields of image dehazing, photoelectric image sensing, memristor in-memory computing, and analog computing circuits. Specifically, it relates to a method and apparatus that maps a cross-channel harmonic mean dehazing algorithm onto a photoelectric sensing and memristor storage array and a two-stage peripheral analog circuit, thereby completing background light estimation, transmittance estimation, and image restoration within the sensor. Background Technology

[0002] Fog and haze reduce image contrast and obscure structural details, affecting the imaging and subsequent visual analysis of mobile terminal cameras, smart surveillance cameras, consumer drones, and vehicle-mounted cameras under adverse weather conditions. Existing dehazing methods based on atmospheric scattering models typically estimate background light and transmittance pixel by pixel; when using dark channel priors, local minimum pooling can disrupt the continuity of local features and may produce artifacts at the boundaries of high-contrast objects. While deep networks can achieve better restoration quality, they usually require significant computational, storage, and data transmission bandwidth.

[0003] As image resolution increases, reading the complete original frame from the sensor first and then having an external processor perform dehazing increases the processor load and external data transfer. Memristor cross arrays can utilize Ohm's law and Kirchhoff's current law to perform analog summation in parallel; integrating photodetector arrays with memristor arrays also allows scene brightness to be encoded as memristor states at the sensing end. However, the digital computation flow of traditional dehazing algorithms does not directly match the analog computation method of cross arrays, especially for local window reciprocal summation, cross-channel feature extraction, transmittance calculation, and restoration operations, which require establishing a clear algorithm-circuit correspondence. Summary of the Invention

[0004] The purpose of this invention is to provide a method and apparatus for image dehazing within a sensor. It uses a cross-channel harmonic mean algorithm to replace the traditional dark channel minimum pooling and maps the algorithm to memristor array current and peripheral analog circuitry. This allows the sensing of the red, green, and blue channels, the calculation of the local harmonic mean, the estimation of the background light, the estimation of the cross-channel transmittance, and the recovery output to be performed in a two-stage process within the sensor.

[0005] The device includes photoelectric sensing and memristor storage arrays for red, green, and blue color channels; a first-stage background light estimation circuit; a second-stage transmittance and image restoration circuit; a control circuit for applying drive and programming voltages; peripheral operational amplifiers and analog division branches; and an analog-to-digital converter module for digitizing the three restored channels. Unless otherwise specified, scalar and matrix operations, and matrix-matrix operations, are performed element-wise in this specification.

[0006] For color channels The atmospheric scattering model is as follows:

[0007] (1)

[0008] In equation (1), For the acquired haze image channels, For the haze-free image channel to be recovered, Transmittance, This is the background light estimate. The input image is first processed... The lower limit is protected to prevent the denominator of subsequent reciprocal operations from being zero.

[0009] Let the target pixel coordinates be The local window radius is ,by The biased harmonic mean operator is defined as follows: (This represents a local window centered on the target pixel.)

[0010] (2)

[0011] In equation (2), This is the bias coefficient used to adjust the exposure level. An example of atmospheric haze imagery is taken... Examples of underwater fogging images Using a harmonic mean window can avoid the abrupt changes in local features caused by traditional minimum pooling.

[0012] The dark features of the red, green, and blue channels are formed by the following formula:

[0013] (3)

[0014] The background light estimates for the three color channels are directly determined by the ratio of the corresponding channel image to the corresponding channel dark feature:

[0015] (4)

[0016] Send the dark features of three channels at the same spatial location into The normalized features are obtained by cross-channel harmonic mean calculation. :

[0017] (5)

[0018] Transmittance and its lower limit protection are as follows:

[0019] (6)

[0020] In equation (6), For the defogging intensity parameter, this embodiment takes... After obtaining the background light estimate and transmittance, each color channel is recovered using the inverse process of the atmospheric scattering model:

[0021] (7)

[0022] (8)

[0023] In hardware mapping, pixel intensity Corresponding memristor Array drive voltage Take 0.01V; average conductivity of the harmonic mean kernel. Feedback resistor Bias voltage When the dark features of the channel output from the first stage are written into the subsequent array, .in This embodiment is determined by the available resistance range of the memristor. , .

[0024] In the blue channel Taking the local harmonic mean kernel centered at the center as an example, in Under its influence, the core operational amplifier output is:

[0025] (9)

[0026] After substituting the aforementioned parameter mapping, equation (9) becomes:

[0027] (10)

[0028] The following analog voltages are given at the corresponding output terminals of the first stage:

[0029] (11)

[0030] (12)

[0031] (13)

[0032] Multiple local harmonic mean kernels operate in parallel in the first stage, and their positions are dynamically moved via programmable voltage control to scan the entire image. The first stage yields... according to Write the corresponding resistor states of the second-stage memristor array; and It is simultaneously sent to the second-stage peripheral circuit input.

[0033] In the second stage, the harmonic mean kernels of the red, green, and blue channels at the same spatial location are interconnected, and simultaneously applied... The output of the external operational amplifier is:

[0034] (14)

[0035] Based on the mapping between dark characteristics and the state of the subsequent resistor, equation (14) can be written as:

[0036] (15)

[0037] The analog voltage corresponding to the transmittance is:

[0038] (16)

[0039] The background light voltage, original pixel voltage, and transmittance voltage are combined in the second-stage peripheral circuit, and the output of the recovery circuit is:

[0040] (17)

[0041] exist Then, equation (17) becomes:

[0042] (18)

[0043] Equation (18) shows that the output of the recovery circuit differs from the algorithm recovery value of Equation (7) by a fixed ratio coefficient of 0.01. This fixed ratio is compensated during the final ADC conversion. The recovery voltages of the three color channels are converted by the ADC and combined in RGB order to obtain the final dehazed image.

[0044] Compared with existing technologies, this invention uses a harmonic mean window instead of traditional minimum pooling, which can suppress edge artifacts and preserve structural information; through photoelectric sensing and memristor memory array, two-stage parallel analog circuit and final ADC readout, it reduces external data transfer and processor workload; the harmonic mean, cross-channel features, transmittance and recovery relationship all have clear algorithm-circuit ratio mapping, which is suitable for execution in high-resolution sensors. Attached Figure Description

[0045] Figure 1 A complete workflow diagram of the two-stage parallel image dehazing framework within the sensor.

[0046] Figure 2 This is an integrated structure diagram of the photoelectric detection array, memristor storage array, and in-situ processing array within the sensor.

[0047] Figure 3 This is a connection structure diagram of the in-situ processing array unit within the sensor.

[0048] Figure 4 This is a structural diagram of the background light estimation circuit and the local harmonic mean kernel in the first stage.

[0049] Figure 5This is a structural diagram of the second-stage transmittance estimation and image restoration circuit.

[0050] Figure 6 Comparison of haze images and defogging results in different scenarios. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings. It should be understood that these descriptions are merely exemplary and not intended to limit the scope of the invention. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concept of the invention; moreover, the technical features involved in the different embodiments of the invention described below can be combined with each other as long as they do not conflict with each other. The following parameters and circuit nodes are used to illustrate the embodiments verified in the paper and do not constitute additional limitations on the scope of protection; different color channels use the same operational relationships.

[0052] like Figure 1 and Figure 2 As shown, the smog scene is first decomposed into red, green, and blue color channels. Each channel is sensed by a photodetector array, and the local light intensity is encoded into the analog resistance state in the memristor array. After photodetection is completed, the sensing branch and the array calculation branch are switched as shown in the figure, so that the written memristor state is used as the operand for the first stage harmonic mean calculation.

[0053] In the first stage, the array is configured with multiple independent locally harmonic mean kernels. (Example: ...) Thus, each kernel corresponds to a 5×5 local window; atmospheric haze images are captured. . When applied to the array's transverse lines, the column ends of each core are connected to peripheral operational amplifiers, simultaneously forming harmonic mean characteristics according to equations (9) to (13). Original pixels and background light The position of each core is moved by programmable voltage control, and the entire image is scanned sequentially.

[0054] After the first stage completes the output at the current position... according to The corresponding memristor states are stored as part of the second-stage array; and The signal is simultaneously sent to the second-stage peripheral circuit input. This inter-stage path operates in the fixed manner described above, eliminating the need to construct an external digital intermediate image between the two stages.

[0055] The second stage connects the harmonic mean kernels of the red, green, and blue channels at the same spatial location, and simultaneously applies them to the three color channel arrays. The array currents are summed according to equation (14) to form a value corresponding to the reciprocal of the harmonic mean across the channels. The peripheral circuit is obtained according to equation (16). and combine according to formula (17) , and The recovery voltage shown in equation (18) is generated.

[0056] Each analog output pulse exhibits a transient spike near its rising edge caused by the operational amplifier's step response, lasting approximately 1 μs. To avoid the influence of the step response, the output value is obtained from the stable region in the latter part of the pulse. The stable outputs of the three color channels are converted by an ADC, compensated by a fixed ratio of 0.01, and then merged in RGB order to form a dehazed image.

[0057] In a series of circuit simulations, the operational amplifier was modeled using a 180nm process with a supply voltage of 1.8V. Under ideal conditions, the output accuracy was approximately 99.61%, and the average power consumption was 19.1nJ / pixel. The output accuracy remained above 91.5% with 20% voltage source noise; approximately 90.13% with 20% memristor programming error; and above 94% with a line resistance of 2.5Ω. These results were used to verify the feasibility of the two-stage circuit under non-ideal conditions.

[0058] In the hardware configuration used for efficiency evaluation, 200 cross arrays are deployed in the first and second stages for each color channel, with each array consisting of 2000 memristors. When the number of image pixels exceeds the capacity of a single array, the image is cropped into segments matching the array capacity, and unprocessed segments are queued; after the current segment is processed, the next segment is mapped to the array. Under this configuration, the estimated processing speeds for 1920×1080, 3840×2160, and 7680×4320 images are 625.84, 178.33, and 45.06 frames per second, respectively.

[0059] The algorithm execution order in this implementation is as follows: For Perform lower limit protection; calculate the local harmonic mean of equation (2) in each of the three color channels and form the dark feature of equation (3); strictly follow the channel ratio relationship of equation (4) to form the background light estimate; extract the cross-channel normalized feature according to equation (5) and obtain the transmittance according to equation (6); restore the three color channels according to equation (7) and merge them according to equation (8). The hardware execution sequence is: sense and write the memristor state; first stage parallel calculation; put Write the state of the subsequent resistor and transfer it. , Second stage cross-channel calculation and recovery; stable interval value selection; ADC conversion and RGB merging; move the local kernel to the next position.

[0060] The device can be integrated into resource-constrained edge imaging systems such as mobile terminal cameras, smart surveillance cameras, consumer drone cameras, and vehicle-mounted cameras to perform dehazing preprocessing before high-resolution images enter an external processor.

Claims

1. An image dehazing device within a sensor, characterized in that, The system includes a photoelectric sensing and memristor storage array with three color channels (red, green, and blue), a first-stage circuit, a second-stage circuit, peripheral analog circuits, and an analog-to-digital conversion module. The photoelectric sensing and memristor storage array transmits the channel pixel intensity data for haze scenes. Mapped to the corresponding memristor state ,in , This is the scaling factor; The first-stage circuit uses programmable voltage control to form multiple isolated local harmonic mean kernels in the memristor array of each color channel. Each local harmonic mean kernel is centered on the target pixel and its window is [missing information]. ,according to Calculate the local harmonic mean, and according to Forming channel dark features, according to , , Forming background light estimates for red, green, and blue channels; in Under the mapping conditions, the first stage circuit output , and satisfies the corresponding background light estimate The analog voltage; the dark characteristic output of the first stage. The original pixel output is stored as the corresponding resistor state in the second-stage memristor array. and background light output It is sent to the second-stage peripheral circuit input; The second-stage circuit connects the red, green, and blue channel harmonic mean kernels at the same spatial location, according to... Obtain cross-channel normalized features, according to Obtain the transmittance and make the transmittance voltage satisfy Then according to A recovery channel is formed, so that the output of the external recovery circuit meets the requirements. The analog-to-digital conversion module converts the three color channels... The image is converted to digital intensity and merged in RGB order to form a dehazed image; the position of the local harmonic mean kernel is dynamically moved by programmable voltage control to scan the image.

2. A method for dehazing an image within a sensor using the device described in claim 1, characterized in that, include: S1 performs photoelectric sensing of the red, green, and blue channels of the haze scene, maps the pixel intensity of each channel to memristor states, and executes the input image. The lower limit protection; S2, forming local harmonic mean kernels in the three channels respectively, according to To obtain the local harmonic mean, according to Three-channel dark features are formed, and directly according to , , S3: Obtain the background light estimates of the three channels; S4: Write the dark feature output generated in the first stage into the resistance state in the second stage array, and send the original pixel output and background light output to the peripheral circuit of the second stage. S4, the harmonic mean kernel connecting three color channels at the same spatial location, according to and Obtain normalized characteristics and transmittance, and perform... S5, respectively according to Restore the red, green, and blue channels so that the analog output of the recovery circuit is the corresponding... 0.01 times; S6, obtain the output value from the middle and later stable range of the output pulse, and merge the three recovery channels into a dehazed image after analog-to-digital conversion; and S7, dynamically move the position of the local harmonic mean kernel through programmable voltage control, and execute S2 to S6 for the next image position.

3. An imaging device, characterized in that, The device includes an imaging optical component, a color channel separation component, an in-sensor image dehazing device as described in claim 1, and an image output interface; the imaging optical component and the color channel separation component transmit hazy scene light to a photoelectric sensing and memristor storage array for the red, green, and blue color channels; the in-sensor image dehazing device sequentially performs local harmonic mean estimation and background light estimation, cross-channel transmittance estimation, and channel analog output restoration; the image output interface outputs a dehazed image after analog-to-digital conversion and RGB merging; the imaging device is a mobile terminal camera, a smart surveillance camera, a consumer-grade drone camera, or a vehicle-mounted camera.