Method and system for expanding imaging dynamic range by using light scattering

Abstract

The invention provides an imaging dynamic range expansion method and system using light scattering, and the method comprises the steps: introducing a spatial light modulator and a scattering modulator in an imaging process, and dynamically adjusting the distribution of a light field entering a subsequent imaging system through adjusting the regulation and control mode of an internal device of the spatial light modulator; and a mask is generated, so that interference of strong background light is removed, information light of the target is reserved, and micron-level target light information rapid accumulation and background light suppression are realized. By introducing the scattering modulator, large-dynamic-range scene light in the imaging system can be dispersed, strong light energy can be diffused, overexposure of the detector is avoided, information originally exceeding the response range of the detector can enter a detectable range, the dynamic range of imaging detection is enlarged, and high-efficiency and high-performance large-dynamic-range imaging is achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of optical imaging, and in particular relates to a method and system for extending the dynamic range of imaging using light scattering. Background Art

[0002] In the past decade, the application of optoelectronic imaging systems has been restricted by the dynamic range of optoelectronic sensors. When using optoelectronic imaging devices such as cameras and optical sights to image scenes with bright objects or light sources in the field of view, the bright light will cause the intensity of the photoelectric sensor to be saturated, resulting in overexposure, which will seriously reduce the image quality. Overexposure of the bright area not only causes the loss of image details, but may also trigger a series of chain reactions, such as measurement errors, difficulty in target recognition, and even affect the stability and reliability of the entire system. Therefore, large dynamic range scene imaging has become a research hotspot. In the field of traffic vision, large dynamic range imaging technology can effectively cope with complex road conditions where strong light and shadows coexist, and improve the recognition accuracy and safety of autonomous driving systems. In the field of automated machinery manufacturing, due to the presence of glare, for detection equipment, it may cause the equipment to be unable to accurately capture targets or images, thereby affecting its detection accuracy and efficiency. At the same time, if it is in a glare environment for a long time, the visual system of the detection equipment may become fatigued, further reducing its working performance, which will bring about impacts including reduced work efficiency, reduced product quality, and increased safety hazards. In smart manufacturing and security monitoring, facing the ever-changing lighting conditions, wide dynamic range imaging technology can ensure the clarity and stability of real-time monitoring images, and ensure production and safety. In the field of aerospace, strong light sources such as searchlights produce extremely strong glare, making it difficult for technicians to see the instruments clearly. In low visibility conditions, excessive light intensity may even interfere with the technicians' line of sight, causing glare and affecting their judgment of the surrounding environment. Therefore, wide dynamic range imaging technology is of great significance for detecting and analyzing complex space environments, ground targets, etc. In computational photography, this technology can help photographers shoot works with more artistic expression and visual impact in high-contrast environments. At present, the types of technologies for large dynamic range imaging include: intensity attenuation imaging technology, large dynamic range imaging technology, event camera imaging technology, etc.

[0003] Intensity attenuation imaging technology uses optical materials such as attenuation sheets to effectively adjust the intensity of light entering the imaging system. This technology can evenly reduce all wavelengths of light, thereby avoiding image saturation and loss of details in scenes with a large dynamic range.

[0004] High dynamic range imaging technology (HDR) captures and synthesizes a series of images taken at different exposure levels to retain details in both low-brightness and high-brightness areas in one image, thereby obtaining a wider dynamic range than a single exposure image. This technology is very effective in scenes that require capturing details in both dark and bright areas at the same time, such as shooting in high-contrast environments, and can significantly improve the image's detail expression and visual effects. Its core advantage lies in the ability to achieve accurate synthesis and processing of multiple images through the optimization of software algorithms, providing a more realistic visual experience.

[0005] Event camera imaging technology only outputs data when it detects that the brightness change exceeds the preset threshold. This working mechanism makes the event camera have the advantages of extremely low latency, high dynamic range, and low power consumption. These characteristics of event cameras make them particularly outstanding in application scenarios with high real-time requirements and drastic lighting changes, and are suitable for dynamic scene monitoring and rapid response imaging tasks.

[0006] However, the above technology has certain defects:

[0007] Intensity attenuation imaging technology uses optical materials such as attenuation sheets to reduce the light intensity entering the imaging system. While suppressing background light interference, it may also reduce the contrast of the imaging target, making it difficult to effectively detect the target's detailed information. Although this technology can solve the imaging problem in high-brightness environments to a certain extent, its impact on the overall image quality cannot be ignored, especially in scenes where contrast is critical. This defect is particularly obvious.

[0008] HDR technology requires the detector to collect multiple frames of data. If the detector shakes or the target moves during this process, the data synthesized using this imaging technology will produce motion blur or ghosting. At the same time, large dynamic range imaging technology only better integrates existing information and cannot display information that is not in the image used for synthesis. This limitation makes it impossible to use it in dynamic scenes or scenes that require high-precision static details. Although these problems can be alleviated to a certain extent through algorithm optimization, its basic limitations still exist.

[0009] Event camera imaging technology is a bionic sensor (brain-like) with a microsecond response time that can record an asynchronous stream of brightness changes per pixel, called an "event", and generates an event by detecting the brightness change of each pixel. However, event cameras can only output moving pixels, which is not applicable to traditional visual methods. At the same time, event cameras only output "positive and negative" signals, without the intensity information in traditional optical imaging, which brings great difficulties to the subsequent identification and confirmation of key parts of the target. For example, when performing tasks such as object recognition and feature extraction, traditional cameras rely on rich grayscale or color information to determine the shape, texture and edges of objects, while event cameras can only output event signals of brightness changes, making these tasks more complex and difficult.

[0010] Although these technologies have made some progress in improving the adaptability of optoelectronic imaging systems to scenes with a large dynamic range, they still need to be further optimized and improved in terms of image quality, real-time performance, system cost, environmental adaptability, and application scope. Therefore, studying an imaging technology that extends the dynamic range has important application value and prospects. Summary of the invention

[0011] In order to solve the above problems existing in the prior art, the present invention provides a method and system for extending the dynamic range of imaging using light scattering. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0012] In a first aspect, the present invention provides an imaging dynamic range extension method using light scattering, which is applied to an imaging dynamic range extension system using light scattering, the system comprising a spatial light modulator, a spatial filter, a scattering modulator and a detector, and the imaging dynamic range extension method using light scattering comprises:

[0013] S100, regulating a spatial light modulator so that an incident light field entering the spatial light modulator is reflected into the spatial filter; the incident light field is light reflected by all objects in a large dynamic range scene;

[0014] S200, filtering the incident light field fed back by the spatial light modulator using a spatial filter to obtain a filtered light field;

[0015] S300, using a scattering modulator to diffuse the filtered light field to obtain a first speckle pattern, and then using a detector to detect target contour information in the first speckle pattern to obtain a target mask;

[0016] S400, regulating the spatial light modulator based on the target mask so that a portion of the light field containing the target information in the incident light entering the spatial light modulator is reflected to the spatial filter;

[0017] S500, filtering a portion of the light field fed back by the spatial light modulator using a spatial filter to obtain a filtered portion of the light field;

[0018] S600, using a scattering modulator to diffuse the filtered part of the light field to obtain a second speckle pattern, and then using a detector to detect target information in the second speckle pattern to obtain target detail information;

[0019] S700: Combining the target mask and the target detail information into an original target image.

[0020] In a second aspect, the present invention provides an imaging dynamic range extension system using light scattering, wherein the imaging dynamic range extension system using light scattering performs imaging using the imaging dynamic range extension method using light scattering described in the first aspect.

[0021] Beneficial effects:

[0022] 1. The imaging dynamic range expansion method using light scattering proposed in the present invention does not introduce complicated optical components during application and has a simple process.

[0023] 2. The present invention introduces a scattering modulator to disperse the strong light energy, and then based on the scattering calculation imaging theory, it can not only restore and reconstruct the target under a large dynamic range, improve the imaging quality and the dynamic range of detection, but also avoid damage to the detector target surface.

[0024] 3. The spatial light modulator introduced in the present invention can select the target in real time in a large dynamic range scene, which can improve the real-time and accuracy of imaging.

[0025] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a schematic diagram of the application of the imaging dynamic range expansion method using light scattering provided by the present invention;

[0027] Figure 2 It is a flow chart of a method for extending the dynamic range of imaging using light scattering provided by the present invention;

[0028] Figure 3 It is a schematic diagram of the process of the imaging dynamic range expansion method using light scattering provided by the present invention. DETAILED DESCRIPTION

[0029] The present invention is further described in detail below with reference to specific embodiments, but the embodiments of the present invention are not limited thereto.

[0030] The present invention provides an imaging dynamic range expansion method using light scattering, which is applied to an imaging dynamic range expansion system using light scattering, referring to Figure 1 As shown, the system includes a spatial light modulator, a spatial filter, a scattering modulator and a detector. The scattering modulator is used to diffuse strong light so that information that originally exceeds the detector response range can enter the detectable range. Any equipment that diffuses scenes with a large dynamic range can replace the scattering modulator. The spatial light modulator is used to select targets in scenes with a large dynamic range. Any device that can select targets can replace the spatial light modulator.

[0031] exist Figure 1 In the figure, 1 is a strong light source scene with dim target information; 2 is a spatial light modulator, whose main function is to regulate the scene with a large dynamic range. 3 is a spatial filter, whose main function is to filter the light field after the spatial light modulator regulates it. 4 is an optical system, which can be appropriately added according to the actual imaging conditions. 5 is a scattering modulator, which can diffuse the light field of the scene with a large dynamic range, thereby avoiding overexposure of the detector and damage to the detector target surface. 6 is a detector, which is used to collect the diffused light field.

[0032] Combination Figure 2 and Figure 3 The present invention provides a method for extending the dynamic range of imaging using light scattering, comprising:

[0033] S100, regulating a spatial light modulator so that an incident light field entering the spatial light modulator is reflected into the spatial filter; the incident light field is light reflected by all objects in a large dynamic range scene;

[0034] The present invention uses a high-precision spatial light modulator to select the target imaging area. By accurately adjusting the angle and position of the spatial light modulator, the spatial light modulator can dynamically adjust the distribution of light entering the imaging system, effectively suppress the interference of background light, and achieve rapid accumulation of target light information and background light suppression at the micron level.

[0035] S200, filtering the incident light field fed back by the spatial light modulator using a spatial filter to obtain a filtered light field;

[0036] S300, using a scattering modulator to diffuse the filtered light field to obtain a first speckle pattern, and then using a detector to detect target contour information in the first speckle pattern to obtain a target mask;

[0037] The target contour generates a corresponding target mask on the scatter modulator, thereby avoiding the interference of strong background light information in a large dynamic range scene, making the strong light background attenuated and the target area enhanced.

[0038] The introduction of the scattering modulator diffuses the large dynamic range scene, allowing information that originally exceeded the detector response range to enter the detectable range, thereby increasing the dynamic range of the detector and achieving large dynamic range imaging.

[0039] In an optional implementation manner of the present invention, S300 includes:

[0040] S310, using the scattering modulator to diffuse the energy of the filtered light field to obtain a first speckle pattern;

[0041] S320, using the detector to detect target contour information in the first speckle pattern, and obtain a pixel value of each pixel point in the first speckle pattern;

[0042] S330, according to the pixel value of each pixel point in the first speckle image, restore the target mask through a target reconstruction algorithm. The target reconstruction algorithm includes a speckle restoration algorithm, the speckle restoration algorithm uses a deconvolution algorithm to restore the image, and the deconvolution algorithm includes a Wiener filter algorithm, a regularized deconvolution algorithm, a blind deconvolution algorithm, etc.

[0043] Specifically, when a beam of incoherent light is irradiated onto the target, the object light passing through the target is incident on the scattering modulator. The light emitted from the scattering modulator is scattered, so speckle I can be detected on the camera. According to the optical memory effect of speckle, when the object is small, the light emitted from two points on the object passes through the frosted glass to form two speckles. The two speckles are translationally invariant and have a high correlation. Approximately, the two speckles are almost the same, but have undergone a slight displacement. Therefore, the speckle emitted by the entire object can be regarded as a simple superposition of the speckle emitted by each point on the object. The speckle emitted by each point on the object can be regarded as the point spread function (PSF) of this imaging system.

[0044] Therefore, the speckle I detected by the detector is approximately equal to the convolution of the object's intensity distribution O and the system's PSF:

[0045] I=O*PSF

[0046] Where “*” indicates convolution operation.

[0047] If the PSF of the system has been obtained in advance, based on this, the intensity distribution image O of the object can be restored from the speckle I by a deconvolution algorithm. The restoration process of the speckle restoration algorithm is expressed by the formula:

[0048] O≈deconv(I,PSF)

[0049] Where O is the recovered target mask, deconν(·) represents the deconvolution operation, I is the first speckle pattern, and PSF is the point spread function.

[0050] S400, regulating the spatial light modulator based on the target mask so that a portion of the incident light field entering the spatial light modulator containing the target information is reflected to the spatial filter;

[0051] S500, filtering a portion of the light field fed back by the spatial light modulator using a spatial filter to obtain a filtered portion of the light field;

[0052] S600, using a scattering modulator to diffuse the filtered part of the light field to obtain a second speckle pattern, and then using a detector to detect target information in the second speckle pattern to obtain target detail information;

[0053] In an optional implementation manner of the present invention, S600 includes:

[0054] S610, using the scattering modulator to diffuse the filtered part of the light field to obtain a second speckle pattern;

[0055] S620, detecting the second speckle pattern using the detector to obtain a pixel value of each pixel in the second speckle pattern;

[0056] S630: Based on the pixel value of each pixel point in the second speckle image, target detail information is restored by using a target reconstruction algorithm.

[0057] The principle of S600 and S300 to restore the original target image by using the target reconstruction algorithm is the same. The target reconstruction algorithm includes a speckle restoration algorithm. The speckle restoration algorithm uses a deconvolution algorithm to restore the image. The deconvolution algorithm includes a Wiener filter algorithm, a regularized deconvolution algorithm, a blind deconvolution algorithm, etc. The restoration process of the speckle restoration algorithm is expressed by a formula:

[0058] O≈deconv(I,PSF)

[0059] Where O is the restored original target image, deconv(·) represents the deconvolution operation, I is the second speckle pattern, and PSF is the point spread function.

[0060] It is worth noting that the target mask is restored in S300, and the target mask reflects the surrounding contour of the target. In S600, the target contour information is combined and the details within the target contour are reconstructed using a target reconstruction algorithm.

[0061] S700: Combining the target mask and the target detail information into an original target image.

[0062] In this step, the target mask and the target detail information are simply combined to obtain the original target image.

[0063] In an optional embodiment of the present invention, the imaging dynamic range extension method using light scattering further includes:

[0064] refer to Figure 2 According to actual imaging conditions, an optical system is arranged between the scattering modulator and the spatial filter, and the optical system is arranged to allow the filtered light field to be incident on the scattering modulator in its entirety along an optical path.

[0065] The actual imaging conditions include: the distance between the scattering modulator and the spatial filter, the width of the filtered light field output by the spatial filter, and whether re-filtering is required.

[0066] It is worth noting that if the distance between the scatter modulator and the spatial filter is far, that is, exceeds the predetermined distance, the filtered light field enters the scatter modulator less, so it is necessary to increase the optical system. If the width of the filtered light field output by the spatial filter is wide, that is, the same filtered light field enters the scatter modulator less, it is necessary to increase the optical system. The purpose is to allow the filtered light field to be incident on the scatter modulator in its entirety according to the optical path. If filtering is required again, an optical system can be added to perform secondary filtering on the filtered light field.

[0067] The present invention uses a light scattering imaging dynamic range extension system to select target contours and restore target information in a large dynamic range scene. The core is to generate a mask at the corresponding position of the spatial light modulator according to the target contour restored in a large dynamic range scene to avoid interference from strong background light and leave information light containing the target. The introduction of the scattering modulator is mainly to disperse the energy of the strong light field to avoid overexposure of the detector.

[0068] In a second aspect, the present invention provides an imaging dynamic range extension system using light scattering, which performs imaging using the imaging dynamic range extension method using light scattering described in the first aspect.

[0069] In a specific embodiment of the present invention, the imaging dynamic range extension system using light scattering further includes:

[0070] An optical system is arranged between the scattering modulator and the spatial filter according to actual imaging conditions, and is used to allow the filtered light field to be incident on the scattering modulator in its entirety along an optical path.

[0071] The actual imaging conditions include: the distance between the scattering modulator and the spatial filter, the width of the filtered light field output by the spatial filter, and whether re-filtering is required.

[0072] The present invention proposes a method and system for extending the dynamic range of imaging using light scattering. By introducing a spatial light modulator and a scattering modulator during the imaging process, and by precisely adjusting the angle and position of the spatial light modulator, the spatial light modulator can dynamically adjust the distribution of light entering the spatial filter, generate a mask to avoid interference from strong background light, and leave information light containing the target, which can effectively suppress interference from background light, and achieve rapid accumulation of target light information and background light suppression at the micron level. The introduction of a scattering modulator can diffuse a large dynamic range scene, disperse the energy of a strong light field, avoid overexposure of the detector, and allow information that originally exceeded the detector's response range to enter the detectable range, thereby increasing the dynamic range of the detector and achieving large dynamic range imaging. Therefore, the imaging efficiency and imaging effect can be improved.

[0073] It is worth noting that the terms "first" and "second" in the present invention are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0074] Although the present application is described herein in conjunction with various embodiments, in the process of implementing the claimed application, those skilled in the art may understand and implement other variations of the disclosed embodiments by reviewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality of components or steps.

[0075] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A method for extending the dynamic range of imaging using light scattering, characterized in that: The invention is applied to an imaging dynamic range extension system using light scattering, the system comprising a spatial light modulator, a spatial filter, a scattering modulator and a detector, and the imaging dynamic range extension method using light scattering comprises: S100, regulating a spatial light modulator so that an incident light field entering the spatial light modulator is reflected into the spatial filter; the incident light field is light reflected by all objects in a large dynamic range scene; S200, filtering the incident light field fed back by the spatial light modulator using a spatial filter to obtain a filtered light field; S300, using a scattering modulator to diffuse the filtered light field to obtain a first speckle pattern, and then using a detector to detect target contour information in the first speckle pattern to obtain a target mask; S400, regulating the spatial light modulator based on the target mask so that a portion of the light field containing the target information in the incident light entering the spatial light modulator is reflected to the spatial filter; S500, filtering a portion of the light field fed back by the spatial light modulator using a spatial filter to obtain a filtered portion of the light field; S600, using a scattering modulator to diffuse the filtered part of the light field to obtain a second speckle pattern, and then using a detector to detect target information in the second speckle pattern to obtain target detail information; S700: Combining the target mask and the target detail information into an original target image.

2. The imaging dynamic range extension method using light scattering according to claim 1, characterized in that: The imaging dynamic range expansion method using light scattering also includes: An optical system is arranged between the scatter modulator and the spatial filter according to actual imaging conditions, and the optical system is arranged to allow the filtered light field to be incident on the scatter modulator in its entirety along an optical path.

3. The imaging dynamic range extension method using light scattering according to claim 2, characterized in that: The actual imaging conditions include: the distance between the scattering modulator and the spatial filter, the width of the filtered light field output by the spatial filter, and whether re-filtering is required.

4. The imaging dynamic range extension method using light scattering according to claim 1, characterized in that: S300 includes: S310, using the scattering modulator to diffuse the energy of the filtered light field to obtain a first speckle pattern; S320, using the detector to detect target contour information in the first speckle pattern, and obtain a pixel value of each pixel point in the first speckle pattern; S330: restore a target mask according to the pixel value of each pixel point in the first speckle image and by using a target reconstruction algorithm.

5. The imaging dynamic range extension method using light scattering according to claim 1, characterized in that: S600 includes: S610, using the scattering modulator to diffuse the filtered part of the light field to obtain a second speckle pattern; S620, detecting the second speckle pattern using the detector to obtain a pixel value of each pixel in the second speckle pattern; S630: Based on the pixel value of each pixel point in the second speckle image, target detail information is restored by using a target reconstruction algorithm.

6. The imaging dynamic range extension method using light scattering according to claim 4 or 5, characterized in that: The target reconstruction algorithm includes a speckle restoration algorithm. The speckle restoration algorithm uses a deconvolution algorithm to restore an image. The restoration process of the speckle restoration algorithm is expressed as follows: O≈deconv(I,PSF) Where O is the restored target mask or the original target image, deconν(·) represents the deconvolution operation, I is the first speckle pattern or the second speckle pattern, and PSF is the point spread function.

7. The imaging dynamic range extension method using light scattering according to claim 6, characterized in that: The deconvolution algorithms include Wiener filtering algorithm, regularized deconvolution algorithm and blind deconvolution algorithm.

8. An imaging dynamic range extension system using light scattering, characterized in that: The imaging dynamic range extension system using light scattering performs imaging using the imaging dynamic range extension method using light scattering as described in any one of claims 1 to 7.

9. The imaging dynamic range extension system using light scattering according to claim 8, characterized in that: The imaging dynamic range extension system using light scattering also includes: An optical system is arranged between the scattering modulator and the spatial filter according to actual imaging conditions, and is used to allow the filtered light field to be incident on the scattering modulator in its entirety along an optical path.

10. The imaging dynamic range extension system using light scattering according to claim 9, characterized in that: The actual imaging conditions include: the distance between the scattering modulator and the spatial filter, the width of the filtered light field output by the spatial filter, and whether re-filtering is required.

Images