Scatter imaging system and scatter imaging method

Abstract

This invention discloses a scattering imaging system and a scattering imaging method. The scattering imaging system includes: a light source module for generating dynamic speckle patterns illuminating various target objects in a scattering medium; and an imaging module for acquiring a mixed speckle pattern formed after the target objects are illuminated by the dynamic speckle. After acquiring the mixed speckle pattern containing the target objects, fingerprint speckle patterns of each target object are extracted. The fingerprint speckle patterns are then scaled, and the relative depths between the target objects are determined based on the scaling ratio. A deconvolution algorithm is used to obtain the lateral relative positional relationships between the target objects, and finally, a three-dimensional image with a large field of view is obtained.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of optical imaging technology, and particularly relates to a scattering imaging system and a scattering imaging method. BACKGROUND

[0002] In many scenarios, the target to be imaged is in a non-uniform medium, and the traditional visible light imaging cannot observe the target. Such non-uniform medium is usually complex, such as rain and snow weather, fog and haze weather, turbid water, and the like. In such medium, the propagation of light no longer follows the basic principle of "propagating along a straight line", and the propagation direction of photons will be disturbed by the non-uniform medium and changed, thereby becoming randomized. This phenomenon is called "scattering", and such non-uniform medium is called "scattering medium". When an object is in a scattering medium, the traditional scattering imaging method cannot obtain the information of the hidden target. Therefore, how to overcome the scattering effect and obtain the information of the hidden target through the scattering medium has important scientific research significance and value.

[0003] There are various scattering imaging technologies in the prior art that can overcome the scattering effect and obtain the information of the hidden target. One of the technologies is speckle correlation technology, which has attracted widespread attention of researchers due to its simple system and the ability to realize single-frame speckle imaging. However, the conventional two-dimensional speckle correlation imaging method only focuses on the planar shape of the hidden target, and cannot obtain the depth information of the hidden target, so it cannot obtain a three-dimensional image. SUMMARY

[0004] The technical problem solved by the present application is how to obtain the depth information of the target in the scattering medium to construct a three-dimensional image.

[0005] The present application discloses a scattering imaging system, which comprises:

[0006] a light source module for generating dynamic speckles for illuminating each target object in the scattering medium;

[0007] an imaging module for obtaining a mixed speckle pattern formed after the each target object is illuminated by the dynamic speckles.

[0008] Preferably, the light source module comprises a laser and a spatial light modulator, and the spatial light modulator is used to convert the laser beam generated by the laser into dynamic speckles.

[0009] Preferably, the imaging module comprises a microscope objective, a filter, a tube lens, and a camera, and the light beam generated after the target object is illuminated passes through the microscope objective, the filter, and the tube lens in sequence and then reaches the camera.

[0010] Preferably, the imaging system further comprises a dichroic mirror between the microscope objective and the filter, the dichroic mirror is used to reflect the dynamic speckle to the target object in the scattering medium, and the dichroic mirror is used to transmit the light beam generated after the target object is illuminated.

[0011] The application also discloses a scattering imaging method, which comprises the following steps:

[0012] Fingerprint speckle patterns of each target object are extracted according to a plurality of mixed speckle images obtained in advance, and each mixed speckle image contains speckle patterns of a plurality of target objects in the scattering medium.

[0013] A fingerprint speckle pattern of one of the target objects is selected as a calibration pattern, and the fingerprint speckle patterns of the remaining target objects are scaled so that the correlation between each scaled fingerprint speckle pattern and the calibration pattern satisfies a predetermined condition.

[0014] The calibration pattern and each scaled fingerprint speckle pattern are deconvoluted to obtain a transverse relative position relationship image of the remaining target objects relative to the target object serving as the calibration, and the axial relative position of the remaining target objects relative to the target object serving as the calibration is determined according to the scaling ratio of each scaled fingerprint speckle pattern.

[0015] The transverse relative position relationship images are combined according to the axial relative positions to obtain a three-dimensional image.

[0016] Preferably, the scattering imaging method further comprises: obtaining a plurality of mixed speckle images by using the scattering imaging system.

[0017] Preferably, the method for extracting the fingerprint speckle pattern of each target object according to a plurality of mixed speckle images obtained in advance comprises the following steps:

[0018] The non-speckle pattern region in each mixed speckle image is cropped to retain the speckle pattern region.

[0019] After the cropped mixed speckle images are filtered and combined, the fingerprint speckle pattern of each target object is obtained by using an eigenmatrix extraction algorithm.

[0020] Preferably, the predetermined condition is that the correlation between the scaled fingerprint speckle pattern and the calibration pattern is maximum.

[0021] The scattering imaging system and the scattering imaging method disclosed by the application have the following technical effects:

[0022] After obtaining the mixed speckle pattern containing each target object, the fingerprint speckle pattern of each target object is extracted, the fingerprint speckle pattern is scaled, the relative depth before each target object is determined according to the scaling ratio, and the transverse relative position relationship between each target object is obtained by combining the deconvolution algorithm, and finally a three-dimensional image in a large field of view is further obtained. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 A schematic diagram of a scattering imaging system of an embodiment of the present application;

[0024] Figure 2 A flowchart of a scattering imaging system in an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical scheme and advantages of the present application clearer, further detailed description will be made below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0026] Before describing the various embodiments of the present application in detail, first briefly describe the technical concept of the present application: the existing speckle correlation technology can usually only obtain the two-dimensional planar shape of the hidden target, and cannot obtain the depth information. Therefore, the present application provides a scattering imaging system and a scattering imaging method, which generates dynamic speckle by a light source module to illuminate each target object in a scattering medium, and obtains a mixed speckle pattern of each target object by an imaging module. Then the fingerprint speckle pattern of each target object is extracted from the mixed speckle pattern, the fingerprint speckle pattern is scaled, the relative depth before each target object is determined according to the scaling ratio, and the transverse relative position relationship between each target object is obtained by combining the deconvolution algorithm, and finally a three-dimensional image is further obtained.

[0027] Specifically, as shown in Figure 1 The scattering imaging system of the present embodiment includes a light source module and an imaging module, the light source module is used to generate dynamic speckle to illuminate each target object in a scattering medium, and the imaging module is used to obtain a mixed speckle pattern formed after each target object is illuminated by dynamic speckle.

[0028] Exemplarily, the light source module includes a laser 10 and a spatial light modulator 20, the spatial light modulator 20 is used to convert the laser beam generated by the laser 10 into dynamic speckle. In other embodiments, the light source module can also be composed of other components to generate dynamic speckle.

[0029] Further, the imaging module comprises a microscope objective 30, a filter 40, a tube lens 50 and a camera 60. The light beams generated after the target objects are illuminated pass through the microscope objective 30, the filter 40 and the tube lens 50 in sequence and then reach the camera 60, forming a mixed speckle pattern. Exemplarily, the speckle generated by the light emitted by a single target object passing through the scattering medium is defined as the fingerprint speckle of the target object. When a certain object is illuminated using incoherent light, at a certain moment, the image captured by the camera is the incoherent sum of the fingerprint speckles of multiple target objects, i.e., the mixed speckle pattern obtained by superimposing the fingerprint speckle patterns of multiple target objects.

[0030] For example, assuming that the object to be imaged is a fluorescent object in the scattering medium M1, when the dynamic speckle illumination generated by the spatial light modulator 20 reaches the fluorescent object, multiple fluorescent particles in the fluorescent object (i.e., the multiple target objects in the first embodiment) will excite to generate fluorescent light beams. The fluorescent light beams pass through the microscope objective 30, the filter 40 and the tube lens 50 in sequence and finally form a mixed speckle image on the camera 60, which is formed by superimposing multiple fluorescent light beams.

[0031] Further, the imaging system further comprises a dichroic mirror 70, which is located between the microscope objective 30 and the filter 40. The dichroic mirror 70 is used to reflect the dynamic speckle to the target objects in the scattering medium, and the dichroic mirror 70 is used to transmit the light beams generated after the target objects are illuminated.

[0032] As shown in FIG. 2, the second embodiment discloses a scattering imaging method, which comprises the following steps: Figure 2

[0033] Step S10: Extracting the fingerprint speckle pattern of each target object according to the multiple frames of mixed speckle images obtained in advance. Each frame of mixed speckle image contains the speckle patterns of multiple target objects in the scattering medium.

[0034] Step S20: Selecting the fingerprint speckle pattern of one of the target objects as a calibration pattern, and performing scaling processing on the fingerprint speckle patterns of the remaining target objects, so that the correlation between each scaled fingerprint speckle pattern and the calibration pattern satisfies a predetermined condition.

[0035] Step S30: Performing deconvolution processing on each scaled fingerprint speckle pattern and the calibration pattern to obtain a lateral relative position relationship image of the remaining target objects relative to the target object serving as the calibration, and determining the axial relative position of the remaining target objects relative to the target object serving as the calibration according to the scaling ratio of each scaled fingerprint speckle pattern.

[0036] Step S40: Merging the lateral relative position relationship images according to the axial relative positions to obtain a three-dimensional image. ​

[0037] Specifically, the mixed speckle images containing speckle patterns of multiple target objects in the scattering medium are obtained by using the scattering imaging system in Embodiment One, and the number of mixed speckle images can be selected according to actual needs. In this embodiment, thousands of mixed speckle images are collected.

[0038] Further, in step S10, since each frame of mixed speckle image contains a speckle pattern region and a non-speckle pattern region, in order to reduce the calculation time, the non-speckle pattern region is cropped and the speckle pattern region is retained. Then, after filtering and merging processing of each frame of mixed speckle image after cropping, the fingerprint speckle pattern of each target object is obtained by using the eigenmatrix extraction algorithm. Exemplarily, each frame of mixed speckle image after cropping is subjected to high-pass filtering processing to improve the contrast of the speckle pattern.

[0039] Further, according to the lateral optical memory effect, the speckle patterns generated by two target objects within the memory effect range are highly correlated, and the spatial offset depends on the relative position between the two scattering targets. According to the axial optical memory effect, the speckle patterns formed by point light sources in different spatial planes are highly correlated after scaling processing. Therefore, in step S20, one of the multiple target objects is selected as a calibration object, and the corresponding fingerprint speckle pattern is taken as a calibration pattern. The scaling ratio of each of the remaining fingerprint speckle patterns is adjusted for scaling processing, and after each adjustment of the scaling ratio, the correlation between the scaled fingerprint speckle pattern and the calibration pattern is calculated until the correlation meets a predetermined condition. Exemplarily, the predetermined condition is that the correlation between the scaled fingerprint speckle pattern and the calibration pattern is maximum. When the predetermined condition is met, the scaling is stopped, and the final scaled fingerprint speckle pattern is obtained.

[0040] Further, the deconvolution processing is performed on each of the scaled fingerprint speckle patterns and the calibration pattern to obtain the lateral relative position relationship image of each of the remaining target objects relative to the calibration object, and the axial relative position of each of the remaining target objects relative to the calibration object is determined according to the scaling ratio of each of the scaled fingerprint speckle patterns. It can be concluded that there is a corresponding relationship between the scaling ratio and the relative depth, and when the scaling ratio of each fingerprint speckle pattern is determined, the axial relative position of each of the remaining target objects relative to the calibration object can be determined.

[0041] Finally, in step S40, according to the axial relative position of each of the two target objects, the relative position relationship image representing the lateral relative position of each of the two target objects is merged to obtain a three-dimensional image.

[0042] In the embodiment two, the relative position relationship between the two scattering targets beyond the range of the optical memory effect is determined by the traditional method. The position of the remaining speckles relative to the calibration speckle is determined. Since the position of the calibration speckle is determined, the position between any two remaining speckles is also determined. That is, the relative position between the two scattering targets beyond the range of the optical memory effect is determined by the method of the embodiment two, thereby increasing the image field of view.

[0043] The specific embodiments of the present application are described above in detail, although some embodiments have been shown and described, it should be understood by those skilled in the art that modifications and improvements can be made to these embodiments without departing from the principles and spirit of the present application, which are defined by the claims and their equivalents, and these modifications and improvements should also be within the scope of protection of the present application.

Claims

1. A method of scatter imaging, characterized by, The scattering imaging method comprises: According to the pre-acquired multiple frames of mixed speckle images, the fingerprint speckle patterns of each target object are extracted, and each frame of mixed speckle image contains the speckle patterns of multiple target objects in the scattering medium, which comprises: cutting the non-speckle pattern area in each frame of mixed speckle image, and retaining the speckle pattern area; after filtering and merging processing of each frame of mixed speckle image after cutting, the fingerprint speckle pattern of each target object is obtained by using the intrinsic matrix extraction algorithm; The fingerprint speckle pattern of one of the target objects is selected as a calibration pattern, and the fingerprint speckle patterns of the remaining target objects are scaled to make the correlation between each scaled fingerprint speckle pattern and the calibration pattern meet a predetermined condition, and the predetermined condition is that the correlation between the scaled fingerprint speckle pattern and the calibration pattern is the largest; The deconvolution processing is performed on each scaled fingerprint speckle pattern and the calibration pattern to obtain the transverse relative position relationship image of the remaining target objects relative to the target object as the calibration, and the axial relative position of the remaining target objects relative to the target object as the calibration is determined according to the scaling ratio of each scaled fingerprint speckle pattern; According to the axial relative positions, the transverse relative position relationship images are merged to obtain a three-dimensional image.

2. The scatter imaging method of claim 1, wherein, The scattering imaging method further comprises: obtaining multiple frames of mixed speckle images by using a scattering imaging system, and the imaging system comprises: A light source module for generating dynamic speckles for illuminating each target object in the scattering medium; An imaging module for acquiring mixed speckle patterns formed by the target objects being illuminated by the dynamic speckles.

3. The scatter imaging method of claim 2, wherein, The light source module comprises a laser and a spatial light modulator, and the spatial light modulator is used to convert the laser beam generated by the laser into dynamic speckles.

4. The scatter imaging method of claim 2, wherein, The imaging module comprises a microscope objective, a filter, a tube lens and a camera, and the light beam generated after the target object is illuminated passes through the microscope objective, the filter and the tube lens in sequence and then reaches the camera.

5. The scatter imaging method of claim 4, wherein, The imaging system further comprises a dichroic mirror, which is located between the microscope objective and the filter, and is used to reflect the dynamic speckles to the target objects in the scattering medium, and is used to transmit the light beam generated after the target objects are illuminated.

Images