Scattering imaging method, device and computer readable storage medium

Abstract

The present application provides a scattering imaging method, device and computer-readable storage medium. The scattering imaging method includes: obtaining a speckle pattern; generating an aliased autocorrelation image according to the speckle pattern; inputting the aliased autocorrelation image into a target convolutional neural network, separating the autocorrelation images of each object from the aliased autocorrelation image through the target convolutional neural network; reconstructing all object autocorrelation images respectively according to a phase recovery algorithm to obtain object images of all target objects; wherein the speckle pattern is formed on the image plane by incident light carrying object information of multiple target objects through a scattering medium, the sizes of all target objects are within the effective range of optical memory, and the distance between any two target objects exceeds the effective range of optical memory; the aliased autocorrelation image is the superposition of the object autocorrelation images of all target objects. The present application is not easily limited by the effective range of the optical memory effect, and can achieve scattering imaging of multiple target objects and a large field of view.

Description

[Technical field]

[0001] The present application relates to the field of optical imaging technology, and in particular to a scattering imaging method, device and computer-readable storage medium. [Background technology]

[0002] The optical memory effect is an inherent property of scattering media. There are usually two scattering imaging methods based on the optical memory effect, namely deconvolution imaging technology and speckle autocorrelation imaging technology. Both scattering imaging technologies have very broad development prospects and have been widely used in various occasions of imaging through scattering media.

[0003] In the related art, the effective range of the optical memory effect will have a great impact on the field of view of the scattered imaging, that is, when imaging through a strong scattering medium, the effective range of the optical memory effect will become very small, thereby limiting the application of the above two technologies in actual scattered imaging scenarios. In order to expand the field of view of scattering imaging, many researchers have proposed different solutions, which can be roughly divided into: the first category, using different pre-calibrated point spread functions to obtain multiple images in the horizontal and vertical directions, and then obtaining a large field of view by stitching multi-lens multi-view images; the second category, using direct or indirect additional prior information to extract the information of hidden objects from the original speckle pattern recorded by the camera; the third category, using high-order correlation to establish a complex nonlinear relationship between the input field and the output field; among them, the first category of solutions requires precise calibration, otherwise it will seriously affect the quality of the reconstructed image; the prior information in the second category of solutions can improve the quality of large field of view imaging to a certain extent, but it cannot solve the problem of speckle aliasing (when multiple target objects are imaged through scattering media, the speckle pattern formed on the image plane does not include only one target object, but includes multiple target objects, which is speckle aliasing); the third category of solutions mainly modulates light waves in the scattering area, and the measurement of its transmission matrix is ​​usually complicated and time-consuming. It can be seen that how to break through the limitation of the effective range of the optical memory effect to achieve multi-target object and large field of view imaging through scattering media is still an important research topic in the field of scattering imaging.

[0004] Therefore, it is necessary to improve the above scattering imaging method. [Summary of the invention]

[0005] The present application provides a scattering imaging method, device and computer-readable storage medium, aiming to solve the problem in the related art that when imaging through a scattering medium, the effective range of the optical memory effect is severely limited and imaging of multiple targets and a large field of view cannot be achieved.

[0006] In order to solve the above technical problems, the first aspect of the embodiment of the present application provides a scattering imaging method, including:

[0007] Acquire a speckle pattern; wherein the speckle pattern is formed on an image plane by incident light carrying object information of a plurality of target objects passing through a scattering medium, the sizes of all the target objects are within an effective range of optical memory, and the distance between any two of the target objects exceeds the effective range of optical memory;

[0008] Generate an aliased autocorrelation image according to the speckle pattern; wherein the aliased autocorrelation image is a superposition of object autocorrelation images of all the target objects;

[0009] Inputting the aliased autocorrelation image into a target convolutional neural network, and separating each of the object autocorrelation images from the aliased autocorrelation image through the target convolutional neural network;

[0010] All the object autocorrelation images are reconstructed respectively according to the phase recovery algorithm to obtain object images of all the target objects.

[0011] A second aspect of an embodiment of the present application provides an electronic device, comprising a storage device and at least one processor; the storage device is used to store at least one program, and when the at least one program is executed by the at least one processor, the at least one processor executes the scattering imaging method described in the first aspect of the embodiment of the present application.

[0012] A third aspect of the embodiments of the present application provides a computer-readable storage medium, on which executable instructions are stored. When the executable instructions are executed, the scattering imaging method described in the first aspect of the embodiments of the present application is executed.

[0013] From the above description, it can be seen that compared with the related art, the beneficial effects of the present application are:

[0014] First, a speckle pattern is obtained, and an aliased autocorrelation image is generated according to the speckle pattern; then, the aliased autocorrelation image is input into a target convolutional neural network, and the autocorrelation image of each object is separated from the aliased autocorrelation image by the target convolutional neural network; finally, all object autocorrelation images are reconstructed according to a phase recovery algorithm to obtain object images of all target objects; wherein, the speckle pattern is formed on the image plane by incident light carrying object information of multiple target objects through a scattering medium, the sizes of all target objects are within the effective range of optical memory, and the distance between any two target objects exceeds the effective range of optical memory; the aliased autocorrelation image is the superposition of the object autocorrelation images of all target objects. It can be seen that although the distance between any two target objects exceeds the effective range of optical memory, the present application can still obtain the object images of all target objects, which means that the present application is not easily limited by the effective range of the optical memory effect; moreover, the present application separates the object autocorrelation image of each target object in the aliased autocorrelation image through the target convolutional neural network, so that the object autocorrelation image of each target object does not contain autocorrelation information of other target objects other than the corresponding target object, thereby being able to smoothly obtain the object images of all target objects, and further being able to well realize scattering imaging of multiple target objects and a large field of view.

Brief Description of the Drawings

[0015] In order to more clearly illustrate the technical solutions in the related art or the embodiments of the present application, the drawings required for use in the description of the related art or the embodiments of the present application are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, not all embodiments. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0016] Figure 1 Schematic diagram of the principle of deconvolution imaging technology;

[0017] Figure 2 Schematic diagram of the principle of speckle autocorrelation imaging technology;

[0018] Figure 3 It is a flowchart of the GS algorithm;

[0019] Figure 4 It is a flowchart of the ER algorithm and the HIO algorithm;

[0020] Figure 5 This is the experimental result of imaging two target objects through scattering media using traditional technology;

[0021] Figure 6 This is the experimental result diagram of U-net convolutional neural network separating autocorrelation;

[0022] Figure 7 A graph showing the experimental results of separating autocorrelations using a Y-net convolutional neural network provided in an embodiment of the present application;

[0023] Figure 8 This is a graph of experimental results of separating autocorrelations using a T-net convolutional neural network provided in an embodiment of the present application;

[0024] Fig. 9 A schematic diagram of the structure of an experimental device for performing scattering imaging of multiple target objects under ambient light interference provided in an embodiment of the present application;

[0025] Fig.10 This is a graph showing the experimental results of separating autocorrelations of a T-net convolutional neural network under ambient light interference provided in an embodiment of the present application;

[0026] Fig.11 A schematic diagram of a scattering imaging method according to an embodiment of the present application;

[0027] Fig.12 A module block diagram of an electronic device provided in an embodiment of the present application;

[0028] Fig.13 A module block diagram of a computer-readable storage medium provided in an embodiment of the present application. [Specific implementation method]

[0029] In order to make the purpose, technical solutions and advantages of the present application more obvious and easy to understand, the present application will be clearly and completely described below in conjunction with the embodiments of the present application and the corresponding drawings; wherein the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. It should be understood that the various embodiments of the present application described below are only used to explain the present application and are not used to limit the present application, that is, based on the various embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application. In addition, the technical features involved in the various embodiments of the present application described below can be combined with each other as long as they do not conflict with each other.

[0030] Optical imaging is the main way to record light information. In the process of traditional optical imaging, the optical imaging system can be optimized to achieve the purpose of eliminating aberrations and improving resolution, thereby achieving high-quality imaging effects. However, when imaging in some complex environments, due to the scattering effect of light, the optimization method of traditional optical imaging is no longer applicable. Unlike traditional optical imaging, where light waves propagate in a space with uniform refractive index distribution, in the process of scattering imaging, when light waves carrying target object information pass through scattering media with uneven spatial distribution of refractive index (such as clouds, fog, biological tissues, etc.), photons interact with microparticles in the scattering medium, which causes the movement of photons to deviate from the original transmission direction. The accumulation of multiple interactions leads to the randomization of the propagation direction of photons, so the detector records a randomly distributed speckle pattern, and cannot directly record the intensity distribution of the object like traditional optical imaging. As a result, scattering imaging methods based on the optical memory effect (an inherent property of scattering media) came into being. They usually include two types, namely deconvolution imaging technology and speckle autocorrelation imaging technology. Both scattering imaging technologies have very broad development prospects and have been widely used in various occasions of imaging through scattering media.

[0031] In the related art, the effective range of the optical memory effect will have a great impact on the field of view of the scattered imaging, that is, when imaging through a strong scattering medium, the effective range of the optical memory effect will become very small, thereby limiting the application of the above two technologies in actual scattered imaging scenarios. In order to expand the field of view of scattering imaging, many researchers have proposed different solutions, which can be roughly divided into: the first category, using different pre-calibrated point spread functions to obtain multiple images in the horizontal and vertical directions, and then obtaining a large field of view by stitching multi-lens and multi-view images; the second category, using direct or indirect additional prior information to extract the information of hidden objects (i.e., target objects) from the original speckle pattern recorded by the camera; the third category, using high-order correlation to establish a complex nonlinear relationship between the input field and the output field; among them, the first category of solutions requires precise calibration, otherwise it will seriously affect the quality of the reconstructed image; although the prior information in the second category of solutions can improve the quality of large-field imaging to a certain extent, it cannot solve the problem of speckle aliasing (when multiple target objects are imaged through a scattering medium, the speckle pattern formed on the image plane does not include only one target object, but includes multiple target objects, which is speckle aliasing); the third category of solutions mainly modulates light waves in the scattering area, and the measurement of its transmission matrix is ​​usually complex and time-consuming. It can be seen that how to break through the limitation of the effective range of the optical memory effect to achieve multi-target object and large field of view imaging through scattering media is still an important research topic in the field of scattering imaging. To this end, the embodiment of the present application provides a scattering imaging method.

[0032] Before elaborating on the scattering imaging method provided in the embodiment of the present application in detail, the research process and the principle based on which the scattering imaging method provided in the embodiment of the present application is briefly described.

[0033] In the process of scattering imaging, after the incident light passes through the scattering medium, a part of the photons in the incident light do not collide with the particles in the scattering medium, which is called ballistic light; while another part of the photons deviate from the original propagation direction due to the collision with the particles in the scattering medium, which is called scattered light. There is a random phase difference between the photons of the outgoing light received by the image plane, and they are still coherent, so light interference will occur, in which constructive interference forms bright spots and destructive interference forms dark spots. Therefore, in the actual process of imaging through the scattering medium, the image plane receives a chaotic speckle pattern. The optical memory effect is an inherent property of the scattering medium. As the name suggests, different incident lights will form different speckle patterns after passing through the scattering medium. However, when incident lights with certain correlation pass through the same area of ​​the scattering medium, the speckle pattern obtained on the image plane also has certain correlation. In 1988, Freund et al. first confirmed the optical memory effect phenomenon through optical experiments. They used a beam of light emitted by a helium-neon laser light source as the incident light. On the premise of ensuring that the incident light passed through the same area of ​​the scattering medium, they slightly adjusted the incident angle of the incident light and used a camera to record the speckle pattern formed on the image plane before and after the incident angle was adjusted. The experimental results proved that there was a strong correlation between the speckle patterns formed on the image plane before and after the incident angle was adjusted. In short, the optical memory effect can be described as follows: the incident light passes through the scattering medium to form a speckle pattern on the image plane. When the incident angle of the incident light is slightly changed, the intensity distribution of the speckle pattern formed on the image plane will not change significantly, but the speckle pattern will move as a whole as the incident angle changes. As the incident angle gradually increases, the movement distance of the speckle pattern also gradually increases. At the same time, the correlation between the speckle patterns formed on the image plane will gradually decrease until the correlation disappears completely.

[0034] The range in which the optical memory effect takes effect can be described by the correlation function of the speckle pattern, which is expressed by the following first formula:

[0035]

[0036] Wherein, k = 2π / λ, Δθ is the deflection angle of the incident light, λ is the central wavelength of the light source emitting the incident light, and L is the effective thickness of the scattering medium. According to the first formula, the condition for the optical memory effect to take effect is: Δθ≤λ / 2πL. It can be seen that under the condition that the central wavelength of the light source emitting the incident light is fixed, the size of the effective range of the optical memory effect is only related to the effective thickness of the scattering medium.

[0037] When a point light source is imaged through a static scattering medium, the intensity distribution of the speckle pattern formed on the image plane can be regarded as the point spread function (PSF) of the imaging system. In the actual scattering imaging process, the target object hidden behind the scattering medium can be regarded as composed of countless point light sources. Therefore, the speckle pattern formed on the image plane is actually the superposition of the speckle patterns formed by countless point light sources passing through the scattering medium. When the size of the target object hidden behind the scattering medium is within the effective range of the optical memory effect, it can be approximately considered that the point spread functions corresponding to the countless point light sources constituting the target object are similar. Therefore, the imaging system is an imaging system with a spatial translation-invariant point spread function. The speckle pattern formed on the image plane can be expressed by the following second formula:

[0038] I = O * S;

[0039] Among them, I represents the speckle pattern formed on the image plane, O represents the actual image of the target object, S represents the spatial translation invariant point spread function of the imaging system, and * represents the convolution operation. It can be seen from the second formula that within the effective range of the optical memory effect, the speckle pattern formed on the image plane is the convolution of the target object hidden behind the scattering medium and the point spread function of the imaging system. The deconvolution imaging technology uses the convolution model of scattering imaging. A point light source is set at the position of the target object in advance to obtain the point spread function of the imaging system, and then the point spread function of the imaging system is used to reconstruct the object image of the target object from the speckle pattern formed after the target object passes through the scattering medium (the principle diagram of the deconvolution imaging technology can be referred to). Figure 1 ). Since the convolution operation is slow in computational imaging, deconvolution imaging technology generally converts the speckle pattern formed on the image plane from the spatial domain to the frequency domain, and then recovers the target object information (i.e., reconstructs the object image of the target object). According to the characteristics of Fourier transform, the convolution operation in the spatial domain is equivalent to the product operation in the frequency domain, so the second formula can be converted into the following third formula:

[0040] FFT(I)=FFT(O)·FFT(S);

[0041] Wherein, FFT() represents Fourier transform. When the point spread function of the imaging system is known, the object image of the target object can be reconstructed from the speckle pattern, which is expressed by the following fourth formula:

[0042]

[0043] Among them, FFT -1 () represents inverse Fourier transform, () * Represents the conjugate operation.

[0044] Although deconvolution imaging technology is simple and convenient, it requires a point light source to be set at the position of the target object in advance to obtain the point spread function of the imaging system. This is a typical invasive scattering imaging technology and is difficult to apply in most practical scattering imaging scenarios. In 2014, Israeli scientist Katz and his colleagues proposed a non-invasive image reconstruction method based on single-frame speckle autocorrelation (i.e., speckle autocorrelation imaging technology). The schematic diagram of its principle can be found in Figure 2 The autocorrelation of the speckle pattern recorded by the camera is calculated, and the autocorrelation of the speckle pattern is expressed by the following fifth formula:

[0045]

[0046] in, represents the autocorrelation operation. From the fifth formula, we know that the autocorrelation of the speckle pattern is equal to the convolution of the autocorrelation of the target object and the autocorrelation of the point spread function of the imaging system. Here, the autocorrelation of the point spread function of the imaging system (i.e. ) can be regarded as the autocorrelation of broadband random noise in essence. Therefore, the autocorrelation of the point spread function of the imaging system can be approximately regarded as an impulse response function δ, that is, the fifth formula can be further simplified into the sixth formula as follows:

[0047]

[0048] It can be seen from the sixth formula that the autocorrelation of the target object can be obtained by calculating the autocorrelation of the speckle pattern. According to the Wiener-Khinchin theorem, the power spectral density of the function (i.e., the square of the modulus of the Fourier transform) can be calculated by performing Fourier transform on the autocorrelation of the function, which is expressed by the following seventh formula:

[0049]

[0050] It can be seen from the seventh formula that the Fourier amplitude information of the target object can be directly obtained by calculating the autocorrelation of the speckle pattern recorded by the camera. However, in addition to the Fourier amplitude information, the Fourier phase information is also required to reconstruct the object image of the target object. Therefore, it is necessary to further use the phase recovery algorithm to reconstruct the object image of the target object.

[0051] The basic idea of ​​the phase recovery algorithm is to perform Fourier transform and inverse Fourier transform on the initial function, and impose constraints in the spatial domain and frequency domain respectively, and make the function converge to the true value in both the spatial domain and frequency domain through cyclic iterative calculation. Among them, the most classic phase recovery algorithm is the GS algorithm, and its operation steps can be found in Figure 3 , as shown below:

[0052] ⑴ Randomly select an initial phase function and combine it with the known amplitude to form an initial complex function g t (x,y);

[0053] ⑵Through two-dimensional Fourier transform, g t (x,y) is transformed to the frequency domain, and G is obtained t (k x ,k y );

[0054] ⑶ for G t (k x ,k y ) imposes a spectral surface constraint, that is, discarding G t (k x ,k y ), retaining only its phase part, and G t (k x ,k y ) is combined with the Fourier amplitude |F| obtained when calculating the autocorrelation of the speckle pattern to obtain a new G t ′(k x ,k y );

[0055] ⑷ For G t ′(k x ,k y ) performs a two-dimensional inverse Fourier transform operation and transforms it back to the spatial domain to obtain a new complex function g t ′(x,y);

[0056] ⑸ For the new complex function g t ′(x,y) imposes surface constraints, that is, discarding g t ′(x,y) and retain only its phase part, and t The phase part of ′(x,y) is combined with the known object plane amplitude |g| to obtain the new g t (x,y) is used as the initial complex function for the next iteration;

[0057] Repeat the above steps (2) to (5) until the GS algorithm converges.

[0058] The quality of the object image reconstructed by the GS algorithm and its convergence speed are largely affected by the initial value of the iteration, and there may be problems such as slow convergence speed or even failure of convergence of the algorithm. In order to solve these problems, the convergence result of the algorithm can be optimized and the convergence speed of the algorithm can be improved by operating the initial value of the iteration, such as introducing some low-frequency information of the target object into the initial value of the iteration. In 1982, Fienup et al. proposed an error reduction (ER) algorithm based on the GS algorithm, which introduced the contour range of the target object as a constraint condition into the step of the object surface constraint, that is, into step ⑸, which is expressed by the following eighth formula:

[0059]

[0060] Where S is the contour range of the target object. From the eighth formula, we can see that the part of the function within the contour range of the target object keeps its value unchanged, while the part beyond the contour range of the target object sets its value to 0. Compared with the GS algorithm, the ER algorithm can gradually reduce the frequency domain error as the number of algorithm iterations increases, and has a faster convergence speed; however, the ER algorithm is prone to fall into a local optimal solution. For this reason, Fienup et al. proposed a hybrid input-output (HIO) algorithm, the operation steps of which can be found in Figure 4 , the constraint condition of the HIO algorithm in the step of the object plane constraint (i.e., step ⑸) is expressed by the following ninth formula:

[0061]

[0062] Among them, β represents the parameter used to control the convergence of the algorithm, and its value range is [0,1]. Compared with the ER algorithm, after the HIO algorithm introduces the parameter β, the contour range of the target object gradually decreases with the increase of the number of algorithm iterations, which can effectively solve the problem that the ER algorithm is prone to fall into the local optimal solution. In addition, it should be noted that the experiments involved in this article all use the HIO algorithm.

[0063] According to the content recorded in the previous article, within the effective range of the optical memory effect, it can be considered that the point spread functions corresponding to all the point light sources that make up the target object are similar, so the scatter imaging process can be simplified to a linear convolution model, that is, the speckle pattern on the image plane can be represented by the convolution of the object image of the target object and the point spread function of the imaging system. In the speckle autocorrelation imaging technology, since the autocorrelation of the point spread function can be regarded as an impulse response function, the autocorrelation of the speckle pattern is equivalent to the autocorrelation of the target object, and the object image of the target object can be reconstructed from the autocorrelation of the speckle pattern by the phase recovery algorithm. However, when the effective range of the optical memory effect is exceeded, the correlation between the point spread functions corresponding to different point light sources drops sharply, and the linear convolution model of scatter imaging is no longer applicable. If there is an image O on the object plane composed of n discrete target objects O located at different spatial positions i The size of each target object is within the effective range of the optical memory effect, but the distance between any two target objects exceeds the effective range of the optical memory effect. Then, let the point spread function corresponding to different target objects be S i , then for incoherent illumination, the speckle pattern recorded by the camera on the image plane is expressed by the following tenth formula:

[0064]

[0065] From the tenth formula, we can know that the speckle pattern recorded by the camera is actually the superposition of the speckle patterns corresponding to different target objects whose spatial distance exceeds the effective range of the optical memory effect. Taking the scattered imaging of two target objects as an example, the speckle pattern recorded by the camera is autocorrelated, which is expressed by the following eleventh formula:

[0066]

[0067] Due to the autocorrelation of the point spread function (i.e. and ) can be approximately regarded as the impulse response function δ, so the eleventh formula can be further simplified to the following twelfth formula:

[0068]

[0069] Among them, the first item is the autocorrelation of the first target object, and the second item is the autocorrelation of the second target object. The above two items are located in the central position of the autocorrelation of the speckle pattern, and the remaining items are cross-correlation items. Since the intensity of the autocorrelation item is much higher than the cross-correlation item, the cross-correlation item can be ignored and regarded as weak background noise. The central part of the autocorrelation image of the speckle pattern recorded by the camera is cut off, and what is actually obtained is the autocorrelation of the two target objects (i.e. and ) is called aliased autocorrelation. To verify this conclusion, we conducted an experiment to image two target objects through scattering media and tried to reconstruct the object images of the two target objects using the phase recovery algorithm. The experimental results can be found in Figure 5 ( Figure 5 (a) are two target objects whose spatial distance on the object plane exceeds the effective range of the optical memory effect. Figure 5 (b) is the speckle pattern recorded by the camera, 5(c) is the autocorrelation image of the speckle pattern, and 5(d) is the reconstructed result of the object images of the two target objects).

[0070] The experimental results show that we cannot directly use the phase recovery algorithm to reconstruct the target object image from the aliased autocorrelation of two target objects. Similarly, for the scattered imaging of more than two multi-target objects, the autocorrelation of the speckle pattern recorded by the camera is expressed by the following twelfth formula:

[0071]

[0072] After ignoring the cross-correlation terms in the twelfth formula, the twelfth formula can be simplified to the following thirteenth formula:

[0073]

[0074] It can be seen from the thirteenth formula that the autocorrelation of the speckle pattern recorded by the camera is approximately equivalent to the superposition of the autocorrelations corresponding to multiple target objects. It is impossible to directly use the speckle autocorrelation imaging technology to image multiple target objects beyond the effective range of the optical memory effect. Therefore, to achieve scattering imaging of multiple target objects, it is necessary to think about how to separate the autocorrelations of different target objects. Recently, Wang et al. successfully separated the autocorrelations of two target objects from the speckle pattern of dual target objects by introducing Fourier spectrum estimation and iterative energy constraint compensation. After that, they further explored the reconstruction and positioning of multiple target objects using two speckle patterns recorded at different imaging distances. At the same time, Li et al. proposed to use independent component analysis to separate the speckles corresponding to different target objects from the speckle pattern recorded by the camera, and then calculate the autocorrelations of different target objects respectively, and then reconstruct the object image of each target object through the phase recovery algorithm. The embodiments of the present application are different from this. In order to break through the imaging field limitation of the effective range of the optical memory effect, the embodiments of the present application propose a scattering imaging method of multiple target objects based on deep learning and speckle autocorrelation, which uses a convolutional neural network to separate the autocorrelations of different target objects in the aliased autocorrelation image to obtain the autocorrelations corresponding to different target objects, and then reconstruct the object images of each target object through a phase recovery algorithm. The embodiments of the present application design a "one-to-many" type convolutional neural network of the Y-net and Tree-net (abbreviated as T-net) structure according to actual task requirements, and experimentally demonstrate the results of scattering imaging of dual-target objects and multiple-target objects beyond the effective range of the optical memory effect by this method, verifying the feasibility of this method.

[0075] Before introducing the convolutional neural network (i.e., Y-net and Tree-net) of the embodiment of the present application, it is necessary to first introduce a commonly used convolutional neural network, i.e., U-net. U-net consists of an encoder and a decoder, the encoder is used to extract image features from the input image, and the decoder is used to reconstruct the extracted image features into an output image. U-net has been proven to be very effective in image processing tasks in end-to-end scenarios, but U-net cannot perform the learning task of separating individual correlations from aliased autocorrelations very well. We used simulated data to test whether U-net can separate the autocorrelations of different target objects from the aliased autocorrelations. The digital images in the MNITS handwritten digit dataset are selected as the target objects, their autocorrelations are calculated respectively, and then two autocorrelation images are randomly selected for superposition to obtain an aliased autocorrelation image. We simulated and generated a total of 5,000 aliased autocorrelation images as training data, where the aliased autocorrelation image is used as the input of U-net, and the corresponding two autocorrelation images of different target objects are used as the output labels of U-net. The output results of U-net after training are as follows Figure 6 As shown ( Figure 6 (a) is the aliased autocorrelation image in the test data set. Figure 6 (b) and (c) are the real autocorrelation images of the two target objects. Figure 6 (d) and (e) are the autocorrelation images of the two target objects output by the trained U-net, respectively. The experimental results show that the autocorrelation images of the two target objects output by the U-net still contain other autocorrelation information, which is significantly different from the real autocorrelation images of the two target objects. There is a large error between the two, which shows that the trained U-net cannot separate the autocorrelation of different target objects from the aliased autocorrelation very well. Here, we believe that the reason why the U-net cannot separate the autocorrelation of different target objects from the aliased autocorrelation is that in the process of object image reconstruction by the U-net decoder, the image features of the autocorrelation of different target objects are simultaneously involved in the object image reconstruction. Since the image features of the autocorrelation images of different target objects are not separated in the process of object image reconstruction, the autocorrelation images of each target object output by the U-net still contain other autocorrelation information at the same time.

[0076] Unlike the learning task where the input-output quantity relationship is "one-to-one" which U-net is good at, here we are facing the problem of "one-to-many" input-output quantity relationship. In order to enable the convolutional neural network to better learn the task of separating the autocorrelations of different target objects, the embodiment of the present application proposes convolutional neural networks with Y-net and T-net structures based on U-net. Prior to this, Wang et al. have used a convolutional neural network with a Y-net structure to directly reconstruct the amplitude information and phase information of the object from the hologram at the same time.

[0077] Y-net still follows the "encoding-decoding" form of U-net in its overall structure. The encoder of Y-net consists of four convolution blocks, each of which contains four operation layers: pooling layer, convolution layer, activation function layer and batch normalization layer. The main function of the pooling layer is to perform downsampling operations, which can remove redundant information of the image and compress the image features to achieve the purpose of simplifying the complexity of the network, reducing the amount of calculation, and speeding up the network training speed. The pooling layer generally uses average pooling or maximum pooling; among them, average pooling is to average the grayscale values ​​of pixels in the neighborhood, which can well preserve the background information of the image, but it is easy to blur the picture; maximum pooling is to take the maximum value of the grayscale value of pixels in the neighborhood, which can well preserve the texture features of the image. Here, in order to retain the autocorrelated texture features as much as possible, we choose to use maximum pooling and set the neighborhood size of the pooling layer to 2×2, that is, each time through the pooling layer, the size of the image feature becomes half of the original. The main function of the convolution layer is to extract image features. The number of convolution kernels in the convolution layer corresponding to the four convolution blocks in the encoder is 12, 24, 48, and 96 respectively. That is, as the number of network layers increases, the number of channels of image features also increases, which is conducive to allowing the encoder to extract more image features. The activation function layer can perform nonlinear mapping on the output results of the convolution layer and introduce nonlinear characteristics into the neural network, so that the neural network can learn complex problems that simple linear models cannot solve. Here, all activation function layers use the linear rectifier function (Rectified Linear Unit, ReLU) as the activation function, which can effectively avoid problems such as gradient explosion and gradient disappearance during gradient descent and back propagation, and improve the efficiency of network training. The batch normalization layer includes two operations: batch processing and normalization. Batch processing means that the convolutional neural network receives multiple images as input at the same time and performs parallel operations, which can greatly improve the training speed of the convolutional neural network. The normalization operation is to adjust the mean of the pixel grayscale value of the input data of each layer of the convolutional neural network to 0 and a standard normal distribution with a variance of 1, which can avoid problems such as gradient disappearance during the training process of the convolutional neural network.

[0078] After the encoder of Y-net extracts the image features of the input image, the decoder reconstructs the image features into the output image. Unlike U-net, the decoder of Y-net contains two upsampling paths, which output different images respectively. This is equivalent to Y-net being designed as a fusion of two U-nets to achieve the purpose of reconstructing different images separately (i.e., reconstructing the autocorrelation images of different target objects). Each upsampling path in the encoder consists of five convolution blocks, and each convolution block contains four operation layers: convolution layer, activation function layer, batch normalization layer and upsampling layer. Specifically, the number of convolution kernels in the convolution layers corresponding to the five convolution blocks are 192, 96, 48, 24, and 12 respectively, that is, as the number of network layers increases, the number of channels of image features also decreases; at the same time, each time the upsampling layer is passed, the size of the image features will become twice the original size, and finally after a convolution layer with a convolution kernel size of 1×1 and an activation function layer, the output image of the convolutional neural network is generated.

[0079] From the structure of convolutional neural network, Y-net can better adapt to the learning task of separating the autocorrelation of different target objects from the aliased autocorrelation. From the above analysis, it is known that the reason why U-net cannot complete this learning task is that when the decoder of the neural network reconstructs the image, there is only one upsampling path in the decoder, and the image features corresponding to different autocorrelations are not distinguished. Therefore, the reconstructed image inevitably carries other autocorrelation information at the same time. In Y-net, the encoder is still responsible for extracting image features from aliased autocorrelations, but in the decoder, since two upsampling paths are included, during the training process of the neural network, each upsampling path can learn to only select the image features corresponding to the autocorrelation image of the output label of the path, and then reconstruct the image features, which can effectively solve the problem of image feature confusion in the image reconstruction process. It should be noted here that deep neural networks can learn complex problems more effectively than shallow neural networks, but when training deep neural networks, the performance of the model often decreases with the increase of network depth, which is called the degradation problem. In order to alleviate the degradation problem of neural networks, U-net uses skip connections to connect the network layers with the same image feature size in the encoder and decoder, and uses the output of the network layer in the encoder as the input of the network layer in the decoder, so that the fine-grained image features learned in the encoder part can participate in image reconstruction in the decoder part, ensuring the reusability of the features. This is equivalent to adding constraints to the parameter learning of the deep network, which can effectively retain the high-frequency detail information of the image and reduce the output error. However, in Y-net, in order to avoid the interference of the unseparated image features in the encoder in the image reconstruction of the separated autocorrelated image features in the decoder, skip connections are not used in Y-net.

[0080] At the same time, it should be emphasized that Y-net also has good scalability. In Y-net, the number of upsampling paths in the decoder is adjustable and can be easily expanded to more upsampling paths to achieve the purpose of one input to multiple outputs. We call the convolutional neural network containing more than two upsampling paths T-net, which can be used to separate the autocorrelations of multiple (greater than 2) target objects. When training Y-net or T-net, we select the mean absolute error (MAE) as the loss function to train the parameters of the convolutional neural network. The loss function is expressed by the following fourteenth formula:

[0081]

[0082] Where n represents the number of images output by the convolutional neural network, N is the number of pixels in each image, and x is i and i The pixel values ​​of each pixel in the input image and output image of the convolutional neural network are respectively. The Adam optimizer is selected to update the weights and biases of the convolutional neural network by back-propagating the loss function in the convolutional neural network to minimize the MAE value. For example, the learning rate of the Adam optimizer is set to 0.0001 and the number of iterations is set to 10; the convolutional neural network is implemented using Python 3.7 on the Pytorch platform, and a GPU (NVIDIAGeForce GTX 1050Ti) is used to accelerate the training process.

[0083] In order to verify that Y-net and T-net can separate the autocorrelation images of different target objects from the aliased autocorrelation images, a set of simulation data is first generated to train and test Y-net or T-net. Taking Y-net as an example, 10,000 digital images with a pixel size of 128×128 are selected from the MNIST handwritten digit dataset, their autocorrelations are calculated respectively, and two autocorrelation images are randomly superimposed, and finally 5,000 aliased autocorrelation images of two target objects are obtained. During the training process, the aliased autocorrelation images are used as the input of Y-net, and the corresponding two independent autocorrelation images of the target objects are used as the output labels of Y-net. The experimental results are shown in the figure. Figure 7 As shown ( Figure 7 (a) is the aliased autocorrelation image, Figure 7 (b) and (c) are the real autocorrelation images of two different target objects. Figure 7 (d) and (e) are the autocorrelation images of the two target objects output by the trained Y-net. Figure 7(f) and (g) are the object images of the two target objects reconstructed from (d) and (e) by the phase recovery algorithm, respectively. The experimental results show that the autocorrelation images of the two target objects output by the trained Y-net are basically consistent with the real autocorrelation images of the two target objects, so that the object images of each target object can be reconstructed from the respective correlation images output by the Y-net by the phase recovery algorithm. This proves the feasibility of using deep learning and speckle autocorrelation to achieve scattering imaging of multiple target objects beyond the effective range of the optical memory effect.

[0084] In addition, we have also verified through experiments that T-net can separate the autocorrelations corresponding to different target objects from the aliased autocorrelation images of multiple target objects. Similarly, digital images of the MINIST handwritten digit dataset are selected, and their autocorrelations are calculated respectively. Then, 3, 4, and 5 autocorrelation images are randomly selected for superposition to obtain triple-aliased autocorrelation images, quadruple-aliased autocorrelation images, and penta-aliased autocorrelation images; among them, there are 5,000 triple-aliased autocorrelation images, quadruple-aliased autocorrelation images, and penta-aliased autocorrelation images. Correspondingly, we control the number of images that T-net finally outputs by setting the number of upsampling paths of the decoder in T-net. 5,000 triple-aliased autocorrelation images, 5,000 quadruple-aliased autocorrelation images, and 5,000 penta-aliased autocorrelation images are used to train T-nets containing different numbers of upsampling paths (i.e., 3, 4, and 5). The experimental results are shown as follows. Figure 8 As shown ( Figure 8 ⑴, ⑵, and ⑶ correspond to the cases of separating the autocorrelations of 3, 4, and 5 target objects, respectively; Figure 8 (a) in (1), (2) and (3) are all aliased autocorrelation images in their respective cases; Figure 8 (b)(c)(d) in (1), Figure 8 (b)(c)(d)(e) in (2), Figure 8 (b)(c)(d)(e)(f) in (⑶) are the real autocorrelation images of the target object in their respective cases; Figure 8 (e)(f)(g) in (⑴), Figure 8 (f)(g)(h)(i) in (2), Figure 8 (g)(h)(i)(j)(k) in (⑶) are the autocorrelation images output by the trained T-net in their respective cases).

[0085] The experimental results show that T-net can effectively separate the autocorrelation of triple-aliased autocorrelation images and quadruple-aliased autocorrelation images. The autocorrelation output by T-net is basically the same as the real autocorrelation of the target object. Figure 8 ⑴ and ⑵. However, when T-net separates the autocorrelation of the five-aliased autocorrelation image, as shown in Figure 8As shown in (3), when the contour structure of the autocorrelation is relatively simple, the autocorrelation output by T-net and the true autocorrelation of the target object are still highly similar, but when the contour structure of the autocorrelation is relatively complex, there is an obvious error between the autocorrelation output by T-net and the true autocorrelation of the target object, and the autocorrelation image of the target object cannot be reconstructed from the output autocorrelation by the phase recovery algorithm. In order to more accurately describe the effect of T-net in separating different amounts of autocorrelation, we calculated the mean absolute error (MAE) and structural similarity (SSIM) between the autocorrelation image output by T-net and the true autocorrelation image of the target object in different cases, as shown in the following table:

[0086]

[0087] We have demonstrated the feasibility of Y-net and T-net through simulation experiments, and will conduct actual optical experiments to further demonstrate the effectiveness of this method. At the same time, it is worth noting that in actual scattering imaging scenarios, the presence of ambient light usually interferes with the speckle pattern recorded by the camera, forming background noise in the autocorrelation of the speckle pattern, affecting the reconstruction results of the phase recovery algorithm.

[0088] Therefore, we conducted an optical experiment on scattering imaging of multiple targets under the condition of ambient light interference. The schematic diagram of the experimental setup can be found in Fig. 9 . The laser emits a laser beam with a wavelength of 632.8nm. After passing through the rotating ground glass and the collimating lens, the laser beam is irradiated on the spatial light modulator. The digital image of the MNITS handwritten digit dataset is displayed on the spatial light modulator, and then the speckle is formed after passing through the ground glass as a scattering medium. It is recorded by a camera placed on the image plane (such as a scientific CMOS camera, PCO edge 4.2, 2048×2048 pixels, 6.5μm×6.5μm pixel size), and the ambient light is introduced by an LED light source. The speckle pattern is recorded with and without ambient light, and then their autocorrelations are calculated. Three autocorrelation images of the target object with background noise are randomly selected and superimposed to obtain an aliased autocorrelation image as the input of T-net, and the corresponding three autocorrelation images of the target object without background noise are used as the output labels of T-net. 5000 sets of aliased autocorrelation images are selected as training data to train T-net. Then, in the presence of ambient light, three target objects are displayed simultaneously on the spatial light modulator, and their speckle patterns are recorded by the camera. Finally, the autocorrelations of the three target objects are calculated as test data to verify whether the trained T-net can separate the autocorrelations of different target objects from the aliased autocorrelations with background noise. The experimental results are shown in Figure 2. Fig.10 As shown ( Fig.10 (a) shows three target objects displayed simultaneously on the spatial light modulator. Fig.10 (b) is the speckle pattern with background noise recorded by the camera, 10(c) is the autocorrelation of the speckle pattern with background noise, Fig.10 (d), (e) and (f) are all autocorrelation images output by the trained T-net. Fig.10 (g), (h) and (i) are all object images of the target object reconstructed from the autocorrelation image output by T-net through the phase recovery algorithm). The experimental results show that T-net can effectively separate the autocorrelations of different target objects in actual optical scattering imaging experiments and can resist the interference of ambient light.

[0089] Finally, in actual scatter imaging, it is often unclear how many target objects are hidden behind the scattering medium. In this case, it is impossible to decide how many upsampling paths a T-net should be used to separate the autocorrelations of the target objects. To address this problem, we use a classification convolutional neural network that can classify aliased autocorrelations of different numbers of target objects. First, the image features of aliased autocorrelations are extracted through six convolutional layers in the classification convolutional neural network, and then classified through two fully connected layers so that the classification convolutional neural network outputs the probabilities of different categories (i.e., different numbers of target objects), and the category with the highest probability is the classification result of the classification convolutional neural network (for example, the classification convolutional neural network outputs "the probability of 2 target objects is 30%, the probability of 3 target objects is 60%, and the probability of 5 target objects is 90%, then the classification result of the classification convolutional neural network is "5 target objects"). We selected 2000 images and 500 images from the aliasing autocorrelations of 2, 3, and 4 target objects for training and testing of the classification convolutional neural network, respectively. That is, the training data set contains 6000 images and the test data set contains 1500 images. The classification accuracy of the trained classification convolutional neural network for the test data set reached 92%.

[0090] According to the above description, in order to break through the imaging field of view limitation of the effective range of the optical memory effect, the embodiment of the present application proposes a multi-target object scattering imaging method based on deep learning and speckle autocorrelation, which uses convolutional neural networks (i.e., Y-net and T-net) to separate different autocorrelation components in the aliased autocorrelation image, obtains the autocorrelation corresponding to each target object, and then reconstructs the object image of each target object through a phase recovery algorithm. The multi-target object scattering imaging method based on deep learning and speckle autocorrelation proposed in the embodiment of the present application has the following advantages:

[0091] It is insensitive to the position change between the target objects; because this method separates the autocorrelation of the target object, and after the autocorrelation calculation of the speckle pattern, the autocorrelation of the target object is incoherent intensity superposition in the center of the autocorrelation image of the speckle pattern, so no matter how the position of the target object changes, the calculated aliased autocorrelation is the same. This method has good robustness to ambient light noise; in the case of ambient light interference, the ambient light will form serious background noise in the autocorrelation image of the speckle pattern, drowning the autocorrelation information of the target object, and because the convolutional neural network itself has a strong denoising ability, this method can use data training to let the neural network learn to extract and separate the autocorrelation corresponding to different target objects from the aliased autocorrelation with background noise. This method has good scalability and can realize the separation of autocorrelations of different numbers of target objects; this method can flexibly change the structure of the neural network (that is, the number of upsampling paths) to control the number of outputs of the neural network, and then train the neural network to flexibly separate the autocorrelations of different numbers of target objects according to actual conditions. A classification convolutional neural network is used to classify the speckle patterns of different numbers of target objects, so there is no need to know the number of target objects hidden behind the scattering medium as prior information.

[0092] In summary, please refer to Fig.11 , Fig.11 A schematic diagram of a scattering imaging method provided in an embodiment of the present application. The present application provides a scattering imaging method, comprising the following steps 1101 to 1104.

[0093] Step 1101: Obtain a speckle pattern.

[0094] In the embodiment of the present application, when performing scattering imaging of multiple target objects, it is necessary to first obtain a speckle pattern; wherein the obtained speckle pattern is formed on the image plane by incident light carrying object information of multiple target objects passing through a scattering medium, the sizes of all target objects are within the effective range of optical memory, and the distance between any two target objects exceeds the effective range of optical memory.

[0095] Step 1102: Generate an aliased autocorrelation image according to the speckle pattern.

[0096] In the embodiment of the present application, after the speckle pattern is acquired, it is also necessary to generate an aliased autocorrelation image according to the speckle pattern; wherein the aliased autocorrelation image is a superposition of object autocorrelation images of all target objects.

[0097] Step 1103: input the aliased autocorrelation image into the target convolutional neural network, and separate the autocorrelation images of each object from the aliased autocorrelation image through the target convolutional neural network.

[0098] In the embodiment of the present application, after the aliased autocorrelation image is generated according to the speckle pattern, it is also necessary to input the aliased autocorrelation image into the target convolutional neural network (i.e., Y-net or T-net), and separate the autocorrelation images of each object from the aliased autocorrelation image through the target convolutional neural network.

[0099] Step 1104: reconstruct all object autocorrelation images respectively according to the phase recovery algorithm to obtain object images of all target objects.

[0100] In an embodiment of the present application, after the aliased autocorrelation image is input into the target convolutional neural network and the autocorrelation images of each object are separated from the aliased autocorrelation image by the target convolutional neural network, all the object autocorrelation images need to be reconstructed separately according to the phase recovery algorithm to obtain the object images of all target objects; at this point, scattering imaging of multiple target objects is realized.

[0101] In the embodiment of the present application, a speckle pattern is first acquired, and an aliased autocorrelation image is generated according to the speckle pattern; the aliased autocorrelation image is then input into a target convolutional neural network, and the autocorrelation image of each object is separated from the aliased autocorrelation image by the target convolutional neural network; finally, all the object autocorrelation images are reconstructed according to a phase recovery algorithm to obtain object images of all target objects; wherein the speckle pattern is formed on the image plane by incident light carrying object information of multiple target objects passing through a scattering medium, the sizes of all target objects are within the effective range of optical memory, and the distance between any two target objects exceeds the effective range of optical memory; the aliased autocorrelation image is the superposition of the object autocorrelation images of all target objects. It can be seen that although the distance between any two target objects exceeds the effective range of optical memory, the embodiment of the present application can still obtain the object images of all target objects, which means that the embodiment of the present application is not easily limited by the effective range of the optical memory effect; moreover, the embodiment of the present application separates the object autocorrelation image of each target object in the aliased autocorrelation image through the target convolutional neural network, so that the object autocorrelation image of each target object does not contain autocorrelation information of other target objects other than the corresponding target object, so that the object images of all target objects can be smoothly obtained, and then the scattering imaging of multiple target objects and a large field of view can be well realized.

[0102] As an implementation manner, step 1102, i.e., generating an aliasing autocorrelation image according to the speckle pattern, may include: performing autocorrelation processing on the speckle pattern to obtain a speckle autocorrelation image of the speckle pattern; and intercepting the speckle autocorrelation image (specifically intercepting the central part of the speckle autocorrelation image) to obtain the aliasing autocorrelation image. It can be understood that the central part of the speckle autocorrelation image is the aliasing autocorrelation image.

[0103] As an implementation method, step 1103, i.e., separating each object autocorrelation image from the aliased autocorrelation image through the target convolutional neural network, may include: extracting image features of each object autocorrelation image from the aliased autocorrelation image through an encoder; reconstructing the image features of all object autocorrelation images respectively through multiple upsampling paths in the decoder to obtain all object autocorrelation images; wherein the target convolutional neural network includes an encoder and a decoder, the decoder includes multiple upsampling paths, the number of upsampling paths is consistent with that of the target object, and there is a one-to-one correspondence between the upsampling paths and the image features of the reconstructed object autocorrelation images.

[0104] It should be understood that the above-mentioned implementation mode is only a preferred implementation of the embodiment of the present application, and is not the only limitation of the specific process of steps 1102 and 1103 in the embodiment of the present application; in this regard, those skilled in the art can flexibly set it according to the actual application scenario based on the embodiment of the present application.

[0105] In some embodiments, step 1103, i.e., before inputting the aliasing autocorrelation image into the target convolutional neural network, may also include: inputting the aliasing autocorrelation image into a classification convolutional neural network; determining the image category of the aliasing autocorrelation image through the classification convolutional neural network; selecting a convolutional neural network from a preset convolutional neural network library as a target convolutional neural network according to the image category of the aliasing autocorrelation image; wherein the image category indicates the number of target objects corresponding to the aliasing autocorrelation image, and the number of upsampling paths contained in the target convolutional neural network corresponds to the image category of the aliasing autocorrelation image.

[0106] As an implementation, determining the image category of the aliasing autocorrelation image by a classification convolutional neural network may include: extracting image features of the aliasing autocorrelation image by a classification convolutional neural network; deriving the probability of the aliasing autocorrelation image in different image categories based on the image features of the aliasing autocorrelation image by a classification convolutional neural network; and determining the image category of the aliasing autocorrelation image according to the principle of highest probability by a classification convolutional neural network. Exemplarily, the classification convolutional neural network may derive the probability of the aliasing autocorrelation image in different image categories based on the number of image features of the aliasing autocorrelation image.

[0107] As an implementation manner, after selecting a convolutional neural network as a target convolutional neural network from a preset convolutional neural network library according to the image category of the aliasing autocorrelation image, the method may also include: training the target convolutional neural network.

[0108] As a specific implementation of this embodiment, training the target convolutional neural network can include: based on the minimum mean absolute error, back-propagating a preset loss function in the target convolutional neural network, and updating the weights and bias of the target convolutional neural network; wherein, the preset loss function can refer to the fourteenth formula shown above.

[0109] It should be understood that the above-mentioned implementation mode is only a preferred implementation of the embodiment of the present application, and is not the only limitation of the embodiment of the present application to additional processes other than steps 1101 to 1104; in this regard, those skilled in the art can flexibly set it according to the actual application scenario based on the embodiment of the present application.

[0110] See also Fig.12 , Fig.12 A module block diagram of an electronic device provided in an embodiment of the present application.

[0111] like Fig.12 As shown, an embodiment of the present application further provides an electronic device 1200, including a storage device 1210 and at least one processor 1220; wherein the storage device 1210 is used to store at least one program, and when the at least one program is executed by the at least one processor 1220, the at least one processor 1220 executes the scattering imaging method provided in the embodiment of the present application.

[0112] In some embodiments, the electronic device 1200 may further include a bus 1230 for communication connection between the storage device 1210 and the at least one processor 1220 .

[0113] See also Fig.13 , Fig.13 A module block diagram of a computer-readable storage medium provided in an embodiment of the present application.

[0114] like Fig.13 As shown, the embodiment of the present application further provides a computer-readable storage medium 1300, on which executable instructions 1310 are stored. When the executable instructions 1310 are executed, the scattering imaging method provided in the embodiment of the present application is executed.

[0115] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0116] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in this application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integration. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk), etc.

[0117] It should be noted that each embodiment in the present application is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For product-type embodiments, since they are similar to method-type embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method-type embodiments.

[0118] It should also be noted that, in the present application, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or equipment. In the absence of further restrictions, the elements defined by the statement "comprise one..." do not exclude the existence of other identical elements in the process, method, article or equipment including the elements.

[0119] The above description of the disclosed embodiments enables professionals and technicians in the field to implement or use the present application. Various modifications to these embodiments will be apparent to professionals and technicians in the field, and the general principles defined in the present application can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown in the present application, but will conform to the widest range consistent with the principles and novel features disclosed in the present application.

Claims

1. A scattering imaging method, characterized in that: include: Acquire a speckle pattern; wherein the speckle pattern is formed on an image plane by incident light carrying object information of a plurality of target objects passing through a scattering medium, the sizes of all the target objects are within an effective range of optical memory, and the distance between any two of the target objects exceeds the effective range of optical memory; Generate an aliased autocorrelation image according to the speckle pattern; wherein the aliased autocorrelation image is a superposition of object autocorrelation images of all the target objects; Inputting the aliased autocorrelation image into a target convolutional neural network, and separating each of the object autocorrelation images from the aliased autocorrelation image through the target convolutional neural network; Reconstructing all the object autocorrelation images respectively according to the phase recovery algorithm to obtain object images of all the target objects; The step of generating an aliased autocorrelation image according to the speckle pattern comprises: performing autocorrelation processing on the speckle pattern to obtain a speckle autocorrelation image of the speckle pattern; intercepting a central portion of the speckle autocorrelation image to obtain an aliased autocorrelation image; The step of separating each of the object autocorrelation images from the aliased autocorrelation image by the target convolutional neural network comprises: Extracting image features of each of the object autocorrelation images from the aliased autocorrelation images through an encoder; wherein the target convolutional neural network includes the encoder; Reconstructing the image features of all the object autocorrelation images respectively through multiple upsampling paths in the decoder to obtain all the object autocorrelation images; wherein the target convolutional neural network also includes the decoder, the decoder includes the multiple upsampling paths, the number of the upsampling paths is consistent with the target object, and there is a one-to-one correspondence between the upsampling paths and the image features of the reconstructed object autocorrelation images; Before inputting the aliased autocorrelation image into the target convolutional neural network, the method further includes: Inputting the aliased autocorrelation image into a classification convolutional neural network; Determining the image category of the aliasing autocorrelation image by the classification convolutional neural network; wherein the image category indicates the number of the target objects corresponding to the aliasing autocorrelation image; A convolutional neural network is selected from a preset convolutional neural network library as a target convolutional neural network according to the image category of the aliasing autocorrelation image, and the target convolutional neural network is trained; wherein the number of upsampling paths contained in the target convolutional neural network corresponds to the image category of the aliasing autocorrelation image.

2. The scattering imaging method according to claim 1, characterized in that: Determining the image category of the aliased autocorrelation image by the classification convolutional neural network includes: Extracting image features of the aliased autocorrelation image by the classification convolutional neural network; Determining the probability of the aliasing autocorrelation image in different image categories according to the image features of the aliasing autocorrelation image by the classification convolutional neural network; The image category of the aliasing autocorrelation image is determined according to the highest probability principle by the classification convolutional neural network.

3. The scattering imaging method according to claim 1, characterized in that: The training of the target convolutional neural network includes: Based on the minimum mean absolute error, the preset loss function is back-propagated in the target convolutional neural network to update the weights and biases of the target convolutional neural network; wherein the preset loss function is represented by the following first formula: Wherein, loss(x,y) represents the preset loss function, n represents the number of output images of the target convolutional neural network, N is the number of pixels per image, x is i and i are the pixel values ​​of each pixel in the input image and output image of the target convolutional neural network, respectively.

4. The scattering imaging method according to claim 1, characterized in that: The optical memory effective range is expressed by the following second formula: Δθ≤λ / 2πL; Wherein, Δθ is the deflection angle of the incident light, λ is the central wavelength of the light source emitting the incident light, and L is the effective thickness of the scattering medium.

5. An electronic device, characterized in that: The method comprises a storage device and at least one processor; the storage device is used to store at least one program, and when the at least one program is executed by the at least one processor, the at least one processor executes the method according to any one of claims 1 to 4.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores executable instructions, and when the executable instructions are executed, the method according to any one of claims 1 to 4 is performed.

Images