Image recovery method and apparatus, computer device, and storage medium

Abstract

The present application relates to the technical field of image processing, and relates to an image recovery method and apparatus, a computer device, and a storage medium. The method comprises: acquiring a target speckle image containing at least two objects to be tested; on the basis of the target speckle image, determining central autocorrelation data of the objects contained in the target speckle image; and, on the basis of a sidelobe recovery network, performing sidelobe recovery processing on the central autocorrelation data to obtain a target image containing the objects to be tested. According to the present application, speckle autocorrelation sidelobes of the objects to be tested can be recovered by means of the central autocorrelation data, thereby implementing scattering imaging processing of a customized field of view, allowing scattering imaging technology to be applied to imaging application scenarios involving multiple objects to be tested, and ensuring the accuracy of scattering imaging for the multiple objects.

Description

Image restoration method, device, computer equipment and storage medium Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image restoration method, apparatus, computer equipment, and storage medium. Background Art

[0002] Scattering imaging is an imaging method based on the principle of scattering. It achieves imaging by measuring the scattering behavior of a sample in response to incident light. When a sample is exposed to light, its composition and structure reflect and scatter the light, affecting its propagation. After multiple scatterings, these rays constitute the scattered light emitted by the sample.

[0003] However, existing scattering imaging technology cannot image multiple different target objects in a space through incident light scattering, which in turn limits the application scenarios of scattering imaging technology.

[0004] Summary of the Invention

[0005] Based on this, it is necessary to provide an image restoration method, device, computer equipment and storage medium that can image multiple different target objects in a space through incident light scattering to address the above technical problems.

[0006] In a first aspect, the present application provides an image restoration method. The method comprises:

[0007] Acquiring a target speckle image containing at least two objects to be detected;

[0008] determining central autocorrelation data of an object contained in the target speckle image according to the target speckle image;

[0009] Based on a sidelobe recovery network, sidelobe recovery processing is performed on the central autocorrelation data to obtain a target image containing each object to be detected.

[0010] In one embodiment, determining central autocorrelation data of an object contained in the target speckle image according to the target speckle image includes:

[0011] acquiring, according to the target speckle image, a candidate object intensity map of each of the objects to be detected;

[0012] According to the object distances of the objects to be detected, a correlation operation is performed on the candidate object intensity maps of the objects to be detected to obtain the central autocorrelation data.

[0013] In one embodiment, determining central autocorrelation data of an object contained in the target speckle image according to the target speckle image includes:

[0014] performing autocorrelation operations on the candidate object intensity maps of the objects to be detected respectively to obtain first autocorrelation data;

[0015] performing cross-correlation operations on the candidate object intensity maps of the objects to be detected respectively to obtain second autocorrelation data;

[0016] The sum of the first autocorrelation data and the second autocorrelation data is used as the central autocorrelation data.

[0017] In one embodiment, acquiring a target speckle image of at least two objects to be detected includes:

[0018] Acquiring an initial speckle image containing at least two objects to be detected;

[0019] Performing envelope correction processing on the initial speckle image to obtain the target speckle image.

[0020] In one embodiment, performing envelope correction on the initial speckle image to obtain the target speckle image includes:

[0021] Using Zernike polynomials to fit the initial speckle image to obtain a fitting surface;

[0022] The initial speckle image is divided by the fitting surface to obtain the target speckle image.

[0023] In one embodiment, the training process of the sidelobe recovery network includes:

[0024] Acquire a reference image and a standard image of the sample object; wherein the reference image is an image obtained by performing sidelobe recovery processing on sample autocorrelation data of the sample object by an initial recovery network, and the standard image is an image including the sample sidelobe of the sample object;

[0025] Comparing the similarity between the reference image and the standard image to obtain the image similarity between the reference image and the standard image;

[0026] determining an average error margin between the reference image and the standard image based on a reference pixel value of the reference image and a standard pixel value of the standard image;

[0027] According to the image similarity and the average error magnitude, parameters of the initial restoration network are adjusted to obtain the sidelobe restoration network.

[0028] In a second aspect, the present application further provides an image restoration device. The device comprises:

[0029] an acquisition module, configured to acquire a target speckle image containing at least two objects to be detected;

[0030] a determination module, configured to determine central autocorrelation data of an object contained in the target speckle image based on the target speckle image;

[0031] The recovery module is used to perform sidelobe recovery processing on the central autocorrelation data based on a sidelobe recovery network to obtain a target image containing each object to be detected.

[0032] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are performed:

[0033] Acquiring a target speckle image containing at least two objects to be detected;

[0034] determining central autocorrelation data of an object contained in the target speckle image according to the target speckle image;

[0035] Based on a sidelobe recovery network, sidelobe recovery processing is performed on the central autocorrelation data to obtain a target image containing each object to be detected.

[0036] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following steps:

[0037] Acquiring a target speckle image containing at least two objects to be detected;

[0038] determining central autocorrelation data of an object contained in the target speckle image according to the target speckle image;

[0039] Based on a sidelobe recovery network, sidelobe recovery processing is performed on the central autocorrelation data to obtain a target image containing each object to be detected.

[0040] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the following steps:

[0041] Acquiring a target speckle image containing at least two objects to be detected;

[0042] determining central autocorrelation data of an object contained in the target speckle image according to the target speckle image;

[0043] Based on a sidelobe recovery network, sidelobe recovery processing is performed on the central autocorrelation data to obtain a target image containing each object to be detected.

[0044] The above-mentioned image restoration method, apparatus, computer device, and storage medium determine central autocorrelation data of objects contained in the target speckle image through the target speckle image. Then, based on the sidelobe recovery network, sidelobe recovery processing is performed on the central autocorrelation data to obtain a target image containing each object to be detected. In the above process, the present application does not directly obtain the target image of the object to be detected based on the target speckle image; instead, the complete amplitude information of each object to be detected is recovered by determining the central autocorrelation data of the object contained in the target speckle image, and sidelobe recovery processing is performed on each object to be detected. This ensures that the target image containing each object to be detected can be successfully obtained based on the sidelobe recovery network. Therefore, the present application can restore the speckle autocorrelation sidelobes of each object to be detected through the central autocorrelation data, realize imaging of multiple different objects to be detected through incident light scattering in a space, enable the application of scattering imaging technology in imaging application scenarios of multiple objects to be detected, and ensure the accuracy of scattering imaging of multiple objects to be detected. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] FIG1 is a flow chart of an image restoration method according to an embodiment of the present application;

[0046] FIG2 is a schematic diagram of a process for obtaining an initial speckle image according to an embodiment of the present application;

[0047] FIG3 is a schematic diagram of the structure of a sidelobe recovery network provided in an embodiment of the present application;

[0048] FIG4 is a schematic diagram of a process for determining central autocorrelation data according to an embodiment of the present application;

[0049] FIG5 is a schematic diagram of a process for determining a target speckle image according to an embodiment of the present application;

[0050] FIG6 is a flow chart of a sidelobe recovery network training process according to an embodiment of the present application;

[0051] FIG7 is a schematic diagram of a flow chart of another image restoration method provided in an embodiment of the present application;

[0052] FIG8 is a structural block diagram of a first image restoration device provided in an embodiment of the present application;

[0053] FIG9 is a structural block diagram of a second image restoration device provided in an embodiment of the present application;

[0054] FIG10 is a structural block diagram of a third image restoration device provided in an embodiment of the present application;

[0055] FIG11 is a structural block diagram of a fourth image restoration device provided in an embodiment of the present application;

[0056] FIG12 is a structural block diagram of a fourth image restoration device provided in an embodiment of the present application;

[0057] FIG13 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0059] It should be understood that the specific embodiments described herein are merely used to explain the present application and are not intended to limit the present application. In the description of the present application, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they contradict each other.

[0060] In one embodiment, as shown in FIG1 , an image recovery method is provided. This embodiment uses the method applied to a terminal as an example for illustration. It is understandable that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. Among them, the terminal can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server can be implemented as an independent server or a server cluster consisting of multiple servers. In this embodiment, the method includes the following steps:

[0061] S101: Acquire a target speckle image containing at least two objects to be detected.

[0062] It should be noted that the target speckle image refers to the image obtained by performing envelope correction processing on the initial speckle image captured by the target camera; therefore, compared with the initial speckle image, the target speckle image reduces the occlusion of the envelope on the speckle texture, so that a more accurate target image can be obtained based on the target speckle image.

[0063] To further illustrate, the process of acquiring the initial speckle image may specifically include the following: an object is projected onto a lens by a projector to form an image of the object; the image of the object is magnified by the lens, and the magnified image of the object is scattered by a scattering medium to form a scattered image; and the scattered image is captured by a target camera to obtain the initial speckle image.

[0064] In one embodiment of the present application, to further improve the scattering imaging effect of the initial speckle image, as shown in FIG2 , an aperture and an attenuation plate may be added during the scattering imaging process. The aperture can limit the propagation range of the scattered light, control the incident angle and light intensity distribution of the scattered light, and improve the scattering imaging effect. Furthermore, the attenuation plate can adjust the light intensity of different regions in the image of the object, making the light intensity distribution of each region of the object image more uniform, further improving the scattering imaging effect.

[0065] In another embodiment of the present application, in order to reduce the occlusion effect of the envelope on the speckle texture, the present application may use Zernike polynomials to perform envelope correction processing on the initial speckle image to obtain a target speckle image after envelope correction.

[0066] S102: Determine central autocorrelation data of an object contained in the target speckle image according to the target speckle image.

[0067] When performing sidelobe recovery processing on a target speckle image, the size of the OME (Optical Memory Effect) range of each object to be detected in the target speckle image is affected. Furthermore, noise in the target speckle image can also affect the accuracy of sidelobe recovery, thereby affecting the recovery effect. Therefore, the present application uses deep learning to determine the corresponding relationship between the OME ranges of each object to be detected, expand the OME range of each object to be detected, and obtain central autocorrelation data. This allows the target image containing each object to be detected to be successfully acquired based on the central autocorrelation data.

[0068] Specifically, when the field of view of the object to be detected exceeds the OME range, the proposed method needs to be used to customize the field of view. The method may include the following steps: obtaining a candidate object intensity map of each object to be detected based on the target speckle image; and performing a correlation operation on the candidate object intensity map of each object to be detected to obtain central autocorrelation data.

[0069] S103 , performing sidelobe recovery processing on the central autocorrelation data based on a sidelobe recovery network to obtain a target image containing each object to be detected.

[0070] It should be noted that the sidelobe recovery network can be a U-Net++ (densely linked neural network) network. The sidelobe recovery network consists of an encoder and a decoder with dense skip connections. The feature maps in the encoder and decoder are fused through the skip connection, thereby achieving the complementarity of the high-resolution feature map and the low-resolution feature map. The sidelobe recovery network can be shown in Figure 3, where the sidelobe recovery network contains multiple x (i,j) Module, each x (i,j) The modules all contain relu (linear rectification function) activation function, convolution layer and batch normalization layer; and the number of channels of the sidelobe recovery network is completely symmetrically distributed.

[0071] In one embodiment of the present application, the central autocorrelation data can be input into the input end of the sidelobe recovery network terminal to obtain the output result output by the output end of the sidelobe recovery network, and the output result is the target image containing each object to be detected.

[0072] To further illustrate, since the target image contains the complete Fourier amplitude information of each object to be detected, the phase recovery algorithm can be used to perform phase recovery on each object to be detected contained in the target image, thereby reconstructing the shape of each object to be detected in the target image and the relative position of each object to be detected.

[0073] The above-mentioned image restoration method determines the central autocorrelation data of the objects contained in the target speckle image through the target speckle image. Then, based on the sidelobe recovery network, the central autocorrelation data is processed for sidelobe recovery to obtain a target image containing each object to be detected. In the above-mentioned process, the present application does not directly obtain the target image of the object to be detected based on the target speckle image; instead, the complete amplitude information of each object to be detected is recovered by determining the central autocorrelation data of the objects contained in the target speckle image, and the sidelobe recovery processing is performed on each object to be detected. This ensures that the target image containing each object to be detected can be successfully obtained based on the sidelobe recovery network. Therefore, the present application can restore the speckle autocorrelation sidelobes of each object to be detected through the central autocorrelation data, realize the imaging of multiple different objects to be detected through incident light scattering in a space, enable the application of scattering imaging technology in imaging application scenarios of multiple objects to be detected, and ensure the accuracy of scattering imaging of multiple objects to be detected.

[0074] In one embodiment, because existing scatter imaging technology cannot image multiple different target objects in a space through incident light scattering, which limits the application scenarios of scatter imaging technology, to solve the above technical problems, the computer device of this embodiment can determine the central autocorrelation data of the object contained in the target speckle image based on the target speckle image in the manner shown in FIG4, which specifically includes the following steps:

[0075] S401 : Obtaining candidate object intensity maps of respective objects to be detected according to the target speckle image.

[0076] It should be noted that the candidate object intensity map of each object to be detected can be represented by calculation formula (1). Specifically, calculation formula (1) can be as follows:

[0077] Candidate object intensity map = O n (r-△r n ) (1)

[0078] Among them, O n Refers to the nth object to be detected in the target speckle image; r refers to the position of the image center in the target speckle image; △r n Refers to the positional relationship of the nth object to be detected in the target speckle image.

[0079] S402 , performing correlation calculation on each object to be detected according to the object distance of each object to be detected, to obtain central autocorrelation data.

[0080] It should be noted that when it is necessary to determine the central autocorrelation data, the following may be specifically included: performing autocorrelation operations on the candidate object intensity maps of each object to be detected to obtain first autocorrelation data; performing cross-correlation operations on the candidate object intensity maps of each object to be detected to obtain second autocorrelation data; and taking the sum of the first autocorrelation data and the second autocorrelation data as the central autocorrelation data.

[0081] In one embodiment of the present application, taking the scattering system containing two objects to be detected as an example, the above content can be expressed as calculation formula (2). Therefore, the central autocorrelation data can be obtained by substituting the two objects to be detected into calculation formula (2). Calculation formula (2) is as follows:

[0082] in, and Refers to the first autocorrelation data obtained by performing autocorrelation calculation on each object to be detected; and Refers to the second autocorrelation data obtained by performing cross-correlation calculations on each object to be detected. O1 refers to the first object to be detected in the target speckle image; O2 refers to the second object to be detected in the target speckle image; r refers to the position of the image center in the target speckle image; △r1 refers to the position relationship of the first object to be detected in the target speckle image; △r2 refers to the position relationship of the second object to be detected in the target speckle image; Refers to the cross-correlation operation.

[0083] In one embodiment of the present application, in order to ensure that when the target speckle image contains two objects to be detected, the function representation of the target speckle image can be used to perform noise analysis and fitting approximation to obtain calculation formula (2); wherein, the function representation (3) of the target speckle image can be as follows:

[0084] I(r)=O1(r-Δr1)*psf1(r-Δr1)+O2(r-Δr2)*psf2(r-Δr2) (3)

[0085] Among them, O1(r-Δr1) refers to the first object to be detected; O2(r-Δr2) refers to the second object to be detected; psf1(r-Δr1) refers to the point spread function of the field of view where the first object to be detected is located; psf2(r-Δr2) refers to the point spread function of the second object to be detected; O1 refers to the first object to be detected in the target speckle image; O2 refers to the second object to be detected in the target speckle image; r refers to the position of the image center in the target speckle image; △r1 refers to the position relationship of the first object to be detected in the target speckle image; △r2 refers to the position relationship of the second object to be detected in the target speckle image.

[0086] Furthermore, the target speckle image is autocorrelated and the calculation formula (4) is obtained:

[0087] Among them, O1(r-Δr1) refers to the candidate object intensity map of the first object to be detected; O2(r-Δr2) refers to the candidate object intensity map of the second object to be detected; psf1(r-Δr1) refers to the point spread function of the first object to be detected; psf2(r-Δr2) refers to the point spread function of the second object to be detected; O1 refers to the first object to be detected in the target speckle image; O2 refers to the second object to be detected in the target speckle image; r refers to the position of the image center in the target speckle image; △r1 refers to the position relationship of the first object to be detected in the target speckle image; △r2 refers to the position relationship of the second object to be detected in the target speckle image.

[0088] Furthermore, according to the optical memory effect, the point spread functions of the objects to be detected within the same OME range are highly correlated. Moreover, since the point spread functions of the objects to be detected are randomly distributed speckle patterns, the point spread functions of the objects to be detected within the same OME range can be approximated as pulse functions. However, the point spread functions of the objects to be detected within different OME ranges have low correlation. Therefore, the point spread functions of the objects to be detected within different OME ranges can be approximated as background noise. The above content can be expressed by calculation formula (5):

[0089] Among them, psf i (r-Δr i ) refers to the point spread function of the i-th object to be detected; psf j (r-Δr j ) refers to the point spread function of the jth object to be detected; r refers to the position of the image center in the target speckle image; △r i Refers to the position relationship of the i-th object to be detected in the target speckle image; △r j Refers to the positional relationship of the jth object to be detected in the target speckle image.

[0090] Furthermore, formula (4) is simplified by formula (5), and the cross-correlation operation between the point spread functions of different objects to be detected is used as background noise; the cross-correlation operation between the point spread functions of the same object to be detected is used as the impulse function. The candidate formula (6) is obtained, which is as follows:

[0091] in, and Refers to the first autocorrelation data obtained by performing autocorrelation calculation on the candidate object intensity map of each object to be detected; O1 refers to the first object to be detected in the target speckle image; O2 refers to the second object to be detected in the target speckle image; r refers to the position of the image center in the target speckle image; △r1 refers to the position relationship of the first object to be detected in the target speckle image; △r2 refers to the position relationship of the second object to be detected in the target speckle image; refers to the cross-correlation operation; C refers to the background noise in the target speckle image.

[0092] Furthermore, since background noise is introduced into the candidate calculation formula (6), the autocorrelation sidelobes of the objects to be detected do not exist in the candidate calculation formula (6). Therefore, when performing sidelobe recovery processing on each object to be detected, the background noise C in the candidate calculation formula (6) can be used as the cross-correlation information of the annihilated objects in the center. Then, through deep learning, the background noise C corresponding to the object spacing of each object to be detected is learned and analyzed to determine the corresponding relationship of the OME range of each object to be detected. The process of determining the corresponding relationship can be regarded as fitting and approximating the point spread function of the two positions, so that psf2(x2, y2)≈psf1(x1, y1), and then the calculation formula (2) is obtained. Furthermore, the neural network can be trained by using samples with different object spacings to realize customized image restoration operations for multi-scale multi-purpose isolated or continuous fields of view according to the object spacing of each object to be detected.

[0093] The above image restoration method determines the candidate object intensity map of each object to be detected, and determines the central autocorrelation data based on the candidate object intensity map of each object to be detected, thereby restoring the complete amplitude information of each object to be detected and performing sidelobe recovery processing on each object to be detected, ensuring that the target image containing each object to be detected can be successfully obtained based on the sidelobe recovery network.

[0094] In one embodiment, as shown in FIG5 , when it is necessary to obtain a target speckle image containing at least two objects to be detected, the following may be specifically included:

[0095] S501: Acquire an initial speckle image containing at least two objects to be detected.

[0096] In one embodiment of the present application, an aperture, an attenuation plate, a lens, an attenuation plate, and a scattering medium are sequentially arranged between a projector and a target camera, so that the target camera can capture and shoot the scattered image to obtain an initial speckle image.

[0097] Specifically, an object is projected onto an aperture through a projector so that the aperture limits the propagation range of the scattered light. The scattered light passes through the aperture and reaches the attenuation plate. The attenuation plate adjusts the light intensity of the scattered light in different areas. The scattered light passes through the attenuation plate and reaches the lens to form an image of the object. The image of the object is magnified by the lens and then passes through the attenuation plate to reach the scattering medium. The scattering medium scatters the magnified image of the object to form a scattered image. The scattered image is captured by the target camera to obtain the initial speckle image.

[0098] S502: Perform envelope correction on the initial speckle image to obtain a target speckle image.

[0099] It should be noted that, when envelope correction processing is required for the initial speckle image, the following steps may be specifically included: fitting the initial speckle image with Zernike polynomials to obtain a fitting surface; and dividing the initial speckle image by the fitting surface to obtain a target speckle image.

[0100] To further illustrate, before performing envelope correction on the initial speckle image, the initial speckle image can be preprocessed to improve the accuracy of subsequent envelope correction on the initial speckle image. The preprocessing may include, but is not limited to, noise removal, image enhancement, and other steps.

[0101] In one embodiment of the present application, since the Zernike polynomials are a set of basis functions that describe the speckle shape, the Zernike polynomial coefficients can be obtained based on the speckle shape of the initial speckle image. Furthermore, a fitting surface can be obtained based on the Zernike polynomial coefficients. By dividing the initial speckle image by the fitting surface, the shape features described by the Zernike polynomials are subtracted from the initial speckle image, thereby obtaining a target speckle image.

[0102] It is further explained that after obtaining the target speckle image, the target speckle image may be post-processed to achieve the purpose of improving the clarity and image quality of the target speckle image; wherein the post-processing may include but is not limited to: smoothing processing, sharpening processing, etc.

[0103] The above image restoration method reduces the occlusion effect of the envelope on the speckle texture by performing envelope correction processing on the initial speckle image, ensuring that the target image containing each object to be detected can be successfully obtained based on the target speckle image.

[0104] In one embodiment, as shown in FIG6 , the training process of the sidelobe recovery network may specifically include the following:

[0105] S601: Acquire a reference image and a standard image of a sample object.

[0106] The reference image is an image obtained by performing sidelobe recovery processing on the sample autocorrelation data of the sample object by the initial recovery network, and the standard image is an image containing the sample sidelobe of the sample object.

[0107] S602 : performing a similarity comparison between the reference image and the standard image to obtain an image similarity between the reference image and the standard image.

[0108] It should be noted that when comparing the similarity between the reference image and the standard image, the similarity between the reference image and the standard image can be compared from three aspects: image brightness, image contrast, and image structure.

[0109] Further explanation: the similarity between the reference image and the standard image can be compared by calculating formula (7); the calculation formula (7) is as follows:

[0110] Among them, SSIM(x, y) refers to the image similarity between the reference image and the standard image; μ x and μ y Refers to the mean of the measurement items of the reference image and the standard image (the measurement items can be image brightness, image contrast and image structure); σ x and σ y Refers to the standard deviation of the measurement items of the reference image and the standard image; σ xyIt refers to the covariance of the measurement items of the reference image and the standard image; C1 and C2 refer to constant terms.

[0111] S603 , determining an average error margin between the reference image and the standard image according to the reference pixel value of the reference image and the standard pixel value of the standard image.

[0112] It should be noted that the reference pixel value of the reference image and the standard pixel value of the standard image can be substituted into the calculation formula (8), and then the average error amplitude between the reference image and the standard image can be determined by the calculation formula (8). The calculation formula (8) is as follows:

[0113] Among them, MAE(x, y) refers to the average error between the reference image and the standard image; i (x, y) refers to the pixel prediction value of the reference image and the standard image; I` i (x, y) refers to the true pixel value of the reference image and the standard image; n refers to the sum of the pixels of the reference image and the standard image.

[0114] S604: Adjust parameters of the initial restoration network according to the image similarity and the average error magnitude to obtain a sidelobe restoration network.

[0115] It should be noted that, since the average error amplitude between the reference image and the standard image is relatively small, in order to better adjust the parameters of the initial recovery network, the multiple n can be pre-set, and then, according to the average error amplitude between the n-fold reference image and the standard image and the image similarity between the reference image and the standard image, the loss function (9) is determined, and then, according to the loss function (9), the parameters of the initial recovery network are adjusted to obtain the sidelobe recovery network.

[0116] Specifically, the loss function (9) is as follows:

[0117] LOSS=MAE*n+(1-SSIM) (9)

[0118] Here, n refers to a pre-set multiple (for example, n can be 10); MAE refers to the average error between the reference image and the standard image; SSIM refers to the image similarity between the reference image and the standard image.

[0119] It is further explained that in the process of adjusting the parameters of the sidelobe recovery network, it can be divided into at least two training stages, and different training stages adopt different learning rates. Then, according to the training stages with different learning rates, the sidelobe recovery network is subjected to multi-cycle parameter adjustment to obtain the trained sidelobe recovery network.

[0120] For example, the parameter adjustment process of the sidelobe recovery network can be divided into three training stages, namely the first training stage, the second training stage and the third training stage, wherein the first training stage corresponds to the first ten rounds of iterative training of the sidelobe recovery network; the second training stage corresponds to the tenth to thirtieth rounds of iterative training of the sidelobe recovery network; the third training stage corresponds to the thirty-first to fiftieth rounds of iterative training of the sidelobe recovery network; and the learning rate of the first training stage can be 0.001; the learning rate of the second training stage can be 0.0003; and the learning rate of the third training stage can be 0.00003.

[0121] For example, when it is necessary to train the sidelobe recovery network based on sample objects, the sample objects can be divided into three groups. Specifically: the distance between two objects in the first group of sample objects is 2 mm; and the point spread functions of objects that do not belong to the same OME range in the first group are set to have no correlation; the distance between two objects in the second group of sample objects includes: 1 mm, 2 mm and 3 mm; and the point spread functions of objects in the second group with a distance less than 2 mm are set to have correlation; the point spread functions of objects with a distance greater than 1 mm are set to have no correlation; the distance between two objects in the third group of sample objects is 2 mm; and the three sample objects are distributed in a triangular pattern, and a single object is within a memory effect range, and the point spread functions of the three objects at their respective positions have no correlation.

[0122] The above image restoration method adjusts the parameters of the initial restoration network by determining image similarity and average error magnitude, thereby obtaining a sidelobe restoration network. This ensures that the sidelobe restoration network can be used to perform sidelobe restoration on the central autocorrelation data, thereby obtaining a target image containing each object to be detected.

[0123] In one embodiment, as shown in FIG7 , when it is necessary to determine a target image containing each object to be detected, the following may be specifically included:

[0124] S701: Acquire an initial speckle image containing at least two objects to be detected.

[0125] S702 , fitting the initial speckle image using Zernike polynomials to obtain a fitting surface.

[0126] S703: Divide the initial speckle image by the fitting surface to obtain a target speckle image.

[0127] S704 : Determine candidate object intensity maps of respective objects to be detected according to the target speckle image.

[0128] S705 , performing autocorrelation operations on the candidate object intensity maps of the objects to be detected to obtain first autocorrelation data.

[0129] S706 , performing cross-correlation operations on the candidate object intensity maps of the objects to be detected to obtain second autocorrelation data.

[0130] S707: Taking the sum of the first autocorrelation data and the second autocorrelation data as central autocorrelation data.

[0131] S708 , performing sidelobe recovery processing on the central autocorrelation data based on a sidelobe recovery network to obtain a target image containing each object to be detected.

[0132] The above-mentioned image restoration method determines the central autocorrelation data of the objects contained in the target speckle image through the target speckle image. Then, based on the sidelobe recovery network, the central autocorrelation data is processed for sidelobe recovery to obtain a target image containing each object to be detected. In the above-mentioned process, the present application does not directly obtain the target image of the object to be detected based on the target speckle image; instead, the complete amplitude information of each object to be detected is recovered by determining the central autocorrelation data of the objects contained in the target speckle image, and the sidelobe recovery processing is performed on each object to be detected. This ensures that the target image containing each object to be detected can be successfully obtained based on the sidelobe recovery network. Therefore, the present application can restore the speckle autocorrelation sidelobes of each object to be detected through the central autocorrelation data, realize the imaging of multiple different objects to be detected through incident light scattering in a space, enable the application of scattering imaging technology in imaging application scenarios of multiple objects to be detected, and ensure the accuracy of scattering imaging of multiple objects to be detected.

[0133] It should be understood that, although the steps in the flowcharts of the above embodiments are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the flowcharts of the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily to be performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0134] Based on the same inventive concept, embodiments of the present application also provide an image restoration device for implementing the aforementioned image restoration method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more of the following image restoration device embodiments can be found in the aforementioned limitations of the image restoration method and will not be further elaborated here.

[0135] In one embodiment, as shown in FIG8 , an image restoration device is provided, comprising: an acquisition module 10 , a determination module 20 , and a restoration module 30 , wherein:

[0136] The acquisition module 10 is configured to acquire a target speckle image containing at least two objects to be detected.

[0137] The determination module 20 is configured to determine the central autocorrelation data of the object contained in the target speckle image according to the target speckle image.

[0138] The recovery module 30 is used to perform sidelobe recovery processing on the central autocorrelation data based on a sidelobe recovery network to obtain a target image containing each object to be detected.

[0139] The training process of the sidelobe recovery network includes: obtaining a reference image and a standard image of the sample object; wherein the reference image is an image obtained by the initial recovery network performing sidelobe recovery processing on the sample autocorrelation data of the sample object, and the standard image is an image containing the sample sidelobe of the sample object; performing a similarity comparison between the reference image and the standard image to obtain the image similarity between the reference image and the standard image; determining the average error amplitude between the reference image and the standard image based on the reference pixel value of the reference image and the standard pixel value of the standard image; and adjusting the parameters of the initial recovery network based on the image similarity and the average error amplitude to obtain the sidelobe recovery network.

[0140] In one embodiment, as shown in FIG9 , an image restoration device is provided. In the image restoration device, a determination module 20 includes a first determination unit 21 and a second determination unit 22 , wherein:

[0141] The first determining unit 21 is configured to respectively obtain a candidate object intensity map of each object to be detected according to the target speckle image.

[0142] The second determining unit 22 is configured to perform a correlation operation on the candidate object intensity map of each object to be detected according to the object distance of each object to be detected, so as to obtain central autocorrelation data.

[0143] In one embodiment, as shown in FIG10 , an image restoration device is provided. In the image restoration device, the second determining unit 22 includes: a first determining subunit 221 , a second determining subunit 222 , and a third determining subunit 223 , wherein:

[0144] The first determining subunit 221 is configured to perform autocorrelation operations on the candidate object intensity maps of each object to be detected to obtain first autocorrelation data.

[0145] The second determining subunit 222 is configured to perform a cross-correlation operation on the candidate object intensity map of each object to be detected to obtain second autocorrelation data;

[0146] The third determining subunit 223 is configured to use the sum of the first autocorrelation data and the second autocorrelation data as central autocorrelation data.

[0147] In one embodiment, as shown in FIG11 , an image restoration device is provided. The acquisition module 10 in the image restoration device includes an acquisition unit 11 and a correction unit 12 , wherein:

[0148] The acquiring unit 11 is configured to acquire an initial speckle image containing at least two objects to be detected.

[0149] The correction unit 12 is configured to perform envelope correction on the initial speckle image to obtain a target speckle image.

[0150] In one embodiment, as shown in FIG12 , an image restoration device is provided. In the image restoration device, the correction unit 12 includes a fitting subunit 121 and a processing subunit 122 , wherein:

[0151] The fitting subunit 121 is configured to perform fitting processing on the initial speckle image using Zernike polynomials to obtain a fitting surface.

[0152] The processing subunit 122 is configured to perform a division process on the initial speckle image and the fitting surface to obtain a target speckle image.

[0153] Each module in the above-mentioned image restoration device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0154] In one embodiment, a computer device is provided, which may be a terminal. Its internal structure diagram may be as shown in FIG13 . The computer device includes a processor, memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are connected to the system bus via the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The input / output interface of the computer device is configured to exchange information between the processor and an external device. The communication interface of the computer device is configured to communicate with an external terminal via wired or wireless communication, where the wireless communication may be achieved via Wi-Fi, a mobile cellular network, NFC (near field communication), or other technologies. When executed by the processor, the computer program implements an image restoration method. The display unit of the computer device is configured to produce a visually visible image and may be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse.

[0155] Those skilled in the art will understand that the structure shown in FIG13 is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0156] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0157] Acquiring a target speckle image containing at least two objects to be detected;

[0158] determining central autocorrelation data of an object contained in the target speckle image according to the target speckle image;

[0159] Based on the sidelobe recovery network, the central autocorrelation data is processed to obtain a target image containing each object to be detected.

[0160] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0161] According to the target speckle image, a candidate object intensity map of each object to be detected is obtained respectively;

[0162] According to the object distances of the objects to be detected, correlation operations are performed on the candidate object intensity maps of the objects to be detected to obtain central autocorrelation data.

[0163] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0164] performing autocorrelation operations on the candidate object intensity maps of each object to be detected to obtain first autocorrelation data;

[0165] performing cross-correlation operations on the candidate object intensity maps of each object to be detected to obtain second autocorrelation data;

[0166] The sum of the first autocorrelation data and the second autocorrelation data is taken as the central autocorrelation data.

[0167] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0168] Acquiring an initial speckle image containing at least two objects to be detected;

[0169] The initial speckle image is subjected to envelope correction to obtain the target speckle image.

[0170] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0171] The Zernike polynomials are used to fit the initial speckle image to obtain the fitting surface;

[0172] The initial speckle image is divided by the fitting surface to obtain the target speckle image.

[0173] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0174] Acquire a reference image and a standard image of the sample object; wherein the reference image is an image obtained by performing sidelobe recovery processing on the sample autocorrelation data of the sample object by the initial recovery network, and the standard image is an image including the sample sidelobe of the sample object;

[0175] Comparing the similarity between the reference image and the standard image to obtain the image similarity between the reference image and the standard image;

[0176] Determining an average error margin between the reference image and the standard image based on a reference pixel value of the reference image and a standard pixel value of the standard image;

[0177] According to the image similarity and the average error magnitude, the parameters of the initial restoration network are adjusted to obtain the sidelobe restoration network.

[0178] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0179] Acquiring a target speckle image containing at least two objects to be detected;

[0180] determining central autocorrelation data of an object contained in the target speckle image according to the target speckle image;

[0181] Based on the sidelobe recovery network, the central autocorrelation data is processed to obtain a target image containing each object to be detected.

[0182] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0183] According to the target speckle image, a candidate object intensity map of each object to be detected is obtained respectively;

[0184] According to the object distances of the objects to be detected, correlation operations are performed on the objects to be detected to obtain autocorrelation data.

[0185] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0186] performing autocorrelation operations on the candidate object intensity maps of each object to be detected to obtain first autocorrelation data;

[0187] performing cross-correlation operations on the candidate object intensity maps of each object to be detected to obtain second autocorrelation data;

[0188] The sum of the first autocorrelation data and the second autocorrelation data is taken as the central autocorrelation data.

[0189] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0190] Acquiring an initial speckle image containing at least two objects to be detected;

[0191] The initial speckle image is subjected to envelope correction to obtain the target speckle image.

[0192] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0193] The Zernike polynomials are used to fit the initial speckle image to obtain the fitting surface;

[0194] The initial speckle image is divided by the fitting surface to obtain the target speckle image.

[0195] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0196] Acquire a reference image and a standard image of the sample object; wherein the reference image is an image obtained by performing sidelobe recovery processing on the sample autocorrelation data of the sample object by the initial recovery network, and the standard image is an image including the sample sidelobe of the sample object;

[0197] Comparing the similarity between the reference image and the standard image to obtain the image similarity between the reference image and the standard image;

[0198] Determining an average error margin between the reference image and the standard image based on a reference pixel value of the reference image and a standard pixel value of the standard image;

[0199] According to the image similarity and the average error magnitude, the parameters of the initial restoration network are adjusted to obtain the sidelobe restoration network.

[0200] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0201] Acquiring a target speckle image containing at least two objects to be detected;

[0202] determining central autocorrelation data of an object contained in the target speckle image according to the target speckle image;

[0203] Based on the sidelobe recovery network, the central autocorrelation data is processed to obtain a target image containing each object to be detected.

[0204] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0205] According to the target speckle image, a candidate object intensity map of each object to be detected is obtained respectively;

[0206] According to the object distances of the objects to be detected, correlation operations are performed on the candidate object intensity maps of the objects to be detected to obtain central autocorrelation data.

[0207] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0208] performing autocorrelation operations on the candidate object intensity maps of each object to be detected to obtain first autocorrelation data;

[0209] performing cross-correlation operations on the candidate object intensity maps of each object to be detected to obtain second autocorrelation data;

[0210] The sum of the first autocorrelation data and the second autocorrelation data is taken as the central autocorrelation data.

[0211] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0212] Acquiring an initial speckle image containing at least two objects to be detected;

[0213] The initial speckle image is subjected to envelope correction to obtain the target speckle image.

[0214] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0215] The Zernike polynomials are used to fit the initial speckle image to obtain the fitting surface;

[0216] The initial speckle image is divided by the fitting surface to obtain the target speckle image.

[0217] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0218] Acquire a reference image and a standard image of the sample object; wherein the reference image is an image obtained by performing sidelobe recovery processing on the sample autocorrelation data of the sample object by the initial recovery network, and the standard image is an image including the sample sidelobe of the sample object;

[0219] Comparing the similarity between the reference image and the standard image to obtain the image similarity between the reference image and the standard image;

[0220] Determining an average error margin between the reference image and the standard image based on a reference pixel value of the reference image and a standard pixel value of the standard image;

[0221] According to the image similarity and the average error magnitude, the parameters of the initial restoration network are adjusted to obtain the sidelobe restoration network.

[0222] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0223] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0224] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0225] The above embodiments merely illustrate several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art may make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. An image restoration method, characterized in that: The method comprises: Acquire a target speckle image including at least two objects to be detected; Determining central autocorrelation data of an object contained in the target speckle image according to the target speckle image; Based on the sidelobe recovery network, the central autocorrelation data is processed by sidelobe recovery to obtain a target image containing each object to be detected.

2. The method according to claim 1, characterized in that The step of determining central autocorrelation data of an object contained in the target speckle image according to the target speckle image comprises: According to the target speckle image, respectively obtaining a candidate object intensity map of each of the objects to be detected; According to the object distances of the objects to be detected, correlation operations are performed on the candidate object intensity maps of the objects to be detected to obtain the central autocorrelation data.

3. The method according to claim 2, characterized in that The step of determining central autocorrelation data of an object contained in the target speckle image according to the target speckle image comprises: Performing autocorrelation operations on the candidate object intensity maps of the objects to be detected respectively to obtain first autocorrelation data; Performing cross-correlation operations on the candidate object intensity maps of the objects to be detected respectively to obtain second autocorrelation data; The sum of the first autocorrelation data and the second autocorrelation data is used as the central autocorrelation data.

4. The method according to any one of claims 1 to 3, characterized in that: The step of acquiring a target speckle image containing at least two objects to be detected comprises: Acquire an initial speckle image including at least two objects to be detected; The initial speckle image is subjected to envelope correction processing to obtain the target speckle image.

5. The method according to claim 4, characterized in that The performing envelope correction processing on the initial speckle image to obtain the target speckle image includes: Using Zernike polynomials to fit the initial speckle image to obtain a fitting surface; The initial speckle image is divided by the fitting surface to obtain the target speckle image.

6. The method according to any one of claims 1 to 3, characterized in that: The training process of the sidelobe recovery network includes: Acquire a reference image and a standard image of the sample object; wherein the reference image is an image obtained by performing sidelobe recovery processing on sample autocorrelation data of the sample object by an initial recovery network, and the standard image is an image containing sample sidelobes of the sample object; Compare the similarity between the reference image and the standard image to obtain the image of the reference image and the standard image Similarity; Determining an average error margin between the reference image and the standard image according to a reference pixel value of the reference image and a standard pixel value of the standard image; According to the image similarity and the average error amplitude, the parameters of the initial restoration network are adjusted to obtain the sidelobe restoration network.

7. An image restoration device, characterized in that: The device comprises: An acquisition module, used for acquiring a target speckle image containing at least two objects to be detected; A determination module, configured to determine central autocorrelation data of an object contained in the target speckle image according to the target speckle image; The recovery module is used to perform sidelobe recovery processing on the central autocorrelation data based on a sidelobe recovery network to obtain a target image containing each object to be detected.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images