Diffraction image data processing method and device and storage medium

Abstract

The invention provides a diffraction image data processing method and device and a storage medium, and the method comprises the steps: scanning a plurality of scanning points of a sample through an electron beam or a photon beam to obtain a first diffraction image data set, performing an insertion operation of an image corresponding to a non-scanning point between scanning points by using an image insertion model of a pre-constructed spatial domain to increase the number of diffraction images, and obtaining a second diffraction image data set based on the diffraction images after the number is increased, the first diffraction image data comprises a two-dimensional diffraction image corresponding to each scanning point in two-dimensional scanning points formed by a plurality of scanning points in a spatial domain, and the second diffraction image data comprises a two-dimensional diffraction image corresponding to each position point in two-dimensional position points formed by a plurality of scanning points and a plurality of non-scanning points; the diffraction image is a frequency domain image, and each pixel position is a frequency vector of the sample structure; and processing the second diffraction image data to obtain the structural information of the sample in the spatial domain.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image enhancement, and in particular to a diffraction image data processing method and device and storage medium. BACKGROUND

[0002] Transmission electron microscopes, optical microscopes, and microscopes of X-ray, synchrotron light sources are important observation tools in different dimensions for materials and biology. Taking X-ray and transmission electron microscopes as examples, by scanning imaging to record two-dimensional diffraction images at each position, the structural characteristics of the sample can be observed, and the atomic arrangement, defect distribution and interface characteristics inside the material can be revealed, providing key basis for material design, performance optimization and failure analysis.

[0003] In recent years, due to the improvement of the performance of image collectors, such as the improvement of image acquisition speed and linear response range, two-dimensional images at different spatial positions can be collected. In such imaging process, the sample surface is usually continuously scanned with a small converging or under (off) focus spot, and the two-dimensional diffraction image of the back surface of the sample is collected using the image collector, and then algorithms are used for analysis and processing, so that the structural information of the sample in the two-dimensional projection plane or three-dimensional direction can be obtained. Taking the four-dimensional scanning transmission electron microscope (4D-STEM: four-dimensional Transmission Electron Microscopy) technology as an example, for micro-nano level electron microscope samples, the 4D-STEM technology can be used to collect two-dimensional diffraction images of the sample at different sampling positions. Different step lengths and beam spots can be selected for diffraction image collection during imaging, such as micron diffraction, nanometer diffraction and stacked coherent diffraction. The beam spot size and step length of the three kinds of diffraction imaging are from large to small. By further using algorithms to analyze the two-dimensional diffraction images obtained at different sampling positions, the sample structure can be obtained at the micron scale, nanometer scale, or even at the atomic scale of angstrom or sub-angstrom scale. If higher resolution sample structure information is required, the beam spot size needs to be reduced, the step length needs to be reduced, and the number of scanning points needs to be increased. The increase in the number of scanning points will inevitably increase the electron radiation received per unit area, and will reduce the imaging efficiency. However, since the electron beam has a radiation damage effect on the nanoscale sample, if the number of scanning points is increased, the electron radiation received per unit area will inevitably increase, and the sample can withstand the illumination (or radiation dose) is limited, for example, light toxicity, radiation damage, etc. are not conducive to long-term and large-area illumination of the sample. How to improve the sampling efficiency and imaging accuracy without increasing or even effectively reducing the light toxicity and radiation damage is a problem to be solved and currently difficult to solve. SUMMARY

[0004] In view of this, the embodiments of the present application provide a diffraction image data processing method and device to eliminate or improve one or more defects in the prior art.

[0005] One aspect of the present application provides a diffraction image data processing method, comprising the steps of:

[0006] For a first diffraction image data set obtained by scanning a plurality of scanning points of a sample by an electron beam or a photon beam, an image insertion operation of non-scanning point corresponding images between scanning points is performed by using a pre-constructed image insertion model in a spatial domain, so as to increase the number of diffraction images, and a second diffraction image data set is obtained based on the diffraction images after the number is increased, wherein the first diffraction image data comprises a two-dimensional diffraction image corresponding to each scanning point in a two-dimensional scanning point formed by a plurality of scanning points in the spatial domain, the second diffraction image data comprises a two-dimensional diffraction image corresponding to each position point in a two-dimensional position point formed by a plurality of scanning points and a plurality of non-scanning points, the diffraction image is a frequency domain image, and each pixel position is a frequency vector of a sample structure; and the second diffraction image data is processed to obtain structure information of the sample in the spatial domain.

[0007] In some embodiments of the present application, the image insertion operation of non-scanning point corresponding images between scanning points by using the pre-constructed image insertion model in the spatial domain to increase the number of diffraction images comprises: the image insertion operation of non-scanning point corresponding images between scanning points is performed by using the pre-constructed image insertion model in the spatial domain according to a predetermined scanning point path or a two-dimensional correlation of scanning points in the spatial domain, so as to increase the number of diffraction images.

[0008] In some embodiments of the present application, the image insertion operation of non-scanning point corresponding images between scanning points by using the pre-constructed image insertion model in the spatial domain according to a predetermined scanning point path or a two-dimensional correlation of scanning points in the spatial domain to increase the number of diffraction images comprises: an inter-frame operation of diffraction images in the first diffraction image data between scanning points is performed according to one or more predetermined one-dimensional paths or a two-dimensional correlation of scanning points in the spatial domain among the two-dimensional scanning points in the spatial domain, so as to obtain a plurality of two-dimensional diffraction images corresponding to the two-dimensional position points including the scanning points and the non-scanning points.

[0009] In some embodiments of the present application, the method for increasing the number of diffraction images by inserting images between non-scan points according to a predetermined scan point path or a two-dimensional correlation of scan points in the spatial domain includes: converting the two-dimensional diffraction images in the frequency domain of the first diffraction image data into images in the spatial domain by using Fourier or inverse Fourier transform operation; performing an interpolation operation on the images in the spatial domain between scan points according to one or more predetermined one-dimensional paths or a two-dimensional correlation of scan points in the spatial domain, to obtain interpolated spatial domain image data; and performing an operation opposite to the Fourier or inverse Fourier transform operation on the spatial domain image data to obtain a plurality of two-dimensional diffraction images corresponding to two-dimensional position points including scan points and non-scan points.

[0010] In some embodiments of the present application, the one or more one-dimensional paths include: a snake shape, a zigzag shape, or a spiral shape; and the two-dimensional correlation of scan points in the spatial domain includes: a cross shape or a hoshi shape correlation

[0011] In some embodiments of the present application, the interpolation method is used to perform an interpolation operation between adjacent images to be interpolated; or the adjacent images are input into a pre-trained interpolation model based on machine learning or deep learning, so as to output an interpolated image corresponding to a non-scan point between scan points. The pre-trained interpolation model is trained in the following manner: taking experimental images corresponding to a plurality of scan points of a sample in one scanning direction as a training set, taking the L+1th and L-1th experimental images as input, and taking the Lth experimental image as real annotation data to train the interpolation model; or taking experimental images of a plurality of continuous scan points of a sample in accordance with the two-dimensional correlation as a training set, taking an image of an associated scan point of a scan point corresponding to an interpolation position in the plurality of scan points as an input image, and taking an image of the scan point corresponding to the interpolation position as real annotation data to train the interpolation model; wherein the experimental image is an experimental frequency domain diffraction image or a corresponding spatial domain image obtained by conversion.

[0012] In some embodiments of the present application, the method further includes: before performing the interpolation operation, pre-processing a plurality of two-dimensional diffraction images corresponding to a plurality of scan points in the first diffraction image data to reduce the numerical range of a single two-dimensional diffraction image; and after performing the interpolation operation, performing an operation opposite to the reduction of the numerical range of a single two-dimensional diffraction image on a plurality of two-dimensional diffraction images corresponding to two-dimensional position points, to restore the numerical space of the two-dimensional diffraction images, thereby obtaining the second four-dimensional diffraction image data.

[0013] In some embodiments of the present application, the method further comprises: before or after the interpolation operation, based on the sample structure information, iteratively optimizing the obtained two-dimensional diffraction image corresponding to the non-scanning point using the imaging principle of the scanning diffraction image.

[0014] In some embodiments of the present application, the image insertion operation between scanning points using the pre-constructed spatial domain image insertion model to increase the number of diffraction images comprises: extracting image intensity-position information of a specific frequency from the first diffraction image dataset to obtain image data reflecting the sample structure; increasing the number of pixels of each image in the obtained image data using a super-resolution algorithm to obtain an expanded image; corresponding the intensity of the expanded image to the specific frequency of the two-dimensional diffraction image, wherein the intensity of the increased pixels is inserted into the specific frequency of the new diffraction image; traversing all frequencies to correspond the intensity of the expanded image to the traversed frequencies of the two-dimensional diffraction image to fill each pixel intensity of the new diffraction image, thereby obtaining the expanded diffraction image data; or

[0015] The image insertion operation between scanning points using the pre-constructed spatial domain image insertion model to increase the number of diffraction images comprises: a transformation operation: applying a forward Fourier transform or an inverse Fourier transform to the diffraction image of a specific position coordinate in the first diffraction image dataset to obtain a complex image in the spatial domain, and extracting the image intensity of a specific frequency from the complex image in the spatial domain and corresponding the intensity to the specific position coordinate; performing the transformation operation on the diffraction image of other all position coordinates in the first diffraction image dataset to obtain image data reflecting the sample structure; increasing the number of pixels of each image in the obtained image data using a super-resolution algorithm to obtain an expanded image reflecting the sample structure; corresponding each pixel of the intensity of the expanded image reflecting the sample structure to the corresponding frequency of the two-dimensional diffraction image, wherein the intensity of the increased pixels is inserted into the specific frequency of the new diffraction image; traversing all coordinates to fill each pixel intensity of the new diffraction image to finally obtain the expanded two-dimensional complex image data; performing a transformation opposite to the forward Fourier transform or the inverse Fourier transform on the two-dimensional complex image data to obtain the expanded frequency domain diffraction image data.

[0016] In embodiments of the present application, the super-resolution algorithm is an interpolation algorithm or a machine learning network or a deep network learning network.

[0017] Another aspect of the present application provides a diffraction image data processing device, which comprises a processor, a memory and a computer program / instruction stored on the memory, wherein the processor is configured to execute the computer program / instruction, and when the computer program / instruction is executed, the device implements the steps of the method as described above.

[0018] Another aspect of the present application provides a computer readable storage medium having stored thereon computer programs / instructions, characterized in that the computer programs / instructions, when executed by a processor, implement the steps of the method as described above.

[0019] The diffraction image data processing method and device of the present application, for the two-dimensional diffraction images collected at different positions in the two-dimensional space of the sample, performs the frame interpolation operation in space to increase the number of diffraction images, and then restores the sample structure based on the increased diffraction images, analyzes the image to analyze the sample structure, which can greatly improve the image information resolution of the restored object real image space sample structure, thereby improving the image collection efficiency and reducing the radiation dose of the sample in the image collection stage.

[0020] Additional advantages, objects, and features of the application will be set forth in part in the description which follows, and in part will become apparent to those having ordinary skill in the art upon examination of the following or can be learned from practice of the application. The objects and other advantages of the application can be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

[0021] It will be understood by those within the art that the objects and advantages of the application can be met by other embodiments not specifically described herein. Those skilled in the art will recognize, or be able to ascertain using no more than routine experimentation, the met by other embodiments not specifically described herein. BRIEF DESCRIPTION OF DRAWINGS

[0022] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description, serve to explain the principles of the application.

[0023] Figure 1 Flowchart of the diffraction image data processing method in an embodiment of the present application.

[0024] Figure 2 Schematic diagram of forming four-dimensional scanning diffraction imaging in an embodiment of the present application.

[0025] Figure 3 Schematic diagram of the frame interpolation path in an embodiment of the present application.

[0026] Figure 4 Schematic diagram of using super-resolution algorithm to perform frequency domain image expansion in an embodiment of the present application.

[0027] Figure 5 Schematic diagram of using super-resolution algorithm to perform spatial domain image expansion in an embodiment of the present application.

[0028] Figure 6 Example of integrating part of the diffraction intensity in the diffraction image to obtain the sample structure graph in the spatial domain.

[0029] Figure 7 For an embodiment of the present application, the effect of the obtained spatial domain sample structure after the insertion of the diffraction image is compared with the spatial domain sample structure obtained by insertion. DETAILED DESCRIPTION

[0030] In order to make the objectives, technical solutions and advantages of the present application clearer, further detailed description will be given below in combination with embodiments and drawings. Herein, the illustrative embodiments of the present application and their descriptions are used to explain the present application, but not as a limitation of the present application.

[0031] It should also be noted that, in order to avoid obscuring the present application due to unnecessary details, only structures and / or processing steps closely related to the solutions according to the present application are shown in the drawings, and other details not closely related to the present application are omitted.

[0032] It should be emphasized that the term "comprises / comprising" as used herein is used to indicate the presence of a feature, element, step or component, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0033] In order to improve the acquisition efficiency of four-dimensional scanning diffraction images, reduce the irradiation dose of the sample from the incident beam, and improve the resolution of the spatial domain structure information of the sample, the present application provides a method and device for processing diffraction image data, which inserts two-dimensional diffraction images of non-scanning points between scanning points by performing an insertion operation of images corresponding to non-scanning points between scanning points in the spatial domain between a series of scanning positions, thereby increasing the number of diffraction images. This method can be used to improve the image acquisition efficiency, and can improve the resolution of the recovered sample structure without increasing or even effectively reducing the photo toxicity and irradiation damage, so that the super-resolution analysis goal of the sample structure can be achieved.

[0034] Figure 1 For an embodiment of the present application, a flowchart of a diffraction image data processing method is shown in FIG. 1, which includes the following steps S110 and S120: Figure 1

[0035] Step S110, for a first diffraction image data set (also referred to as a first four-dimensional diffraction image data set) obtained by scanning a plurality of scanning points of a sample by an electron beam or a photon beam, an insertion operation of images corresponding to non-scanning points between scanning points is performed by using a pre-constructed image insertion model in the spatial domain to increase the number of diffraction images, and a second diffraction image data set (also referred to as a second four-dimensional diffraction image data set) is obtained based on the diffraction images after the number is increased.

[0036] ​The method of the present invention is applicable not only to optical diffraction images collected when scanning samples using converging photon beams, X-rays, synchrotron radiation sources, etc., but also to diffraction images obtained when scanning samples using electron beams.

[0037] Scanning is performed at different locations in the two-dimensional space of the sample. Each scanning point acquires a two-dimensional diffraction image. Thus, the data corresponding to different scanning positions in the entire two-dimensional space of the sample contains four dimensions. The resulting scanned image can be represented in four dimensions, hence it can be defined as a four-dimensional scanned image, or a four-dimensional diffraction image. Each diffraction image is collected in the frequency domain, and each pixel position represents the frequency vector of the sample structure. For example, a 4D-STEM image is a four-dimensional diffraction image acquired using an electron microscope pixel array detector. Here, "four-dimensional" means that each two-dimensional scanning position (scanning point) corresponds to one two-dimensional diffraction image, and the data has a four-dimensional structure.

[0038] By using a converging light beam, X-rays, synchrotron radiation source, or electron beam to move across the sample surface and scan the sample, two-dimensional diffraction images collected at different scanning positions of the sample can be represented as I(u,v,x). m ,y n ), which is in four-dimensional form, where (u,v) is the frequency vector on the two-dimensional diffraction pattern, (x m ,y n ) is the position coordinate of the scanning point when scanning the sample. It can be directly used to represent the scanning point. Here, m and n represent the number of rows and columns where the scanning point is located, respectively. m and n are integers, with values ​​ranging from [1,M] and [1,N], respectively. M and N represent the total number of rows and columns of the scanning point, respectively. Figure 2 This is a schematic diagram of four-dimensional scanning diffraction imaging. Figure 2 In this process, the sample is located in the "spatial domain," and the diffraction image records the intensity of the light or electron beam in the frequency domain. Figure 2 The data records M×N two-dimensional diffraction images corresponding to M×N scanning points.

[0039] In this embodiment of the invention, the first four-dimensional diffraction image data includes the two-dimensional diffraction image corresponding to each scanning point among multiple scanning points in the spatial domain; that is, the first four-dimensional diffraction image data is the four-dimensional diffraction image data before the frame interpolation operation. The second four-dimensional diffraction image data includes the two-dimensional diffraction image corresponding to each position point among multiple scanning points and multiple non-scanning points; that is, the second four-dimensional diffraction image data is the entire four-dimensional diffraction image data after the frame interpolation operation. The two-dimensional diffraction images at different positions, including scanning points and non-scanning points, after frame interpolation can be represented as I(u,v,x). m’ ,y nwherein m' and n' represent the row number and column number of each position corresponding to the diffraction image, m' and n' are integers, and the value ranges of m' and n' are [1, M'] and [1, N'] respectively, M' and N' represent the total row number and total column number of the scanning points and non-scanning points corresponding to the diffraction image. Figure 2 The two-dimensional diffraction image corresponding to the newly added non-scanning point between two adjacent scanning points through the spatial interpolation operation is also shown in FIG. 4, as shown by the green box in FIG. 4. Figure 2

[0040] In the present application, the interpolation algorithm used in the spatial domain is referred to as a spatial interpolation algorithm. Here, "frame" refers to a series of spatially related diffraction images. Therefore, the essential meaning of "interpolation" in the present application is the operation of adding new position points (non-scanning points) between scanning points in the spatial domain of the sample and obtaining the corresponding two-dimensional diffraction images. Therefore, it needs to be emphasized that the "interpolation" concept proposed in the present application is not the same as the current traditional video interpolation concept. In traditional video technology, "frame" refers to a series of time-continuous real images. In addition, the spatial interpolation algorithm of the present application also has the following main differences from the traditional video interpolation algorithm: (1) The interpolation algorithm in the traditional video field uses the motion continuity of the object in the time dimension for interpolation; however, the spatial domain interpolation algorithm of the present application is based on the continuity between two-dimensional diffraction images at different positions during scanning of the sample. Although there is a time difference in the collection process, the difference between two-dimensional diffraction images comes from the difference in scanning spatial position, rather than the difference in time. (2) The interpolation algorithm in the traditional video field is based on a relatively stable numerical value range of the image, usually a natural color (or black and white) image with a numerical value range of 0 to 1 or 0 to 255; while the present application is based on diffraction images, and the numerical value range of each image can differ by several orders of magnitude. (3) The image after interpolation in the traditional video field shows the newly added image in the time domain, which can directly describe the motion change of the object; the algorithm of the present application is based on two-dimensional interpolation, which needs to refer to image information from two dimensions and adjacent positions, and after completing the spatial domain interpolation algorithm, the present application still obtains diffraction images at different spatial positions, which need to be analyzed through specific information to show the structural information of the sample to be tested. Therefore, the spatial interpolation of two-dimensional diffraction images in the present application is significantly different from the interpolation in the traditional video field in terms of both the operation object and the interpolation method.

[0041] ​In some embodiments of the present application, the step S110 of using the pre-constructed image insertion model in the spatial domain to perform the image insertion operation between the scanning points corresponding to the non-scanning points to increase the number of diffraction images can include: using the pre-constructed image insertion model in the spatial domain (i.e., the spatial domain interpolation algorithm for two-dimensional diffraction images) to perform the image insertion operation between the scanning points corresponding to the non-scanning points according to the predetermined scanning point path or the two-dimensional correlation of the scanning points in the spatial domain to increase the number of diffraction images. As an example, the predetermined scanning point path can be one or more predetermined one-dimensional scanning point paths, which can be a snake shape, a zigzag shape, or a spiral shape, etc. As an example, the two-dimensional correlation of the scanning points in the spatial domain can be a cross shape or a pinwheel shape correlation formed between the scanning points, etc., but the present application is not limited thereto.

[0042] The spatial domain interpolation algorithm for two-dimensional diffraction images can be an interpolation method, etc., or a pre-trained interpolation model based on machine learning or deep learning.

[0043] In the case of using the interpolation method, interpolation can be performed based on the pixels (frequency vectors) of the diffraction images corresponding to the adjacent scanning points to obtain the newly added diffraction images corresponding to the non-scanning points.

[0044] In the case of using the spatial domain interpolation algorithm as a pre-trained interpolation model based on machine learning or deep learning, the images corresponding to the adjacent scanning points are input into the pre-trained interpolation model based on machine learning or deep learning, and the images (intermediate images) corresponding to the non-scanning points between the scanning points are output. As an example, the pre-trained interpolation model can be trained in the following way:

[0045] 1) In the case of performing the image insertion operation according to the predetermined scanning point path, taking the one-dimensional scanning point path (such as a snake shape or a zigzag shape, etc.) as an example, preferably: converting the scanning points in the spatial domain into one-dimensional sequential scanning points, using the multiple experimental images (two-dimensional diffraction images) corresponding to the multiple adjacent scanning points in the selected one-dimensional path direction of the sample as the training set, using the previous experimental image (L-1th) and the next experimental image (L+1th) in the scanning direction of the Lth experimental image as the input, and using the Lth experimental image as the true annotation data to train the interpolation model. As an example, the training interpolation model is a convolutional neural network, an IFNet network, or a neural network based on a generative adversarial network framework, but the present application is not limited thereto.

[0046] In this way, during the interpolation operation, using the trained network and the diffraction images of the adjacent two positions (scanning points), the diffraction images of the intermediate one or several positions (non-scanning points) can be generated.

[0047] Along the predetermined scanning point path, the adjacent scanning points requiring the interpolation operation are traversed, and the interpolation operation is repeatedly performed to obtain as many two-dimensional diffraction images corresponding to the non-scanning points as possible.

[0048] 2) In the image interpolation operation according to the two-dimensional correlation of the scanning points in the spatial domain, the scanning points have two-dimensional correlation when the incident beam moves on the sample surface. Taking the "cross" type and the "rice" type as examples, preferably:

[0049] For the "cross" type, the experimental images of the continuous 5 scanning points of the sample in the "cross" type relationship (such as the positional correlation of a certain scanning point with the 4 adjacent scanning points above, below, left and right) are taken as the training set, wherein the images of the scanning points above, below, left and right are the input images, and the image of the "cross" intersection point (the image corresponding to the "certain scanning point") is taken as the true labeled data to train the interpolation model.

[0050] For the "rice" type, the experimental images of the continuous 9 scanning points of the sample in the "rice" type relationship (such as the positional correlation of a certain scanning point with the 8 adjacent scanning points of the upper left, upper, upper right, left, right, lower left, lower and lower right) are taken as the training set, wherein the images of the scanning points of the upper left, upper, upper right, left, right, lower left, lower and lower right are the input images, and the image of the "rice" intersection point (the image corresponding to the "certain scanning point") is taken as the true labeled data to train the interpolation model.

[0051] As an example, the trained interpolation model is a convolutional neural network, an IFNet network or a neural network based on a generative adversarial network framework, but the present application is not limited thereto.

[0052] In this way, in the interpolation operation, the trained network is used to generate the image of the intermediate frame (intermediate position) using the images of the adjacent four positions.

[0053] According to the two-dimensional correlation in the spatial domain, all adjacent scanning points requiring the interpolation operation are traversed, and the interpolation operation is repeatedly performed to obtain as many two-dimensional diffraction images corresponding to the non-scanning points as possible.

[0054] After the interpolation operation, the two-dimensional diffraction images corresponding to all two-dimensional position points including the plurality of scanning points and the plurality of non-scanning points form a diffraction image data set applied to restore the sample structure.

[0055] In some embodiments of the present application, the step of increasing the number of diffraction images by using the pre-constructed image insertion model of spatial domain to perform the image insertion operation between the scanning points corresponding to the non-scanning points can be implemented in the following way: first, obtain the image data of spatial domain related to the sample structure based on the diffraction images, and then increase the number of image pixels by using the super-resolution algorithm based on the image data of spatial domain related to the sample structure, thereby increasing the number of diffraction images. In this way, the super-resolution algorithm is not applied to the diffraction images, but to the sample structure images of spatial domain. This method can be referred to as the super-resolution algorithm method.

[0056] More specifically, based on the super-resolution algorithm method, in some embodiments of the present application, the step of increasing the number of diffraction images by using the pre-constructed image insertion model of spatial domain to perform the image insertion operation between the scanning points corresponding to the non-scanning points can include: obtaining the image data of spatial domain related to the sample structure based on the intensity-position information of each frequency in the image of four-dimensional diffraction data; increasing the number of pixels of each image in the obtained image data by using the super-resolution algorithm to obtain an expanded image; and corresponding the intensity of the expanded image to the traversed frequencies of the two-dimensional diffraction image to obtain the expanded diffraction image data.

[0057] Based on the super-resolution algorithm method, in some other embodiments of the present application, the step of increasing the number of diffraction images by using the pre-constructed image insertion model of spatial domain to perform the image insertion operation between the scanning points corresponding to the non-scanning points can include: obtaining the image data of spatial domain related to the sample structure based on the intensity-position information of each frequency in the image of spatial domain; increasing the number of pixels of each image in the obtained image data by using the super-resolution algorithm to obtain an expanded image; corresponding the intensity of the expanded image to the traversed frequencies of the diffraction image to obtain the expanded complex image data; and performing the inverse transform of the forward Fourier transform or the inverse Fourier transform on the complex image data to obtain the expanded frequency domain four-dimensional diffraction image data.

[0058] The implementation modes will be described in more details later. In the embodiments of the present application, the super-resolution algorithm is not a specific algorithm, but refers to a possible algorithm capable of expanding the size of the spatial domain image, including existing algorithms. The super-resolution algorithm may, for example, be an interpolation algorithm, or be implemented through machine learning or deep network learning. In the case where the super-resolution algorithm is an interpolation algorithm, the intensity of the increased pixels is calculated using the interpolation algorithm, and the image on the sample structure space corresponding to the scanning point is used to display the sample structure image with denser pixels. Before and after the super-resolution processing, the size of each pixel changes from large to small, and the number of pixels of the image changes from less to more. If implemented through machine learning or deep network learning, the training data may, for example, be the Fourier transform image of the diffraction pattern of the scanning point, and the network model used may, for example, be a deep learning-based super-resolution algorithm network such as Realesrgan or swinir.

[0059] In step S120, the four-dimensional diffraction image data is processed to obtain the structure information of the sample in the spatial domain.

[0060] Generally, according to the prior art, after obtaining the first four-dimensional diffraction image data, the sample structure is further analyzed based on the four-dimensional diffraction image data. For example, the intensity integral information or position information of each two-dimensional diffraction image in a specific frequency range can be extracted, or part of the information signal or intensity position information can be extracted after being transformed to the spatial domain by using (inverse) Fourier transform, or the reconstruction algorithm of the stacked diffraction coherent imaging can be used, and so on, so that the structure information of the sample in the spatial domain can be obtained, that is, the sample structure recovery can be realized. For example, for the four-dimensional scanning electron diffraction image, the low-frequency signal of the diffraction image is extracted and integrated, and the bright-field scanning electron micrograph can be obtained, that is, the sample structure imaging information can be obtained.

[0061] In the embodiments of the present application, the difference from the prior art is only that the object of processing is different, that is, the second four-dimensional diffraction image data is processed, which includes the two-dimensional diffraction image corresponding to each position point formed by the plurality of scanning points and the plurality of non-scanning points. However, the processing mode can be the same as that of processing the first four-dimensional diffraction image data using the prior art, and the non-scanning points are treated as scanning points, so that the sample structure imaging information with higher resolution can be obtained. Since the processing mode can use the prior art, it will not be described here.

[0062] The diffraction image data processing method of the present application is used for the two-dimensional diffraction images collected at different positions in the two-dimensional space of the sample, performs the frame interpolation operation in space to increase the number of diffraction patterns, and then performs the recovery of the sample structure based on the increased diffraction images, analyzes the image to analyze the sample structure, which can greatly improve the image information resolution of the recovered object real image space sample structure, so that the image collection efficiency can be improved and the radiation dose of the sample can be reduced in the image collection stage.

[0063] The spatial domain interpolation algorithm provided by the application aims to make full use of the diffraction image information of the scanning points, further increase the number of diffraction images, and thus improve the diffraction image quality of the sample structure, stress and / or density and the like for analysis. The application realizes spatial domain interpolation by using a calculation method, obtains a larger number of diffraction images, and excavates potential structure information as much as possible, which has important practical significance and scientific research value.

[0064] The application mainly aims to solve three problems: first, the structure of some materials has strict requirements on the energy dose of the incident beam, and the application can effectively reduce the irradiation of the sample; second, the application generates diffraction images between adjacent scanning positions based on the spatial continuity of the scanning points, and reduces the acquisition time; third, the increased diffraction images improve the details of the analyzed or restored sample structure. By using the method of the application, the effect of the restored image can be significantly improved, the image acquisition efficiency is improved, the sample structure is protected from being damaged by the incident beam, and the method has important significance for microstructure analysis.

[0065] In some embodiments of the application, the step of increasing the number of diffraction images by using a pre-constructed spatial domain image interpolation model to perform an interpolation operation of non-scanning point corresponding images between scanning points according to a predetermined scanning point path or a two-dimensional correlation of scanning points in the spatial domain can include:

[0066] According to one or more predetermined one-dimensional paths or the two-dimensional correlation of scanning points in the spatial domain, the interpolation operation is performed on the diffraction images in the first diffraction image data between the scanning points, and a plurality of two-dimensional diffraction images corresponding to the two-dimensional position points including the scanning points and the non-scanning points are obtained.

[0067] For example, referring to the 4D-STEM, the schematic diagram of the scanning point path (interpolation path) of an embodiment of the application is shown in Figure 3 The interpolation path shown in Figure 3 is a snake-shaped one-dimensional scanning point path, Figure 3 (a) of is a schematic diagram of the two-dimensional diffraction images corresponding to each two-dimensional scanning point based on the original four-dimensional diffraction scanning data, Figure 3 (b) of is a schematic diagram of the two-dimensional diffraction images corresponding to each position based on the four-dimensional diffraction scanning data obtained after interpolation containing scanning points and non-scanning points by using the spatial domain interpolation algorithm, the interpolation path direction shown by the arrow in (a) is the first interpolation path direction, the interpolation path direction shown by the arrow in (b) is the second interpolation path for continuing to supplement the intermediate non-scanning point position, and the blank area is the area without forming the two-dimensional diffraction image. For example, Figure 3In the case of the illustrated interpolation path, the diffraction pattern data collected by the experiment is used to obtain the diffraction pattern of the non-scan point by the spatial domain interpolation algorithm as follows:

[0068] First, the two-dimensional scan points in the spatial domain are converted into one-dimensional sequential scan points according to the set interpolation path, and then interpolation is performed along the one-dimensional direction to obtain the diffraction pattern of the non-scan point. For example, first, along the x direction, between the adjacent scan points with x m and x m+1 , a two-dimensional diffraction pattern is added for each y n coordinate, and the interpolated diffraction pattern can be represented as I(u, v, x m’ , y n ), where m' is an integer, and the value range is [1, M'] and M'>M; then, along the y direction, between the adjacent positions with y n and y n+1 , a two-dimensional diffraction pattern is added for each x m’ coordinate, and the interpolated diffraction pattern can be represented as I(u, v, x m’ , y n '), where n' is an integer, and the value range is [1, N'] and N'>N. The specific interpolation path is not limited, and in addition to the snake type, the "Z" type or spiral type can also be used. In addition, since the two-dimensional scan points in the spatial domain also have two-dimensional correlation, the interpolation method can also be determined by the "cross" or "meter" type correlation between the scan points. More diffraction pattern data I(u, v, x m’ , y n ') is obtained by using the spatial domain interpolation algorithm, and the value ranges are [1, M'] and [1, N'], respectively.

[0069] As described above, the two-dimensional diffraction pattern between the adjacent positions is directly interpolated to obtain the newly added diffraction pattern. In an alternative embodiment of the present application, the two-dimensional diffraction pattern in the frequency domain corresponding to each position can be first converted into a complex image in the spatial domain by using Fourier transform or inverse Fourier transform, and then the real image and the imaginary image (or the amplitude image and the phase image) of the complex image are interpolated to obtain the interpolated image. Then, the newly added non-scan point image is transformed from the spatial domain to the frequency domain by using the inverse transform operation of the Fourier transform, to obtain the newly added non-scan point diffraction pattern.

[0070] In this case, the number of diffraction patterns is increased by using a pre-constructed image insertion model in the spatial domain to perform an insertion operation of the image corresponding to the non-scan point between the scan points according to the predetermined scan point path or the two-dimensional correlation of the scan points in the spatial domain, which can include:

[0071] The frequency domain diffraction image in the first diffraction image data is converted into a spatial domain image using Fourier transform or inverse Fourier transform operations.

[0072] In the two-dimensional scanning points of the spatial domain, frame interpolation is performed between the scanning points according to one or more predetermined one-dimensional paths or the two-dimensional relationship between the scanning points in the spatial domain, to obtain the framed image data of the spatial domain.

[0073] The image data in the spatial domain is subjected to the opposite operation to the Fourier or inverse Fourier transform operation to obtain multiple two-dimensional diffraction images corresponding to two-dimensional position points, including scanned points and non-scanned points.

[0074] If a super-resolution algorithm is used to augment the diffraction image data, then in one embodiment, such as Figure 4 As shown, the diffraction image data augmentation operation (performing an image insertion operation to increase the number of diffraction images) includes the following steps:

[0075] 1) Extract image intensity location information at specific frequencies from the diffraction image dataset to obtain image data related to the spatial domain structure of the sample.

[0076] The diffraction image dataset can be either raw, uninterpolated diffraction images or preprocessed, uninterpolated diffraction images.

[0077] For example, the diffraction pattern in the first diffraction image set located at the scanning point (x0, y0) is written as I(u, v, x0, y0). The intensity at a specific frequency (u0, v0) is extracted from this pattern, and this intensity is assigned to the image O reflecting the sample structure at (x0, y0). [u0,v0] (x0,y0). Figure 4 In the diagram, the selected frequency (u0, v0) is located at position [92, 59] of the diffraction image spectrum.

[0078] Then, by iterating through all diffraction patterns in the first diffraction image dataset at this specific frequency, the response (x) is obtained. m ,y n Image of the sample structure at point O [u0,v0] (x m ,y n If m = 0, 1, 2, 3, ..., M-1 and n = 0, 1, 2, 3, ..., N-1, then the size of the image reflecting the sample structure is M×N.

[0079] 2) Use super-resolution algorithms to increase the number of pixels in each image in the obtained image data to obtain an enlarged image.

[0080] For example, in the case of using super-resolution algorithm for the image O [u0,v0] (x m ,y n ) reflecting the sample structure based on the original diffraction image data set: for the image O [u0,v0] (x m ,y n ), the size of the image is increased by increasing the number of pixels of the image using the super-resolution algorithm, so as to increase the details of the image and improve the resolution of the image. After increasing the details, the image can be represented as O' [u0,v0] (x m′ ,y n′ ), and the size becomes M'xN'; the super-resolution algorithm can increase the details of the image. For example, in the case of not using the super-resolution algorithm, the originally obtained image reflecting the sample structure has a size of 50*50, and if 1 new non-scanning point is inserted between the scanning points, then after the super-resolution algorithm processing, an image with a size of 99*99 can be obtained.

[0081] 3) Corresponding the intensity of the expanded image reflecting the sample structure to the traversed frequencies of the two-dimensional diffraction image, to obtain the expanded diffraction image data.

[0082] More specifically, the intensity of the expanded image O' [u0,v0] (x m′ ,y n′ ) reflecting the sample structure is corresponded to the specific frequency (u0, v0) of the two-dimensional diffraction image, and the intensity of the increased pixel is inserted into the (u0, v0) frequency of the new diffraction image I(u, v, x m′ ,y n′ ); then all frequencies are traversed, and each pixel intensity of the new diffraction image is filled, and finally the number of the expanded diffraction image data is obtained.

[0083] If the expansion is based on the MxN original diffraction image data, the number of the diffraction image obtained after the expansion is M'xN' as described above.

[0084] In the above steps of diffraction image data expansion, the coordinates of the image reflecting the sample structure are actually pixel position coordinates, and after calculation from physical quantities, they actually correspond to the coordinates of the frequency vector.

[0085] If the super-resolution algorithm is used to expand the diffraction image data, in another embodiment, as shown in Figure 5 , the diffraction image data can also be expanded based on the super-resolution algorithm by the following steps:

[0086] 1) Transform operation: applying a forward Fourier transform or inverse Fourier transform to the diffraction pattern of a specific position coordinate in the first diffraction image data set, obtaining a complex image in spatial domain, and obtaining image data reflecting the sample structure based on the intensity-position information of each frequency in the image in spatial domain.

[0087] For example, the diffraction pattern of the scanning point at (x0, y0) in the first diffraction image set is written as I(u, v, x0, y0), and a forward Fourier transform or inverse Fourier transform is applied to the pattern to obtain a complex image J(u', v', x0, y0) in spatial domain. The intensity of a specific coordinate (u0', v0') is extracted from the complex image J(u', v', x0, y0) in spatial domain, and the intensity is assigned to the image reflecting the sample structure at (x0, y0) All diffraction patterns in the first diffraction image data set are traversed to obtain the image reflecting the sample structure corresponding to the intensity (due to the operation of the forward Fourier transform or inverse Fourier transform, the image is a complex image) with a size of M x N; Figure 5 wherein the coordinate (u0', v0') is [92, 59] position.

[0088] 2) Processing the obtained image data reflecting the sample structure using a super-resolution algorithm to increase the image pixels to obtain an enlarged image reflecting the sample structure.

[0089] For the image The size of the image reflecting the sample structure is increased by increasing the number of image pixels using the super-resolution algorithm, which is used to increase the image details and improve the resolution of the image. The image after increasing the details is written as The size becomes M' x N';

[0090] The super-resolution algorithm can increase the image details. For example, without using the super-resolution algorithm, the originally obtained image reflecting the sample structure has a size of 50*50, and if 1 new non-scanning point is inserted between the scanning points, a 99*99 size image can be obtained after the super-resolution algorithm processing.

[0091] 3) Corresponding each pixel of the intensity of the enlarged image reflecting the sample structure to the specific frequency of the diffraction pattern, and inserting the intensity of the increased pixel into the specific frequency of the new diffraction pattern.

[0092] From the enlarged image reflecting the sample structure , the value of each pixel (x m′ , y n′ ) of the image intensity is put back to the complex image J'(u', v', x m′ , y n′The corresponding frequency (u0′, v0′) coordinates of the added pixel are inserted into the newly added complex image J′(u′, v′, x). m′ ,y n′ );

[0093] 4) Iterate through all coordinates, filling in the intensity of each pixel in the newly added diffraction image, and finally obtain the expanded two-dimensional complex image data J′(u′,v′,x). m′ ,y n′ ).

[0094] 5) Transform the two-dimensional complex image J′(u′,v′,x) m′ ,y n′ Perform the inverse Fourier transform or the direct Fourier transform, which is the opposite of the previous operation, to obtain the frequency domain image I′(u,v,x). m′ ,y n′ The total number of two-dimensional diffraction images is M'×N'.

[0095] The diffraction image data augmentation steps described above can further improve the resolution of the sample structure.

[0096] In some embodiments of the present invention, the frame interpolation operation performed on the diffraction image in the spatial domain can also be combined with the enlargement operation of the image reflecting the sample structure using a super-resolution algorithm.

[0097] Since each pixel position on the diffraction image actually corresponds to the frequency vector information of the sample structure, the values ​​of different pixels can differ by several orders of magnitude. Using the raw data for calculation may lead to inaccuracies. Therefore, in some embodiments of this invention, before performing frame interpolation, preprocessing can be performed on multiple two-dimensional diffraction images corresponding to multiple scan points in the four-dimensional diffraction image data to reduce the numerical range of a single two-dimensional diffraction image. For example, taking the logarithm or square root can be used to appropriately reduce the numerical range of a single diffraction image. After frame interpolation, the two-dimensional diffraction images corresponding to each position point, including multiple scan points and multiple non-scan points, undergo the opposite operation to reducing the numerical range of a single two-dimensional diffraction image to restore the numerical space of the two-dimensional diffraction image, thereby obtaining the second four-dimensional diffraction image data.

[0098] Furthermore, in some embodiments of the present invention, the preprocessing may further include: before the frame interpolation operation, iteratively optimizing the two-dimensional diffraction image corresponding to each scanning point based on the sample structure information and utilizing the imaging principle of scanning diffraction images. Since the iterative optimization method for the two-dimensional diffraction image can be performed using existing technologies, it will not be elaborated upon here.

[0099] In addition, in some embodiments of the present application, the method can further comprise: after the interpolation operation, iteratively optimizing the obtained two-dimensional diffraction image corresponding to the non-scanning point according to the sample structure information using the imaging principle of the scanning diffraction image; or, after the interpolation operation, iteratively optimizing all the obtained two-dimensional diffraction images corresponding to the scanning point and the non-scanning point according to the sample structure information using the imaging principle of the scanning diffraction image. The optimization method can use the prior art.

[0100] Using the obtained diffraction image data of the scanning point, the diffraction image of the non-scanning point is obtained by the spatial domain interpolation algorithm of the present application, which comprises: inputting the original diffraction image into a pre-constructed spatial domain interpolation model, and obtaining a new diffraction image from the spatial domain interpolation model according to each original diffraction image. For example, if the number of diffraction images obtained by experiment scanning is MxN, the number of two-dimensional diffraction images obtained by interpolation becomes M'xN', and M'>M, N'>N, and M', M, N', N are all integers. Here, the pre-constructed spatial domain interpolation model can be an interpolation model based on interpolation method, or a spatial domain interpolation model pre-trained by machine learning or deep neural network.

[0101] For example, a set of (128x128x50x50) four-dimensional scanning diffraction image data set represents a total of 50x50 positions scanned, each position has a two-dimensional diffraction image with a size of 128x128 pixels, that is, the number of diffraction images is 50x50. After processing by the spatial domain interpolation algorithm of the present application, if 1 new non-scanning point is inserted between the scanning points, the number of diffraction images obtained is 99x99, which represents an increase of nearly 4 times the number of diffraction images. After simple signal processing on the above original data set and the interpolated data set, for example, extracting the image intensity of a specific frequency from the diffraction image, the structure information related to the sample spatial domain can be obtained. The size of the structure information graph of the original data is 50x50, and if 1 new non-scanning point is inserted between the scanning points, the size of the structure information graph after using the spatial domain interpolation algorithm is 99x99, thereby realizing higher resolution display of the sample structure information.

[0102] For the original diffraction image data, the structure image function Object(x, y) of the sample in the spatial domain can be obtained by integrating part of the frequency, or calculating the center of each disc diffraction, or taking the maximum value coordinates of the image after inverse Fourier transform, etc. Therefore, in step S120 of the present application, the specific way of processing the diffraction image data to obtain the structure information of the sample in the spatial domain can include operations such as integrating part of the frequency of the diffraction image data after spatial domain interpolation, or calculating the center of each disc diffraction, or taking the maximum value coordinates of the image after inverse Fourier transform, etc. Thus, the structure image function Object'(x', y') of the sample in the spatial domain can be obtained, and the size and details of the object wave function Object'(x', y') are increased.

[0103] Figure 6 Fig. (a) and (b) respectively show the extracted two-dimensional diffraction image and the sample structure information obtained after integrating the extracted intensity. The integration process includes: extracting the intensity information of a specific frequency range of each two-dimensional diffraction image, integrating the intensity of the specific frequency range, and then inserting the integrated intensity into the space according to the scanning point position of the diffraction image in the sample spatial domain, thereby obtaining the image of the sample structure.

[0104] The reconstruction algorithm of the stacked diffraction coherent imaging can obtain the structure information of the sample in the spatial domain, i.e. realize the sample structure recovery. The stacked diffraction coherent imaging (ptychography) is a method for recovering the sample amplitude and phase using diffraction images, and the phase recovery method of the electron incident wave. Its main principle is that when the electron beam is scanned, there is a certain overlap between each adjacent scanning area. The sample structure and the electron incident wave function are recovered by using this constraint term of the overlapping area. The traditional stacked diffraction coherent imaging requires a large overlapping area, but if the data set is expanded by the method of the present application, the scanning time and the size of the overlapping area can be reduced to reduce the electron radiation dose on the sample.

[0105] The stacked diffraction coherent imaging can obtain the sample structure image function Object'(x', y'), recover the sample structure, and simulate and calculate the diffraction image of the newly added non-scanning point from the structure image function Object'(x', y') according to the imaging principle of the scanning diffraction image. For example, the deviation between the newly added non-scanning point diffraction image and the diffraction image predicted by deep learning is compared to improve the accuracy of the non-scanning point diffraction image. If there is a deviation between the simulated diffraction image and the newly added diffraction image, the deviation is fed back to the spatial interpolation model of the newly added non-scanning point diffraction image for iterative optimization to improve the accuracy of these newly added diffraction images. Figure 7 Fig. (a) is an example of analyzing the sample structure in the spatial domain after interpolating the diffraction image by the method of the present application, Figure 7(b) in FIG. 2 is the original sample structure analyzed by the diffraction pattern without the interpolation processing. As shown in FIG. 2, the resolution of the sample structure obtained after the interpolation operation is obviously improved. Figure 7

[0106] The present application can be used to increase the number of diffraction patterns for electron beam, optical diffraction images, and two-dimensional diffraction images collected by X-ray, synchrotron, etc. The present application can help to improve the imaging efficiency, mine information in a more efficient way, and effectively reduce the photo toxicity and irradiation damage.

[0107] Corresponding to the above method, the present application also provides a device for processing diffraction image data, which comprises a computer device, the computer device comprising a processor and a memory, the memory storing computer instructions, and the processor being configured to execute the computer instructions stored in the memory, so that the device implements the steps of the above method.

[0108] The present application also provides a computer readable storage medium storing a computer program, which is executed by a processor to implement the steps of the above edge computing server deployment method. The computer readable storage medium can be a tangible storage medium, such as random access memory (RAM), internal memory, read only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, floppy disk, hard disk, removable memory disk, CD-ROM, or any other form of storage medium known in the art.

[0109] Those of ordinary skill in the art should understand that the exemplary components, systems and methods described in connection with the embodiments disclosed herein can be implemented in hardware, software, or a combination thereof. The decision to implement particular applications in hardware or software depends on the particular application and design constraints imposed on the overall system. Those of ordinary skill in the art can use different methods to implement the described functions for each particular application, but such implementation should not be considered beyond the scope of the present application. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a functional card, etc. When implemented in software, the elements of the present application are program or code segments used to perform the required tasks. The program or code segments can be stored in a machine readable medium or transmitted through a data signal carried in a carrier wave in a transmission medium or communication link.

[0110] ​It is to be expressly understood that the invention is not limited to the specific configurations and process described above and illustrated in the accompanying drawings. For the sake of clarity, detailed descriptions of known methods are omitted. In the above-described embodiments, several specific steps are described and illustrated as examples. However, the method processes of the present invention are not limited to the specific steps described and illustrated, and various changes, modifications and additions can be made thereto by one of ordinary skill in the art without departing from the spirit of the present invention, and the order of the steps can be changed.

[0111] In the present invention, features described and / or illustrated with respect to one embodiment can be used in the same or a similar way in one or more other embodiments, and / or in combination with or instead of features of other embodiments.

[0112] The above description is merely illustrative of the application, and is not intended to limit the scope of the application. Various modifications and changes can be made by one of ordinary skill in the art without departing from the spirit and scope of the application. Any modification, equivalent replacement, improvement, and the like made within the spirit and principle of the application should be included in the scope of the application.

Claims

1. A diffracted image data processing method, characterized by, The method comprises the following steps: The method comprises the following steps: The method comprises the following steps:

2. The method of claim 1, wherein, The method comprises the following steps:

3. The method of claim 2, wherein, The method comprises the following steps: The method comprises the following steps:

4. The method of claim 2, wherein, The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps:

5. The method according to any one of claims 2-4, characterized in that, The method comprises the following steps: The method comprises the following steps:

6. The method of claim 5, wherein, The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the inputting the adjacent images into a pre-trained machine learning or deep learning based interpolation model, to output interpolated images corresponding to non-scanning points between the scanning points, wherein the pre-trained interpolation model is trained based on a manner that: taking experimental images corresponding to multiple scanning points of a sample in one scanning direction as a training set, taking the L+1th and L-1th experimental images as input, and taking the Lth experimental image as true labeled data to train the interpolation model; or taking experimental images of continuous multiple scanning points of the sample conforming to the two-dimensional correlation relationship as a training set, wherein the image of the correlated scanning point corresponding to the scanning point at the position to be interpolated in the multiple scanning points is taken as input, and the image of the scanning point at the position to be interpolated is taken as true labeled data to train the interpolation model; wherein the experimental image is an experimental frequency domain diffraction image or a corresponding spatial domain image obtained by conversion.

7. The method of claim 1, wherein, The image interpolation operation between the scanning points using the pre-constructed spatial domain image interpolation model to increase the number of diffraction images includes: extracting image intensity-position information of a specific frequency from the first diffraction image dataset to obtain image data reflecting the structure of the sample; increasing the number of pixels of each image in the obtained image data using a super-resolution algorithm to obtain an expanded image; corresponding the intensity of the expanded image to the specific frequency of the two-dimensional diffraction image, wherein the intensity of the added pixels is inserted into the specific frequency of the new diffraction image; traversing all frequencies to correspond the intensity of the expanded image to the traversed frequencies of the two-dimensional diffraction image to fill each pixel intensity of the new diffraction image, to obtain the expanded diffraction image data; or The image interpolation operation between the scanning points using the pre-constructed spatial domain image interpolation model to increase the number of diffraction images includes: transformation operation: performing Fourier transform or inverse Fourier transform on the diffraction image of a specific position coordinate in the first diffraction image dataset to obtain a spatial domain complex image, and extracting image intensity at a specific frequency from the spatial domain complex image, and corresponding the intensity to the specific position coordinate; performing the transformation operation on the diffraction images of all other position coordinates in the first diffraction image dataset to obtain image data reflecting the structure of the sample; increasing the number of pixels of each image in the obtained image data using a super-resolution algorithm to obtain an expanded image reflecting the structure of the sample; corresponding each pixel of the intensity of the expanded image reflecting the structure of the sample to the corresponding frequency of the two-dimensional diffraction image, wherein the intensity of the added pixels is inserted into the specific frequency of the new diffraction image; traversing all coordinates to fill each pixel intensity of the new diffraction image to finally obtain the expanded two-dimensional complex image data; performing a transformation opposite to the Fourier transform or inverse Fourier transform on the two-dimensional complex image data to obtain the expanded frequency domain diffraction image data.

8. The method of claim 7, wherein, The super-resolution algorithm is an interpolation algorithm or a machine learning network or a deep network learning network.

9. The method according to any one of claims 1 to 4, 7, wherein, The method further comprises: The first four-dimensional diffraction image data is obtained by performing the following steps on the first three-dimensional diffraction image data: The method further comprises:

10. The method according to any one of claims 1-4, 7, wherein, Before or after the interpolation operation, based on the sample structure information, the obtained two-dimensional diffraction image corresponding to the non-scanning point is iteratively optimized using the imaging principle of the scanning diffraction image. The processor is configured to execute the computer program / instructions, and when the computer program / instructions are executed, the device implements the steps of the method according to any one of claims 1 to 10.

11. A diffractogram data processing apparatus comprising a processor, a memory and a computer program / instructions stored on the memory, wherein the computer program / instructions are arranged, when executed by the processor, to perform the method of any one of claims 1 to 10. The computer program / instructions are executed by the processor to implement the steps of the method according to any one of claims 1 to 10.

12. A computer readable storage medium having stored thereon computer programs / instructions, characterized in that, ​

Images