Diffraction image analysis method and device, diffraction image processing method and device, equipment and storage medium

Abstract

The invention provides a diffraction image analysis method and device, a diffraction image processing method and device, equipment and a storage medium. The diffraction image analysis method comprises the following steps: acquiring multiple frames of diffraction images acquired in the growth process of a sample, wherein the diffraction images comprise electron diffraction patterns generated by interaction of a high-energy electron beam and the sample; determining an ROI in the diffraction image, and obtaining an image frame sequence and light intensity time sequence data of the ROI; and performing multi-modal analysis on the image frame sequence and the light intensity time sequence data of the ROI to obtain an analysis result. By adopting the technical scheme provided by the invention, the accuracy of diffraction image analysis can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of semiconductor technology, and in particular to a diffraction image analysis method, a processing method, a device, an apparatus and a storage medium. BACKGROUND

[0002] With the development of semiconductor materials and molecular beam epitaxy (MBE) thin film technology, real-time monitoring of material growth process and surface structure is crucial for quality control and performance optimization of epitaxial wafers. Researchers usually observe diffraction images, such as reflection high-energy electron diffraction (RHEED) images, to observe the electron diffraction pattern of the electron beam hitting the substrate surface, to provide real-time feedback on surface topography and growth rate, and to convert the invisible atomic-level growth process into quantifiable dynamic signals.

[0003] In related technologies, although diffraction image analysis tools play an important role in monitoring thin film growth, simple crystal structure analysis and surface defect detection, there are still problems of insufficient image analysis accuracy, which limits their application in semiconductor materials and MBE technology. SUMMARY

[0004] Embodiments of the present application provide a diffraction image analysis method, a processing method, a device, an apparatus and a storage medium.

[0005] In a first aspect, embodiments of the present application provide a diffraction image analysis method, which comprises:

[0006] Obtaining a plurality of frames of diffraction images collected during the growth process of a sample, the diffraction images containing an electron diffraction pattern generated by the interaction of a high-energy electron beam with the sample;

[0007] Determining a ROI (Region of Interest) in the diffraction image and obtaining an image frame sequence and light intensity time series data of the ROI;

[0008] Performing multi-modal analysis on the image frame sequence and light intensity time series data of the ROI to obtain an analysis result.

[0009] In some embodiments, before performing multi-modal analysis, the method further comprises:

[0010] Fitting the light intensity time series data of the ROI to generate an oscillation curve describing the change of light intensity with time, and separating out a noise component;

[0011] Based on the noise period of the noise component, generating a correction function describing the change of displacement error with time;

[0012] performing a displacement correction operation on the collected diffraction images based on the correction function to make the positions of the ROI in different frames of the diffraction images consistent.

[0013] In some embodiments, the collecting process of the diffraction images comprises:

[0014] sampling the video of the sample to obtain the diffraction images according to an image sampling frame rate, the video being obtained based on recording the electron diffraction patterns reflected by the sample during growth; the image sampling frame rate being used to synchronize the image sampling period with the rotation period of a sample rotation stage used to hold the sample for rotation.

[0015] In some embodiments, the generating of the correction function for describing the displacement error change over time based on the noise period of the noise component comprises:

[0016] when the noise period is less than a preset period threshold, performing equal-division segmented sampling on the noise period to obtain the peak position corresponding to each segmented time;

[0017] generating the correction function for describing the displacement error change over time based on the coordinate difference of the peak position corresponding to each segmented time relative to the initial center reference point of the ROI.

[0018] In some embodiments, the method further comprises:

[0019] when the correction function is enabled, performing the displacement correction operation and outputting a comparison view of the oscillation curves before and after the displacement correction of the ROI.

[0020] In some embodiments, the determining of the ROI in the diffraction images comprises:

[0021] in response to a region selection operation of a user on the diffraction images through a ROI selection tool, determining the ROI in the diffraction images, the ROI selection tool being any one of a rectangular region selection tool and a non-rectangular region selection tool.

[0022] In some embodiments, the method further comprises:

[0023] performing frequency domain analysis on the light intensity time series data of the ROI to obtain period information;

[0024] using a selected fitting function in a plurality of fitting functions to fit the light intensity time series data to obtain a fitting curve, and identifying a peak in the fitting curve;

[0025] Based on the peak value and the period information, a trigonometric function fitting curve is generated, and the trigonometric function fitting curve and the corresponding mathematical expression are output.

[0026] In some embodiments, the analysis results include: sample material structure prediction results and / or sample growth behavior prediction results; the sample material structure prediction results indicate at least one of lattice type and surface defect type; the sample growth behavior prediction results indicate at least one of oscillation period and growth rate.

[0027] In some embodiments, performing multimodal analysis on the image frame sequence and light intensity temporal data of the ROI and outputting analysis results includes:

[0028] Visual spatial features are extracted from the image frame sequence through the feature extraction layer of the image processing model, and light intensity temporal features are extracted from the light intensity temporal data.

[0029] The visual spatial features and the temporal features of light intensity are fused through the attention-based feature fusion layer of the image processing model to obtain multimodal features;

[0030] The analysis results are obtained by inferring based on the multimodal features through the inference layer of the image processing model.

[0031] In some embodiments, the analysis results are obtained by inferring based on the multimodal features through the inference layer of the image processing model, including:

[0032] The multimodal features are input into the pattern classifier in the inference layer for prediction to obtain the predicted result of the sample material structure.

[0033] The multimodal features are input into the regression predictor in the inference layer for prediction, and the prediction result of the sample growth behavior is obtained.

[0034] In some embodiments, the method further includes:

[0035] The diffraction image is processed using a convolutional neural network to obtain diffraction fringe arrangement information;

[0036] The step of inputting the multimodal features into the pattern classifier in the inference layer for prediction to obtain the sample growth behavior prediction result includes:

[0037] The diffraction fringe arrangement information and the multimodal features are input into the pattern classifier for prediction to obtain the sample material structure prediction result, wherein the sample material structure prediction result also includes the fringe arrangement type.

[0038] In some embodiments, the analysis result further comprises a recommended value of an experimental parameter; and the method further comprises:

[0039] matching the historical experimental data and the current experimental data based on the experimental parameter values, to obtain a plurality of sets of experimental parameter values respectively corresponding to a sample growth behavior prediction result;

[0040] fitting a trend curve based on the plurality of sets of experimental parameter values and the sample growth behavior prediction results respectively corresponding to the plurality of sets of experimental parameter values;

[0041] finding a set of experimental parameter values in the trend curve that optimizes the sample growth behavior prediction result, to obtain a recommended value of an experimental parameter; wherein the experimental parameter comprises at least one of an image exposure time, a sample rotation table rotation speed, a high-energy electron beam energy, and an electron beam irradiation angle.

[0042] In a second aspect, the embodiments of the present application provide a diffraction image processing method, which comprises:

[0043] obtaining a plurality of frames of diffraction images collected during a growth process of a sample, wherein the diffraction images contain electron diffraction patterns generated by interaction between a high-energy electron beam and the sample;

[0044] determining a ROI in the diffraction images and obtaining light intensity time series data of the ROI;

[0045] fitting an oscillation curve for describing changes of light intensity with time based on the light intensity time series data of the ROI, and separating a noise component;

[0046] generating a correction function for describing changes of displacement error with time based on a noise period of the noise component;

[0047] performing a displacement correction operation on the collected diffraction images based on the correction function, so that the positions of the ROI in different diffraction images are consistent.

[0048] In some embodiments, the diffraction image collection process comprises:

[0049] sampling a video of the sample according to an image sampling frame rate to obtain the diffraction images, wherein the video is obtained based on recording electron diffraction patterns reflected by the sample during the growth process; and the image sampling frame rate is used to synchronize an image sampling period with a rotation period of a sample rotation table, wherein the sample rotation table is used to support the sample for rotation.

[0050] In some embodiments, the generating, based on the noise period of the noise component, a correction function for describing a change of displacement error over time comprises:

[0051] When the noise period is less than a preset period threshold, the noise period is equally divided into segmented sampling to obtain a peak position corresponding to each segmented time;

[0052] Based on a coordinate difference value of the peak position corresponding to each segmented time relative to an initial center reference point of the ROI, a correction function for describing a change of displacement error over time is generated.

[0053] In some embodiments, the performing, based on the correction function, a displacement correction operation on the collected diffraction images comprises:

[0054] For each frame of the collected diffraction images, the horizontal displacement error and the vertical displacement error of the ROI in the diffraction image are calculated according to the correction function and the sampling time of the diffraction image;

[0055] Based on the horizontal displacement error and the vertical displacement error, each frame of the collected diffraction images is adjusted in displacement.

[0056] In some embodiments, after the displacement correction operation is performed, the method further comprises:

[0057] For the ROI with the same position in different diffraction images, image frame sequences and light intensity time sequence data of the ROI are obtained;

[0058] The image frame sequences and the light intensity time sequence data of the ROI are subjected to multi-modal analysis to obtain an analysis result.

[0059] In a third aspect, the embodiments of the present application provide a diffraction image analysis device, which comprises:

[0060] An acquisition module is configured to acquire a plurality of frames of diffraction images collected during a growth process of a sample, wherein the diffraction images contain electron diffraction patterns generated by interaction between a high-energy electron beam and the sample;

[0061] A processing unit is configured to determine an ROI in the diffraction images and acquire image frame sequences and light intensity time sequence data of the ROI;

[0062] An analysis unit is configured to perform multi-modal analysis on the image frame sequences and the light intensity time sequence data of the ROI to obtain an analysis result.

[0063] In a fourth aspect, the embodiments of the present application provide a diffraction image analysis device, which comprises:

[0064] An acquisition unit is configured to acquire a plurality of diffraction images collected in a growth process of a sample, the diffraction images containing electron diffraction patterns generated by interaction of a high-energy electron beam with the sample;

[0065] A processing unit is configured to determine a ROI in the diffraction images and acquire light intensity time-series data of the ROI;

[0066] A correction unit includes:

[0067] A first generating sub-unit is configured to fit an oscillation curve for describing variation of light intensity with time to the light intensity time-series data of the ROI and separate a noise component;

[0068] A second generating sub-unit is configured to generate a correction function for describing variation of displacement error with time based on a noise period of the noise component;

[0069] A correction sub-unit is configured to perform a displacement correction operation on the collected diffraction images based on the correction function so as to make positions of the ROI in different frames of the diffraction images consistent.

[0070] In a fifth aspect, an electronic device is provided, which includes a processor, a memory, and an executable program stored in the memory and capable of being executed by the processor, and the processor executes the executable program to implement the diffraction image analysis method according to any one of the first aspect or the diffraction image processing method according to any one of the second aspect.

[0071] In a sixth aspect, a computer readable storage medium is provided, which stores an executable program, and the executable program is executed by a processor to implement the diffraction image analysis method according to any one of the first aspect or the diffraction image processing method according to any one of the second aspect.

[0072] In the technical scheme provided by the embodiments of the present application, in the diffraction image analysis process, a plurality of diffraction images collected in a growth process of a sample are acquired, the diffraction images containing electron diffraction patterns generated by interaction of a high-energy electron beam with the sample; a ROI is determined in the diffraction images, and an image frame sequence and light intensity time-series data of the ROI are acquired. Since the image frame sequence can reflect key characteristics of the sample in a spatial dimension, and the light intensity time-series data can reflect dynamic change characteristics of the sample in a time dimension, by performing multi-modal analysis on the image frame sequence and the light intensity time-series data of the ROI, an analysis result is obtained, so that multi-modal analysis is performed in combination with data in the spatial and time dimensions, compared with analysis using single-dimensional data, the accuracy of diffraction image analysis can be improved; in addition, by determining the ROI in the diffraction images, multi-modal analysis is focused on a specific region of the image, so that interference of irrelevant information can be reduced, the analysis efficiency is improved, and the accuracy of diffraction image analysis can be further improved.

[0073] Further, in the processing of the diffraction image, by fitting the light intensity time series data of the ROI, the oscillation curve representing the periodic change rule is extracted, and the noise component is separated. Not only can the real dynamic behavior of the sample surface be accurately reflected, but also the error introduced by the slight vibration of the equipment can be effectively analyzed; the correction function generated based on the periodic characteristics of the noise component can quantitatively describe the characteristics of the displacement error changing with time, and the displacement correction of the collected diffraction image using the correction function can compensate the displacement error of the ROI in each frame of diffraction image, that is, by introducing the adaptive micro-motion denoising algorithm, the automatic correction of the data is realized, thereby improving the stability of the data, so as to provide more reliable data support for the multi-modal analysis of the diffraction image, and help to further improve the accuracy and reliability of the diffraction image data analysis. BRIEF DESCRIPTION OF DRAWINGS

[0074] Figure 1 FIG. 1 is a flowchart of a diffraction image analysis method according to an embodiment.

[0075] Figure 2 FIG. 2 is a schematic diagram of a diffraction image correction process according to an embodiment.

[0076] Figure 3 FIG. 3 is a schematic diagram of a multi-modal data analysis process according to an embodiment.

[0077] Figure 4 FIG. 4 is a schematic diagram of an experimental parameter recommended value acquisition process according to an embodiment.

[0078] Figure 5 FIG. 5 is a flowchart of a diffraction image processing method according to an embodiment.

[0079] Figure 6 FIG. 6 is a schematic diagram of a correction function generation process according to an embodiment.

[0080] Figure 7a FIG. 7 is a schematic diagram of the architecture of a diffraction image analysis system according to an embodiment.

[0081] Figure 7b FIG. 8 is a schematic diagram of a rotation matching and micro-motion denoising process according to an embodiment.

[0082] Figure 7c FIG. 9 is a schematic diagram of the internal processing logic of an AI multi-modal large model according to an embodiment.

[0083] Figure 7d FIG. 10 is a schematic diagram of the display of a ROI region and its oscillation curve according to an embodiment.

[0084] Figure 8 is a structural schematic diagram of a diffraction image analysis device according to an embodiment.

[0085] Figure 9 is a structural schematic diagram of a diffraction image processing device according to an embodiment. DETAILED DESCRIPTION

[0086] In order to make the technical solutions and beneficial effects of the present application more obvious and easy to understand, the following will be described in detail by listing specific embodiments. The drawings are not necessarily drawn to scale, and local features can be enlarged or reduced to more clearly show the details of the local features; unless otherwise defined, the technical and scientific terms used herein have the same meaning as the technical and scientific terms in the technical field to which the present application belongs.

[0087] The embodiments of the present application are not exhaustive, but only illustrate some embodiments, and are not specific limitations on the protection scope of the present disclosure. In the case of no contradiction, each step in an embodiment can be implemented as an independent embodiment, and the steps can be combined arbitrarily, for example, the scheme after removing part of the steps in an embodiment can also be implemented as an independent embodiment, and the order of the steps in an embodiment can be exchanged arbitrarily, in addition, the optional implementation manners in an embodiment can be combined arbitrarily; in addition, the embodiments can be combined arbitrarily, for example, part or all steps of different embodiments can be combined arbitrarily, an embodiment can be combined with optional implementation manners of other embodiments.

[0088] In each embodiment of the present application, the terms and / or descriptions of the embodiments are consistent and can be referred to each other if there is no special description and logical conflict, and the technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationship.

[0089] In the description of the embodiments of the present application, it should be understood that the terms "first", "second" are only for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features limited by "first", "second" can explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, the meaning of "multiple" is two or more, unless otherwise specifically limited.

[0090] Figure 1 is a flowchart of a diffraction image analysis method according to an embodiment. As shown in Figure 1 , the method comprises steps S101 to S103.

[0091] Step S101: acquiring a plurality of diffraction images collected in the growth process of a sample.

[0092] The diffraction image comprises an electron diffraction pattern generated by the interaction of the high-energy electron beam with the sample.

[0093] The diffraction image analysis method provided by the embodiments of the present application can be applied to any electronic device with a diffraction image analysis function, such as a server or a terminal installed with diffraction image analysis software. The server can be a single server, a server cluster composed of multiple servers, or a cloud computing service center; and the terminal can be a PC (Personal Computer), a smart phone, a PDA (Personal Digital Assistant), a tablet computer, and the like. The embodiments of the present application do not limit the specific type of the electronic device.

[0094] The sample can be a material capable of surface reconstruction or structure evolution during growth. For example, the sample can include a semiconductor thin film or a crystal material.

[0095] The energy range of the high-energy electron beam is related to high-energy electron diffraction technology. For example, the RHEED (Reflection High-Energy Electron Diffraction) technology uses an electron beam with an energy of 10 keV to 50 keV, which is incident on the sample surface at a small angle (close to parallel, for example, less than 3°), and forms an electron diffraction pattern after reflection, which can be applied to real-time monitoring of material surface properties and thin film growth. For another example, the HEED (High-Energy Electron Diffraction) electron beam energy range is usually 50 keV to 300 keV or higher, the electron beam is incident on the sample at a larger angle, and can penetrate deeper into the sample area, which can be applied to analyze the bulk crystal structure of thick samples or multi-layer materials.

[0096] The electron diffraction pattern is formed by the reflection of the high-energy electron beam after interacting with the sample surface. For example, the RHEED system is used to reflect the high-energy electron beam after being incident on the sample surface to form a reflection high-energy electron diffraction pattern.

[0097] The diffraction image formed by the interaction of the RHEED high-energy electron beam with the sample can be referred to as a “RHEED image”.

[0098] It can be understood that, as the sample grows, the crystal structure (such as thin film deposition, crystal growth, etc.), orientation or morphology (such as grain size, surface steps, defect density) of the sample surface will dynamically evolve, and the electron diffraction pattern will also change accordingly. The dynamic changes of the crystal structure and morphology of the sample during growth will be directly reflected in the characteristics of the diffraction pattern.

[0099] In a possible implementation, during the sample growth, the diffraction images generated by the interaction between the high-energy electron beam and the sample surface can be continuously collected by an image collection device such as an industrial camera.

[0100] For example, the sample can be fixed on a sample rotating table, and during the rotation of the sample rotating table, the incident angle of the electron beam remains constant, and the diffraction images generated during the sample growth are uniformly collected by the camera at different rotation periods, for example, one frame of diffraction image is collected in each rotation period.

[0101] In a possible implementation, before step S101 is performed, the method can further include:

[0102] It is determined whether the terminal has a software license of the diffraction image analysis software, and if the terminal has the valid software license, the terminal is allowed to call the function of the diffraction image analysis software, and if the terminal does not have the valid software license, the terminal is limited to call the function of the diffraction image analysis software. The software license is generated based on a unique identifier of the terminal, and the unique identifier can be a universally unique identifier (UUID) for example. By generating the software license based on the unique identifier of the terminal, the confidentiality and integrity of the software can be maintained in actual application, and the unauthorized terminal is prevented from running the software or illegally copying the function of the software.

[0103] Step S102: determining the ROI in the diffraction image, and obtaining the image frame sequence and the light intensity time sequence data of the ROI.

[0104] The ROI can be a selected key region in the diffraction image. For example, for the first frame of collected diffraction image, the ROI can be determined in the first frame of diffraction image based on the region selection operation of the user, for example, in the RHEED image, the region with the clearest stripe indicated by the region selection operation can be taken as the ROI. Alternatively, the key region in the diffraction image can be automatically identified based on an image processing algorithm (such as edge detection, threshold segmentation) to take as the ROI, for example, the highlight region is automatically extracted according to the spot intensity distribution to take as the ROI.

[0105] The shape of the ROI can be rectangular or non-rectangular, for example, elliptical, circular, or the like, and the ROI can be multiple, and the shapes of the ROIs at different positions on the same diffraction image can be the same or different.

[0106] After determining the ROI in the first frame of the collected diffraction images, for the ROI corresponding to the same physical region on the sample, the ROI in the first frame of the diffraction images can be mapped to the subsequent frames of the diffraction images. For example, for the diffraction images collected in a rotating collection mode, if the diffraction images are collected at a fixed rotation angle (e.g., 60 degrees) in each rotation cycle, the ROI determined in the first frame of the diffraction images can be mapped to the subsequent frames of the diffraction images to obtain the ROI in the subsequent frames of the diffraction images corresponding to the same physical region as the ROI in the first frame of the diffraction images.

[0107] It can be understood that the positions of the ROIs corresponding to the same physical region on the sample in different frames of the diffraction images can change due to factors such as material evaporation, rotation of the sample rotation stage, and the like. Such changes in position reflect the performance of the same ROI at different time points. Therefore, for the ROIs corresponding to the same physical region on the sample, although their positions can change over time in different frames of the diffraction images, they can still be referred to as “the same ROI”.

[0108] In addition, in order to ensure the consistency of the ROIs corresponding to the same physical region in different frames of the diffraction images, an image correction algorithm can be used to dynamically adjust the positions of the ROIs in the frames of the diffraction images, for example, a correction function can be generated based on the light intensity time series data of the ROIs to compensate for the displacement errors of the ROIs in the frames of the diffraction images.

[0109] In one possible implementation, the process of obtaining the image frame sequence of the ROI can include:

[0110] Based on the position and shape of the ROI, the image frames of the ROI are extracted from the collected multiple frames of the diffraction images in the order of the collection time to obtain the image frame sequence of the ROI.

[0111] The image frame sequence of the ROI can reflect the characteristics of the sample in the spatial dimension, for example, can reflect the characteristics of the crystal structure, the arrangement of the stripes, and / or the defect distribution, and the like.

[0112] In one possible implementation, the light intensity time series data of the ROI includes light intensity statistical values of the ROI sorted by the collection time.

[0113] The process of obtaining the light intensity time series data of the ROI can include: for each frame of the collected diffraction images, the light intensity values of all the pixel points in the ROI in the frame of the diffraction images are counted to obtain a light intensity statistical value (such as an average value, a total value, or a standard deviation, and the like), and the light intensity statistical values are sorted by the collection time to form the light intensity time series data of the ROI.

[0114] The light intensity time series data of the ROI can reflect the characteristics of the sample in the time dimension, for example, can reflect the growth rate and / or the oscillation period, and the like.

[0115] In one possible implementation, in order to improve the quality of the diffraction image so as to further improve the accuracy of the diffraction image analysis, before the ROI is determined in the diffraction image, the method can further include:

[0116] The diffraction image collected by the connection line is subjected to enhancement processing; the enhancement processing can include at least one of contrast adjustment, brightness adjustment, exposure adjustment, and parameter adjustment.

[0117] Step S103: performing multi-modal analysis on the ROI image frame sequence and the light intensity time sequence data to obtain an analysis result.

[0118] In one possible implementation, a deep learning model (such as a convolutional neural network, a recurrent neural network, etc.) can be used to extract spatial features of the image frame sequence and temporal features of the light intensity time sequence data and perform multi-modal fusion to predict and generate the analysis result, or a time-space correlation analysis can also be used, for example, by calculating the spatial features of the image frame sequence and the temporal features of the light intensity time sequence data and performing joint analysis to obtain the analysis result. For example, by extracting the temporal features (such as the oscillation period) of the light intensity time sequence data and the spatial features (such as the fringe spacing) of the image frame sequence and performing joint analysis, if the analysis result indicates that the fringe spacing remains stable and the oscillation period of the light intensity time sequence data is regular, it indicates that the growth process is stable.

[0119] It can be understood that if there are multiple ROIs in the same diffraction image, the multi-modal analysis needs to be performed on the image frame sequence and the light intensity time sequence data of each ROI to obtain the analysis result corresponding to the ROI, and then all the analysis results corresponding to the ROIs are integrated to obtain the final analysis result.

[0120] In one possible implementation, the analysis result can include but is not limited to an analysis result related to the sample structure (such as the lattice type, surface defects), and / or an analysis result related to the sample growth behavior (such as the oscillation period, growth rate, etc.).

[0121] In one possible implementation, the method can further include: performing a natural language processing (NLP) module on the analysis result, for example, the analysis result can be processed based on image-text joint learning to form an explainability report and output to assist users to quickly understand the experimental state and adjust the experimental result.

[0122] In the embodiment, by acquiring a plurality of diffraction images collected in a growth process of a sample, the diffraction images containing electron diffraction patterns generated by interaction of a high-energy electron beam with the sample, determining a ROI in the diffraction images, acquiring image frame sequences and light intensity time series data of the ROI, and performing multi-modal analysis on the image frame sequences and the light intensity time series data of the ROI to obtain an analysis result, since the image frame sequences can reflect key characteristics of the sample in the spatial dimension, and the light intensity time series data can reflect dynamic change characteristics of the sample in the time dimension, so that multi-modal analysis is performed by combining data in the spatial and time dimensions, compared with analysis using single-dimensional data, the accuracy of diffraction image analysis can be improved. In addition, by determining the ROI in the diffraction images, the multi-modal analysis is focused on a specific region of the image, which can reduce the interference of irrelevant information, improve the analysis efficiency, and further improve the accuracy of diffraction image analysis.

[0123] In some embodiments, the acquisition process of the diffraction images can include:

[0124] According to an image sampling frame rate, a video of the sample is sampled to obtain diffraction images, the video being obtained based on recording electron diffraction patterns reflected by the sample in a growth process; the image sampling frame rate is used to synchronize an image sampling period with a rotation period of a sample rotation stage, the sample rotation stage being used to carry the sample for rotation.

[0125] Here, the video of the sample can be a real-time video, or can also be an offline video, which is a video of the electron diffraction patterns reflected by the sample in a pre-recorded rotation process.

[0126] Here, the image sampling period refers to a time interval between adjacent two frames of diffraction images. The rotation period of the rotation stage is the time required for one rotation of the rotation stage.

[0127] In one possible implementation, a rotation speed parameter (such as revolutions per minute, RPM) of the sample rotation stage for carrying the sample for rotation can be acquired by the user input through the interactive interface. According to the input rotation speed of the sample rotation stage, the rotation period T (unit: second) of the sample is calculated, that is, the time required for the sample to complete one rotation is obtained. Based on the image sampling period synchronized with the rotation period of the sample, the target image sampling frame rate is calculated, and the image sampling frame rate is adjusted to the target image sampling frame rate to extract the diffraction images from the video of the sample. In this way, by synchronizing the image sampling period with the rotation period of the sample rotation stage, a fixed number of image frames can be collected in each rotation period, that is, a fixed number of diffraction images are uniformly collected from the video of the sample in each rotation period of the sample rotation stage.

[0128] For example, if one frame of image is collected per rotation cycle, the image sampling period is equal to the rotation period; if N frames of image are collected per rotation cycle (N is greater than 1), the rotation period is N times of the image sampling period.

[0129] In the embodiment, by adjusting the image sampling frame rate in response to the input of the sample rotation stage speed, and extracting the diffraction images from the video according to the adjusted frame rate, the synchronization of the image sampling period and the rotation period is achieved, which can synchronize the diffraction images collected in different rotation periods, thus ensuring the consistency and comparability of the data, and reducing unnecessary data processing, significantly reducing data redundancy, providing a reliable basis for subsequent multi-modal analysis (such as the combination of spatial features and time features), and thus improving the monitoring accuracy and analysis ability of the sample growth behavior.

[0130] In high-precision semiconductor processes such as molecular beam epitaxy, material evaporation, sample rotation stage rotation, and other situations can cause slight vibrations of the equipment, and these micro-motions can cause slight changes in the electron beam irradiation angle of RHEED, which in turn affects the stability of the diffraction stripes. Real-time video recording will also capture these small errors, affecting diffraction image analysis and ultimately affecting the correct judgment of the test personnel on the growth of the substrate surface material. During long-term experiments, data drift and error accumulation can occur, ultimately affecting the accuracy of film thickness measurement and growth rate calculation. In addition, due to the complex working environment of the equipment, engineers often need to manually adjust parameters for error correction, which increases the difficulty of the experiment and the possibility of misjudgment.

[0131] Therefore, in order to reduce the influence of the slight vibration of the equipment on the stability of the diffraction stripes and improve the reliability of the diffraction image analysis, in some embodiments, before performing multi-modal analysis, as shown in Figure 2 Figure 2 is a diffraction image correction process diagram according to an embodiment, the method can further include:

[0132] Step S201: fitting the light intensity time series data of the ROI to generate an oscillation curve describing the change of light intensity with time, and separating out the noise component;

[0133] Step S202: generating a correction function for describing the change of displacement error with time based on the noise period of the noise component;

[0134] Step S203: performing a displacement correction operation on the collected diffraction images based on the correction function to make the positions of the ROI in different frames of the diffraction images consistent.

[0135] ​In a possible implementation, the light intensity time series data of the ROI in step S201 can be fitted with a polynomial using the least squares method, an oscillation curve representing the periodic variation is extracted, and a noise component is separated. The oscillation curve can reflect the dynamic behavior of the sample surface, such as the oscillation period or the change in stripe definition during thin film growth. The noise component can include periodic characteristics (such as displacement errors caused by equipment vibration).

[0136] For example, the noise component can be subjected to spectral analysis (such as Fourier transform) to obtain a noise period of the noise component. The Fourier transform is used to convert a time domain signal into a frequency domain signal, and by analyzing the main frequency in the frequency spectrum, the period of the noise component is extracted.

[0137] In a possible implementation, in step S202, when the noise period of the noise component is less than a preset period threshold, a correction function for describing the change of the displacement error with time is generated.

[0138] The period threshold can be preset according to expert experience, for example, the period threshold is set to 1 s.

[0139] In a possible implementation, when the noise period of the noise component is greater than or equal to the period threshold, a warning prompt can be generated to remind the experimental personnel to check the equipment stability, so as to reduce the problem of excessive noise affecting the data quality caused by equipment vibration and the like.

[0140] The equipment vibration can cause the ROI in the collected diffraction image to drift in the horizontal and vertical directions, and therefore the correction function can include: a first correction function describing the change of the horizontal displacement error with time, and a second correction function describing the change of the vertical displacement error with time.

[0141] In a long-time experiment, the equipment vibration can cause the error to accumulate, and therefore in some possible implementations, in step S203, performing a displacement correction operation on the collected diffraction image based on the correction function can include: taking the correction function as a global correction function to perform a displacement correction operation on the collected diffraction image and the diffraction image to be collected, so as to reduce the influence of the error on data analysis.

[0142] In some examples, the optional implementation of steps S201 to S203 described above can refer to the description of the optional implementation of steps S401 to S403 in the diffraction image processing method embodiment described below.

[0143] In some embodiments, the optional implementation of step S201 can refer to the optional implementation of step S503 in the method for processing a diffraction image described in Figure 5 and other related parts of the embodiments involved, which will not be described in detail here. Figure 5

[0144] ​In some embodiments, the optional implementation of step S202 can refer to the optional implementation of step S504 in Figure 5 and other related parts of the embodiments involved, which are not described in detail here for the time being. Figure 5

[0145] In some embodiments, the optional implementation of step S203 can refer to the optional implementation of step S505 in Figure 5 and other related parts of the embodiments involved, which are not described in detail here for the time being. Figure 5

[0146] It should be noted that after the displacement correction operation is completed on all collected diffraction images based on the correction function, the image frame sequence and the light intensity time sequence data of the ROI can be reacquired based on the corrected multiple frames of diffraction images, for multi-modal analysis.

[0147] The mean filtering, Gaussian filtering and other methods are used to denoise the whole image, which is simple to operate, but lacks targeted correction, and when processing detailed pictures, key information is easily lost. In the embodiments of the present application, the oscillation curve representing the periodic variation rule is extracted by fitting the light intensity time sequence data of the ROI, and the noise component is separated, which not only accurately reflects the real dynamic behavior of the sample surface, but also effectively analyzes the error introduced by the slight vibration of the equipment; the correction function is generated based on the periodic characteristics of the noise component, which can quantitatively describe the characteristics of the displacement error changing with time, and the collected diffraction images are corrected by using the correction function, which can compensate the displacement error of the ROI in each frame of diffraction image, that is, by introducing the adaptive micro-motion denoising algorithm, the automatic correction of data is realized, thereby improving the stability of the data, so as to provide more reliable data support for multi-modal analysis of diffraction images, which helps to improve the accuracy and reliability of diffraction image data analysis.

[0148] In order to intuitively present the displacement correction effect, in some embodiments, the method can further include:

[0149] In the case where the correction function is enabled, the displacement correction operation is performed, and a comparison view of the oscillation curves of the ROI before and after the displacement correction is output.

[0150] In one possible implementation, in the case where the correction function is enabled, the displacement correction operation is performed on the collected frames of diffraction images based on the generated correction function, so as to adjust the position of the ROI in different frames of diffraction images, so that the ROI remains consistent in different frames of diffraction images.

[0151] ​​In a possible implementation, the correction function can be enabled or disabled by providing a function control, for example, adding a switch button or a check box in the user interface for the user to manually select whether to enable the correction function.

[0152] Here, the oscillation curve before displacement correction is generated by fitting the light intensity time series data of the ROI in the diffraction image before correction. The oscillation curve after displacement correction is generated by fitting the light intensity time series data of the ROI in the diffraction image after correction, that is, the oscillation curve after displacement correction is generated by re-extracting the light intensity time series data of the ROI in the diffraction image after performing displacement correction on the diffraction image based on the correction function, and fitting the light intensity time series data.

[0153] For example, the oscillation curves before and after correction can be displayed in the same chart, and the two oscillation curves can be distinguished by using different colors (for example, red and black) and / or line types (for example, dotted line and solid line).

[0154] In this embodiment, when the correction function is enabled, the displacement correction effect can be intuitively presented by performing the displacement correction operation and outputting the oscillation curve comparison view of the ROI before and after displacement correction, and the interpretability of the displacement correction operation is improved.

[0155] To meet the diversified demand for the shape of the ROI, in some embodiments, determining the ROI in the diffraction image can include:

[0156] In response to the user's region selection operation on the diffraction image through the ROI selection tool, the ROI is determined in the diffraction image, and the ROI selection tool is any one of a rectangular region selection tool and a non-rectangular region selection tool.

[0157] The rectangular region selection tool can be used for rectangular shape ROI selection, and the rectangular region selection tool supports circular, elliptical, polygonal selection or free drawing shape, and can be used for ROI selection in complex scenes.

[0158] In a possible implementation, the target region of the region selection operation can be highlighted as the ROI in response to the user's region selection operation on the diffraction image through the ROI selection tool.

[0159] In this embodiment, by providing a plurality of ROI selection tools of rectangular and non-rectangular shapes, the non-rectangular region analysis demand under different substrate materials and growth conditions can be flexibly adapted, and the flexibility of user interaction operation is improved.

[0160] In some embodiments, the method can further include:

[0161] The period information is obtained by performing frequency domain analysis on the light intensity time series data of the ROI; a fitting function selected from a plurality of fitting functions is used to fit the light intensity time series data to obtain a fitting curve, and a peak value in the fitting curve is identified; a trigonometric function fitting curve is generated based on the peak value and the period information, and the trigonometric function fitting curve and a corresponding mathematical expression are output.

[0162] In one possible implementation, the period information can be used to describe the periodic characteristics in the light intensity time series data. For example, Fourier transform can be performed on the light intensity time series data of the ROI to extract the period information in the light intensity time series data.

[0163] The light intensity time series data can be fitted based on a fitting function selected by the user from a plurality of fitting functions, wherein the plurality of fitting functions can include at least two of a Gaussian function, a Lorentz function, a sine function, and a custom function. In the case where the fitting curve is obtained, the peak value can be identified by a point where the first derivative of the fitting curve is zero. Then, based on the peak value and the period information, a trigonometric function fitting curve can be generated using a sine or cosine function to represent the periodicity of the light intensity variation, and a corresponding mathematical expression is output.

[0164] In this embodiment, by introducing diversified fitting functions, the flexibility of the fitting function is enhanced compared to a single fitting method (such as Gaussian fitting); in addition, by generating a trigonometric function fitting curve and outputting the trigonometric function fitting curve and a corresponding mathematical expression to intuitively present to the user, the user can easily perform quantitative analysis, which can enable the user to quickly and accurately analyze the oscillation mode in the growth process, reduce the user's understanding cost, and thus improve the decision-making efficiency in actual work.

[0165] In some embodiments, the analysis result can include a sample material structure prediction result and / or a sample growth behavior prediction result.

[0166] The sample material structure prediction result indicates at least one of a lattice type and a surface defect type; the sample growth behavior prediction result indicates at least one of an oscillation period and a growth rate.

[0167] The lattice type can include face-centered cubic, body-centered cubic, etc. The surface defect type can include dislocation, vacancy, grain boundary, etc.

[0168] The oscillation period can reflect the periodic characteristics of the surface or interface evolution law of the sample in the growth process. For example, in the process of thin film growth, the oscillation period can reflect the atomic layer deposition rate in the process of thin film growth. The growth rate can reflect the thickness or volume of the sample per unit time.

[0169] In the above embodiments, by performing multi-modal analysis on the ROI image frame sequence and the light intensity time series data, sample material structure prediction results and / or sample growth behavior prediction results are obtained, which can help users to deeply understand the micro characteristics and growth dynamics of the material, thereby helping users to optimize material design or process parameters.

[0170] In some embodiments, as shown in Figure 3 Figure 3 is a multi-modal data analysis process diagram according to an embodiment. The multi-modal analysis is performed on the ROI image frame sequence and the light intensity time series data, and the analysis results are output, which can include:

[0171] Step S301: Extract visual spatial features from the image frame sequence and light intensity time series features from the light intensity time series data through the feature extraction layer of the image processing model;

[0172] Step S302: Fuse the visual spatial features and the light intensity time series features through the attention mechanism-based feature fusion layer of the image processing model to obtain multi-modal features;

[0173] Step S303: Perform inference based on the multi-modal features through the inference layer of the image processing model to obtain the analysis results.

[0174] In one possible implementation, the image processing model is a pre-trained AI (Artificial Intelligence) model, which can be obtained by using training data to perform end-to-end joint training strategy on the feature extraction layer, the attention mechanism-based feature fusion layer, and the inference layer of the AI model; The training data can include image frame sequences and light intensity time series data of ROIs and corresponding labeled data, which is used to represent known analysis results.

[0175] In other examples, in the absence of labeled data (e.g., known lattice type or growth rate), a self-supervised learning method can be used to train the image processing model. After the self-supervised learning is completed, if there is labeled data, a small amount of labeled data can be used to fine-tune the image processing model. In this way, the dependence on labeled data can be reduced, while the model generalization ability can be improved.

[0176] In one possible implementation, the feature extraction layer can include a first feature extraction network for extracting visual spatial features (such as stripe clarity, spot distribution, etc. in the image frame) and a second feature extraction network for extracting light intensity time series features (such as oscillation mode, periodic variation law, etc.). The first feature extraction network can include a convolutional neural network, such as a ResNet convolutional neural network. The second feature extraction network can include a Transformer or an LSTM network (Long Short-Term Memory Neural Network), etc. ​

[0177] In a possible implementation, the feature fusion layer can utilize an attention mechanism to perform weighted summation or concatenation on the visual spatial features and the light intensity time sequence features, to generate the multi-modal features.

[0178] In some embodiments, the inference based on the multi-modal features by the inference layer of the image processing model to obtain the analysis result in step S303 can include:

[0179] The multi-modal features are input into a regression predictor in the inference layer for prediction to obtain a sample material structure prediction result; and the multi-modal features are input into a regression predictor in the inference layer for prediction to obtain a sample growth behavior prediction result.

[0180] It can be understood that, if only the sample growth behavior needs to be analyzed, the light intensity time sequence features can be directly input into the regression predictor for inference, without the need for multi-modal fusion.

[0181] The image processing model can directly output the categories and confidence of the lattice type or surface defect type for the sample growth behavior prediction result. The image processing model can directly output the specific values of the oscillation period and growth rate for the sample material structure prediction result, and can further output a visual chart (such as an oscillation curve).

[0182] In this embodiment, the multi-modal features are obtained by performing feature fusion based on the attention mechanism on the visual spatial features and the light intensity time sequence features, and the inference is performed based on the multi-modal features, thereby realizing efficient joint analysis of the image frame sequence and the light intensity time sequence data, effectively improving the accuracy and reliability of the analysis result, and thereby providing strong technical support for high-precision material growth monitoring scenarios.

[0183] In some embodiments, the method can further include:

[0184] The diffraction image is processed based on a convolutional neural network to obtain diffraction stripe arrangement information.

[0185] For example, the diffraction image can be classified based on a ResNet convolutional neural network, and a grid structure of m*n diffraction stripes is output, that is, the diffraction stripe arrangement information is obtained, where m and n are positive integers.

[0186] In the above step, the multi-modal features are input into a regression predictor in the inference layer for prediction to obtain a sample growth behavior prediction result, which can include:

[0187] The diffraction stripe arrangement information and the multi-modal features are input into a pattern classifier for prediction to obtain a sample material structure prediction result, wherein the sample material structure prediction result further includes a stripe arrangement type.

[0188] In a possible implementation, the pattern classifier can identify the fringe arrangement type (such as regular fringe, irregular fringe, or multi-level fringe, etc.) based on the diffraction fringe arrangement information; then the fringe arrangement type is combined with the multi-modal features to further analyze the lattice type (such as face-centered cubic, body-centered cubic, etc.) and / or the surface defect type (such as dislocation, vacancy, grain boundary, etc.). Finally, the sample material structure prediction result not only contains the lattice type to which the sample belongs and / or the surface defect type to which the sample belongs, but also contains the fringe arrangement type to which the sample belongs, so that the microstructure characteristics of the sample material can be more comprehensively characterized.

[0189] In some embodiments, as shown in Figure 4 Figure 4 is a flowchart of an experimental parameter recommended value acquisition process according to an embodiment, and the analysis result further includes recommended values of experimental parameters; the method can further include steps S401-S404.

[0190] Step S401: match the historical experimental data and the current experimental data based on the experimental parameter values to obtain sample growth behavior prediction results corresponding to a plurality of sets of experimental parameter values.

[0191] Each experimental data includes a set of experimental parameter values and a sample growth behavior prediction result of the sample under the set of experimental parameter values.

[0192] For example, a diffraction image collected during the growth process of the sample under a set of experimental parameter values can be input into an image processing model as a model input to obtain a sample growth behavior prediction result. For example, the sample growth behavior prediction result can include an oscillation period and / or a growth rate.

[0193] Exemplarily, a plurality of experiments can be performed on the same sample to obtain experimental data of the plurality of experiments. A set of experimental parameters used in different experiments can be completely the same or partially the same or completely different, for example, the exposure time and the rotation speed of the sample rotation table used in a historical experiment are the same as the exposure time and the rotation speed of the sample rotation table used in the current experiment.

[0194] In a possible implementation, in the above step S401, the historical experimental data and the current experimental data can be matched based on the experimental parameter values, and target experimental data containing a set of experimental parameter values identical to the current experimental data can be determined from the historical experimental data; for the set of experimental parameter values of the current experimental data, the sample growth behavior prediction results corresponding to the set of experimental parameter values are obtained from the target experimental data and the current experimental data, respectively.

[0195] ​It can be understood that, under the same experimental conditions (for example, the experimental parameter values such as exposure time, rotation table speed are all the same), due to the slight differences of the samples themselves (such as initial state, material property difference), different growth behaviors can also be caused, that is, the same set of experimental parameter values can correspond to different sample growth behavior prediction results (such as oscillation period, growth rate, etc.).

[0196] Step S402: Based on the multiple sets of experimental parameter values and the sample growth behavior prediction results corresponding to the multiple sets of experimental parameter values respectively, a trend curve is fitted and generated.

[0197] Among them, the trend curve can be used to describe the relationship between different sets of experimental parameter values and sample growth behavior prediction results, that is, it can reflect the influence trend of experimental parameters on sample growth behavior.

[0198] Exemplarily, when fitting and generating the trend curve in step S402, a regression method (such as polynomial regression) can be used to fit and generate the trend curve based on different sets of experimental parameter values and their corresponding sample growth behavior prediction results.

[0199] Among them, if there is a set of experimental parameter values corresponding to different sample growth behavior prediction results, the mean value of the sample growth behavior prediction results corresponding to the set of experimental parameters can be used as the fitting value, for example, for a set of experimental parameter values corresponding to multiple different oscillation periods, the multiple different oscillation periods can be averaged to obtain the fitting value.

[0200] Step S403: Find a set of experimental parameter values in the trend curve that makes the sample growth behavior prediction result optimal to obtain the recommended value of the experimental parameter.

[0201] Among them, the experimental parameters include at least one of image exposure time, sample rotation table speed, high-energy electron beam energy and electron beam irradiation angle.

[0202] Exemplarily, the sample growth behavior prediction result is optimal, for example, it can be the most stable oscillation period or the highest growth rate.

[0203] In one possible implementation, if a single behavior index is optimized (such as maximizing the growth rate or minimizing the oscillation period fluctuation), the peak or valley value can be directly searched on the trend curve to find a set of experimental parameter values that makes the sample growth behavior prediction result optimal. If multiple behavior indexes are optimized (such as maximizing the growth rate and minimizing the oscillation period fluctuation at the same time), a multi-objective optimization method (such as combining multiple objective functions into a comprehensive objective function by weighting) can be used, and then a set of experimental parameter values that makes the sample growth behavior prediction result optimal can be found on the trend curve.

[0204] In the embodiment, the dynamic behavior (such as stability of oscillation period, uniformity of growth rate) in the sample growth process can be improved by finding a set of experimental parameter values in the trend curve that optimizes the sample growth behavior prediction result to obtain recommended values of the experimental parameters, so that the sample experiment quality can be improved, the sample growth behavior can be more accurately controlled, and the reliability of sample growth behavior monitoring can be improved. In addition, the optimal experimental parameters can also be directly found through the trend curve, so that the inefficiency of blind trial and error can be improved, and the experimental efficiency can be improved.

[0205] Figure 5 FIG. 1 is a flow diagram of a diffraction image processing method according to an embodiment. Each of the embodiments or implementations in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. For the same or similar parts between the embodiments, refer to each other.

[0206] As shown in Figure 5 , the method comprises steps S501 to S504.

[0207] Step S501: Obtain a plurality of diffraction images collected in a growth process of a sample.

[0208] The diffraction images contain electron diffraction patterns generated by interaction of a high-energy electron beam and the sample.

[0209] The diffraction image processing method provided by the embodiments of the present application can be applied to any electronic device with a diffraction image analysis function, such as a server or a terminal installed with a diffraction image analysis software. The server can be a server, a server cluster composed of multiple servers, or a cloud computing service center; and the terminal can be a PC (Personal Computer), a smart phone, a PDA (Personal Digital Assistant), a tablet computer, etc. The embodiments of the present application do not make any limitation on the specific type of the above-mentioned electronic device.

[0210] In some embodiments, the optional implementation of step S501 can refer to the optional implementation of step S101 and other related parts of the embodiments involved in Figure 1 , which will not be repeated here. Figure 1

[0211] Step S502: Determine an ROI in the diffraction image and obtain light intensity time sequence data of the ROI.

[0212] In some embodiments, the optional implementation of step S502 can refer to the optional implementation of step S102 and other related parts of the embodiments involved in Figure 1 , which will not be repeated here. Figure 1 ​Other related parts in the embodiments involved are not described here again.

[0213] Step S503: fitting the light intensity time series data of the ROI to generate an oscillation curve describing the change of light intensity over time, and separating out the noise component.

[0214] In some embodiments, the optional implementation of step S503 can refer to the optional implementation of step S201 of Figure 2 , and Figure 2 other related parts in the embodiments involved are not described here again.

[0215] Step S504: generating a correction function describing the change of displacement error over time based on the noise period of the noise component.

[0216] In some embodiments, the optional implementation of step S504 can refer to the optional implementation of step S202 of Figure 2 , and Figure 2 other related parts in the embodiments involved are not described here again.

[0217] Step S505: performing a displacement correction operation on the collected diffraction images based on the correction function to make the positions of the ROI in different frame diffraction images consistent.

[0218] In some embodiments, the optional implementation of step S505 can refer to the optional implementation of step S203 of Figure 2 , and Figure 2 other related parts in the embodiments involved are not described here again.

[0219] In some embodiments, the collection process of the diffraction images can include:

[0220] According to the image sampling frame rate, a video of the sample is sampled to obtain the diffraction images, and the video is obtained based on recording the electron diffraction patterns reflected by the sample during growth; the image sampling frame rate is used to synchronize the image sampling period with the rotation period of the sample rotation table, and the sample rotation table is used to load and rotate the sample.

[0221] In some embodiments, the optional implementation of the collection process of the diffraction images can refer to the same or similar parts in the foregoing diffraction image analysis method embodiments, which are not described here again.

[0222] In some embodiments, as shown in Figure 6 , Figure 6 is a correction function generation flow diagram according to an embodiment. The above step of generating a correction function describing the change of displacement error over time based on the noise period of the noise component can include:

[0223] Step S601: when the noise period is less than a preset period threshold, the noise period is equally divided and sampled to obtain peak positions corresponding to each segment time;

[0224] Step S602: based on coordinate difference values of the peak positions corresponding to each segment time relative to an initial center reference point of the ROI, a correction function is generated to describe displacement error changes over time.

[0225] For example, the period threshold can be pre-set according to expert experience, for example, the period threshold is set to 1s.

[0226] In some examples, in step S601, the noise period T can be equally divided into a preset number of segments (for example, the preset number can be 4 or 5, etc.), and the corresponding peak position of each segment is extracted. Among them, the influence degree of the noise component on the ROI at a certain time can be used to quantify the displacement error.

[0227] In some examples, in step S602, the coordinate difference values can include horizontal coordinate difference values and vertical coordinate difference values, a first correction function f_x(t) can be generated to describe horizontal displacement error changes over time based on the horizontal coordinate difference values of the peak positions corresponding to each segment time relative to the initial center reference point of the ROI, and a second correction function f_y(t) can be generated to describe vertical displacement error changes over time based on the vertical coordinate difference values of the peak positions corresponding to each segment time relative to the initial center reference point of the ROI. Here, the horizontal coordinate difference values and the vertical coordinate difference values can be expressed in pixel points.

[0228] The horizontal coordinate difference values corresponding to each segment time can be used to determine the drift trajectory of the ROI in the Y-axis direction. The vertical coordinate difference values corresponding to each segment time can be used to determine the drift trajectory of the ROI in the Y-axis direction.

[0229] For example, each horizontal coordinate difference value can be fitted (such as polynomial fitting) to generate the first correction function f_x(t), and each vertical coordinate difference value can be fitted (such as polynomial fitting) to generate the second correction function f_y(t).

[0230] In this embodiment, by equally dividing and sampling the noise period when the noise period is less than the preset period threshold, the local change characteristics in the noise period can be more accurately captured, and by generating the correction function based on the coordinate difference values of the peak positions corresponding to each segment time relative to the initial center reference point of the ROI, the displacement error can be accurately quantified. Therefore, a more accurate correction function can be generated to correct the diffraction image.

[0231] In some embodiments, the above step of performing displacement correction operation on the collected diffraction image based on the correction function comprises:

[0232] For each collected frame of diffraction image, the horizontal displacement error and the vertical displacement error of the ROI in the frame of diffraction image are calculated according to the correction function and the sampling time of the frame of diffraction image; and the frame of diffraction image is adjusted in displacement based on the horizontal displacement error and the vertical displacement error.

[0233] Exemplarily, for each collected frame of diffraction image, the horizontal displacement error and the vertical displacement error of the frame of diffraction image are calculated according to the correction function, and the displacement error is (Δxi, Δyi), and the image is moved by -Δxi and -Δyi along the x direction (horizontal direction) and the y direction (vertical direction) respectively, that is, the displacement correction of the frame of diffraction image is completed.

[0234] It should be noted that after the displacement correction operation is completed on all collected diffraction images based on the correction function, the image frame sequence and the light intensity time sequence data of the ROI can be reacquired based on the corrected diffraction images for multi-modal analysis.

[0235] The image as a whole is denoised by using mean filtering, Gaussian filtering and the like, which is simple to operate but lacks targeted correction, and when processing a detailed picture, key information is easily lost. In the embodiment of the present application, the oscillation curve representing the periodic variation law is extracted by fitting the light intensity time sequence data of the ROI, and the noise component is separated, which not only accurately reflects the real dynamic behavior of the sample surface, but also effectively analyzes the error introduced by the slight vibration of the equipment; the correction function is generated based on the periodic characteristics of the noise component, which can quantitatively describe the characteristics of the displacement error changing with time, and the collected diffraction images are corrected in displacement by using the correction function, which can compensate the displacement error of the ROI in each frame of diffraction image, that is, by introducing the adaptive micro-motion denoising algorithm, the automatic correction of data is realized, thereby improving the stability of the data, so as to provide more reliable data support for the multi-modal analysis of the diffraction image, which helps to improve the accuracy and reliability of the diffraction image data analysis.

[0236] In a long time experiment, the vibration of the equipment may cause error accumulation, therefore, in some possible embodiments, the displacement correction operation is performed on the collected diffraction images based on the correction function, which can include: taking the correction function as a global correction function to perform the displacement correction operation on the collected diffraction images and the diffraction images to be collected, so as to reduce the influence of the error on the data analysis.

[0237] For example, it is assumed that S frames of diffraction images can be sampled from the video of the sample, and the correction function is generated based on the light intensity time sequence data of the ROI in the first to m-th frames of diffraction images (m is greater than or equal to 3, and m is less than 1 / n of S, and n can be not less than 3), which can be used to correct the displacement error of the first to m-th frames of diffraction images, so that the positions of the ROI in the S frames of diffraction images after displacement correction are consistent.

[0238] In order to reduce the complexity of global calculation, effectively capture the local error characteristics, and dynamically generate a more accurate correction function, in some possible embodiments, a sliding window (the window step can be equal to the window width) can be used to generate the correction function, for example, after a preset number of frames (the preset number of frames is greater than or equal to 3) of diffraction images are accumulated, the preset number of frames of diffraction images obtained by the current sampling are used as the diffraction images in the nth window. The correction function f n is generated based on the light intensity time sequence data of the ROI in the preset number of frames of diffraction images in the nth window. The correction function f n can be used to describe the characteristics of the displacement error of the ROI in the nth window changing with time. The displacement correction operation is performed on each frame of diffraction image in the nth window, so that the ROI remains consistent between different frames of diffraction images in the nth window. In this way, by using the sliding window, the correction function can be dynamically adjusted, which can more accurately capture the drift characteristics of the ROI in a local range, thereby reducing the error accumulation problem caused by equipment vibration in a long time experiment. n

[0239] For example, assuming that a plurality of frames of diffraction images can be sampled from the video of the sample, if a sliding window is used to generate the correction function, for example, s frames of diffraction images are accumulated as a window (i.e., the preset number of frames is s), the first window contains the first to s frames of images; the second window contains the s+1 to 2s frames of images (the sliding window moves s frames backward), and so on. Based on the light intensity time sequence data of the ROI in the s frames of images in the nth window, the correction function is generated, which can be used to correct the displacement error of the s frames of diffraction images in the nth window, so that the positions of the ROI in the plurality of frames of diffraction images after displacement correction are consistent. With the sliding of the window and the updating of the correction function, the positions of the ROI in the diffraction images in each window tend to be consistent, and finally the consistency of the ROI between different frames of diffraction images is realized. n

[0240] In some possible embodiments, after the correction of the nth window is completed, the correction function f n can be used as reference information, and a new correction function f n+1 is generated based on the preset number of frames of diffraction images sampled in the nth+1 window. The correction of the nth+1 window, since each time a new correction function is generated, it is based on the correction function of the previous window and the data of the current window, and this recursive manner ensures that the correction function can gradually adapt to the change trend of the ROI drift, thereby improving the correction accuracy. Moreover, the correction function has continuity in time, which can avoid the sudden change problem caused by independent processing of the window, thereby more effectively reducing the error accumulation problem caused by equipment vibration in a long time experiment. n+1

[0241] In some embodiments, the method can further comprise:

[0242] In the case where the correction function is enabled, performing the displacement correction operation, and outputting a comparison view of the oscillation curve of the ROI before displacement correction and the oscillation curve of the ROI after displacement correction. The implementation of this step can refer to the same or similar parts in the foregoing embodiments of the diffraction image analysis method, and will not be described here again.

[0243] In some embodiments, after performing the displacement correction operation, the method can further comprise:

[0244] For the ROI with the same position in different diffraction images, obtaining the image frame sequence and the light intensity time sequence data of the ROI; performing multi-modal analysis on the image frame sequence and the light intensity time sequence data of the ROI to obtain an analysis result.

[0245] The implementation of obtaining the image frame sequence and the light intensity time sequence data of the ROI in this step can refer to the optional implementation of step S102 in the foregoing embodiments of the diffraction image analysis method, and Figure 1 other associated parts in the embodiments involved, and will not be described here again.

[0246] The implementation of performing multi-modal analysis on the image frame sequence and the light intensity time sequence data of the ROI to obtain an analysis result in this step can refer to the optional implementation of step S103 in the foregoing embodiments of the diffraction image analysis method, and Figure 1 other associated parts in the embodiments involved, and will not be described here again.

[0247] It is worth noting that, without contradiction, each embodiment of the diffraction image analysis method and each embodiment of the diffraction image processing method can be arbitrarily combined, for example, part or all steps of different embodiments can be arbitrarily combined, and a certain embodiment can be arbitrarily combined with the optional implementation of other embodiments.

[0248] Next, the diffraction image analysis method provided by the embodiments of the present application will be described in combination with Figures 7a to 7d .

[0249] To solve at least one of the problems of the traditional RHEED image analysis software, such as limited functions, insufficient intelligent prediction ability, and data precision affected by device vibration interference, the embodiments of the present application provide a diffraction image analysis method, which can be applied to a diffraction image analysis system.

[0250] The embodiments of the present application introduce deep learning and multi-modal large model AI (artificial intelligence) methods by adopting a combination of deep learning algorithms, multi-modal large model analysis, and adaptive micro-motion denoising image processing scheme to provide more accurate RHEED image analysis and real-time prediction for users. Through the micro-motion denoising algorithm and AI optimization module, the software can intelligently identify the small vibrations and other noises of the equipment during the experiment, automatically adjust the data acquisition and processing strategy, solve the unavoidable equipment error problem in the existing analysis software, and ensure the reproducibility and high precision of the experimental results. By providing more diverse ROI selection tools such as elliptical, circular frame selection, and supporting user-defined function analysis of image data, providing language selection function, adapting to various language environments, the application range and flexibility of the existing software are expanded, and strong technical support is provided for various high-precision material growth monitoring scenarios.

[0251] Figure 7a is a schematic diagram of the architecture of a diffraction image analysis system according to an embodiment. Referring to Figure 7a , the system includes a license verification module, an image acquisition subsystem, an image analysis subsystem, and a data management subsystem.

[0252] The license verification module is used for the software license of the electronic device, wherein the software license generated based on the universally unique identifier (UUID) of different electronic devices is different, so as to ensure the confidentiality and integrity of the diffraction image analysis software.

[0253] The image acquisition subsystem is used to call the industrial camera interface, and real-time transfer the acquired original RHEED image to the subsequent module. The OpenCV library is used to realize the acquisition of high-frame-rate video, and the image clarity is improved through image enhancement algorithms such as green channel enhancement, brightness, contrast, and Gamma adjustment, so as to ensure that the fine diffraction stripes in the image can be clearly displayed under complex working conditions, thereby providing data support for subsequent analysis. The image after image enhancement processing can be exported as a screenshot or a video or in CSV format (Comma-Separated Values, a data exchange format) for subsequent analysis.

[0254] The image analysis subsystem includes an ROI region analysis module, a straight line light intensity analysis module, a rotation matching and adaptive denoising module, and an AI multi-modal large model (such as the image processing model in the foregoing embodiments).

[0255] The ROI region analysis module can be used to receive the image after image enhancement processing, allow the user to select multiple ROI regions (regions of interest) through front-end interaction, and analyze the ROI image frame sequence. By providing ROI selection tools such as rectangle, ellipse, and circle, the user can flexibly select the region to be analyzed on the interface.

[0256] The ROI region analysis module can also be used for time series data collection of light intensity in the ROI region, obtaining light intensity time series data, extracting periodic information from the light intensity time series data using Fast Fourier Transform (FFT), obtaining a fitting curve by Gaussian, Lorentz or custom function fitting and automatically finding a peak, drawing a trigonometric function fitting curve, and intuitively presenting a mathematical expression for easy quantitative analysis by engineers. By increasing various fitting algorithms, the function of automatically outputting the curve combined with the mathematical expression is provided, and users can accurately understand the analysis model and parameters, improving the accuracy of data interpretation.

[0257] The straight-line light intensity analysis module can be used for classification of diffraction images based on a ResNet convolutional neural network, and output a grid structure of m*n diffraction fringes, i.e., obtaining diffraction fringe arrangement information.

[0258] The rotation matching and adaptive denoising module is located after the straight-line light intensity analysis module and forms a closed loop feedback with the data acquisition subsystem for automatic correction of image data to eliminate errors caused by device vibration and ensure data stability.

[0259] The AI multi-modal large model can perform ROI-based image frame processing and light intensity time series data prediction to obtain analysis results and generate experiment reports, trend prediction and parameter optimization suggestions, etc.

[0260] The data management subsystem includes a data storage module, an abnormality detection and warning module, and a real-time playback module.

[0261] The data storage module can be used to receive and store data output by the image analysis subsystem and interact with the front-end display interface. It supports multiple data formats for export, such as CSV files, images, AVI (Audio Video Interleave) videos, etc., and can store data accurately according to the user's custom FPS (Frames Per Second).

[0262] The abnormality detection and warning module can be used for state monitoring, rule checking and abnormality detection, and through automatic pop-up reminders and log records to ensure reasonable operation and accurate data.

[0263] The real-time playback module can be used to play back the data stored during the experiment in a visual manner in real time, and can feed the data back to the image analysis subsystem for model optimization.

[0264] Figure 7b is a flowchart of a rotation matching and micro-motion denoising according to an embodiment. Referring to FIG. 8, the rotation matching and micro-motion denoising can include the following steps. Figure 7b, the data acquisition is adjusted by user parameters (such as RPM, FPS, ROI setting), and a micro-motion compensation function is fitted by time series analysis and mathematical methods to realize automatic correction.

[0265] After the user inputs the RPM of the current sample rotation table rotation, the rotation period is calculated and the data acquisition frequency FPS is automatically adjusted to ensure accurate sampling of one frame of image per rotation period to synchronize multi-period data; this function can not only be applied to real-time video, but also to recorded video. By time series sampling of light intensity data in the fixed ROI, light intensity time series data is obtained, and the light intensity time series data of the ROI is fitted by a polynomial (for example, using the least squares method) to generate an oscillation curve describing the change of light intensity with time, and the noise component is separated. The period information of the noise component is obtained by Fourier transform; the noise period of the noise component is usually not more than 1s, and if the period is too large, a warning prompt is directly given. If the period is less than 1s, then segment sampling is performed for each period, and the peak position coordinates of each equal division time are extracted, and the position coordinates are compared with the center reference point coordinates to solve the micro-motion correction functions f_x(t), f_y(t) in the x-axis and y-axis directions. The micro-motion correction function (i.e. the correction function in the foregoing embodiment, which can also be called: compensation function) is applied to the real-time sampled image frame to correct the drift of the ROI with time, thereby eliminating the displacement error caused by the correction of mechanical vibration or environmental noise. The user can choose whether to apply the micro-motion correction function, and in the case of applying the micro-motion correction function, a comparison image of the oscillation curves (i.e. intensity curves) before and after correction can be output. Compared with the traditional RHEED image analysis software which is difficult to deal with signal drift introduced by equipment vibration, the present application realizes automatic correction of data by introducing an adaptive micro-motion denoising algorithm, greatly improving the stability and repeatability of experimental data.

[0266] Figure 7c is a schematic diagram of the internal logic of an AI multi-modal large model according to an embodiment. Referring to Figure 7c , the AI multi-modal large model performs prediction based on ROI image frame processing and light intensity time series data, obtains analysis results, and sends the analysis results to a data storage module.

[0267] The input of the AI multimodal large model comes from the image frame sequence output by the ROI region analysis module and the light intensity time series data provided by the straight light intensity analysis module, forming a multi-source input data stream. The visual spatial features of the ROI image frame sequence can be extracted based on the ResNet convolutional neural network, and the light intensity time series data of the ROI can be encoded by the Transformer or LSTM network (Long Short Term Memory neural network) to capture the dynamic change trend of the diffraction signal, i.e., to obtain the light intensity time series features of the ROI. The visual spatial features and the light intensity time series features are subjected to deep multimodal interaction through a joint attention mechanism (Cross-Attention) in a fusion layer to obtain multimodal features.

[0268] The multimodal features and the light intensity time series features are respectively sent to two intelligent reasoning branches: one is a pattern classifier, which can obtain diffraction stripe arrangement information obtained by classifying diffraction images based on a ResNet convolutional neural network, and perform pattern classification based on the diffraction stripe arrangement information and the multimodal features, and finally automatically identify lattice types, surface defect forms, and stripe arrangement types, etc., thereby significantly improving the accuracy of engineers in judging the material state; the other is a regression predictor, which outputs key quantitative indicators such as oscillation period and growth rate by analyzing the light intensity time series features. In addition, parameter optimization suggestions such as recommended exposure time and RPM value can be generated based on historical samples and prediction trends.

[0269] All analysis results can be further transmitted to an NLP (Natural Language Processing) module to form an explainable report based on image-text joint learning, to assist users in efficiently understanding the experimental state and adjusting the strategy.

[0270] In addition, the AI multimodal large model supports self-supervised learning, and the system automatically records data for model fine-tuning after each acquisition and analysis, realizes long-term stable evolution of the model, and improves the adaptability under different material systems and experimental conditions.

[0271] Traditional RHEED image analysis software generally lacks the application of deep learning, and even more lacks the cross-modal data fusion and intelligent reasoning mechanism. The multimodal large model based on visual spatial features and light intensity time series features is integrated into the RHEED image analysis process in the embodiments of the present application, which not only realizes the lightweight deployment and real-time calling on the local server, but also realizes an integrated closed-loop system from data acquisition to intelligent decision-making, significantly improves the automation, accuracy and application breadth of image analysis, and provides strong technical support for the semiconductor material growth monitoring scene.

[0272] Figure 7d is a ROI region and its oscillation curve display schematic diagram according to an embodiment. Referring to Figure 7dIn the left display area of the UI interface, multiple ROIs selected by the user are displayed, each of which is identified by a different style of frame line, such as a border of different colors or line types (for ease of display, in Figure 7d different dashed frame lines are used in the figure to distinguish different ROIs). After multiple ROIs are selected, the software automatically analyzes and processes the time-series light intensity data of each ROI, and after the analysis and processing are completed, a trend curve window is popped up to display the normalized light intensity change curves corresponding to each ROI. Each curve is drawn in a style matching the corresponding ROI frame to facilitate user identification. After normalization, all curves can be compared in a unified vertical coordinate range: areas with larger changes show obvious fluctuations, and areas with smaller changes tend to be stable (showing slight fluctuations or approximately straight lines), thus more intuitively reflecting the dynamic characteristics of each area. In addition, the right display area of the UI interface displays the actual light intensity values corresponding to each ROI.

[0273] In summary, the embodiments of the present application can adopt modular design, with efficient data interfaces between the functional modules to achieve a perfect closed-loop system from image acquisition to intelligent analysis and data storage, compact structure, strong expandability, and internal integration of deep learning and AI multi-modal large model modules, which can achieve higher level of intelligent upgrade.

[0274] In terms of function, real-time image acquisition and adaptive image enhancement technology ensure high-quality RHEED image acquisition, provide multiple ROI selection and signal analysis methods, and realize accurate oscillation period detection in combination with Fourier transform and automatic peak searching algorithm. Deep learning and large model support realize intelligent extraction of image features and pattern recognition, automatically generate experiment reports, trend prediction and parameter optimization suggestions, and significantly improve the intelligent level of the system. In terms of process, the software can be developed based on Python and Qt, using existing open source libraries and large model interfaces, with short development cycle and controllable cost. At the same time, full use is made of existing MBE equipment in-situ monitoring means to realize real-time data feedback and automatic correction, ensuring stable process flow. In terms of cost, no additional hardware upgrade is needed, and function expansion can be realized through software algorithm upgrade, greatly reducing the overall application cost; and specific functional modules can be selected according to actual needs, with high customization degree and avoiding resource waste.

[0275] Compared with traditional RHEED image analysis software, the technical solution provided by the embodiments of the present application is more suitable for semiconductor material growth monitoring, surface structure analysis and related high-precision experimental scenarios.

[0276] Figure 8 is a structural schematic diagram of a diffraction image analysis device according to an embodiment. As Figure 8As shown, the diffraction image analysis apparatus 800 comprises an acquisition unit 801, a processing unit 802 and an analysis unit 803.

[0277] The acquisition unit 801 is configured to acquire a plurality of frames of diffraction images collected in a growth process of a sample, the diffraction images containing electron diffraction patterns generated by interaction of a high-energy electron beam with the sample;

[0278] The processing unit 802 is configured to determine a region of interest (ROI) in the diffraction images, and acquire a sequence of image frames and light intensity time series data of the ROI;

[0279] The analysis unit 803 is configured to perform multi-modal analysis on the sequence of image frames and the light intensity time series data of the ROI to obtain an analysis result.

[0280] In some embodiments, the apparatus further comprises a correction unit, which comprises:

[0281] A first generating sub-unit configured to fit an oscillation curve for describing variation of light intensity with time to the light intensity time series data of the ROI, and separate a noise component;

[0282] A second generating sub-unit configured to generate a correction function for describing variation of displacement error with time based on a noise period of the noise component;

[0283] A correction sub-unit configured to perform a displacement correction operation on the collected diffraction images based on the correction function, so that positions of the ROI in different frames of the diffraction images are consistent.

[0284] In some embodiments, the acquisition unit 801 is configured to:

[0285] Sample diffraction images from a video of the sample based on an image sampling frame rate, the video being obtained based on recording electron diffraction patterns reflected by the sample in a growth process; the image sampling frame rate is used to synchronize an image sampling period with a rotation period of a sample rotation stage, the sample rotation stage being used to hold and rotate the sample.

[0286] In some embodiments, the second generating sub-unit is configured to:

[0287] When the noise period is less than a preset period threshold, perform equal-division segmentation sampling on the noise period to obtain peak position corresponding to each segmented time;

[0288] Generate a correction function for describing variation of displacement error with time based on coordinate difference values of the peak positions corresponding to each segmented time relative to an initial center reference point of the ROI.

[0289] In some embodiments, the correction sub-unit is configured to:

[0290] For each frame of the collected diffraction images, according to the correction function and the sampling time of the diffraction image, the horizontal displacement error and the vertical displacement error of the ROI in the diffraction image are calculated;

[0291] Based on the horizontal displacement error and the vertical displacement error, each frame of the collected diffraction images is adjusted in displacement.

[0292] In some embodiments, the correction subunit is further configured to:

[0293] When the correction function is enabled, the displacement correction operation is performed, and a comparison view of the oscillation curves before and after displacement correction of the ROI is output.

[0294] In some embodiments, the processing unit 802 is configured to:

[0295] In response to a region selection operation of a user on the diffraction image through an ROI selection tool, the ROI is determined in the diffraction image, and the ROI selection tool is any one of a rectangular region selection tool and a non-rectangular region selection tool.

[0296] In some embodiments, the processing unit 802 is further configured to:

[0297] The period information is obtained by performing frequency domain analysis on the light intensity time series data of the ROI;

[0298] A fitting curve is obtained by fitting the light intensity time series data using a selected fitting function from a plurality of fitting functions, and a peak value in the fitting curve is identified;

[0299] A trigonometric function fitting curve is generated based on the peak value and the period information, and the trigonometric function fitting curve and the corresponding mathematical expression are output.

[0300] In some embodiments, the analysis result includes a sample material structure prediction result and / or a sample growth behavior prediction result; the sample material structure prediction result indicates at least one of a lattice type and a surface defect type; and the sample growth behavior prediction result indicates at least one of an oscillation period and a growth rate.

[0301] In some embodiments, the analysis unit 803 is configured to:

[0302] The visual spatial features are extracted from the image frame sequence through a feature extraction layer of the image processing model, and the light intensity time series features are extracted from the light intensity time series data;

[0303] The visual space feature and the light intensity time sequence feature are fused by a feature fusion layer based on an attention mechanism of the image processing model to obtain a multi-modal feature.

[0304] The multi-modal feature is input into an inference layer of the image processing model to perform inference based on the multi-modal feature to obtain the analysis result.

[0305] In some embodiments, the analysis unit 803 is configured to:

[0306] The multi-modal feature is input into a pattern classifier in the inference layer to perform prediction to obtain the sample material structure prediction result, and the multi-modal feature is input into a regression predictor in the inference layer to perform prediction to obtain the sample growth behavior prediction result.

[0307] In some embodiments, the processing unit is further configured to:

[0308] The diffraction image is processed based on a convolutional neural network to obtain diffraction fringe arrangement information.

[0309] The analysis unit 803 is configured to:

[0310] The diffraction fringe arrangement information and the multi-modal feature are input into the pattern classifier to perform prediction to obtain the sample material structure prediction result, wherein the sample material structure prediction result further includes a fringe arrangement type.

[0311] In some embodiments, the analysis result further includes a recommended value of an experimental parameter; and the analysis unit 803 is further configured to:

[0312] Historical experimental data and current experimental data are matched based on experimental parameter values to obtain sample growth behavior prediction results corresponding to a plurality of groups of experimental parameter values respectively, wherein each experimental data includes a group of experimental parameter values and a sample growth behavior prediction result of a sample under the group of experimental parameter values.

[0313] A trend curve is generated based on a plurality of groups of the experimental parameter values and the sample growth behavior prediction results corresponding to the plurality of groups of the experimental parameter values respectively.

[0314] A group of experimental parameter values that optimizes the sample growth behavior prediction result is searched for in the trend curve to obtain a recommended value of an experimental parameter; wherein the experimental parameter includes at least one of an image exposure time, a sample rotation table rotation speed, a high-energy electron beam energy, and an electron beam irradiation angle.

[0315] The diffraction image analysis device provided by the embodiments of the present application belongs to the same application concept as the diffraction image analysis method provided by the above embodiments of the present application, can execute the diffraction image analysis method provided by any of the above embodiments of the present application, and has the corresponding function modules and beneficial effects of executing the diffraction image analysis method. The technical details not described in detail in the embodiments can refer to the specific processing content of the diffraction image analysis method provided by the above embodiments of the present application, which will not be described here.

[0316] Figure 9 FIG. 9 is a structural schematic diagram of a diffraction image processing device according to an embodiment. As shown in FIG. 9, the diffraction image processing device 900 includes: Figure 9

[0317] An acquisition unit 901 is configured to acquire a plurality of frames of diffraction images collected in a growth process of a sample, the diffraction images containing electron diffraction patterns generated by interaction of a high-energy electron beam with the sample.

[0318] A processing unit 902 is configured to determine a region of interest (ROI) in the diffraction images and acquire light intensity time series data of the ROI.

[0319] A correction unit 903 includes:

[0320] A first generation sub-unit 9031 is configured to fit an oscillation curve for describing variation of light intensity with time to the light intensity time series data of the ROI and separate a noise component.

[0321] A second generation sub-unit 9032 is configured to generate a correction function for describing variation of displacement error with time based on a noise period of the noise component.

[0322] A correction sub-unit 9033 is configured to perform a displacement correction operation on the collected diffraction images based on the correction function, so that the position of the ROI in different frames of the diffraction images is consistent.

[0323] In some embodiments, the acquisition unit 901 is configured to:

[0324] sample the diffraction images from a video of the sample according to an image sampling frame rate, the video being obtained based on recording electron diffraction patterns reflected by the sample in the growth process; and the image sampling frame rate is used to synchronize an image sampling period with a rotation period of a sample rotation stage, the sample rotation stage being used to rotate the sample.

[0325] In some embodiments, the second generation sub-unit 9032 is configured to:

[0326] when the noise period is less than a preset period threshold, perform equal-division segmentation sampling on the noise period to obtain peak positions corresponding to each segmented time.​

[0327] generate a correction function for describing the displacement error changing with time based on coordinate difference values of the peak positions corresponding to the segment moments relative to an initial center reference point of the ROI.

[0328] In some embodiments, the correction sub-unit 9033 is configured to:

[0329] For each captured frame of the diffraction image, calculate horizontal displacement error and vertical displacement error of the ROI in the diffraction image according to the correction function and sampling time of the diffraction image;

[0330] Based on the horizontal displacement error and the vertical displacement error, perform displacement adjustment on each captured frame of the diffraction image.

[0331] In some embodiments, the apparatus further comprises an analysis unit;

[0332] The processing unit 902 is further configured to, for the ROI with consistent positions in different diffraction images, acquire image frame sequence and light intensity time series data of the ROI;

[0333] The analysis unit is configured to perform multi-modal analysis on the image frame sequence and light intensity time series data of the ROI to obtain an analysis result.

[0334] In some embodiments, the correction sub-unit is further configured to:

[0335] When the correction function is enabled, perform the displacement correction operation and output a comparison view of the oscillation curve of the ROI before displacement correction and the oscillation curve of the ROI after displacement correction.

[0336] In some embodiments, the processing unit is configured to:

[0337] In response to a user's region selection operation on the diffraction image through an ROI selection tool, determine the ROI in the diffraction image, the ROI selection tool being any one of a rectangular region selection tool and a non-rectangular region selection tool.

[0338] In some embodiments, the processing unit is further configured to:

[0339] Perform frequency domain analysis on the light intensity time series data of the ROI to obtain period information;

[0340] Fit the light intensity time series data with a selected fitting function from multiple fitting functions to obtain a fitting curve, and identify a peak value in the fitting curve;

[0341] Generate a trigonometric function fitting curve based on the peak value and the period information, and output the trigonometric function fitting curve and the corresponding mathematical expression.

[0342] In some embodiments, the analysis result includes: a sample material structure prediction result and / or a sample growth behavior prediction result; the sample material structure prediction result indicates at least one of a lattice type and a surface defect type; the sample growth behavior prediction result indicates at least one of an oscillation period and a growth rate.

[0343] In some embodiments, the analysis unit is configured to:

[0344] extract visual spatial features from the image frame sequence and light intensity time sequence features from the light intensity time sequence data through a feature extraction layer of the image processing model;

[0345] fuse the visual spatial features and the light intensity time sequence features through an attention mechanism-based feature fusion layer of the image processing model to obtain multi-modal features;

[0346] obtain the analysis result based on the multi-modal features through an inference layer of the image processing model.

[0347] In some embodiments, the analysis unit is configured to:

[0348] input the multi-modal features into a pattern classifier in the inference layer for prediction to obtain the sample material structure prediction result, and input the multi-modal features into a regression predictor in the inference layer for prediction to obtain the sample growth behavior prediction result.

[0349] In some embodiments, the processing unit is further configured to:

[0350] process the diffraction image based on a convolutional neural network to obtain diffraction fringe arrangement information;

[0351] The analysis unit is configured to:

[0352] input the diffraction fringe arrangement information and the multi-modal features into the pattern classifier for prediction to obtain the sample material structure prediction result, wherein the sample material structure prediction result further includes a fringe arrangement type.

[0353] In some embodiments, the analysis result further includes recommended values of experimental parameters; and the analysis unit is further configured to:

[0354] The historical experimental data and the current experimental data are matched based on experimental parameter values to obtain sample growth behavior prediction results corresponding to a plurality of sets of experimental parameter values respectively, wherein each experimental data includes a set of experimental parameter values and a sample growth behavior prediction result of a sample under the set of experimental parameter values.

[0355] A trend curve is generated based on the plurality of sets of experimental parameter values and the sample growth behavior prediction results corresponding to the plurality of sets of experimental parameter values respectively.

[0356] A set of experimental parameter values that optimizes the sample growth behavior prediction results is searched for in the trend curve to obtain recommended values of experimental parameters, wherein the experimental parameters include at least one of an image exposure time, a sample rotating table rotating speed, a high-energy electron beam energy, and an electron beam irradiation angle.

[0357] The diffraction image processing apparatus provided by the embodiments of the present application belongs to the same application concept as the diffraction image processing method provided by the above embodiments of the present application, can execute the diffraction image processing method provided by any of the above embodiments of the present application, and has the corresponding function modules and beneficial effects of executing the diffraction image processing method. Technical details not described in detail in the present embodiment can be referred to the specific processing content of the diffraction image processing method provided by the above embodiments of the present application, which will not be described here.

[0358] It should be understood that one or more units in the above diffraction image processing apparatus can be integrated in the diffraction image analysis apparatus in the above embodiments. Each unit in the diffraction image analysis apparatus and the diffraction image processing apparatus can be implemented in the form of processor calling software, or each unit can be implemented in the form of hardware circuit, and the function of part or all units can be realized through the design of hardware circuit, which can be understood as one or more processors.

[0359] The embodiments of the present application also provide an electronic device, which includes a processor, a memory, and an executable program stored on the memory and capable of being executed by the processor, and the processor executes the steps of the diffraction image analysis method or the diffraction image processing method provided by any one of the above embodiments when executing the executable program.

[0360] The embodiments of the present application also provide a computer readable storage medium, which stores an executable program, and the executable program implements the steps of the diffraction image analysis method or the diffraction image processing method provided by any one of the above embodiments when executed by a processor.

[0361] For the convenience of understanding, the following focuses on the description of the terms used in the present embodiment:

[0362] In the embodiments of the present application, the processor is a circuit with signal processing capability. In one implementation, the processor can be a circuit with instruction reading and running capability, such as a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU) (which can be understood as a microprocessor), a digital signal processing circuit (DSP), or the like. In another implementation, the processor can implement certain functions through a logical relationship of hardware circuits, and the logical relationship of the hardware circuits is fixed or reconfigurable. For example, the processor is a hardware circuit implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as an FPGA. In the reconfigurable hardware circuit, the processor loads a configuration document to implement the configuration of the hardware circuit. It can be understood that the processor loads instructions to implement the functions of the above part or all units or modules. In addition, the hardware circuit can also be designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), a tensor processing unit (TPU), a deep learning processing unit (DPU), or the like.

[0363] The computer readable storage medium provided in the embodiments can execute the diffraction image analysis method or the diffraction image processing method of the above embodiments, and the implementation principle and technical effects are similar to those of the above embodiments, which will not be described here.

[0364] The computer readable storage medium described above can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk or an optical disk. The readable storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0365] An example readable storage medium is coupled to the processor such that the processor can read information from, and write information to, the readable storage medium. Of course, the readable storage medium can be a part of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can exist as discrete components in the electronic device or as part of a control unit.

[0366] Those skilled in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by program instruction related hardware. The foregoing program can be stored in a computer readable storage medium. The program executes to perform the steps of the above-mentioned method embodiments; and the foregoing storage medium includes ROM, RAM, magnetic or optical disk, and various storage media that can store program codes.

[0367] The embodiments or implementations in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts between the embodiments can be referred to each other.

[0368] In the description of the specification, the description of "one embodiment", "some embodiments", "exemplary embodiment", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the specification, the exemplary description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0369] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A diffraction image analysis method, characterized in that, The method includes: Acquire multiple frames of diffraction images during the growth process of the sample, the diffraction images containing electron diffraction patterns generated by the interaction between a high-energy electron beam and the sample; The Region of Interest (ROI) is determined in the diffraction image, and the image frame sequence and light intensity temporal data of the ROI are obtained. Multimodal analysis was performed on the image frame sequence and light intensity time series data of the ROI to obtain the analysis results.

2. The diffraction image analysis method according to claim 1, characterized in that, Prior to performing multimodal analysis, the method further includes: The time-series light intensity data of the ROI is fitted to generate an oscillation curve describing the change of light intensity over time, and the noise component is separated. Based on the noise period of the noise components, a correction function is generated to describe the change of displacement error over time. The acquired diffraction images are subjected to displacement correction based on the correction function so that the position of the ROI is consistent in different frames of the diffraction images.

3. The diffraction image analysis method according to claim 2, characterized in that, The process of acquiring the diffraction image includes: A diffraction image is obtained by sampling the video of the sample according to the image sampling frame rate. The video is obtained by recording the electron diffraction pattern reflected by the sample during the growth process. The image sampling frame rate is used to synchronize the image sampling period with the rotation period of the sample rotating stage, which is used to support the sample for rotation.

4. The diffraction image analysis method according to claim 2, characterized in that, The method further includes: With the correction function enabled, the displacement correction operation is performed, and a comparison view of the oscillation curves of the ROI before and after displacement correction is output.

5. The diffraction image analysis method according to claim 1, characterized in that, Determining the ROI in the diffraction pattern includes: In response to a user's region selection operation on the diffraction image using a ROI selection tool, an ROI is determined in the diffraction image, wherein the ROI selection tool is either a rectangular region selection tool or a non-rectangular region selection tool.

6. The diffraction image analysis method according to claim 1, characterized in that, The method further includes: Frequency domain analysis was performed on the light intensity time series data of the ROI to obtain periodic information; The light intensity time series data is fitted using a selected fitting function from a variety of fitting functions to obtain a fitting curve, and the peak value in the fitting curve is identified. Based on the peak value and the period information, a trigonometric function fitting curve is generated, and the trigonometric function fitting curve and the corresponding mathematical expression are output.

7. The diffraction image analysis method according to any one of claims 1 to 6, characterized in that, The analysis results include: sample material structure prediction results and / or sample growth behavior prediction results; The predicted material structure of the sample indicates at least one of the lattice type and surface defect type; The predicted growth behavior of the sample indicates at least one of the oscillation period and growth rate.

8. The diffraction image analysis method according to claim 7, characterized in that, The multimodal analysis is performed on the image frame sequence and light intensity temporal data of the ROI, and the analysis results are output, including: Visual spatial features are extracted from the image frame sequence through the feature extraction layer of the image processing model, and light intensity temporal features are extracted from the light intensity temporal data. The visual spatial features and the temporal features of light intensity are fused through the attention-based feature fusion layer of the image processing model to obtain multimodal features; The analysis results are obtained by inferring based on the multimodal features through the inference layer of the image processing model.

9. The diffraction image analysis method according to claim 8, characterized in that, The analysis results are obtained by inferring based on the multimodal features through the inference layer of the image processing model, including: The multimodal features are input into the pattern classifier in the inference layer for prediction to obtain the predicted result of the sample material structure. The multimodal features are input into the regression predictor in the inference layer for prediction, and the prediction result of the sample growth behavior is obtained.

10. The diffraction image analysis method according to claim 9, characterized in that, The method further includes: The diffraction image is processed using a convolutional neural network to obtain diffraction fringe arrangement information; The step of inputting the multimodal features into the pattern classifier in the inference layer for prediction to obtain the sample growth behavior prediction result includes: The diffraction fringe arrangement information and the multimodal features are input into the pattern classifier for prediction to obtain the sample material structure prediction result, wherein the sample material structure prediction result also includes the fringe arrangement type.

11. The diffraction image analysis method according to claim 7, characterized in that, The analysis results also include recommended values ​​for experimental parameters; the method further includes: By matching historical experimental data with current experimental data based on experimental parameter values, multiple sets of experimental parameter values ​​are obtained to predict the sample growth behavior. Each set of experimental data includes a set of experimental parameter values ​​and the sample growth behavior prediction result under the set of experimental parameter values. Based on the multiple sets of experimental parameter values ​​and the predicted growth behavior of the sample corresponding to the multiple sets of experimental parameter values, a trend curve is fitted and generated. Find the set of experimental parameter values ​​that best predict the growth behavior of the sample in the trend curve to obtain recommended values ​​for the experimental parameters; wherein the experimental parameters include at least one of the following: image exposure time, sample rotary stage speed, high-energy electron beam energy, and electron beam irradiation angle.

12. A diffraction image processing method, characterized in that, The method includes: Acquire multiple frames of diffraction images during the growth process of the sample, the diffraction images containing electron diffraction patterns generated by the interaction between a high-energy electron beam and the sample; The region of origin (ROI) is determined in the diffraction image, and the temporal data of the light intensity of the ROI is obtained. The time-series light intensity data of the ROI is fitted to generate an oscillation curve describing the change of light intensity over time, and the noise component is separated. Based on the noise period of the noise components, a correction function is generated to describe the change of displacement error over time. The acquired diffraction images are subjected to displacement correction based on the correction function so that the position of the ROI is consistent in different frames of the diffraction images.

13. The diffraction image processing method according to claim 12, characterized in that, The process of acquiring the diffraction image includes: A diffraction image is obtained by sampling the video of the sample according to the image sampling frame rate. The video is obtained by recording the electron diffraction pattern reflected by the sample during the growth process. The image sampling frame rate is used to synchronize the image sampling period with the rotation period of the sample rotating stage, which is used to support the sample for rotation.

14. The diffraction image processing method according to claim 12, characterized in that, The generation of a correction function based on the noise period of the noise components to describe the change of displacement error over time includes: When the noise period is less than a preset period threshold, the noise period is divided into equal segments for sampling to obtain the peak position corresponding to each segment time. Based on the coordinate difference between the peak position corresponding to each segment time point and the initial center reference point of the ROI, a correction function is generated to describe the change of displacement error over time.

15. The diffraction image processing method according to claim 12, characterized in that, The displacement correction operation performed on the acquired diffraction image based on the correction function includes: For each frame of diffraction image acquired, the horizontal displacement error and vertical displacement error of the ROI in the diffraction image are calculated based on the correction function and the sampling time of the diffraction image. The displacement of each frame of diffraction image acquired is adjusted based on the horizontal displacement error and the vertical displacement error.

16. The diffraction image processing method according to any one of claims 12 to 15, characterized in that, After performing the displacement correction operation, the method further includes: For ROIs that are in the same position in different frames of the diffraction image, obtain the image frame sequence and light intensity temporal data of the ROI; Multimodal analysis was performed on the image frame sequence and light intensity time series data of the ROI to obtain the analysis results.

17. A diffraction image analysis device, characterized in that, The device includes: The acquisition unit is used to acquire multiple frames of diffraction images collected during the growth process of the sample, wherein the diffraction images contain electron diffraction patterns generated by the interaction between the high-energy electron beam and the sample. The processing unit is used to determine the Region of Interest (ROI) in the diffraction image and acquire the image frame sequence and light intensity time series data of the ROI; The analysis unit is used to perform multimodal analysis on the image frame sequence and light intensity time series data of the ROI to obtain analysis results.

18. A diffraction image processing apparatus, characterized in that, The device includes: The acquisition unit is used to acquire multiple frames of diffraction images collected during the growth process of the sample, wherein the diffraction images contain electron diffraction patterns generated by the interaction between the high-energy electron beam and the sample. The processing unit is used to determine the region of interest (ROI) in the diffraction image and acquire the temporal data of the light intensity of the ROI. The calibration unit includes: The first generation subunit is used to fit the light intensity time series data of the ROI to generate an oscillation curve describing the change of light intensity over time, and to separate the noise component. The second generation subunit is used to generate a correction function that describes the change of displacement error over time based on the noise period of the noise component. The correction subunit is used to perform a displacement correction operation on the acquired diffraction image based on the correction function, so that the position of the ROI is consistent in different frames of the diffraction image.

19. An electronic device comprising a processor, a memory, and an executable program stored in the memory and executable by the processor, characterized in that, When the processor runs the executable program, it performs the diffraction image analysis method as described in any one of claims 1 to 11, or the diffraction image processing method as described in any one of claims 12 to 16.

20. A computer-readable storage medium having an executable program stored thereon, characterized in that, When the executable program is executed by the processor, it implements the diffraction image analysis method as described in any one of claims 1 to 11, or the diffraction image processing method as described in any one of claims 12 to 16.

Images