Labeled training data collection method and charged particle beam device

Abstract

The present invention discloses a method for collecting labeled training data that can be used to improve the predictive performance of a machine learning model to be used in automatic tuning of a charged particle beam device. With a labeled training data collection starting point set to a tuning completion point or to a point to which is applied a prescribed offset from the tuning completion point, imaging data is acquired while changing a control value in one direction from the collection starting point, and a labeled training data set containing a plurality of labeled training data, namely pairs of imaging data and a label for said imaging data, is collected. The direction in which to apply the offset is set to be the opposite direction of the direction in which to collect the labeled training data set.

Description

Teacher data collection method and charged particle beam device

[0001] The present disclosure relates to a training data method for collecting training data for a machine learning model and a charged particle beam device.

[0002] In the field of electron microscopes, there is a demand for reducing user skill requirements and automating observations. To achieve this, it is necessary to automate adjustments that were previously performed by users visually checking images. Furthermore, it is not enough to simply automate adjustments; there is a demand for automatic adjustments to be performed faster, with higher accuracy and a higher success rate. In recent years, in the field of image processing, it has been reported that methods using machine learning demonstrate higher performance than conventional algorithm-based methods. This is no exception when it comes to image processing for automatic adjustment of electron microscopes.

[0003] Patent Document 1 discloses a method for speeding up autofocus by using images captured under optical conditions in which astigmatism of a predetermined specification is introduced as training data to train a convolutional neural network (CNN) and estimate the amount of defocus from one or a very small number of images.

[0004] Patent Document 2 discloses a method of training a machine learning model using a calculated Ronchigram generated from a simulator as training data, and estimating an aberration value from the trained model.

[0005] JP 2020-187980 A JP 2022-124508 A

[0006] When creating a machine learning-based estimation model and automatically adjusting it in a charged particle beam device, the challenge is how to collect the large amount of training data required.

[0007] When collecting training data from an actual machine, a large number of imaging data sets are acquired continuously using a charged particle beam device while gradually changing the imaging conditions. This inevitably results in errors in the training data. Sources of errors include errors caused by the controlled object, such as magnetic hysteresis in electromagnetic lenses and lost motion in mechanical mechanisms. Furthermore, when the imaging data is, for example, SEM images, long-term observation with an electron beam can cause contamination and charge-up, potentially resulting in images used as training data captured in an environment different from the intended operating environment.

[0008] When training data is collected using a simulator, it is possible to avoid errors caused by the characteristics of these devices. However, it is difficult to perfectly reproduce images acquired from a real device using a simulator. Therefore, if training is performed using only images generated by a simulator, there is a limit to the predictive performance of the machine learning model in the actual operating environment of the real device.

[0009] The present disclosure aims to reduce the time required for automatic adjustment and improve accuracy by improving the predictive performance of machine learning models through improvements to the method of collecting training data from charged particle beam devices.

[0010] A teacher data collection method according to one embodiment of the present invention is a teacher data collection method for collecting teacher data for a machine learning model that performs automatic adjustment of a charged particle beam device, the machine learning model comprising a main body and a control device that controls the main body to acquire imaging data, the machine learning model being a machine learning model that estimates parameters for controlling the main body using imaging data as input data, the control device comprising a collection execution unit that collects teacher data, the collection execution unit controlling the parameters to identify a control value of the parameter when the main body is in a desired state as an adjustment completion point, setting a collection start point for the teacher data as the adjustment completion point or a point added a predetermined offset to the adjustment completion point, setting a data acquisition point for the teacher data by changing the control value of the parameter, Each time a score is set, a pair of imaging data and a label for that imaging data is obtained, and a teacher dataset containing multiple pairs is collected. The control values ​​of the data acquisition points of the teacher data that make up the teacher dataset are all increased or decreased compared to the control value of the collection start point, and the later-set data acquisition point is set to be farther from the collection start point than the earlier-set data acquisition point. The specified offset is set so that if the control value of the data acquisition point is set to be increased compared to the control value of the collection start point, the control value of the collection start point is decreased compared to the control value of the adjustment completion point, and if the control value of the data acquisition point is set to be decreased compared to the control value of the collection start point, the control value of the collection start point is increased compared to the control value of the adjustment completion point.

[0011] According to the present disclosure, the prediction accuracy of the adjustment completion point, which is particularly important in a machine learning model used for automatic adjustment of a charged particle beam device, is improved. Other objects and novel features will become apparent from the description of this specification and the accompanying drawings.

[0012] 1 is a schematic configuration diagram of a charged particle beam device; FIG. 2 is an example of the hardware configuration of a control device; FIG. 3 is a functional block diagram of a control device; FIG. 4 is a flow of automatic adjustment using a trained model; FIG. 5 is a diagram explaining desirable characteristics of a trained model; FIG. 6 is a flow of collecting teacher data; FIG. 7 is a data structure of a teacher dataset; FIG. 8 is a data structure of teacher data; FIG. 9 is an example of a GUI; FIG. 10 is a diagram explaining differences in the accuracy of a trained model that are expected depending on differences in the order in which teacher data is collected; FIG. 11 is a teacher data collection pattern when there are multiple control values; FIG. 12 is a teacher data collection pattern when there are multiple control values; FIG. 13 is a teacher data collection pattern when there are multiple control values.

[0013] Embodiments of the present disclosure will be described in detail with reference to the drawings. In the following embodiments, the configurations (including steps in the flowcharts) are not necessarily essential unless otherwise specified or considered to be clearly essential in principle. Preferred embodiments of the present disclosure will be described below with reference to the drawings. In the drawings, identical parts are generally designated by the same reference numerals, and repeated explanations will be omitted. In the drawings, the depiction of components may not represent their actual positions, sizes, shapes, scopes, etc., in order to facilitate understanding of the present disclosure.

[0014] 1A shows a schematic configuration diagram of a charged particle beam device 1. The charged particle beam device 1 is, for example, a device intended to observe a sample by irradiating the sample with charged particles, such as a scanning electron microscope (SEM) or a transmission electron microscope (TEM), or a device intended to process a sample, such as a focused ion beam (FIB) processing device. The following description will be given using an SEM as an example, unless otherwise specified.

[0015] The main body 2 includes a stage mechanism for placing a sample, an electron optical system for irradiating the sample placed on the stage with an electron beam, and a detector for detecting signal electrons or electromagnetic waves emitted from the sample in response to the electron beam irradiation. The stage mechanism includes, for example, a stepping motor and a piezoelectric element for adjusting the position of the sample, and the electron optical system includes an electromagnetic lens and an electrostatic lens for controlling the focusing state of the electron beam, and a deflector for controlling the irradiation position. Each of these components is controlled by an individual controller 3. A user can use the main body 2 to acquire a desired SEM image by comprehensively controlling the individual controllers 3 from a control device 4.

[0016] FIG. 1B shows an example of the hardware configuration of the control device 4. The control device 4 has a configuration as an information processing device, and its main components include a processor (Central Processing Unit: CPU) 11, memory 12, storage device 13, input interface (I / F) 14, output I / F 15, communication I / F 16, and bus 17. The processor 11 functions as a functional unit that provides a predetermined function by executing processing in accordance with a program loaded in the memory 12. The storage device 13 stores data and programs used by the functional unit. The input I / F 14 is connected to an input device 18 such as a keyboard, pointing device, or operation panel, and the output I / F 15 is connected to a display device 19. The communication device I / F 16 enables communication with other information processing devices via a network. These are connected to each other so that they can communicate with each other via the bus 17.

[0017] In the following description, when describing processing by a program, the program or functional units may be described as the main components, but the main hardware components are a processor or an information processing device configured to include the processor. The information processing device executes processing according to a program loaded into memory using resources such as memory and a communication interface as appropriate. While FIG. 1B shows an example of a CPU as the processor, a GPU (Graphical Processing Unit) or the like may also be used. Furthermore, processing to realize a function is not limited to software program processing, and can also be implemented using a dedicated circuit. Examples of the dedicated circuit include a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), and a complex programmable logic device (CPLD).

[0018] The program may be installed as data in advance in the control device 4, or may be distributed as data from a program source to the control device 4 and then installed. The program source may be a program distribution server connected via a network, or a non-volatile computer-readable storage medium (e.g., a memory card or a magnetic disk). The program may be composed of multiple modules. The control device 4 may be composed of multiple computers. For example, part of the learning control function of this embodiment may be implemented in a cloud computing system.

[0019] 1C is a functional block diagram of the control device 4 of this embodiment. Fig. 1C shows the functional units realized by the processor 11 and the data stored in the storage device 13 in the processing of this embodiment.

[0020] The device control unit 21 controls controlled objects included in the main body unit 2 via the individual controllers 3. The controlled objects are lenses, deflectors, or actuators such as motors included in the main body unit 2. The controlled objects are not limited to the content of their mechanisms as long as they are mechanisms included in the main body unit 2, and may have a structure that combines multiple actuators, for example.

[0021] The imaging data acquisition unit 22 acquires imaging data from the main body 2. The imaging data is data acquired by a detector included in the main body 2, and there are no limitations on the content or type of the imaging data as long as it is imaging data acquired from a detector included in the main body 2. Any imaging data acquired by the main body 2 can be the target, such as an SEM image when the detector is an electron detector, a mapping image when the detector is an EDX (Energy-dispersive X-ray spectroscopy) detector, or a camera image when the detector is a camera that captures images inside the chamber.

[0022] The learning control unit 23 has functions necessary for creating and operating a machine learning model, such as a data collection unit 24, a learning unit 28, and a trained model execution unit 29. These functions may be implemented in a computer on which an operating system (OS) is installed. The functions of the learning control unit 23 may be implemented in a single information processing device (control device 4), or some of its functions (e.g., the learning unit 28) may be implemented in another information processing device or a cloud computing system.

[0023] The data collection unit 24 has a function of collecting teacher data 31 by controlling the device control unit 21 and acquiring imaging data from the imaging data acquisition unit 22. The data collection unit 24 includes a collection condition setting unit 25 that sets the procedure for collecting teacher data, the range of control variables for collecting teacher data, imaging conditions, the number of images to be acquired, etc., a collection execution unit 26 that collects teacher data according to the collection conditions set by the collection condition setting unit 25, and a collection result confirmation unit 27 that allows the user to confirm the collection results.

[0024] The learning unit 28 uses the teacher data 31 to learn (train) the machine learning model 32. Since the learning of the machine learning model does not require information exchange with the main body unit 2 or the like, this function may be executed by a separate information processing device.

[0025] The trained model execution unit 29 outputs an estimated value obtained by inputting the imaging data acquired by the imaging data acquisition unit 22 into a trained machine learning model, and causes the device control unit 21 to control the main body unit 2 based on the output result.

[0026] The automatic adjustment of the main body 2 using the trained model of this embodiment can be applied to the adjustment of any controlled object that is adjusted based on imaging data acquired by a charged particle beam device. Examples include autofocus, astigmatism correction, adjustment of an aberration corrector using a Ronchigram image, stage tilt adjustment for electron beam zone axis incidence, current center axis adjustment, aperture hole position adjustment, and field of view position adjustment using stage or image shift. Therefore, the parameters to be adjusted vary depending on the controlled object. For example, when this embodiment is applied to the autofocus of a charged particle beam device, the controlled object is an objective lens or stage, and the controlled parameters include the objective lens current value and Z stage height. The objective lens may be an electromagnetic lens or an electrostatic lens, and the actuator that changes the Z stage height may be a stepping motor or a piezoelectric element. Furthermore, if the structure of the main body 2 or the algorithm of the device control unit 21 allows adjustment of one or more parameters for a unit combining multiple controlled objects, this embodiment can be applied to the unit as a single controlled object.

[0027] FIG. 2 shows the flow of automatic adjustment using a trained model executed by the trained model execution unit 29. The trained model used in this automatic adjustment flow is a machine learning model that estimates parameters for a controlled object required to obtain desired imaging or processing conditions from imaging data acquired by a charged particle beam device. For example, in an example in which the current value of an objective lens (controlled object) is controlled for autofocus, the machine learning model used is a machine learning model that estimates a control amount for changing the current control value of the parameter (i.e., the current value of the objective lens when the imaging data was acquired) to a control value that satisfies the just-focus condition. Therefore, automatic adjustment is completed when the estimated amount output by the machine learning model can be considered to be zero.

[0028] The trained model execution unit 29 acquires an image (image data) using the image data acquisition unit 22 (S201). Subsequently, the image acquired in step S201 is input to the trained model, and an estimate is calculated (S202). Subsequently, it is determined whether the absolute value of the estimate is equal to or less than a threshold. If it is less than the threshold, the adjustment is considered complete (S203). For example, in the case of autofocus, the magnitude of the threshold can be set based on the focal depth of the objective lens. If the absolute value of the estimate is greater than the threshold in step S203, the control value of the parameter when the image was acquired is changed by the estimate calculated by the trained model in step S202 (S204). Thereafter, the flow from step S201 is repeatedly executed until the absolute value of the estimate is equal to or less than the threshold in step S203.

[0029] It is known that the accuracy of a trained model depends on the training data used by the learning unit 28 to train the machine learning model. For example, if the machine learning model is trained using training data with large errors, the accuracy of the machine learning model will decrease. Furthermore, if the correct answer is similar to a large amount of training data used to train the machine learning model, the machine learning model can predict the correct answer with high accuracy. On the other hand, if the training data used to train the machine learning model does not contain much training data similar to the correct answer, the prediction accuracy of the machine learning model for the correct answer will decrease.

[0030] Fig. 3 shows the preferred characteristics of the trained model used in the flow of Fig. 2. As shown in Fig. 3, it is possible to distinguish between them based on the distance from the adjustment completion point. In the example of autofocus, the adjustment completion point is the in-focus position, and here, the short distance and long distance are distinguished based on the relative distance from the in-focus position.

[0031] In this embodiment, the required accuracy of the trained model is increased when the distance to the adjustment completion point is close, and is decreased when the distance to the adjustment completion point is far. This makes it possible to avoid an increase in the cost of acquiring training data due to uniformly increasing the required accuracy of the trained model. Furthermore, even if the prediction accuracy is low when the distance to the adjustment completion point is far, it is possible to converge to near the adjustment completion point by repeating the flow from steps S201 to S204. On the other hand, since the estimated value by the trained model is used to determine the completion of automatic adjustment in step S203, the prediction accuracy near the adjustment completion point becomes the final automatic adjustment accuracy. Therefore, it is desirable to have high prediction accuracy when the distance to the adjustment completion point is close.

[0032] The accuracy of such a trained model can be obtained by relatively differentiating the error and / or number of training data between when the distance from the calibration completion point is close and when it is far. For this reason, it is desirable to collect training data in which the error of the training data for short distances is smaller than the error of the training data for long distances, and / or training data in which the training data for short distances is greater than the training data for long distances.

[0033] In adjusting a controlled object in a charged particle beam device, it is possible to identify the adjustment completion point, but it is often not easy to determine how much the acquired imaging data deviates from the adjustment completion point. For example, in the case of autofocus, the in-focus position can be searched for as the position where the evaluation value of the image sharpness is maximized. However, it is difficult to estimate how much an image with low sharpness is defocused from just focus, and it is also difficult to determine whether it is deviated to the overfocus side or the underfocus side.

[0034] Furthermore, in order to use the data as training data, a large amount of imaging data is acquired while changing the imaging conditions. However, the error in the imaging data acquired later in the series of imaging data tends to be larger than the error in the imaging data acquired earlier. The causes of such errors include cumulative error and drift that accumulate during repeated control of the controlled object. Furthermore, in the case of SEM images, errors appear due to the accumulation of contamination that occurs when the sample is irradiated with an electron beam and changes in the sample due to damage caused by the electron beam.

[0035] Additionally, the direction in which the control value of the imaging conditions is changed can also affect errors. In the case of mechanical mechanisms, a phenomenon known as lost motion occurs, where the stopping position differs between moving from a positive direction and moving from a negative direction. Furthermore, in the case of electromagnetic lenses, hysteresis causes the magnetic field generated when the control value reached by increasing the current value to be different from the control value reached by decreasing the current value. Therefore, depending on the controlled object, the direction in which data is acquired may also affect the accuracy of the machine learning model.

[0036] FIG. 4 shows a flow for collecting training data to meet the requirements shown in FIG. 3 . The training data is configured as a pair of input data input to a machine learning model and a correct answer (label) that the machine learning model estimates for the input data. The input data is imaging data acquired by a charged particle beam device. The label indicating the correct answer is information that can be used to determine a control amount for automatic adjustment. For example, a machine learning model that predicts a control amount can be trained using the label as the control amount from the current control value to the control value at the adjustment completion point. Alternatively, the label can be classified according to whether the current control value deviates from the adjustment completion point in the positive or negative direction, or according to the distance from the adjustment completion point, and a machine learning model that classifies the control direction and the magnitude of the control amount can be trained.

[0037] As described above, the machine learning model of this embodiment is a supervised learning model that extracts features from input data and predicts a control variable or classifies the input data by training using teacher data. For example, when the machine learning model is a regression model that estimates a control variable (the difference between a control value at an adjustment completion point and a current control value), lasso regression, ridge regression, support vector regression, or a deep learning-based convolutional neural network may be used. Furthermore, these machine learning models may be used in combination. The algorithm of the machine learning model is not particularly limited. The machine learning model has internal parameters, and learning (training) the machine learning model refers to the process of updating the internal parameters so as to reduce the error between the estimated variable when input data of the teacher data is input and the correct answer (label) of the teacher data.

[0038] Here, the flow of Fig. 4 will be explained with reference to an example of autofocusing performed by controlling the amount of current (control value) of the objective lens. The label of the training data is the control amount (the difference between the control value at the adjustment completion point and the current control value), which in the autofocus example is the difference between the amount of current of the objective lens at the in-focus position and the amount of current of the objective lens when the image, which is the input data, was acquired. Therefore, the label of the image at the adjustment completion point is 0.

[0039] This flow is executed by the collection execution unit 26 according to the collection conditions set by the collection condition setting unit 25. In step S401, movement to the adjustment completion point (S401) is performed. In the case of autofocus, for example, the adjustment completion point can be identified as the point with the smallest amount of blur. Movement to the adjustment completion point may be performed manually by visual inspection, by using algorithm-based automatic adjustment, or by using a machine learning model that has already been trained at that time.

[0040] Step S402 is an offset setting process that moves the start point of teacher data collection from the adjustment completion point in the direction opposite to the data acquisition direction. Setting an offset has the advantage that the adjustment completion point can be determined separately from the teacher data after data acquisition. Note that this offset process may be omitted. Next, teacher data collection is performed.

[0041] In step S403, an image (imaging data) is acquired from the main body unit 2. Predetermined imaging conditions are set in the device control unit 21, and the imaging data is acquired by the imaging data acquisition unit 22. The image (imaging data) acquired in this step is input data for the training data.

[0042] In step S404, the correct answer to the content estimated by the machine learning model from the image is obtained. For example, the difference between the control value at the adjustment completion point and the current control value is obtained. As described above, the correct answer varies depending on the machine learning model.

[0043] Steps S405 and S406 determine whether to widen the data acquisition interval when the number of data or the control variable reaches a certain level. This results in the number of teacher data decreasing as the distance from the adjustment completion point increases. For example, until the number of data reaches a predetermined number, teacher data is collected while changing the objective lens current (control value) by ΔI1. After the predetermined number is reached, teacher data is collected while changing the objective lens current by ΔI2 (ΔI2 > ΔI1). Alternatively, the data acquisition interval may be changed based on the control variable, i.e., when the difference between the control value when acquiring the next teacher data and the control value at the adjustment completion point reaches a predetermined value, rather than the number of data.

[0044] In this example, the acquisition interval is widened after a certain amount of teacher data is acquired, but the acquisition interval may be widened each time teacher data is acquired. The acquisition interval may also be changed multiple times. In either case, it is sufficient that the teacher data is collected so that more teacher data is distributed near the adjustment completion point than far from the adjustment completion point across the entire range in which teacher data is acquired.

[0045] In step S407, the control value is increased or decreased by the data acquisition interval. Whether the control value is increased or decreased remains unchanged until data collection within the specified range is completed (S408). This makes it possible to acquire imaging data while changing the control value in one direction.

[0046] Step S408 is a branch to determine whether data collection within the specified range has been completed. If collection has not been completed, the process returns to step S403.

[0047] In step S409, an adjustment completion point is determined from the collected data. For example, in the case of autofocus, a method for determining the adjustment completion point is to determine the point among a series of collected images at which the sharpness evaluation value is highest. The adjustment completion point may be determined manually or automatically. Alternatively, the adjustment completion point may not be determined, and the value obtained in step S401 may be used as the adjustment completion point.

[0048] Step S410 is a determination of whether all data collection is complete. If not all data collection is complete, for example, if the desired number of teacher data items has not been acquired, the process returns to the adjustment completion point from step S401 and teacher data is collected again using the same procedure.

[0049] 4 shows an example in which both the process of widening the data acquisition interval in steps S405 and S406 and the process of acquiring imaging data while changing the control variable in one direction from the data acquisition start point in step S407 are performed. Depending on the accuracy required of the trained model and the characteristics of the controlled object, it is possible to omit one of the processes.

[0050] FIG. 5 shows the data structure of a training dataset. The training dataset identified by the set number 501 corresponds to a group of training data obtained by executing steps S401 to S410 in FIG. 4 once. The acquisition direction 502 indicates whether the control value was moved in the positive or negative direction from the adjustment completion point. The start point 503 indicates the control value at the start of data collection. If no offset setting (step S402) is performed, the adjustment completion point moved in step S401 is used. The adjustment completion point 504 indicates the control value at the adjustment completion point. The control value at the adjustment completion point determined in step S409 is registered as the adjustment completion point 504. However, if step S409 is omitted, the control value at the adjustment completion point moved in step S401 is registered. The end point 505 indicates the control value at the end of data acquisition. The control value at the end of data acquisition indicated by the end point 505 is within the range of automatic adjustment performed by the machine learning model. In the example of FIG. 5, automatic adjustment can be performed within a range of ±5.00 from the adjustment completion point. The number of data 506 indicates the number of data per training data set.

[0051] FIG. 6 shows the data structure of training data. For example, this corresponds to the training data set with set number 1 shown in FIG. 5. Data number 601 is a number that identifies training data (pairs of input data and labels) included in the training data set. Data numbers 601 can be assigned in the order of acquisition. Control value 602 is the control value of the parameter when the imaging data is acquired. In the case of autofocus, this would be the amount of current through the objective lens. Training data (label) 603 is the correct answer that the machine learning model estimates for the input data. In this example, the difference between control value 602 and the control value at the adjustment completion point (10.00 in this example) is set as the correct answer. Training data (input data) 604 is an image captured under the conditions of control value 602.

[0052] 7 shows an example of a GUI (Graphical User Interface) displayed on the display device 19 for confirming the condition settings and collection results for collecting teacher data by the data collection unit 24. The upper section of the display screen 700 is a collection condition setting screen 701 displayed by the collection condition setting unit 25, and the lower section is a collection result confirmation screen 721 displayed by the collection result confirmation unit 27.

[0053] The user sets the conditions for collecting teacher data on a collection condition setting screen 701. The number of sets 702 corresponds to the total number of sets in the teacher data set shown in Figure 5, and the number of acquisitions per set 703 corresponds to the number of data per teacher data set. The data distribution setting 704 is set so that the closer the distance to the adjustment completion point, the more data is acquired, and the user can move the position of the control point 705 to set the degree to which the number of data decreases as the distance increases.

[0054] The collection condition setting screen 701 also includes a collection condition setting table 706, allowing various collection conditions to be set. The estimator 707 specifies the value to be inferred when an image is input to the machine learning model, e.g., the control value for moving to the in-focus position. The control value 708 specifies the parameter for adjusting the controlled object. The image 709 specifies the type of image, imaging conditions, image size, etc., to be used as input data for the machine learning model. The absolute value 710 specifies whether the sign is included in the prediction target. The acquisition order 711 specifies the data acquisition method for each training data set. For example, possible methods include alternately acquiring data in the positive and negative directions, or acquiring half of the total sets in the positive direction and then acquiring half of the remaining sets in the negative direction. The movement to adjustment completion point 712 specifies the method for moving to the adjustment completion point in step S401. For example, the movement method can be manual, automatic, or fixed value. The offset 713 sets the offset amount to be moved in step S402. If set to 0, no offset is set.

[0055] After setting the collection conditions, when the user presses the execute button 714, the collection execution unit 26 executes the collection of teacher data under the set collection conditions.

[0056] The collected teacher data can be checked on a collection result confirmation screen 721. In a set number 722, the user selects a teacher data set to be displayed on the collection result confirmation screen 721. In a teacher data display section 723, the image (input data), control values, and labels (see FIG. 6) included in the teacher data set are displayed. In addition, the teacher data display section 723 clearly shows an image of the adjustment completion point using an adjustment completion point display icon 724. If the user wants to change the adjustment completion point, they can do so by manually dragging the icon 724. This will cause the label to be recalculated.

[0057] Figure 8 uses automatic autofocus adjustment as an example to explain the expected difference in accuracy of the trained model depending on the order in which training data is collected, when the estimated amount of the machine learning model is the defocus amount (absolute value). The comparative example (x) is a case in which training data is collected in the order from underfocus to overfocus. The example (●) is a case in which training data is collected in one direction, either underfocus or overfocus, starting from the in-focus position. The defocus amount on the horizontal axis represents the amount of focus deviation when the in-focus position is set to 0, with the left side representing underfocus and the right side representing overfocus. The prediction error on the vertical axis represents the difference between the defocus amount estimated by the machine learning model and the correct answer (the label of the training data).

[0058] In the comparative example, it can be seen that the distribution of prediction errors is biased between underfocus and overfocus. This is due to errors in the training data itself caused by the influence of magnetic hysteresis of the objective lens. On the other hand, in the example, such bias in prediction errors does not occur. Furthermore, the prediction error at the in-focus position (adjustment completion point), which is a desirable characteristic for a machine learning model used in automatic adjustment, is also small.

[0059] When a machine learning model outputs estimates for multiple parameters from an input image, it is necessary to collect training data while changing the control values ​​of the multiple parameters. There are various possible patterns for collecting training data while changing multiple control values, so the following describes collection patterns when multiple control values ​​exist. Figures 9 to 11 show an example of a training data collection procedure when a machine learning model predicts two parameters, control value A and control value B. In these figures, the intersection (origin) of axis A (vertical axis) and axis B (horizontal axis) is represented as the point where adjustment of both control values ​​is complete.

[0060] The data collection order in Figure 9 is a training data collection pattern that is desirable when relatively high accuracy is required for control value B while relatively low accuracy is sufficient for control value A, or when control value B generates a relatively large cumulative error while control value A generates only a relatively small cumulative error. In this example, imaging data is acquired while changing the control value in one direction, with the training data collection start point set as the adjustment completion point for control value B only. The data collection pattern is shown as a polygonal line, with each vertex of the polygonal line representing a data acquisition point. Dataset 1 is a training data set collected using a collection pattern that starts from a collection start point located at an arbitrary position on the A axis and moves in the positive direction of control value B. Dataset 2 is a training data set collected using a collection pattern that starts from a collection start point located at an arbitrary position on the A axis and moves in the negative direction of control value B. Arrows in the figure indicate example acquisition directions. Furthermore, an adjustment range 901 is shown as a dashed rectangle. The training data collection end point for control value B is set within the adjustment range 901, and the value of control value A is randomly assigned within the adjustment range 901. In this case, too, an offset may be set for the control value B to collect the teaching data.

[0061] 9, when collecting teacher data while changing multiple control values, (1) the teacher data collection start point is set at the adjustment completion point or its offset position only for control value B, which requires higher accuracy, and teacher data is collected while changing the control value in one direction, and (2) teacher data is collected so that the teacher data collection density in an area relatively close to the adjustment completion point is higher than the teacher data collection density in an area relatively far from the adjustment completion point. Note that the collection pattern may be set to satisfy either collection pattern (1) or (2).

[0062] The data collection sequence shown in FIG. 10 is a training data collection pattern that is desirable when two control values ​​require equal accuracy or generate equal cumulative errors. In this example, imaging data is acquired while changing the control values ​​in one direction for both control values ​​A and B, with the training data collection start point set as the adjustment completion point. The data collection pattern is represented by multiple straight lines, and the data acquisition point is indicated by a symbol. The position where both control values ​​A and B are at their adjustment completion points corresponds to the intersection of the A and B axes. Data set 1 is a training data set obtained by fixing control value B and changing control value A in a positive or negative direction from the adjustment completion point to acquire training data. When control value A reaches adjustment range 1001, control value B is changed in a positive direction, and then control value A is changed in a negative or positive direction from the adjustment completion point to acquire training data. This process is repeated until control value B reaches adjustment range 1001. Dataset 2 is a teacher data set obtained by repeatedly acquiring teacher data by fixing control value B and varying control value A in a positive or negative direction from the adjustment completion point of control value A. When control value A reaches adjustment range 1001, control value B is changed in the negative direction. When control value A reaches adjustment range 1001, control value B is changed again in the negative or positive direction from the adjustment completion point of control value A to acquire teacher data. This process is repeated until control value B reaches adjustment range 1001. Note that a group of teacher data collected by fixing control value B and varying control value A is sometimes referred to as a teacher data subset. Teacher data is acquired in the order of datasets 1 and 2, and the acquisition order is indicated in the figure by circled numbers and arrows. Note that in this case, teacher data may also be collected by setting an offset for each of control values ​​A and B. Furthermore, when acquiring the teacher data subset, the fixed control value may be reversed between control value A and control value B.

[0063] 10, when collecting teacher data while changing multiple control values, (1) the teacher data collection start point is set at the adjustment completion point or its offset position, control value B (A) is fixed, and the process of collecting teacher data subsets while changing control value A (B) in one direction is repeated while changing fixed control value B (A) in one direction, and (2) teacher data is collected so that the teacher data collection density in areas relatively close to the adjustment completion point for both control values ​​A and B is higher than the teacher data collection density in areas relatively far from the adjustment completion point. Note that the collection pattern may be set to satisfy either collection pattern (1) or (2).

[0064] The data collection order in Figure 11 is a teacher data collection pattern that is desirable when two control values ​​require equal accuracy and the difference in cumulative error due to the direction of the control values ​​is negligible. In this example, for both control values ​​A and B, the teacher data collection start point is set as the adjustment completion point, and imaging data is acquired so that the distance between the data acquisition point and the adjustment completion point (origin) increases. The teacher data collection pattern is represented by a curve that spreads out in a spiral shape from the intersection of the A and B axes (origin), and the data acquisition points are indicated by symbols.

[0065] Here, the distance between an arbitrary data acquisition point 1102 and the collection start point is set to L, and L is expressed by (Equation 1).

[0066] Here, A and B are the control values ​​A and B, respectively, and a and b are arbitrary coefficients that serve to correct the weight of the calculated value when the magnitude of the numerical values ​​differs between the control variables A and B. In other words, the distance L is the L2 norm obtained by adding an arbitrary coefficient to the control values ​​A and B, and training data is collected so that L increases as data collection progresses. The L2 norm is an example, and the L1 norm of the control values ​​A and B may also be used, for example. In this case, L is expressed by (Equation 2).

[0067] 11, when collecting teacher data while changing multiple control values, (1) the teacher data collection start point is set at the adjustment completion point, and teacher data is collected while gradually increasing the distance between the collection start point and the data acquisition point in the space spanned by control values ​​A and B, and (2) for both control values ​​A and B, the teacher data is collected so that the teacher data collection density in areas relatively close to the adjustment completion point is higher than the teacher data collection density in areas relatively far from the adjustment completion point. Note that the collection pattern may be set to satisfy either (1) or (2). Note that this can be expanded to three or more control values.

[0068] 9 to 11 may be combined. For example, if there are five control values ​​A, B, C, D, and E, the norm calculated from control values ​​A and B and control value C may be changed in one direction from the adjustment completion point, and control values ​​D and E may be changed randomly to perform collection.

[0069] The present invention is not limited to the above-described embodiments and includes various modifications. For example, the above-described embodiments and modifications have been described in detail to make the present invention easier to understand, and are not necessarily limited to those including all of the described configurations. Furthermore, it is possible to replace part of the configuration of one embodiment or modification with the configuration of another embodiment or modification, and it is also possible to add the configuration of another embodiment or modification to the configuration of one embodiment or modification. Furthermore, it is possible to add, delete, or replace part of the configuration of each embodiment or modification with other configurations.

[0070] 1: charged particle beam device, 2: main body, 3: individual controller, 4: control device, 11: processor (CPU), 12: memory, 13: storage device, 14: input interface, 15: output interface, 16: communication interface, 17: bus, 18: input device, 19: display device, 21: device control unit, 22: imaging data acquisition unit, 23: learning control unit, 24: data collection unit, 25: collection condition setting unit, 26: collection execution unit, 27: collection result confirmation unit, 28: learning unit, 29: learned model execution unit, 31: teacher data, 32: machine learning model, 501: set number, 502: acquisition direction, 503: start point, 504: adjustment completion point, 505: end point, 506: number of data 601: Data number, 602: Control value, 603: Teacher data (label), 604: Teacher data (input data), 700: Display screen, 701: Collection condition setting screen, 702: Number of sets, 703: Number of acquisitions per set, 704: Data distribution setting, 705: Control point, 706: Collection condition setting table, 707: Estimator, 708: Control value, 709: Image, 710: Absolute value, 711: Acquisition order, 712: Move to adjustment completion point, 713: Offset, 714: Execute button, 721: Collection result confirmation screen, 722: Set number, 723: Teacher data display section, 724: Adjustment completion point display icon, 901, 1001, 1101: Adjustment range, 1102: Data acquisition point.

Claims

1. A teacher data collection method for collecting teacher data for a machine learning model that performs automatic adjustment of a charged particle beam device comprising a main body and a control device that controls the main body to acquire imaging data, wherein the machine learning model estimates a parameter for controlling the main body using the imaging data as input data, the control device comprises a collection execution unit that collects the teacher data, the collection execution unit: specifies a control value of the parameter when the main body is in a desired state by controlling the parameter as an adjustment completion point; sets a collection start point of the teacher data as the adjustment completion point or a point obtained by adding a predetermined offset to the adjustment completion point; sets a data acquisition point of the teacher data by changing a control value of the parameter; and collects a teacher data set including a plurality of pairs by acquiring a pair of the imaging data and a label of the imaging data each time the collection start point and the data acquisition point are set, wherein the control values ​​of the data acquisition points of the teacher data constituting the teacher data set are all increased or decreased compared to the control value of the collection start point, and a data acquisition point set later is set to be farther from the collection start point than a data acquisition point set earlier, A teacher data collection method in which the specified offset is set so that the control value of the collection start point is decreased below the control value of the adjustment completion point when the control value of the data acquisition point is set to be increased compared to the control value of the collection start point, and so that the control value of the collection start point is increased above the control value of the adjustment completion point when the control value of the data acquisition point is set to be decreased compared to the control value of the collection start point.

2. A teacher data collection method according to claim 1, wherein the collection execution unit determines the label of the imaging data based on a control value when the imaging data was acquired and a control value at the adjustment completion point.

3. A teacher data collection method according to claim 1, wherein the data acquisition points of the teacher data constituting the teacher data set include a first data acquisition point and a second data acquisition point set after the first data acquisition point, and the absolute value of the difference between the control value of the data acquisition point set immediately before the second data acquisition point and the control value of the second data acquisition point is made larger than the absolute value of the difference between the control value of the data acquisition point set immediately before the first data acquisition point and the control value of the first data acquisition point.

4. A teacher data collection method as claimed in claim 1, wherein the machine learning model is a machine learning model which estimates a first parameter and a second parameter, and a first control value of the first parameter when the main body unit is in a desired state by controlling the first parameter is identified as an adjustment completion point, and a data acquisition point of the teacher data is set by changing a first control value of the first parameter and a second control value of the second parameter, and the first control values ​​of the data acquisition points of the teacher data constituting the teacher data set are all increased or decreased compared to the first control value of the collection start point, and a subsequently set data acquisition point is set to be farther from the collection start point than an earlier set data acquisition point, and the second control value of a certain data acquisition point is set to be increased or decreased compared to the second control value of the data acquisition point set immediately before it, at the data acquisition points of the teacher data constituting the teacher data set.

5. In claim 1, the machine learning model is a machine learning model that estimates a first parameter and a second parameter, and a first control value of the first parameter and a second control value of the second parameter when the main body unit is in a desired state by controlling the first parameter and the second parameter are specified as an adjustment completion point, and a data acquisition point of the teacher data is set by changing the first control value of the first parameter and the second control value of the second parameter, the teacher data constituting the teacher data set has a plurality of teacher data subsets, the second control values ​​of the data acquisition points of the teacher data constituting the teacher data subsets are fixed, and all of the first control values ​​are increased or decreased compared to the first control value of the collection start point, and the absolute value of the first control value of the later set data acquisition point is set to be greater than the absolute value of the first control value of the earlier set data acquisition point, A teacher data collection method in which the second control values ​​of the data acquisition points of the teacher data constituting the teacher data subset are all increased or decreased compared to the second control value at the collection start point, and the absolute value of the second control value of the data acquisition point of a later collected teacher data subset is set to be greater than the absolute value of the second control value of the data acquisition point of an earlier collected teacher data subset.

6. A teacher data collection method for collecting teacher data for a machine learning model that performs automatic adjustment of a charged particle beam device comprising a main body unit and a control device that controls the main body unit to acquire imaging data, wherein the machine learning model estimates a plurality of parameters that control the main body unit using the imaging data as input data, the control device comprising a collection execution unit that collects the teacher data, the collection execution unit controlling the plurality of parameters to identify control values ​​of the plurality of parameters at which the main body unit is in a desired state as an adjustment completion point, setting a collection start point of the teacher data as the adjustment completion point, setting a data acquisition point of the teacher data by changing the control values ​​of the plurality of parameters, and collecting a teacher data set including a plurality of said pairs each time the collection start point and the data acquisition point are set, wherein the data acquisition points of the teacher data constituting the teacher data set are set so that a distance between the data acquisition point set later and the origin in a space having each of the plurality of parameters as an axis and the adjustment completion point as an origin is longer than a distance between the data acquisition point set earlier and the origin in the space.

7. According to claim 6, the plurality of parameters include a first parameter and a second parameter, and when a control value of the first parameter at a data acquisition point of the teacher data is A and a control value of the second parameter is B, the distance is expressed as follows, with a and b as coefficients: or A teacher data collection method defined as:

8. A teacher data collection method according to claim 6, wherein the collection execution unit is configured such that the data acquisition points of the teacher data constituting the teacher dataset include a first data acquisition point and a second data acquisition point set after the first data acquisition point, and the absolute value of the difference between the distance between the data acquisition point set immediately before the second data acquisition point in the space and the origin and the distance between the second data acquisition point in the space and the origin is longer than the absolute value of the difference between the distance between the data acquisition point set immediately before the first data acquisition point in the space and the origin and the distance between the first data acquisition point in the space and the origin.

9. A teacher data collection method for collecting teacher data for a machine learning model that performs automatic adjustment of a charged particle beam device comprising a main body and a control device that controls the main body to acquire imaging data, wherein the machine learning model estimates a parameter for controlling the main body using the imaging data as input data, the control device comprises a collection execution unit that collects the teacher data, the collection execution unit: controls the parameter to identify a control value of the parameter when the main body is in a desired state as an adjustment completion point; sets a collection start point of the teacher data as the adjustment completion point or a point obtained by adding a predetermined offset to the adjustment completion point; sets a data acquisition point of the teacher data by changing the control value of the parameter; acquires pairs of the imaging data and a label of the imaging data at the collection start point and the data acquisition point, thereby collecting a teacher dataset including a plurality of the pairs; the control values ​​of the data acquisition points of the teacher data that constitute the teacher dataset are all increased or decreased compared to the control value at the collection start point, A teacher data collection method in which the data acquisition points of the teacher data constituting the teacher data set include a first data acquisition point and a second data acquisition point set farther from the adjustment completion point than the first data acquisition point, and an absolute value of a difference between a control value of the data acquisition point set closest to the second data acquisition point and a control value of the second data acquisition point is larger than an absolute value of a difference between a control value of the data acquisition point set closest to the first data acquisition point and a control value of the first data acquisition point, and the predetermined offset is set so that the control value of the collection start point is decreased below the control value of the adjustment completion point when the control value of the data acquisition point is set to increase compared to the control value of the collection start point, and so that the control value of the collection start point is increased above the control value of the adjustment completion point when the control value of the data acquisition point is set to decrease compared to the control value of the collection start point.

10. A teacher data collection method according to claim 9, wherein the control device is provided with a collection condition setting unit that sets collection conditions for collecting the teacher data, and the collection condition setting unit is capable of setting a data distribution that sets the number of teacher data to be collected depending on the distance from the adjustment completion point.

11. A charged particle beam device comprising: a main body; and a control device for controlling the main body to acquire imaging data, wherein the control device comprises a learned model execution unit for performing automatic adjustment of the main body using a learned machine learning model, and a data collection unit for collecting teacher data for the machine learning model, wherein the machine learning model is a machine learning model for estimating a parameter for controlling the main body using the imaging data as input data, wherein the data collection unit specifies a control value of the parameter when the main body is in a desired state by controlling the parameter as an adjustment completion point, sets a collection start point of the teacher data as the adjustment completion point or a point obtained by adding a predetermined offset to the adjustment completion point, and sets a data acquisition point of the teacher data by changing the control value of the parameter, wherein the data collection unit acquires a pair of the imaging data and a label of the imaging data each time the collection start point and the data acquisition point are set, thereby collecting a teacher dataset including a plurality of the pairs, a control value of each of the data acquisition points of the teacher data constituting the teacher data set is increased or decreased compared to the control value of the collection start point, and a later set data acquisition point is set to be farther from the collection start point than an earlier set data acquisition point, and the predetermined offset is set so that when the control value of the data acquisition point is set to be increased compared to the control value of the collection start point, the control value of the collection start point is decreased compared to the control value of the adjustment completion point, and when the control value of the data acquisition point is set to be decreased compared to the control value of the collection start point, the control value of the collection start point is increased compared to the control value of the adjustment completion point.

12. A charged particle beam device according to claim 11, wherein the trained machine learning model used by the trained model execution unit is trained using the teacher data collected by the data collection unit.

13. A charged particle beam device comprising: a main body; and a control device for controlling the main body to acquire imaging data, wherein the control device comprises a learned model execution unit for performing automatic adjustment of the main body using a learned machine learning model, and a data collection unit for collecting teacher data for the machine learning model, wherein the machine learning model is a machine learning model for estimating a plurality of parameters for controlling the main body using the imaging data as input data, wherein the data collection unit controls the plurality of parameters to identify control values ​​of the plurality of parameters at which the main body is in a desired state as an adjustment completion point, sets a collection start point of the teacher data as the adjustment completion point, and sets a data acquisition point of the teacher data by changing the control values ​​of the plurality of parameters, wherein the data collection unit acquires a pair of the imaging data and a label of the imaging data each time the collection start point and the data acquisition point are set, thereby collecting a teacher dataset including a plurality of the pairs, A charged particle beam device in which, for the data acquisition points of the teacher data constituting the teacher dataset, a distance between the subsequently set data acquisition point and the origin in a space having each of the multiple parameters as an axis and the adjustment completion point as the origin is set to be longer than a distance between the previously set data acquisition point and the origin in the space.

14. A charged particle beam device according to claim 13, wherein the trained machine learning model used by the trained model execution unit is trained using the teacher data collected by the data collection unit.

15. A charged particle beam device comprising: a main body; and a control device for controlling the main body to acquire imaging data, wherein the control device comprises a learned model execution unit for performing automatic adjustment of the main body using a learned machine learning model, and a data collection unit for collecting teacher data for the machine learning model, wherein the machine learning model is a machine learning model for estimating a parameter for controlling the main body using the imaging data as input data, wherein the data collection unit specifies a control value of the parameter when the main body is in a desired state by controlling the parameter as an adjustment completion point, sets a collection start point of the teacher data as the adjustment completion point or a point obtained by adding a predetermined offset to the adjustment completion point, and sets a data acquisition point of the teacher data by changing the control value of the parameter, wherein the data collection unit collects a teacher dataset including a plurality of the pairs by acquiring pairs of the imaging data and a label of the imaging data at the collection start point and the data acquisition point, wherein the control values ​​of the data acquisition points of the teacher data constituting the teacher dataset are all increased or decreased compared to the control value at the collection start point, a control value of the control point set at the closest to the first data acquisition point and a control value of the second data acquisition point set at the closest to the first data acquisition point, the control value of the second data acquisition point being greater than the absolute value of the difference between the control value of the data acquisition point set at the closest to the first data acquisition point and the control value of the first data acquisition point; and the predetermined offset is set so that the control value of the collection start point is decreased below the control value of the adjustment completion point when the control value of the data acquisition point is set to be increased above the control value of the collection start point, and so that the control value of the collection start point is increased above the control value of the adjustment completion point when the control value of the data acquisition point is set to be decreased above the control value of the collection start point.

16. A charged particle beam device according to claim 15, wherein the trained machine learning model used by the trained model execution unit is trained using the teacher data collected by the data collection unit.

Images