A model training method and program product
By adaptively configuring the hyperparameters of the AI model to adapt to the complexity of the training data for micro- and nano-structures, the problems of low training efficiency and insufficient performance of AI models are solved, and more efficient and accurate calculation of micro- and nano-structure parameters is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SKYVERSE TECH CO LTD
- Filing Date
- 2025-07-30
- Publication Date
- 2026-05-12
AI Technical Summary
Existing AI models are inefficient in training when determining the parameter information of micro and nano structures, and their performance cannot meet the measurement requirements of different micro and nano structures. They are also difficult to learn the complex training data features during the training process, and there is a problem of wasting time and cost.
By determining the first indication information to indicate the structural complexity of the AI model, and adjusting the values of hyperparameters according to the indication information, the hyperparameters such as the depth and width of the neural network are adaptively configured to adapt to the complexity of the training data and optimize the training process.
This improved the accuracy and training efficiency of AI models when calculating parameters of micro and nano structures, reduced the risk of gradient vanishing, avoided resource waste, and enhanced model performance.
Smart Images
Figure CN120600190B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of nanotechnology, and in particular to a model training method and program product. Background Technology
[0002] With the rapid development of science and technology, nanotechnology is increasingly being applied to more and more industries and fields. Micro- and nanostructures, as the core application of nanotechnology, have enormous application potential in information technology, energy, and biomedicine due to their unique size effects. For example, semiconductor micro- and nanostructures exhibit excellent performance in electronics and photonics, and have become key materials for high-performance electronic and optoelectronic devices.
[0003] In practical applications, determining the parameter information of micro / nanostructures is crucial. For example, the parameter information of micro / nanostructures can be used to describe their performance or to perform quality inspection. These parameters can include critical dimensions (CD), period, height, or sidewall angles.
[0004] Typically, artificial intelligence (AI) models can be used to determine the parameter information of micro / nano structures. For example, the scattering signals of micro / nano structures can be measured using different measurement techniques or equipment (such as scattering measurement, ellipsometers, CD small-angle X-ray scattering, CD scanning electron microscopes, etc.), and the theoretical signals of the micro / nano structure parameters under different values can be calculated based on the AI model. Then, by comparing and matching the different theoretical signals with the measured signals, the parameter values of the micro / nano structure can be deduced. However, the above AI models may suffer from low training efficiency and performance limitations that cannot meet the measurement requirements of different micro / nano structures. Summary of the Invention
[0005] This application provides a model training method to improve the training efficiency and performance of AI models used to determine micro / nano structure parameter information. Furthermore, this application also provides corresponding apparatus, device storage media, and computer program products.
[0006] In a first aspect, embodiments of this application provide a model training method, comprising: determining first indication information, the first indication information being used to indicate the structural complexity of a first artificial intelligence (AI) model, the first indication information including one or more data indicators of first training data used to train the first AI model; determining a first value of hyperparameters of the first AI model for the first indication information based on the first indication information; setting the hyperparameters of the first AI model to the first value; training the first AI model using the first training data, the first training data including theoretical spectra corresponding to N values of parameters of the micro / nano structure, the first AI model being used to calculate theoretical spectra based on different values of parameters of the micro / nano structure, wherein N is a positive integer.
[0007] In one possible implementation, determining the first value of the hyperparameters of the first AI model based on the first indication information includes: determining second indication information, the second indication information including one or more data indicators of second training data used for pre-training the first AI model; pre-training the first AI model using the second training data, the second training data including theoretical spectra corresponding to M values of the parameters of the micro / nano structure, where M is a positive integer less than N; determining the second value of the hyperparameters of the first AI model for the second indication information based on the training results of the pre-training of the first AI model; and determining the first value based on the second value according to the difference between the first indication information and the second indication information.
[0008] In one possible implementation, the method further includes: during the training of the first AI model using the first training data, when the training status of the first AI model meets a first condition, adjusting the hyperparameters of the first AI model to a third value based on the training status of the first AI model, the third value being used to optimize the training status of the first AI model; the first condition includes the performance improvement of the iterative first AI model being less than a first threshold, or the first condition includes the training error of the first AI model exceeding a second threshold.
[0009] In one possible implementation, the first indication information includes one or more of the following: the amount of data N of the parameters of the micro / nano structure included in the first training data; the value range of the parameters of the micro / nano structure in the first training data; the variation of the theoretical spectrum in the first training data with the value of the parameters of the micro / nano structure; or the variation of the parameter values of each theoretical spectrum in the first training data with the measurement conditions. The hyperparameters of the first AI model include one or more of the following: network width, network depth, learning rate, regularization strength, and batch size.
[0010] In one possible implementation, the second indication information includes one or more of the following: the value range of the parameters of the micro / nano structure in the second training data, the variation of the theoretical spectrum in the second training data with the value of the parameters of the micro / nano structure, or the variation of the parameter values of each theoretical spectrum in the second training data with the measurement conditions.
[0011] In one possible implementation, the method further includes: measuring the measurement spectrum of the micro / nano structure; using the first AI model, performing a fitting algorithm on the measurement spectrum to obtain target values for the parameters of the micro / nano structure, wherein the theoretical spectrum corresponding to the target value matches the measurement spectrum.
[0012] In one possible implementation, before fitting the measured spectrum, the method further includes: determining third indication information, the third indication information including one or more data indicators of the micro / nano structure; and adjusting the values of hyperparameters of the fitting algorithm according to the third indication information, the hyperparameters of the fitting algorithm including iterative compensation parameters.
[0013] In one possible implementation, the step of using the first AI model to perform a fitting algorithm on the measured spectrum to obtain the target values of the parameters of the micro / nano structure includes: using the first AI model to determine a fitting dataset, the fitting dataset including theoretical spectra corresponding to the values of the parameters of the micro / nano structure that satisfy the fitting numerical range; and retrieving the fitting dataset to perform a fitting algorithm on the measured spectrum to obtain the target values within the fitting numerical range.
[0014] Secondly, embodiments of this application provide a model training apparatus, comprising: a determining module, the determining module being configured to determine first indication information, the first indication information being configured to indicate the structural complexity of a first artificial intelligence (AI) model, the first indication information including one or more data indicators of first training data used to train the first AI model; determining a first value of hyperparameters of the first AI model for the first indication information according to the first indication information; a setting module, the setting module being configured to set the hyperparameters of the first AI model to the first value; and a training module, the training module being configured to train the first AI model using the first training data, the first training data including theoretical spectra corresponding to N values of the parameters of the micro / nano structure, the first AI model being configured to calculate theoretical spectra based on different values of the parameters of the micro / nano structure, wherein N is a positive integer.
[0015] In one possible implementation, the determining module is specifically configured to: determine second indication information, the second indication information including one or more data metrics of second training data used for pre-training the first AI model; pre-train the first AI model using the second training data, the second training data including theoretical spectra corresponding to M values of the parameters of the micro / nano structure, where M is a positive integer less than N; determine a second value of the hyperparameters of the first AI model for the second indication information based on the training results of the pre-training of the first AI model; and determine the first value based on the second value according to the difference between the first indication information and the second indication information.
[0016] In one possible implementation, the adjustment module is further configured to, during the process of training the first AI model using the first training data, adjust the hyperparameters of the first AI model to a third value based on the training status of the first AI model when the training status of the first AI model meets a first condition, wherein the third value is used to optimize the training status of the first AI model; the first condition includes the performance improvement of the iterative first AI model being less than a first threshold, or the first condition includes the training error of the first AI model exceeding a second threshold.
[0017] In one possible implementation, the first indication information includes one or more of the following: the amount of data N of the parameters of the micro / nano structure included in the first training data; the value range of the parameters of the micro / nano structure in the first training data; the variation of the theoretical spectrum in the first training data with the value of the parameters of the micro / nano structure; or the variation of the parameter values of each theoretical spectrum in the first training data with the measurement conditions. The hyperparameters of the first AI model include one or more of the following: network width, network depth, learning rate, regularization strength, and batch size.
[0018] In one possible implementation, the second indication information includes one or more of the following: the value range of the parameters of the micro / nano structure in the second training data, the variation of the theoretical spectrum in the second training data with the value of the parameters of the micro / nano structure, or the variation of the parameter values of each theoretical spectrum in the second training data with the measurement conditions.
[0019] In one possible implementation, the method further includes a measurement module for measuring the measurement spectrum of the micro / nano structure; the method further includes an execution module for using the first AI model to perform a fitting algorithm on the measurement spectrum to obtain target values for the parameters of the micro / nano structure, wherein the theoretical spectrum corresponding to the target value matches the measurement spectrum.
[0020] In one possible implementation, the determining module is further configured to determine third indication information before fitting the measured spectrum, the third indication information including one or more data indicators of the micro / nano structure; the adjusting module is further configured to adjust the values of the hyperparameters of the fitting algorithm according to the third indication information, the hyperparameters of the fitting algorithm including iterative compensation parameters.
[0021] In one possible implementation, the execution module is specifically used to: utilize the first AI model to determine a fitting dataset, the fitting dataset including theoretical spectra corresponding to the parameter values of the micro / nano structure that satisfy the fitting numerical range; and by retrieving the fitting dataset, perform a fitting algorithm on the measured spectra to obtain the target value within the fitting numerical range.
[0022] Thirdly, embodiments of this application also provide a computing device, which may include a processor and a memory: the memory is used to store a computer program; the processor is used to execute the method described in the first aspect and any one of the embodiments in the first aspect according to the computer program.
[0023] Fourthly, embodiments of this application also provide a computer-readable storage medium for storing a computer program for performing the methods described in the first aspect and any one of the embodiments of the first aspect.
[0024] Fifthly, embodiments of this application also provide a computer program product including instructions that, when run on a computing device, cause the computing device to perform the methods described in the first aspect and any one of the embodiments of the first aspect.
[0025] In the above implementation of the embodiments of this application, first indication information is determined. The first indication information is used to indicate the complexity of the structure of the first artificial intelligence (AI) model. The first indication information includes one or more data indicators of the first training data used to train the first AI model. Based on the first indication information, a first value of the hyperparameters of the first AI model for the first indication information is determined. The hyperparameters of the first AI model are set to the first value. The first AI model is trained using the first training data, which includes theoretical spectra corresponding to N values of the parameters of the micro / nano structure. The first AI model is used to calculate the theoretical spectra based on different values of the parameters of the micro / nano structure, where N is a positive integer. Thus, considering the complex and variable characteristics of micro / nano structures, before training the AI model used to calculate the theoretical spectra of the parameters of the micro / nano structure under different values, considering the different data indicators of the first training data (e.g., the total amount of data N), different values of hyperparameters (e.g., the depth and width of the neural network) can be adaptively configured for the first AI model. Therefore, it effectively avoids the problem that when the structure of the first AI model is fixed, the relatively simple model structure is difficult to fully learn the features of the complex first training data during the training process, effectively ensuring the performance of the first AI model and improving the accuracy of calculating the theoretical spectrum; and it also effectively avoids the problem that when the structure of the first AI model is fixed, the relatively complex model structure will consume a lot of time and training costs during the training process, and effectively reduces the risk of gradient vanishing and improves the efficiency of model training. Attached Figure Description
[0026] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings.
[0027] Figure 1 This is a schematic diagram of the architecture of a data processing system according to an embodiment of this application;
[0028] Figure 2This is a flowchart illustrating a model training method in an embodiment of this application;
[0029] Figure 3 This is a schematic diagram of the structure of a model training device according to an embodiment of this application;
[0030] Figure 4 This is a schematic diagram of the hardware structure of a computing device according to an embodiment of this application. Detailed Implementation
[0031] See Figure 1 This is a schematic diagram of a data processing system provided in this application. Figure 1 As shown, the data processing system 10 includes a processor and a memory. Figure 1 Taking the processor 100 and memory 200 as an example, in actual application scenarios, the data processing system 10 may include a greater number of processors and memory.
[0032] Optionally, the data processing system 10 may also include measuring devices. Figure 1 Taking the measurement device 300 as an example, in actual application scenarios, the data processing system 10 may include a larger number of measurement devices.
[0033] The processor 100 can be connected to the memory 200 and the measuring device 300 via a bus, such as a peripheral component interconnect express (PCIe) bus. Alternatively, the processor 100 can also be connected to the memory 200 and the measuring device 300 via a network, such as a local area network (LAN) or a wide area network (WAN) in terms of coverage, and via a wired or wireless network in terms of connection method.
[0034] The processor 100 refers to a device with data processing and control capabilities. For example, the processor 100 can be a graphics processing unit (GPU), a central processing unit (CPU), etc., and there is no limitation on this.
[0035] Memory 200 refers to a device with file storage capabilities, such as semiconductor memory, magnetic memory, optical memory, etc., without limitation. Additionally, memory 200 may store a database for data management and retrieval, without limitation.
[0036] The measuring device 300 refers to a device capable of measuring the scattering signals of materials. For example, it can be a device based on scattering measurement technology, an ellipsometry, a device based on critical dimension small angle X-ray scattering (CD-SAXS) technology, a critical dimension scanning electron microscope (CD-SEM), etc., without limitation.
[0037] In practical applications, determining the parameter information of micro / nanostructures can help describe their performance and facilitate quality inspection. Parameters related to micro / nanostructures can include their CD (cathode), period, height, sidewall angles, etc., and the parameter information refers to the actual values of these parameters.
[0038] Typically, algorithms based on electromagnetic field simulation can be used to determine the theoretical scattering signals of micro / nanostructures under different parameter values, and these signals can be stored in a database. The physics-based algorithm can be a rigorous coupled wave analysis (RCWA) algorithm or other algorithms, without limitation. Then, the actual scattering signals of the micro / nanostructure are measured using a measuring device 300, and the actual parameter values of the micro / nanostructure can be obtained by comparing and matching different theoretical and measured signals. However, in the process of building the database, physics-based algorithms may consume excessive computation time and cost when dealing with large-scale data. Therefore, AI models can be used to calculate the theoretical signals of the micro / nanostructure. These AI models can be trained based on the correspondence between relatively small-scale theoretical signals and the parameter values of the micro / nanostructure.
[0039] However, the aforementioned AI models may suffer from low training efficiency and performance limitations that cannot meet the measurement requirements of different micro / nano structures. For example, if the training data used to train the AI model is complex, the model may struggle to achieve feature decoupling, leading to poor performance. Conversely, if the training data is simple, the model may waste too much time on training.
[0040] Based on this, the processor 100 determines first indication information, which is used to indicate the complexity of the structure of the first artificial intelligence (AI) model. The first indication information includes one or more data indicators of the first training data used to train the first AI model. According to the first indication information, the processor 100 determines a first value of the hyperparameters of the first AI model for the first indication information. The processor 100 sets the hyperparameters of the first AI model to the first value. The processor 100 trains the first AI model using the first training data, which includes theoretical signals corresponding to N values of the parameters of the micro / nano structure. The first AI model is used to calculate the theoretical signals based on different values of the parameters of the micro / nano structure, where N is a positive integer.
[0041] Thus, considering the complex and variable nature of micro / nano structures, before training the AI model used to calculate the theoretical signals of micro / nano structures with different parameter values, different hyperparameter values (e.g., the total amount of data N) can be adaptively configured for the first AI model, taking into account the different data indicators of the first training data (e.g., the total amount of data N). Therefore, this effectively avoids the problem that a relatively simple model structure, when the structure of the first AI model is fixed, cannot fully learn the features of the complex first training data during training, effectively ensuring the performance of the first AI model and improving the accuracy of calculating the theoretical signals; it also effectively avoids the problem that a relatively complex model structure, when the structure of the first AI model is fixed, will consume a lot of time and training costs during training, and effectively reduces the risk of gradient vanishing, improving the efficiency of model training.
[0042] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, various non-limiting embodiments of the present application will be described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.
[0043] See Figure 2 , Figure 2 This paper illustrates a flowchart of a model training method according to an embodiment of the present application. This method can be applied to... Figure 1 The data processing system 10 shown can be applied to other applicable data processing systems, as described below. Figure 1 The following explanation uses the data processing system 10 shown as an example for model training. Figure 2 As shown, the method may specifically include the following steps.
[0044] S201: The processor 100 determines first indication information, which is used to indicate the complexity of the structure of the first artificial intelligence (AI) model. The first indication information includes one or more data metrics of the first training data used to train the first AI model.
[0045] In this embodiment, the first AI model is used to calculate the theoretical signal of the micro / nano structure under different values. The first AI model can be a neural network model, or other AI or machine learning (ML) models; there is no limitation on this.
[0046] Before training the first AI model, the processor 100 first determines first instruction information, which includes one or more data metrics of the first training data used to train the first AI model.
[0047] The first training data is described below as an example. The first training data may include theoretical signals corresponding to N values of the parameters of the micro / nano structure. That is, the first training data includes N different sets of correspondences between the parameter values of the micro / nano structure and their theoretical signals. These different correspondences can, to some extent, summarize the nonlinear relationship between the parameter values and the theoretical signals, where N is a positive integer. The aforementioned first training data can be obtained by the processor 100 calculating the theoretical signals of the micro / nano structure under various parameter values based on physical principles. For example, the processor 100 can use the RCWA algorithm to determine the training data. It should be noted that the aforementioned first training data can also be determined through other implementation methods, which are not limited thereto.
[0048] The parameters of a micro / nanostructure can include the critical dimension (CD), which is the lateral dimension of the smallest or core feature unit of the micro / nanostructure; the period, which is the center-to-center spacing between repeating micro / nanostructure units; the height, which is the vertical dimension of the micro / nanostructure; and the sidewall angle, which is the tilt angle between the sidewall of the micro / nanostructure and the substrate plane. Other parameters are not limited to these.
[0049] The theoretical signal of a micro / nanostructure (and similarly for subsequent measurement signals) may include Mueller elements, which describe the polarization change of polarized light after interaction with the micro / nanostructure; the theoretical signal may also include ellipsoidal parameters, which describe the change in polarization state of polarized light after reflection from the surface of the micro / nanostructure; the theoretical signal may also include reflectivity, which describes the ability of the micro / nanostructure surface to reflect light; the theoretical signal may also include transmittance, which describes the ability of light to penetrate the micro / nanostructure; and the theoretical signal may also include scattering intensity, which describes the ability of the micro / nanostructure to scatter incident light in various directions. In addition, the theoretical signal of a micro / nanostructure may also include other parameters, which are not limited thereto.
[0050] It should be noted that the above-mentioned different parameters can be obtained by measuring different types of measuring equipment 300. For example, Mueller elements and ellipsometric parameters can be spectral data obtained by ellipsometer measurement, reflectance and transmittance can be spectral data obtained by spectrophotometer measurement, and scattering intensity can be data obtained by equipment based on CD-SAXS technology, CD-SEM, etc., and there is no limitation on these.
[0051] The first indication information is described in detail below. The first indication information includes one or more data metrics of the first training data. It is understood that the data metrics of the first training data are statistical descriptions of the first training data, which can be used to indicate the scale of the first training data and the characteristics of the data contained within it. For example, the first indication information may include the data volume N of the first training data, the range of values for the parameters of the micro / nano structures in the first training data, the variation of the theoretical signals of the micro / nano structures with the parameter values of the micro / nano structures, and the variation of the parameter values of each theoretical signal in the first training data under different measurement conditions. In addition, the first indication information may include other similar statistical descriptions of the first training data, which are not limited thereto.
[0052] The variation of the theoretical signal of a micro / nanostructure with its parameters can be, for example, the continuity of the theoretical signal as the parameters change linearly, i.e., whether the theoretical signal remains continuous and does not jump during the change. Alternatively, the variation of the theoretical signal with its parameters can be, for example, the linearity of the theoretical signal as the parameters change linearly, i.e., whether the change of the theoretical signal is linear and non-oscillating, or whether the change of the theoretical signal is easily interpretable (easily learned by AI models). Other variations are also possible and are not limited to these specific examples.
[0053] Furthermore, the variation of the parameters of each theoretical signal in the first training data under different measurement conditions can be, for example, the continuity of the values of each parameter contained in the theoretical signal (e.g., Mueller element, ellipsoid parameter, reflectivity, transmittance, scattering intensity, or other parameters mentioned above) as the measurement conditions change linearly. That is, whether the values of each parameter of the theoretical signal remain continuous and do not jump during the change. Alternatively, it can be the linearity of the values of each parameter contained in the theoretical signal as the wavelength changes during the linear change of measurement conditions. That is, whether the changes in the values of each parameter are linear, non-oscillating, or easily interpretable. The above variations can also take other forms, which are not limited here.
[0054] Understandably, the first indication information can describe the complexity of the first training data. For example, if the amount of data N is larger, the range of values for the parameters of the micro / nano structure is wider, and the changes in the theoretical signals of the micro / nano structure with the values of the parameters of the micro / nano structure become more complex (less easily interpreted and learned, including but not limited to jumps and discontinuities in the theoretical signals during changes, oscillations and nonlinearities in the changes of the theoretical signals), or the changes in the values of the parameters of each theoretical signal with the measurement conditions become more complex (less easily interpreted, including but not limited to jumps and discontinuities in the values of each parameter of the theoretical signals during changes, oscillations and nonlinearities in the changes of the values of each parameter), then the features that the first AI model needs to learn from the first training data will be more complex. Therefore, the first indication information can be used to indicate the complexity of the structure of the first AI model; that is, the more complex the first training data, the more complex the structure of the first AI model should be, thereby ensuring the performance of the first AI model.
[0055] S202: Based on the first instruction information, the processor 100 determines the first value of the hyperparameter of the first AI model for the first instruction information.
[0056] S203: Processor 100 sets the hyperparameters of the first AI model to the first value.
[0057] The first indication information can be used to indicate the complexity of the structure of the first AI model. Therefore, the hyperparameter values of the first AI model can be adjusted using the first indication information. That is, as the first training data becomes more complex, the processor 100 can adjust the hyperparameter values of the first AI model to make the model structure used for training more complex to suit the first training data. In other words, the processor 100 determines the first value of the hyperparameters of the first AI model for the first indication information based on the first indication information.
[0058] The hyperparameters of the first AI model refer to the parameters used to control the model training process, describing the specific structure and implementation logic of the AI model. For example, the hyperparameters of the first AI model to be determined may include: network depth (the number of hidden layers in the network); network width (the number of neurons in each layer); learning rate (the step size at which the model updates in each iteration); regularization strength (the weights used to limit overfitting); or batch size (the number of samples included in a single batch of training). In addition, the hyperparameters of the first AI model to be determined may include more parameters, without limitation.
[0059] It should be noted that the first value determined by the processor 100 can be a combination of the values of each parameter item of the hyperparameters of the first AI model.
[0060] Below are several examples of how processor 100 determines the first value.
[0061] As a first possible implementation, the processor 100 can determine the first value based on the correspondence between different first indication information and the values of hyperparameters, wherein the correspondence can be pre-configured in the processor 100 by the developers. Specifically, the first training data can correspond to different hyperparameter values under different complexity conditions. Furthermore, based on the quantized correspondence, the processor 100 can directly determine the first value. For example, the first indication information may include the total amount of data N in the first training data, and the parameters of the hyperparameters may include the depth of the neural network. Therefore, different values of N can correspond to different neural network depths. It is understood that the larger the value of N, the higher the depth of the neural network (i.e., the more hidden layers).
[0062] As a second possible implementation, since the first training data may be large in scale, the processor 100 can pre-train the first AI model and determine the first value based on the first indication information according to the pre-training results. Specifically, the processor 100 can first determine the second indication information, which includes one or more data indicators of the second training data used for pre-training the first AI model. That is, the second indication information is a statistical description of the second training data and can be used to indicate the scale of the second training data and the characteristics of the data contained in the second training data.
[0063] The second training data includes the theoretical signals corresponding to the M values of the parameters of the micro / nano structure. For a detailed description of the second training data, please refer to the relevant description of the first training data in S201, which will not be repeated here.
[0064] In addition, the second indication information may include one or more of the following: the range of values for the parameters of the micro / nano structure in the second training data (M as described above), the variation of the theoretical signals in the second training data with the values of the parameters of the micro / nano structure, or the variation of the parameter values of each theoretical signal in the second training data with the measurement conditions. A detailed description of the second indication information can be found in the relevant description of the first indication information in S201, and will not be repeated here.
[0065] It should be noted that the data indicators included in the first and second indication information can be the same. For example, the first indication information may include the data volume N of the first training data, and the second indication information may include the data volume M of the second training data. Furthermore, the size of the second training data can be smaller or much smaller than the size of the first training data; that is, M is a positive integer less than or much smaller than N.
[0066] Furthermore, the processor 100 can pre-train the first AI model using the second training data, and determine the second values of the hyperparameters of the first AI model in response to the second instruction information based on the training progress of the pre-training of the first AI model. In other words, for small-scale training data such as the second training data, the processor 100 can monitor the convergence accuracy, training time, and other training conditions during the pre-training process to determine the second values of the hyperparameters that conform to the second instruction information.
[0067] As can be easily understood from the above description, compared to directly determining the first value of the hyperparameter using data metrics (i.e., the first indication information) from a large-scale second training data set, the second value of the hyperparameter that conforms to the second indication information can be determined more quickly and easily during pre-training with a small-scale second training data set. Based on this, according to the difference between the first and second indication information (i.e., the magnitude of the difference in data scale), the processor 100 can determine the first value based on the second value. Alternatively, the processor 100 can determine the numerical range for the first value based on the second value, and then select the first value from within that numerical range (for example, by combining the correspondence from the first implementation method).
[0068] For example, the first indication information includes the amount of data N in the first training data, and the second indication information includes the amount of data M in the second training data. The first value and the second value can refer to the depth of the neural network. Thus, the first value can specifically be the ratio of N to M and the product of the second value, that is, the depth of the neural network is proportionally increased based on the pre-training results.
[0069] It should be noted that the processor 100 can also determine the first value of the hyperparameters of the first AI model in other ways, without limitation.
[0070] After determining the first value, the processor 100 can set the hyperparameter values of the first AI model to the aforementioned first value. In this way, based on different data metrics of the first training data, adaptive adjustment of the hyperparameter values of the first AI model is achieved.
[0071] S204: The processor 100 uses the first training data to train the first AI model. The first training data includes theoretical signals corresponding to N values of the parameters of the micro-nano structure. The first AI model is used to calculate the theoretical signals based on different values of the parameters of the micro-nano structure, where N is a positive integer.
[0072] After setting the hyperparameters of the first AI model to a first value that matches the first instruction information, the processor 100 can train the first AI model using the first training data.
[0073] Thus, considering the complex and variable nature of micro / nano structures, before training the AI model used to calculate the theoretical signals of micro / nano structures with different parameter values, different hyperparameter values (e.g., the total amount of data N) can be adaptively configured for the first AI model, taking into account the different data indicators of the first training data (e.g., the total amount of data N). Therefore, this effectively avoids the problem that a relatively simple model structure, when the structure of the first AI model is fixed, cannot fully learn the features of the complex first training data during training, effectively ensuring the performance of the first AI model and improving the accuracy of calculating the theoretical signals; it also effectively avoids the problem that a relatively complex model structure, when the structure of the first AI model is fixed, will consume a lot of time and training costs during training, and effectively reduces the risk of gradient vanishing, improving the efficiency of model training.
[0074] Furthermore, during the training of the first AI model, the values of its hyperparameters can be adjusted accordingly based on the training results.
[0075] For example, during the training of the first AI model using the first training data, if the training status of the first AI model meets a first condition, the hyperparameters of the first AI model can be adjusted to a third value based on the aforementioned training status. This adjusted third value is used to optimize the training status of the first AI model, so that it minimizes the fulfillment of the first condition. The first condition could be, for example, that the performance improvement of the first AI model in each iteration is less than a first threshold. That is, after each iteration, if the current first AI model is used for testing (i.e., evaluating the performance / generalization ability of the first AI model), and the obtained test accuracy does not improve significantly or even decreases over multiple iterations, the hyperparameters of the first AI model can be adaptively adjusted to avoid overfitting. For example, the depth of the neural network can be increased to further improve the performance of the first AI model. Alternatively, the first condition could be that the training error of the first AI model exceeds a second threshold. In other words, during the training of the first AI model, the error between the theoretical signal generated by the first AI model and the theoretical signal given in the first training data is monitored. Once the training error exceeds a second threshold, the hyperparameters of the first AI model can be adjusted accordingly. This second threshold can be a pre-set fixed error threshold or an adaptive second threshold based on the first indication information. For example, the second threshold may vary depending on the parameter values of the micro / nano structure in the first training data.
[0076] It should be noted that the above description of the processor 100 adjusting the values of the hyperparameters of the first AI model is only an illustrative example. In actual applications, the processor 100 may also adjust the values of the hyperparameters of the first AI model based on other implementation methods.
[0077] S205: Processor 100 measures measurement signals of micro / nano structures.
[0078] In practical applications, the processor 100 can use the measurement device 300 in the data processing system 10 to measure the micro-nano structure, or the processor 100 can send a measurement instruction signal to the measurement device 300 to obtain the measurement signal of the micro-nano structure. The measurement signal may include Mueller elements, ellipsoid parameters, reflectivity, etc., which can be referred to in the relevant description in step S201, and will not be repeated here.
[0079] S206: Using the first AI model, the processor 100 performs a fitting algorithm on the measurement signal to obtain the target values of the parameters of the micro / nano structure. The theoretical signal corresponding to the target value matches the measurement signal.
[0080] The processor 100 can use the first AI model to fit the measurement signal obtained in step S205. In practical applications, the processor 100 can use different fitting algorithms for fitting, for example, the processor 100 can use the LM (Levenberg-Marquarelt) nonlinear fitting algorithm for fitting, and there is no limitation on this.
[0081] During the fitting process of the measurement signal, the processor 100 can iterate through multiple rounds of parameter values for the micro / nano structure. By continuously adjusting the parameter values, the theoretical signal corresponding to a certain parameter value of the micro / nano structure eventually matches the measured signal. At this point, the parameter value of the micro / nano structure is the target value, which is the actual parameter value of the micro / nano structure determined by the processor 100. It should be noted that matching the theoretical signal with the measured signal can mean that the sum of the squares of the residuals of each parameter term in the theoretical signal and the measured signal is minimized (below a certain threshold), without any limitation.
[0082] The following are some implementation examples of how the processor 100 uses the first AI model for fitting.
[0083] In the first implementation, the processor 100 can directly calculate the theoretical signal corresponding to the real-time parameter values of the micro / nano structure during each iteration in the fitting process. Then, by continuously adjusting the real-time values, the theoretical signal and the measured signal can be matched to complete the fitting.
[0084] As a second implementation, the processor 100 can first use the first AI model to determine a fitting dataset. This fitting dataset includes theoretical signals corresponding to the parameter values of the micro / nano structure that satisfy the fitting numerical range; that is, multiple theoretical signals corresponding to each value of the micro / nano structure parameter within its defined domain. Then, during the fitting process, the processor 100 can fit the measurement signal by retrieving the aforementioned fitting dataset. It should be noted that the aforementioned fitting dataset can be pre-calculated and stored in the memory 200, and then directly retrieved during the fitting process.
[0085] Furthermore, the above-described implementation of the processor 100 using an AI model for fitting is merely an illustrative example. In actual applications, the processor 100 may also perform fitting through other implementation methods, and no limitation is imposed on this.
[0086] Furthermore, before fitting the measurement signal, the processor 100 can determine third indication information, which includes one or more data indicators of the micro / nano structure. For example, the third indication information may include the range of values for the parameters of the micro / nano structure, or it may include other data indicators without limitation. Then, based on the third indication information, the processor 100 can adaptively adjust the values of the hyperparameters of the fitting algorithm to achieve faster fitting speed and higher fitting accuracy. The hyperparameters of the fitting algorithm may, for example, include iteration compensation parameters, i.e., the step size of each iteration; or, the hyperparameters may include a convergence threshold, i.e., the criterion for stopping iteration. In addition, the hyperparameters may include other parameter terms without limitation. For example, if the range of values for the parameters of the micro / nano structure is large, the step size of each iteration can be adaptively adjusted to achieve a faster fitting speed.
[0087] In addition, the values of hyperparameters can be adaptively adjusted during the execution of the fitting algorithm. For example, if the difference between the measured signal and the theoretical signal is large, the iteration step size can be larger to improve the iteration speed; if the difference between the measured signal and the theoretical signal is small, the iteration step size can be smaller to improve the iteration accuracy.
[0088] It should be noted that the processor 100's adjustment of the hyperparameter values of the fitting algorithm based on the third instruction information is merely an illustrative example. In actual applications, the hyperparameter values can also be adjusted in other ways, and this is not limited.
[0089] Furthermore, embodiments of this application also provide a model training apparatus. See also... Figure 3 , Figure 3 This paper shows a schematic diagram of the structure of a model training device according to an embodiment of the present application. Figure 3 The model training device 300 shown includes:
[0090] The system includes a determining module 301, which determines first indication information, the first indication information indicating the structural complexity of a first artificial intelligence (AI) model, the first indication information including one or more data indicators of first training data used to train the first AI model; determining a first value for the hyperparameters of the first AI model based on the first indication information; a setting module 302, which sets the hyperparameters of the first AI model to the first value; and a training module 303, which trains the first AI model using the first training data, the first training data including theoretical signals corresponding to N values of the parameters of the micro / nano structure, the first AI model calculating theoretical signals based on different values of the parameters of the micro / nano structure, where N is a positive integer.
[0091] In one possible implementation, the determining module 301 is specifically configured to: determine second indication information, the second indication information including one or more data indicators of second training data used for pre-training the first AI model; pre-train the first AI model using the second training data, the second training data including theoretical signals corresponding to M values of the parameters of the micro / nano structure, where M is a positive integer less than N; determine a second value of the hyperparameters of the first AI model for the second indication information based on the training situation of the pre-training of the first AI model; and determine the first value based on the second value according to the difference between the first indication information and the second indication information.
[0092] In one possible implementation, the device further includes an adjustment module 304, which is configured to, during the training of the first AI model using the first training data, adjust the hyperparameters of the first AI model to a third value based on the training status of the first AI model when the training status of the first AI model meets a first condition. The third value is used to optimize the training status of the first AI model. The first condition includes the performance improvement of the first AI model being less than a first threshold, or the first condition includes the training error of the first AI model exceeding a second threshold.
[0093] In one possible implementation, the first indication information includes one or more of the following: the amount of data N of the parameters of the micro / nano structure included in the first training data; the value range of the parameters of the micro / nano structure in the first training data; the variation of the theoretical signal in the first training data with the value of the parameters of the micro / nano structure; or the variation of the parameter values of each theoretical signal in the first training data with the measurement conditions. The hyperparameters of the first AI model include one or more of the following: network width, network depth, learning rate, regularization strength, and batch size.
[0094] In one possible implementation, the second indication information includes one or more of the following: the value range of the parameters of the micro / nano structure in the second training data, the variation of the theoretical signal in the second training data with the value of the parameters of the micro / nano structure, or the variation of the parameter values of each theoretical signal in the second training data with the measurement conditions.
[0095] In one possible implementation, the method further includes a measurement module 305 for measuring the measurement signal of the micro / nano structure; the method further includes an execution module 306 for using the first AI model to perform a fitting algorithm on the measurement signal to obtain target values of the parameters of the micro / nano structure, wherein the theoretical signal corresponding to the target value matches the measurement signal.
[0096] In one possible implementation, the determining module 301 is further configured to determine third indication information before fitting the measurement signal, the third indication information including one or more data indicators of the micro / nano structure; the adjusting module 304 is further configured to adjust the values of the hyperparameters of the fitting algorithm according to the third indication information, the hyperparameters of the fitting algorithm including iterative compensation parameters.
[0097] In one possible implementation, the execution module 306 is specifically used to: use the first AI model to determine a fitting dataset, the fitting dataset including theoretical signals corresponding to the values of the parameters of the micro / nano structure that satisfy the fitting numerical range; and by retrieving the fitting dataset, perform a fitting algorithm on the measurement signal to obtain the target value within the fitting numerical range.
[0098] It should be noted that the information interaction and execution process between the modules and units of the above-mentioned device are based on the same concept as the method embodiment in this application, and the resulting technical effects are the same as those in the method embodiment in this application. For details, please refer to the description in the method embodiment shown above in this application, and it will not be repeated here.
[0099] Furthermore, embodiments of this application also provide a computing device. See also... Figure 4 , Figure 4 A schematic diagram of the hardware structure of a computing device according to an embodiment of this application is shown. Figure 4 As shown, computing device 400 may include processor 401 and memory 402.
[0100] The memory 402 is used to store computer programs;
[0101] The processor 401 is configured to execute the model training method described in the above method embodiments according to the computer program.
[0102] In addition, this application embodiment also provides a computer-readable storage medium for storing a computer program for executing the model training method described in the above method embodiments.
[0103] In addition, this application also provides a computer program product containing instructions that, when run on a computing device, causes the computing device to execute the model training method described in the above method embodiments.
[0104] In the embodiments of this application, the "first" in names such as "first AI model" is only used for naming purposes and does not represent the first in order. This rule also applies to "second," "third," etc.
[0105] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that all or part of the steps in the methods of the above embodiments can be implemented by means of software plus a general-purpose hardware platform. Based on this understanding, the technical solution of this application can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as a read-only memory (ROM) / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, a server, or a network communication device such as a router) to execute the methods described in various embodiments or some parts of the embodiments of this application.
[0106] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The device embodiments described above are merely illustrative. Modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.
[0107] The above description is merely an exemplary implementation of this application and is not intended to limit the scope of protection of this application.
Claims
1. A model training method, characterized in that, The methods include: The first indication information is determined. The first indication information is used to indicate the complexity of the structure of the first AI model. The first indication information includes one or more data indicators of the first training data used to train the first AI model. The one or more data indicators of the first training data are used to indicate the scale of the first training data and the characteristics of the data contained in the first training data. The first indication information includes one or more of the following: the amount of data N of the parameters of the micro-nano structure included in the first training data, the value range of the parameters of the micro-nano structure in the first training data, the change of the theoretical signal in the first training data with the value of the parameters of the micro-nano structure, or the change of the parameter value of each theoretical signal in the first training data with the wavelength. Determine the second indication information, which includes one or more data metrics of the second training data used to pre-train the first AI model; The first AI model is pre-trained using the second training data, which includes the theoretical signals corresponding to M values of the parameters of the micro-nano structure, where M is a positive integer less than N. Based on the training results of the first AI model during pre-training, determine the second value of the hyperparameters of the first AI model for the second instruction information; Based on the difference ratio between the first instruction information and the second instruction information, the second value is amplified to the first value of the hyperparameter of the first AI model for the first instruction information. The first value is the product of the second value and the difference ratio. Set the hyperparameters of the first AI model to the first value; The first AI model is trained using the first training data, which includes theoretical signals corresponding to N values of the parameters of the micro-nano structure. The first AI model is used to calculate the theoretical signals based on different values of the parameters of the micro-nano structure, where N is a positive integer.
2. The method according to claim 1, characterized in that, The method further includes: During the process of training the first AI model using the first training data, when the training status of the first AI model meets the first condition, the hyperparameters of the first AI model are adjusted to a third value according to the training status of the first AI model. The third value is used to optimize the training status of the first AI model. The first condition includes the performance improvement of the first AI model being less than a first threshold, or the first condition includes the training error of the first AI model exceeding a second threshold.
3. The method according to claim 1 or 2, characterized in that, The hyperparameters of the first AI model include one or more of the following: network width, network depth, learning rate, regularization strength, and batch size.
4. The method according to claim 2, characterized in that, The second indication information includes one or more of the following: the value range of the parameters of the micro / nano structure in the second training data, the change of the theoretical signal in the second training data with the value of the parameters of the micro / nano structure, or the change of the parameter values of each theoretical signal in the second training data with the measurement conditions.
5. The method according to any one of claims 1, 2, and 4, characterized in that, The method further includes: The measurement signal of the micro / nano structure is measured; Using the first AI model, a fitting algorithm is performed on the measurement signal to obtain the target values of the parameters of the micro / nano structure. The theoretical signal corresponding to the target value matches the measurement signal.
6. The method according to claim 5, characterized in that, Before fitting the measured signal, the method further includes: Determine the third indication information, which includes one or more data indicators of the micro / nano structure; Based on the third instruction information, the values of the hyperparameters of the fitting algorithm are adjusted, and the hyperparameters of the fitting algorithm include iterative compensation parameters.
7. The method according to claim 5, characterized in that, The step of using the first AI model to perform a fitting algorithm on the measurement signal to obtain the target values of the parameters of the micro / nano structure includes: Using the first AI model, a fitting dataset is determined, which includes theoretical signals corresponding to the values of the parameters of the micro / nano structure that satisfy the fitting numerical range. By retrieving the fitted dataset, a fitting algorithm is executed on the measured signal to obtain the target value within the fitted numerical range.
8. A model training device, characterized in that, The device includes: The determining module is used to determine first indication information. The first indication information indicates the structural complexity of a first AI model. The first indication information includes one or more data metrics of the first training data used to train the first AI model. These data metrics indicate the scale of the first training data and the characteristics of the data contained within it. The first indication information includes one or more of the following: the amount of data N of the parameters of the micro / nanostructures included in the first training data; the value range of the parameters of the micro / nanostructures in the first training data; the variation of theoretical signals in the first training data with the values of the parameters of the micro / nanostructures; or the variation of the values of the parameters of each theoretical signal in the first training data with wavelength. The process involves: determining second indication information, which includes one or more data metrics of second training data used for pre-training the first AI model; pre-training the first AI model using the second training data, which includes theoretical signals corresponding to M values of the parameters of the micro / nano structure, where M is a positive integer less than N; determining a second value of the hyperparameters of the first AI model for the second indication information based on the training results of the pre-training of the first AI model; and amplifying the second value to a first value of the hyperparameters of the first AI model for the first indication information based on the difference ratio between the first and second indication information, where the first value is the product of the second value and the difference ratio. The setting module is used to set the hyperparameters of the first AI model to the first value; The training module is used to train the first AI model using the first training data, which includes theoretical signals corresponding to N values of the parameters of the micro / nano structure. The first AI model is used to calculate the theoretical signals based on different values of the parameters of the micro / nano structure, where N is a positive integer.
9. A computer program product containing instructions, characterized in that, When it is run on a computing device, it causes the computing device to perform the method as described in any one of claims 1 to 7.