Automatic design method for optical integrated device based on large model and black box optimization

Through an automatic design method based on large model and black box optimization, the problem of no professional participation and process limitations in optical integrated device design is solved, and an automated optical integrated device design is realized, meeting user requirements and improving design efficiency and accuracy are improved. .

CN119066862APending Publication Date: 2024-12-03SHAOXING RES INST OF ZHEJIANG UNIV
View PDF 0 Cites 3 Cited by

Patent Information

Application Number
CN202411167389.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-23
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The prior art cannot automatically generate an optimal design of optical integrated device that meets user requirements without the participation of professionals, and the design of optical integrated device requires simulation adjustment and verification, making it difficult to meet process limitations and size requirements in actual scenarios.

Method used

Using automatic design methods based on big model and black box optimization, the automatic design of optical integrated devices is realized by building a professional knowledge base, multimodal big model preprocessing, language big model technology summary, embedded model search, and black box optimization algorithm optimization .

Benefits of technology

It realizes automatic design of optical integrated devices without professional participation, meets user requirements, automatically adjusts parameters and optimizes simulation scripts, and improves design efficiency and accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119066862A_ABST
    Figure CN119066862A_ABST
Patent Text Reader

Abstract

The invention discloses an optical integrated device automatic design method based on a large model and black box optimization. The method comprises the following steps: S1, constructing a professional knowledge base; s2, preprocessing the data in the professional knowledge base; s3, carrying out technical summarization; s4, on the basis of user input, searching an optical integrated device which most meets user requirements by utilizing the language large model and the embedded model; s5, key parameters, a parameter setting range and constraint conditions for designing the optical integrated device are extracted; s6, generating a simulation script by using the large language model; and S7, calling the simulation script by using a black box optimization algorithm, and outputting parameters of the optical integrated device. According to the automatic design method for the optical integrated device based on the large model and black box optimization disclosed by the invention, the optimal design of the optical integrated device meeting the user requirements is automatically generated on the premise of not needing the participation of professionals.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of optical integrated device design, and in particular relates to an automatic design method for optical integrated devices based on a large model and black box optimization. Background Art

[0002] Large language models usually do not have specialized knowledge in specific areas, nor are they up-to-date. Enhanced retrieval generation technology can search for context in private knowledge bases and inject it into the prompt words of large language models to obtain more professional and timely answers. But even so, the question-answering mode of large language models still cannot directly generate practical optical integrated device designs. One of the reasons is that the design of optical integrated devices requires professionals to repeatedly adjust and verify through simulation to achieve optimal performance. The second reason is that in actual scenarios, constraints under process limitations also need to be considered, and user requirements in terms of size and other aspects must be given priority.

[0003] Therefore, further improvements are made to the above problems. Summary of the invention

[0004] The main purpose of the present invention is to provide an automatic design method for optical integrated devices based on large models and black box optimization, which can automatically generate the optimal design of optical integrated devices that meets user requirements without the participation of professionals.

[0005] To achieve the above objectives, the present invention provides an automatic design method for optical integrated devices based on large models and black box optimization, comprising the following steps:

[0006] Step S1: construct a professional knowledge base based on optical integrated devices (collect relevant literature and experimental notes and other materials);

[0007] Step S2: preprocessing the data in the professional knowledge base using a multimodal large model;

[0008] Step S3: using the language macromodel to perform a technical summary of each optical integrated device in the professional knowledge base (including key information such as function, material, structure, size, and operating wavelength);

[0009] Step S4: Based on the user input, the language macro model and the embedding model are used to search for the optical integrated device that best meets the user's needs in the professional knowledge base after the technical summary;

[0010] Step S5: inputting the searched optical integrated device data and user requirements as prompt words into the language macromodel at the same time, extracting key parameters, parameter setting ranges and constraint conditions for designing the optical integrated device, and automatically correcting parameters that do not meet the user requirements;

[0011] Step S6: Based on the key parameters, parameter setting ranges and constraints of the designed optical integrated device, a simulation script is generated using a language macro model;

[0012] Step S7: Use the black box optimization algorithm to call the simulation script, continue to optimize until the best performance is obtained and output the parameters of the optical integrated device.

[0013] As a further preferred technical solution of the above technical solution, in step S2, a multimodal large model is used to convert tables and formulas into text information, and the optical integrated device design drawing is converted into a text description in combination with the context.

[0014] As a further preferred technical solution of the above technical solution, in step S4, when searching for optical integrated devices that meet user requirements in the professional knowledge base, the embedding model is used to convert text information into vectors, and several optical integrated devices that meet the requirements are first obtained through the vector similarity between the user input and the technical summary, and then the large language model is used to screen out the optical integrated device that best meets the requirements.

[0015] As a further preferred technical solution of the above technical solution, in step S7, the simulation output of the optical integrated device is regarded as a black box function, and the simulation script is called using a black box optimization algorithm to continuously optimize the input parameters, and the parameters under the optimal result are output under the premise of satisfying the constraints.

[0016] As a further preferred technical solution of the above technical solution, the embedded model and the large language model are models deployed locally or models deployed on a remote server that are accessed through an API.

[0017] The beneficial effects of the present invention are:

[0018] The present invention subdivides the process of optical integrated device design into the steps of data search, key information extraction, simulation script generation, and parameter optimization, and realizes the automation processing of each step based on a large model and a black box optimization algorithm, thereby achieving the effect of automatically generating an optical integrated device design that meets user requirements without the participation of professionals. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a schematic diagram of the process of the present invention.

[0020] Figure 2 It is a schematic diagram of the process of automatically generating a cross waveguide device design that meets user requirements according to the present invention.

[0021] Figure 3 It is a schematic diagram of preprocessing a table in a professional knowledge base based on a visual macro model according to the present invention.

[0022] Figure 4It is a schematic diagram of preprocessing pictures in the professional knowledge base based on the visual big model of the present invention.

[0023] Figure 5 This is a schematic diagram of the present invention's technical summary of various optical integrated devices in the professional knowledge base based on the language big model. DETAILED DESCRIPTION

[0024] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art can think of other obvious variations. The basic principles of the present invention defined in the following description can be applied to other embodiments, variations, improvements, equivalents, and other technical solutions that do not deviate from the spirit and scope of the present invention.

[0025] In the preferred embodiments of the present invention, those skilled in the art should note that the optical integrated device and the like involved in the present invention may be regarded as prior art.

[0026] Preferred embodiments.

[0027] like Figure 1 As shown, the present invention discloses an automatic design method for optical integrated devices based on large models and black box optimization, comprising the following steps:

[0028] Step S1: construct a professional knowledge base based on optical integrated devices (collect relevant literature and experimental notes, etc., which can be PDF format documents, or plain text documents in other formats, or documents containing charts and formulas);

[0029] Step S2: preprocessing the data in the professional knowledge base using the multimodal large model (including text processing of PDF format documents and converting the charts and formulas contained in the documents into text information);

[0030] Step S3: using the language macromodel to perform a technical summary of each optical integrated device in the professional knowledge base (including key information such as function, material, structure, size, and operating wavelength);

[0031] Step S4: Based on the user input, the language macro model and the embedding model are used to search for the optical integrated device that best meets the user's needs in the professional knowledge base after the technical summary;

[0032] Step S5: inputting the searched optical integrated device data and user requirements as prompt words into the language macromodel at the same time, extracting key parameters, parameter setting ranges and constraint conditions for designing the optical integrated device, and automatically correcting parameters that do not meet the user requirements;

[0033] Step S6: Based on the key parameters, parameter setting ranges and constraints of the optical integrated device, a simulation script is generated using the language macromodel; if necessary, the generated script can be checked and tested manually, or the script can be input into another macromodel for program syntax checking. The simulation software corresponding to the simulation script can be commercial simulation software such as Lumerical, HFSS, etc., or open source simulation software such as Meep.

[0034] Step S7: Use the black box optimization algorithm to call the simulation script, continue to optimize until the best performance is obtained and output the parameters of the optical integrated device.

[0035] Specifically, in step S2, the multimodal large model is used to convert the table and formula into text information, and the optical integrated device design drawing is converted into a text description in combination with the context. When the PDF document is converted into text, the traditional OCR (Optical Character Recognition) text recognition method based on template matching and feature extraction can be used, or the multimodal large model can be used for text recognition.

[0036] When converting tables and formulas into text information, a multimodal large model can be used for automatic conversion.

[0037] When converting images into text descriptions, we can first extract the context in the corresponding document, then input the context information and the image into a multimodal large model, and generate a detailed text description of the image through prompt words.

[0038] The multimodal big model used can be a big model deployed locally, or a big model deployed on a remote server accessed through an API.

[0039] More specifically, in step S4, when searching for optical integrated devices that meet the user's requirements in the professional knowledge base, the text information is converted into a vector using an embedding model, and several optical integrated devices that meet the requirements are first obtained through the vector similarity between the user input and the technical summary, and then the large language model is used to screen out the optical integrated device that best meets the requirements. When searching for optical integrated device information that best meets the user's needs in the professional knowledge base based on user input, the text input by the user can be first converted into a vector using an embedding model, and the technical summary of each device generated above can be converted into a vector, and the technical summary of the device that best meets the user's requirements can be obtained through vector comparison. It is also possible to first obtain the technical summary of several devices that meet the user's requirements through an embedding model, and then input the obtained technical summary of several devices into the large language model, and use prompt words to allow the large language model to screen out the device that best meets the user's requirements.

[0040] Furthermore, in step S7, the simulation output of the optical integrated device is regarded as a black box function, and the black box optimization algorithm is used to call the simulation script and continuously optimize the input parameters, and the parameters under the optimal result are output under the premise of satisfying the constraints. The black box optimization algorithm used can be Bayesian optimization, meta-heuristic algorithm, end-to-end neural network, or the parameters can be adjusted and optimized by inputting a large language model with prompt words. Here is one example of using Bayesian optimization. When using Bayesian optimization as a black box optimization algorithm, it can be based on the mean value μ(x) and variance σ of the input data x. 2 (x) Establish a Gaussian Process Regression model f * (x).

[0041] f * (x)~N(μ(x),σ 2 (x)

[0042] Calculate the Gaussian process regression model f * The mean μ(x) and variance σ of (x) 2 (x), a suitable kernel function can be used to process the input data x. For example, the Gaussian kernel function K(x) is used in the following formula: i ,x j ).

[0043]

[0044] In the above formula, a is the hyperparameter of the Gaussian kernel function (also known as the Radial basis function kernel), d is the dimension of the input data, and x is i,m Represents the mth element of the i-th group of input data.

[0045] The Gaussian process regression model f established above * (x) is used as a proxy model to search for the best potential solution based on the acquisition function. For example, the following formula uses PI (Probability of Improvement) as the sampling function.

[0046]

[0047] In the above formula, Φ is the standard normal distribution function, y best is the best target value among the currently known samples. x that maximizes the sampling function PI * This is the best potential solution, which can be input into the black box function and substituted into the next iteration.

[0048] Similarly, other appropriate sampling functions may be selected according to actual conditions, such as EI (Expected improvement), UCB (Upper condence bound), EHVI (Expected Hypervolume Improvement), etc.

[0049] Similarly, other appropriate proxy models can be selected according to actual conditions. For example, the kernel function of the Gaussian process regression model can be replaced with the Matern kernel, the RationalQuadratic kernel, the sine square kernel, etc.

[0050] Similarly, other appropriate black box optimization algorithms can be selected according to actual conditions, such as genetic algorithms, particle swarm algorithms, simulated annealing algorithms, or end-to-end neural networks, etc. It is also possible to input known data samples into the language model through appropriate prompt words to make it output the best potential solution.

[0051] The termination condition of the optimization process can be that the device performance meets the user requirements, or it can be after a certain number of pre-set iterations, or it can be terminated when the device performance does not meet the requirements but the optimization process has converged.

[0052] In the above step S4, when there is more than one device structure that can realize the function required by the user, more than one device can be processed through steps S5 to S7, and the best performance result is output in step S7.

[0053] Furthermore, the embedded model and the language model are models deployed locally or models deployed on a remote server that are accessed through an API.

[0054] Preferably, when deploying an embedding model, a large language model, or a large multimodal model, the weights of the corresponding open source model can be used directly, or further training and adjustment can be performed based on the open source model for the purpose of the present invention.

[0055] Preferably, the most suitable black box optimization algorithm can be selected according to the device to be optimized.

[0056] It is worth mentioning that Figure 2 As shown, for example:

[0057] Collect thirty relevant documents on silicon-based waveguide device design and include them in the professional knowledge database;

[0058] Thirty of the above documents were preprocessed. Figure 3 As shown in Figure 1, calling GPT4o and using appropriate prompt words can convert the table into text information. Figure 4As shown, by inputting the corresponding context into GPT4o and using appropriate prompt words, the design drawing of the optical integrated device can be converted into a text description.

[0059] After the above preprocessing, a technical summary is made for each device. Figure 5 As shown in the figure, the summary in the original document does not contain all the key information of the device. By calling GPT4o and using appropriate prompt words, a comprehensive technical summary of the waveguide device can be made, which includes all the key information such as the device's function, size, operating wavelength, and final performance.

[0060] like Figure 2 As shown in the figure, the user needs to design a cross waveguide device and has made requirements on size, material, thickness, wavelength, etc. The embedded model BGE-M3 is used to convert the user's text input and the technical summary mentioned above into vectors, and through vector comparison, the device that best meets the user's requirements is obtained, that is, Figure 5 Although the size of the device in the literature is 9×9μm^2, which does not meet the user's requirement of 6×6μm^2, the other elements such as thickness, material, wavelength, etc. meet the user's requirements, so it is still the most suitable device in the professional knowledge base.

[0061] like Figure 2 As shown in the figure, the user's requirements and the device data obtained above are input into GPT4o through prompt words, so that GPT4o outputs the key parameters and constraints for designing the device. In particular, in order to meet the user's requirements on size, GPT4o automatically modifies the device length described in the original document. In particular, in order to meet the user's requirements on the number of parameters, GPT4o automatically reduces the parameters of the device width in the original document from 13 to 7.

[0062] like Figure 2 As shown, based on the output results of GPT4o mentioned above, GPT4o is further prompted to generate a python script for the simulation software Lumerical through prompt words.

[0063] like Figure 2 As shown, the Bayesian optimization algorithm is used to run the above simulation script for 100 iterations. According to the user's requirements, the loss of the cross waveguide device is optimized under the premise of satisfying the constraints, and finally the loss is less than -0.1dB, and the parameters are output. In particular, Figure 2 As shown, the device parameters can also be iteratively adjusted using GPT3.5-turbo and Llama3 through prompt words, and a loss of about -0.1dB can also be achieved after 100 iterations.

[0064] It is worth mentioning that the technical features such as optical integrated devices involved in the patent application of this invention should be regarded as prior art. The specific structure, working principle and possible control method and spatial layout method of these technical features can be selected by conventional methods in the field, and should not be regarded as the inventive point of the patent of this invention. The patent of this invention will not be further elaborated.

[0065] For those skilled in the art, it is still possible to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An automatic design method for optical integrated devices based on large models and black box optimization, characterized in that: The following steps are involved: Step S1: constructing a professional knowledge base based on optical integrated devices; Step S2: preprocessing the data in the professional knowledge base using a multimodal large model; Step S3: using the language big model to make a technical summary for each optical integrated device in the professional knowledge base; Step S4: Based on the user input, the language macro model and the embedding model are used to search for the optical integrated device that best meets the user's needs in the professional knowledge base after the technical summary; Step S5: inputting the searched optical integrated device data and user requirements as prompt words into the language macromodel at the same time, extracting key parameters, parameter setting ranges and constraint conditions for designing the optical integrated device, and automatically correcting parameters that do not meet the user requirements; Step S6: Based on the key parameters, parameter setting ranges and constraints of the designed optical integrated device, a simulation script is generated using a language macro model; Step S7: Use the black box optimization algorithm to call the simulation script, continue to optimize until the best performance is obtained and output the parameters of the optical integrated device.

2. The automatic design method of optical integrated devices based on large model and black box optimization according to claim 1, characterized in that: In step S2, the multimodal large model is used to convert tables and formulas into text information, and the optical integrated device design drawing is converted into a text description in combination with the context.

3. The automatic design method of optical integrated devices based on large model and black box optimization according to claim 2 is characterized in that: In step S4, when searching for optical integrated devices that meet the user's requirements in the professional knowledge base, the text information is converted into a vector using the embedding model. First, several optical integrated devices that meet the requirements are obtained through the vector similarity between the user input and the technical summary, and then the large language model is used to screen out the optical integrated device that best meets the requirements.

4. The automatic design method of optical integrated devices based on large model and black box optimization according to claim 3 is characterized in that: In step S7, the simulation output of the optical integrated device is regarded as a black box function, and the simulation script is called using a black box optimization algorithm to continuously optimize the input parameters, and the parameters under the optimal result are output under the premise of satisfying the constraints.

5. The automatic design method of optical integrated devices based on large model and black box optimization according to claim 4, characterized in that: Embedded models and language models are models deployed locally or models deployed on remote servers that are accessed through APIs.

Citation Information

Cited By

  • Optical system design method and system based on knowledge base retrieval enhancement large model

    CN120522893A

  • Optical system design method and system based on knowledge base retrieval enhanced large model

    CN120522893B

  • Generative design method and device for optical system based on function call

    CN121365608A