Inverse design of metaphotonic matrix processing chips, methods for determining chip structure, in-loop verification and deployment methods, and multi-task adaptive configuration methods.
By inversely designing a metaphotonic matrix processing chip, and utilizing the inverse design objective function and reconfigurable compensation network, the area and power consumption problems in the photonic matrix computing architecture are solved, realizing efficient and low-complexity optical linear transformation functions, which are suitable for multi-task processing.
Patent Information
- Application Number
- CN202611132340.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2026-06-26
- Filing Date
- 2026-07-28
- Publication Date
- 2026-11-17
AI Technical Summary
Existing programmable photonic matrix computing architectures struggle to balance programmability, integration density, and low power consumption. In particular, as the matrix dimension increases, the size of the optical interference module in the photonic processor becomes enormous, leading to increased chip area, thermal tuning power consumption, thermal crosstalk, and calibration complexity.
A reverse-design metaphotonic matrix processing chip solution is adopted. By obtaining the initial complex matrix operator, iterative updates are performed using the reverse design objective function to determine the geometric structure of the medium perturbation structure. Combined with a reconfigurable compensation network, chip parameters are calibrated and trained to achieve multi-task adaptive configuration.
It effectively reduces chip area, lowers thermal tuning power consumption and thermal crosstalk, simplifies calibration complexity, and improves processing efficiency and accuracy.
Smart Images

Figure CN123113144A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of integrated optical path technology, and in particular to a reverse-design metaphotonic matrix processing chip, a method for determining the chip structure, an in-loop verification and deployment method, and a multi-task adaptive configuration method. Background Technology
[0002] With the rapid growth in the scale of artificial intelligence models and the demand for edge intelligence applications, traditional electronic computing systems face continuous pressure in terms of data transfer power consumption, storage bandwidth, interconnect latency, and parallel computing efficiency. Photonic computing, leveraging the inherent parallelism, broadband transmission capability, and low latency of optical signals during propagation, provides a potential hardware acceleration path for matrix operations, neural network inference, multimodal signal processing, and edge intelligent computing.
[0003] Existing programmable photonic matrix computing architectures struggle to balance programmability, integration density, and low power consumption. While photonic processors based on Mach-Zehnder interferometer (MZI) arrays offer strong programmability, the increasing matrix dimension leads to a massive size of the MZI nodes in the optical interference module, resulting in increased chip area, thermal tuning power consumption, thermal crosstalk, and calibration complexity. Summary of the Invention
[0004] In view of this, this disclosure proposes a reverse-design metaphotonic matrix processing chip scheme.
[0005] According to one aspect of this disclosure, a method for determining a chip structure is provided. The chip includes one or more inverse-design metaphotonic matrix processing units, wherein the multiple inverse-design metaphotonic matrix processing units are connected in a cascaded manner. The method includes: obtaining initial complex matrix operators corresponding to each of the one or more inverse-design metaphotonic matrix processing units, the initial complex matrix operators being used to constrain amplitude and phase; obtaining an inverse design objective function; the inverse design objective function including at least: a matrix error term and a transmission efficiency constraint term; performing simulated optical linear transformation based on each of the initial complex matrix operators, and determining a first loss between the optical signal transmission transmittance and the target transmission transmittance of the one or more inverse-design metaphotonic matrix processing units based on the inverse design objective function; iteratively updating each of the initial complex matrix operators based on the first loss until a stopping condition is met, and determining a target complex matrix operator corresponding to the one or more inverse-design metaphotonic matrix processing units to obtain one or more inverse-design metaphotonic matrix processing units with a fixed geometric structure due to a dielectric perturbation structure.
[0006] In one possible implementation, obtaining the initial complex matrix operator corresponding to each of the one or more inverse-designed metaphotonic matrix processing units includes: uniformly dividing the relative phase value interval and the beam splitting ratio value interval with the same number of sub-intervals to obtain the same number of relative phase values and beam splitting ratio values; pairing the relative phase values and the beam splitting ratio values to obtain multiple relative phase-beam splitting ratio combinations to determine multiple candidate complex matrix operators; adding the multiple candidate complex matrix operators to a device library; selecting candidate complex matrix operators from the device library as the initial complex matrix operators; and iteratively updating each of the initial complex matrix operators based on the first loss includes: selecting candidate complex matrix operators from the device library to replace the initial complex matrix operators based on the first loss to update the initial complex matrix operators.
[0007] In one possible implementation, the chip includes a reconfigurable compensation network connected to the inverse-design metaphotonic matrix processing unit. The in-loop verification and deployment method of the chip includes: inputting a calibration optical signal into the chip to obtain a first output signal from the chip; calibrating chip parameters based on the difference between the first output signal and the calibration optical signal, the chip parameters including: input parameters, routing parameters, attenuation parameters, and network parameters in the reconfigurable compensation network, the network parameters including node parameters; training the reconfigurable compensation network for a sample task to obtain node parameters corresponding to the sample task; the training includes: inputting a sample optical signal into the chip based on the sample task to obtain a second output signal from the chip representing the task result; comparing the second output signal with a target output signal to obtain a task loss; updating the node parameters of each node in the reconfigurable compensation network according to the task loss; iteratively executing the steps from inputting the sample optical signal to updating the node parameters until the task loss meets a preset condition.
[0008] In one possible implementation, the step of inputting a sample optical signal into the chip based on a sample task to obtain a second output signal representing the task result from the chip includes: acquiring a perturbation vector; incrementally adjusting the current node parameters based on the perturbation vector to obtain a first fine-tuning parameter; processing the sample optical signal based on the first fine-tuning parameter to obtain a first fine-tuned output signal; decrementing the current node parameters based on the perturbation vector to obtain a second fine-tuning parameter; and processing the sample optical signal based on the second fine-tuning parameter to obtain a second fine-tuned output signal. The step of comparing the second output signal with a target output signal to obtain a task loss includes: comparing the first fine-tuned output signal with the target output signal to obtain a first sub-loss; comparing the second fine-tuned output signal with the target output signal to obtain a second sub-loss; and obtaining the task loss based on the first sub-loss and the second sub-loss.
[0009] In one possible implementation, updating the node parameters of each node in the reconfigurable compensation network based on the task loss includes: determining the loss gradient of each node based on the task loss; determining a first node whose loss gradient is greater than a gradient threshold, and a second node whose loss gradient is not greater than a gradient threshold, wherein the second node does not update its node parameters but performs forward propagation; and updating the node parameters of the first node based on the loss gradient of the first node.
[0010] In one possible implementation, the multi-task adaptive configuration method of the chip includes: training the reconfigurable compensation network contained in the chip in multiple rounds for each type of sample task to obtain a set of gradient vectors corresponding to each type of sample task; the set of gradient vectors includes the gradient vectors of each node for a single type of sample task, and the gradient vectors represent the loss gradients obtained by the same node in each round of training for a single type of sample task; based on each set of gradient vectors, determining shared nodes with similar gradient vectors for each type of sample task.
[0011] In one possible implementation, the node includes: the shared node and the private node, and the node parameters include: shared node parameters and private node parameters; the method further includes: when the current target task and the previous target task are different types of tasks, obtaining the private node parameters corresponding to the current target task; updating the private node parameters corresponding to the current target task to the private node, without updating the shared node parameters of the shared node.
[0012] According to another aspect of this disclosure, a chip structure determination apparatus is provided, the chip comprising one or more inverse design metaphotonic matrix processing units, wherein the plurality of inverse design metaphotonic matrix processing units are connected in a cascaded manner, the apparatus comprising:
[0013] An initial complex matrix operator acquisition module is used to acquire the initial complex matrix operator corresponding to each of the one or more inverse-designed metaphotonic matrix processing units, wherein the initial complex matrix operator is used to constrain the amplitude and phase;
[0014] An inverse design objective function acquisition module is used to acquire the inverse design objective function; the inverse design objective function includes at least: a matrix error term and a transmission efficiency constraint term;
[0015] The first loss determination module is used to perform simulated optical linear transformation based on each of the initial complex matrix operators, and to determine the first loss between the optical signal transmission transmittance and the target transmission transmittance of the one or more inverse design metaphotonic matrix processing units based on the inverse design objective function.
[0016] The connected inverse design metaphotonic matrix processing unit determination module is used to iteratively update each of the initial complex matrix operators based on the first loss until a stopping condition is met, and determine the target complex matrix operator corresponding to the one or more inverse design metaphotonic matrix processing units, so as to obtain one or more inverse design metaphotonic matrix processing units with fixed geometry of the medium perturbation structure.
[0017] In one possible implementation, the initial complex matrix operator acquisition module is further configured to:
[0018] The relative phase value interval and the beam split ratio value interval are evenly divided according to the same number of sub-intervals to obtain the same number of relative phase values and beam split ratio values.
[0019] The relative phase value and the beam splitting ratio value are paired to obtain multiple relative phase-beam splitting ratio combinations, thereby determining multiple candidate complex matrix operators, and adding the multiple candidate complex matrix operators to the device library;
[0020] Candidate complex matrix operators are selected from the device library as the initial complex matrix operators;
[0021] The iterative update of each of the initial complex matrix operators based on the first loss includes:
[0022] Based on the first loss, a candidate complex matrix operator is selected from the device library to replace the initial complex matrix operator, thereby updating the initial complex matrix operator.
[0023] In one possible implementation, the chip includes a reconfigurable compensation network connected to the inverse-design metaphotonic matrix processing unit, and the device includes an in-loop verification and deployment module.
[0024] The in-loop verification and deployment module is used for:
[0025] The calibration optical signal is input into the chip to obtain the first output signal output by the chip;
[0026] Based on the difference between the first output signal and the calibration optical signal, the chip parameters of the chip are calibrated. The chip parameters include: input parameters, routing parameters, attenuation parameters, and network parameters in the reconfigurable compensation network, wherein the network parameters include node parameters.
[0027] For a given sample task, the reconfigurable compensation network is trained to obtain the node parameters corresponding to the sample task.
[0028] The training includes:
[0029] Based on the sample task, the sample optical signal is input into the chip to obtain a second output signal representing the task result output by the chip;
[0030] The second output signal is compared with the target output signal to obtain the task loss;
[0031] Based on the task loss, update the node parameters of each node in the reconfigurable compensation network;
[0032] Iteratively execute the steps from input sample light signal to update node parameters until the task loss meets the preset conditions.
[0033] In one possible implementation, the in-loop verification and deployment module is further configured to:
[0034] Obtain the perturbation vector;
[0035] Based on the perturbation vector, the parameters of the current node are incrementally adjusted to obtain the first fine-tuning parameter;
[0036] Based on the first fine-tuning parameter, the sample light signal is processed to obtain the first fine-tuning output signal;
[0037] Based on the perturbation vector, the parameters of the current node are adjusted by a reduction to obtain the second fine-tuning parameter;
[0038] Based on the second fine-tuning parameter, the sample light signal is processed to obtain the second fine-tuning output signal;
[0039] The step of comparing the second output signal with the target output signal to obtain the task loss includes:
[0040] The first fine-tuned output signal is compared with the target output signal to obtain the first sub-loss.
[0041] The second fine-tuned output signal is compared with the target output signal to obtain the second sub-loss.
[0042] The task loss is obtained based on the first sub-loss and the second sub-loss.
[0043] In one possible implementation, the in-loop verification and deployment module is further configured to:
[0044] Based on the task loss, determine the loss gradient for each node;
[0045] The first node whose loss gradient is greater than the gradient threshold is identified, and the second node whose loss gradient is not greater than the gradient threshold is identified. The second node does not update its node parameters but propagates forward.
[0046] Update the node parameters of the first node based on the loss gradient of the first node.
[0047] In one possible implementation, the device further includes: a multi-task adaptive configuration module;
[0048] The multi-task adaptive configuration module is used for:
[0049] The in-loop verification and deployment module is controlled to perform multiple rounds of training on the reconfigurable compensation network contained in the chip for each type of sample task, so as to obtain a set of gradient vectors corresponding to each type of sample task.
[0050] The multi-task adaptive configuration module is also used for:
[0051] The gradient vector set contains the gradient vectors of each node for a single-type sample task. The gradient vectors represent the loss gradients obtained by the same node in each round of training for a single-type sample task.
[0052] Based on the aforementioned gradient vector sets, shared nodes with similar gradient vectors are identified for each type of sample task.
[0053] In one possible implementation, the node includes: the shared node and the private node, and the node parameters include: shared node parameters and private node parameters;
[0054] The device further includes:
[0055] The private node parameter acquisition module is used to acquire the private node parameters corresponding to the current target task when the current target task and the previous target task are different types of tasks.
[0056] The private node parameter update module is used to update the private node parameters corresponding to the current target task to the private node, without updating the shared node parameters of the shared node.
[0057] According to another aspect of this disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above-described method.
[0058] According to another aspect of this disclosure, a non-volatile computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the above-described method.
[0059] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the above-described method.
[0060] According to another aspect of this disclosure, a reverse-designed metaphotonic matrix processing chip is provided, the chip comprising: an input modulation module, one or more reverse-designed metaphotonic matrix processing units, a reconfigurable compensation network, a photodetector module, and a control module;
[0061] The input modulation module is used to encode the data to be processed to obtain the input optical field;
[0062] The inverse design metaphotonic matrix processing unit includes a medium perturbation structure; the one or more inverse design metaphotonic matrix processing units are used to perform a linear optical transformation on the input optical field based on their respective medium perturbation structures to obtain a first optical field;
[0063] The reconfigurable compensation network is used to adjust the first optical field based on node parameters that match the target task to obtain a second optical field. The reconfigurable compensation network includes an optical interference module with a reduced number of nodes.
[0064] The photoelectric detection module is used to convert the second light field into an electrical signal;
[0065] The control module is used to output inference results for the target task based on the electrical signal.
[0066] In one possible implementation, the multiple inverse-designed metaphotonic matrix processing units are connected in a cascaded, parallel, cross-connected, or block-connected manner.
[0067] Alternatively, the connection method may be a combination of two or more of the following connection methods: cascading, parallel connection, cross connection, and block connection.
[0068] In one possible implementation, the dielectric perturbation structure is one or a combination of the following structures: shallow etched aperture array, groove structure, ridge perturbation structure, subwavelength dielectric pixel, or non-periodic dielectric pattern.
[0069] This disclosure identifies a method comprising one or more inverse-designed metaphotonic matrix processing units to accurately implement the optical linear transformation function in an optical interferometer module. This not only separates the optical linear transformation function from the optical interferometer module, reducing the number of nodes and shrinking the module's area, but also reduces the area of the dielectric perturbation structure within the inverse-designed metaphotonic matrix processing unit compared to the nodes of the optical interferometer module, thus further reducing the area of the device implementing the optical linear transformation function. Consequently, the chip area is reduced. Furthermore, the reduction in nodes lowers thermal tuning power consumption, thermal crosstalk, and calibration complexity.
[0070] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description
[0071] The accompanying drawings, which are included in and form part of this specification, illustrate exemplary embodiments, features, and aspects of this disclosure together with the specification and serve to explain the principles of this disclosure.
[0072] Figure 1 This is a flowchart illustrating the method for determining the chip structure provided in an embodiment of this disclosure.
[0073] Figure 2 This is a schematic diagram of chip multitasking switching provided in an embodiment of this disclosure.
[0074] Figure 3 This is a performance illustration shown on the EMNIST dataset, provided as an embodiment of the present disclosure.
[0075] Figure 4 This is a schematic diagram comparing the shared node distribution patterns of hierarchical sharing and node-level sharing methods under different shared node ratios in the embodiments of this disclosure.
[0076] Figure 5 This is a schematic diagram of the chip structure provided in an embodiment of this disclosure.
[0077] Figure 6 This is another schematic diagram of a chip structure provided for an embodiment of this disclosure.
[0078] Figure 7 This is a schematic diagram of the structure of the chip structure determination device provided in the embodiments of this disclosure.
[0079] Figure 8This is a schematic diagram of the structure of an electronic device for determining chip structure, provided in an embodiment of this disclosure. Detailed Implementation
[0080] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.
[0081] As used herein, the terms “comprising,” “including,” “having,” or variations thereof are open-ended and include one or more of the stated features, integrals, elements, steps, components, or functions, but do not exclude the presence or addition of one or more other features, integrals, elements, steps, components, functions, or groups thereof.
[0082] When an element is referred to as “connected,” “coupled,” “responding,” or a variation thereof relative to another element, it may be directly connected, coupled, or responding to another element, or there may be an intermediate element present.
[0083] Although the terms first, second, third, etc., may be used herein to describe various elements / operations, these elements / operations should not be limited by these terms. These terms are only used to distinguish one element / operation from another. Therefore, without departing from the teachings of the inventive concept, a first element / operation in some embodiments may be referred to as a second element / operation in other embodiments.
[0084] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.
[0085] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.
[0086] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant regions.
[0087] Figure 1This is a flowchart illustrating a method for determining a chip structure according to an embodiment of this disclosure. The chip includes one or more inverse-design metaphotonic matrix processing units, wherein the multiple inverse-design metaphotonic matrix processing units are connected in a cascaded manner. The method includes:
[0088] S11, obtain the initial complex matrix operator corresponding to each of the one or more inverse design metaphotonic matrix processing units, the initial complex matrix operator being used to constrain amplitude and phase.
[0089] The inverse design metaphotonic matrix processing unit can perform linear optical transformations on the optical field (optical signal) based on complex matrix operators used to constrain amplitude and phase.
[0090] For ease of description, the complex matrix operator corresponding to the initial inverse design metaphotonic matrix processing unit is named the initial complex matrix operator. The initial complex matrix operator can be obtained either directly or according to the specific scenario.
[0091] S12, Obtain the inverse design objective function; the inverse design objective function includes at least: matrix error term and transmission efficiency constraint term.
[0092] In this embodiment, the objective function includes a matrix error term and a transmission efficiency constraint term. The matrix error term is used to update the initial complex matrix operator so that the optical transformation effect based on the initial complex matrix operator is close to or equivalent to the desired target optical transformation effect. The transmission efficiency constraint term can impose a scale constraint on the target complex amplitude, establishing a trade-off between the accuracy of complex matrix fitting and insertion loss, avoiding the acquisition of a structure with "very accurate amplitude and phase but extremely low transmission" that has no practical value.
[0093] In this way, when multiple inverse-design metaphotonic matrix processing units are connected in a cascaded manner, or in a manner that includes cascaded connections, the linear optical transformation process of each stage of the inverse-design metaphotonic matrix processing unit is knowable; the output optical field of each stage of the inverse-design metaphotonic matrix processing unit is also accurate and controllable. Therefore, one or more inverse-design metaphotonic matrix processing units can accurately realize the optical linear transformation function in the optical interference module, successfully separating this function from the optical interference module.
[0094] S13, based on each of the initial complex matrix operators, perform simulated optical linear transformation, and based on the inverse design objective function, determine the first loss between the optical signal transmission transmittance and the target transmission transmittance of the one or more inverse design metaphotonic matrix processing units.
[0095] In this embodiment of the disclosure, an electromagnetic simulation model can be used to simulate optical linear transformation based on an initial complex matrix operator, and a first loss can be determined based on an inverse design objective function.
[0096] For ease of understanding, formulas (1)-(3) are used to represent the inverse design objective function.
[0097] (1)
[0098] (2)
[0099] (3)
[0100] in, This represents the transmission efficiency constraint. The target complex transmission coefficient represents the fundamental mode response from the fundamental mode excitation at input port 1 to the fundamental mode response at output port 1. The simulated complex transfer coefficients represent the fundamental mode excitation from input port 1 to the fundamental mode response at output port 1. The target complex transmission coefficient represents the fundamental mode response from the fundamental mode excitation at input port 2 to the fundamental mode response at output port 1. The simulated complex transfer coefficients represent the fundamental mode excitation from input port 2 to the fundamental mode response at output port 1. and This represents the intensity of the reflected light propagating backward from input port 1 for two different fundamental mode excitations. and This represents the intensity of reflected light propagating backwards from input port 2 under two different fundamental mode excitations. L represents the first loss, which characterizes the total difference between the simulated complex transmission coefficients and the target secondary transmission coefficients from each input port to each output port. Indicates the first loss. This represents the second sub-loss. The simulated complex transmission coefficient corresponds to the aforementioned transmission transmittance, the target complex transmission coefficient corresponds to the aforementioned target transmission transmittance, and the fundamental mode excitation corresponds to the optical signal.
[0101] S14, based on the first loss, iteratively update each of the initial complex matrix operators until the stopping condition is met, and determine the target complex matrix operator corresponding to the one or more inverse design metaphotonic matrix processing units to obtain one or more inverse design metaphotonic matrix processing units with fixed geometry of the medium perturbation structure.
[0102] The metaphoton matrix processing unit contains a dielectric perturbation structure. This perturbation structure can be formed according to a continuous dielectric constant distribution. A complex matrix operator can control this dielectric constant distribution. That is, the geometry of the dielectric perturbation structure can be defined by a complex matrix operator. A complex matrix operator can correspond to a metaphoton matrix processing unit. A single complex matrix operator can correspond to one or more metaphoton matrix processing units. The design region containing the dielectric perturbation structure can be discretized into multiple tunable dielectric pixels. Continuous interpolation between the shallowly etched and unetched regions of each dielectric pixel yields a linear system of equations satisfying the frequency-domain Maxwell's equations for the electromagnetic field. The continuous variables of this linear system can be gradually converged into a fabricable approximate binary graph using a projection function, thus obtaining the dielectric perturbation structure.
[0103] During training, the complex matrix operator can be iteratively adjusted multiple times to determine the target complex matrix operator. Using the target matrix operator as the objective, a dielectric perturbation structure is fabricated, enabling the metaphotonic matrix processing unit to achieve the same or approximately the same optical signal processing performance as the target matrix operator. The dielectric perturbation structure within the on-chip metaphotonic matrix processing unit is fixed; that is, once fabricated, it is physically unadjustable.
[0104] In addition, the area of the dielectric perturbation structure is smaller than that of the nodes of the optical interference module, which also reduces the area of the metaphotonic matrix processing unit.
[0105] In this embodiment, iteratively updating the initial complex matrix operator based on a first loss can be achieved by iteratively updating the geometry of the medium perturbation structure in one or more inverse-designed metaphotonic matrix processing units. The stopping condition can be defined according to actual needs; for example, the stopping condition can be that the first loss is less than a loss threshold. The resulting one or more inverse-designed metaphotonic matrix processing units can accurately realize complex, multi-level linear optical transformations. The final obtained medium perturbation structure is not an arbitrary structure, but rather a structure defined by the complex matrix operator and the constraints of the loss function. The medium perturbation structure can be a fabricable nanopattern.
[0106] This disclosure identifies a method comprising one or more inverse-designed meta-processing units (MPUs) to accurately implement the optical linear transformation function within an optical interferometer module. This not only separates the optical linear transformation function from the optical interferometer module, reducing the number of nodes and shrinking the module's area, but also reduces the area of the dielectric perturbation structure within the inverse-designed meta-processing unit compared to the nodes of the optical interferometer module, thus further reducing the area of the device implementing the optical linear transformation function. Consequently, the chip area is reduced. Furthermore, the reduction in nodes lowers thermal tuning power consumption, thermal crosstalk, and calibration complexity.
[0107] In one possible implementation, obtaining the initial complex matrix operator corresponding to each of the one or more inverse-designed metaphotonic matrix processing units includes: uniformly dividing the relative phase value interval and the beam splitting ratio value interval with the same number of sub-intervals to obtain the same number of relative phase values and beam splitting ratio values; pairing the relative phase values and the beam splitting ratio values to obtain multiple relative phase-beam splitting ratio combinations to determine multiple candidate complex matrix operators; adding the multiple candidate complex matrix operators to a device library; selecting candidate complex matrix operators from the device library as the initial complex matrix operators; and iteratively updating each of the initial complex matrix operators based on the first loss includes: selecting candidate complex matrix operators from the device library to replace the initial complex matrix operators based on the first loss to update the initial complex matrix operators.
[0108] In this embodiment, the relative phase value interval and the beam ratio value interval can be preset value intervals. The relative phase value interval and the beam ratio value interval can be uniformly divided according to a preset number of sub-intervals, resulting in multiple first division points in the relative phase value interval and multiple second division points in the beam ratio value interval. The first division point corresponds to the relative phase value in the phase value interval. The second division point corresponds to the beam ratio value in the beam ratio value interval. This yields multiple uniformly distributed relative phase values and multiple uniformly distributed beam ratio values, with the same number of relative phase values and beam ratio values. Combining the relative phase values and beam ratio values yields multiple relative phase-beam ratios. Each relative phase-beam ratio combination contains one relative phase value and one beam ratio value. A complex matrix operator can then be determined based on this relative phase-beam ratio combination. This determination process is a common physical calculation process and is not the focus of this disclosure, therefore it will not be described in detail. For ease of description, the complex matrix operator determined based on the relative phase-beam ratio combination can be named a candidate complex matrix operator.
[0109] For example, using 3-bit quantization, eight relative phase values and eight beam splitting ratio values can be obtained, resulting in 64 relative phase-beam splitting ratio combinations, which in turn yields 64 candidate complex matrix operators. This is merely an example, and this disclosure does not limit the number of candidate complex matrix operators.
[0110] In this embodiment of the disclosure, candidate complex matrix operators can be added to the device library. During the aforementioned iterative update process based on the inverse design objective function, candidate complex matrices can be obtained from the device library as initial complex matrix operators, and candidate complex matrices can also be obtained from the device library as initial complex matrix operators updated during the iterative update process.
[0111] In this embodiment, since the phase and beam splitting ratio values are uniformly and equally distributed across the phase and beam splitting ratio ranges, the phase and amplitude of the candidate complex matrix operators in the device library can uniformly cover their respective value ranges. This enriches the candidate complex matrix operators in the device library and satisfies the design requirements of various dielectric perturbation structures, improving the rationality and applicability of the dielectric perturbation structures and accelerating the aforementioned update iterations.
[0112] In one possible implementation, the chip includes a reconfigurable compensation network connected to the inverse-design metaphotonic matrix processing unit. The in-loop verification and deployment method of the chip includes: inputting a calibration optical signal into the chip to obtain a first output signal from the chip; calibrating chip parameters based on the difference between the first output signal and the calibration optical signal, the chip parameters including: input parameters, routing parameters, attenuation parameters, and network parameters in the reconfigurable compensation network, the network parameters including node parameters; training the reconfigurable compensation network for a sample task to obtain node parameters corresponding to the sample task; the training includes: inputting a sample optical signal into the chip based on the sample task to obtain a second output signal from the chip representing the task result; comparing the second output signal with a target output signal to obtain a task loss; updating the node parameters of each node in the reconfigurable compensation network according to the task loss; iteratively executing the steps from inputting the sample optical signal to updating the node parameters until the task loss meets a preset condition.
[0113] The reconfigurable compensation network can be connected to the inverse design metaphotonic matrix processing unit. The reconfigurable compensation network is used to adjust the optical field output by the inverse design metaphotonic matrix processing unit to obtain the optical field representing the inference result.
[0114] In this embodiment, chip parameters can be calibrated based on calibration optical signals. Chip parameters include input parameters, routing parameters, attenuation parameters, and network parameters in the reconfigurable compensation network. Then, based on the sample optical signal corresponding to the sample task, the node parameters in the network parameters are updated to complete the training of the reconfigurable compensation network and obtain node parameters capable of processing the sample task. This two-stage deployment reduces the impact of chip parameter deviations on the efficiency and accuracy of node parameter updates. It not only improves the efficiency of determining node parameters but also ensures that node parameters accurately match the processing of sample tasks, increasing the probability that node parameters accurately handle the sample tasks.
[0115] During the chip parameter calibration stage, a calibration optical signal can be input into the chip to obtain the first output signal. Based on the difference between the first output signal and the calibration optical signal, the chip parameters are calibrated.
[0116] During the chip parameter calibration stage, the phase shifters in the reconfigurable compensation network can be scanned without connecting the reverse-designed metaphotonic matrix processing unit to the reconfigurable compensation network, yielding multiple driving voltages for the phase shifters. Based on these multiple driving voltages, a periodic first current curve and first parameters are fitted. The first parameters may include the voltage-phase mapping relationship and the half-wave voltage.
[0117] Then, the inverse-designed metaphotonic matrix processing unit is connected after the reconfigurable compensation network and placed in the chip circuit; the fundamental mode optical field is sequentially input from each input port of the chip, and multiple currents of each output port are collected after each input to obtain the second current curve of each output port corresponding to each input, as well as the second parameter, which may include DC current, modulation degree and relative phase.
[0118] Based on the first current curve, the second current curve, the first parameter, and the second parameter, the fitting difference between the reconfigurable compensation network and the inverse-designed metaphotonic matrix processing unit before and after cascading is determined. Based on this fitting difference, the complex matrix operator corresponding to the inverse-designed metaphotonic matrix processing unit is updated. This calibrates the complex matrix operator, improving the accuracy of the inverse-designed metaphotonic matrix processing unit in practical applications and its cascading with the reconfigurable compensation network, further enhancing deployment efficiency and accuracy. For example, the complex matrix operator can be calibrated after chip parameter calibration and before node parameter updates.
[0119] In this embodiment, a sample optical signal can be input into the chip to obtain a second output signal characterizing the task result. The second output signal is an optical signal. The first difference between the second output signal and the target output signal can characterize the first difference between the task result output by the reconfigurable compensation network and the sample label (true value). Based on this first difference, the task loss can be determined, and the node parameters can be updated based on the task loss. The node parameter update stops when the task loss meets preset conditions such as being less than a loss threshold or the task loss no longer decreasing after two consecutive attempts. The obtained node parameters are used as the node parameters matched with the sample task.
[0120] Using the deployment method in the embodiments of this disclosure, the convergence of reconfigurable compensation networks can be accelerated, and deployment efficiency and the accuracy of chip processing of sample tasks can be improved.
[0121] In one possible implementation, the step of inputting a sample optical signal into the chip based on a sample task to obtain a second output signal representing the task result from the chip includes: acquiring a perturbation vector; incrementally adjusting the current node parameters based on the perturbation vector to obtain a first fine-tuning parameter; processing the sample optical signal based on the first fine-tuning parameter to obtain a first fine-tuned output signal; decrementing the current node parameters based on the perturbation vector to obtain a second fine-tuning parameter; and processing the sample optical signal based on the second fine-tuning parameter to obtain a second fine-tuned output signal. The step of comparing the second output signal with a target output signal to obtain a task loss includes: comparing the first fine-tuned output signal with the target output signal to obtain a first sub-loss; comparing the second fine-tuned output signal with the target output signal to obtain a second sub-loss; and obtaining the task loss based on the first sub-loss and the second sub-loss.
[0122] Due to deviations in chip manufacturing and design, problems such as nonlinear arrangement of optoelectronic modulators, and / or coupling imbalance, and differences in photodetector response can occur, making it difficult to accurately match the sample task with task loss modeling during the deployment phase. The reconfigurable compensation network includes a thermally tuned phase shifter. In this embodiment, a perturbation vector can be randomly obtained. In each round of node parameter updates, the perturbation vector is added to the current voltage of the thermally tuned phase shifter to obtain a first fine-tuning parameter, i.e., the incremented voltage. The sample optical signal is processed based on the incremented voltage to obtain an incremented output signal. Based on the second difference between the incremented output signal and the target output signal, a first sub-loss is obtained.
[0123] Furthermore, in each round of node parameter updates, the perturbation vector of the current voltage of the thermally tuned phase shifter is reduced to obtain the second fine-tuning parameter, i.e., the reduced voltage. Based on the reduced voltage, the sample optical signal is processed to obtain the reduced output signal. Based on the third difference between the reduced output signal and the target output signal, the second sub-loss is obtained.
[0124] In this embodiment of the disclosure, the task loss can be determined based on the first sub-loss and the second sub-loss. For example, the average of the first sub-loss and the second sub-loss can be used as the task loss.
[0125] Adjusting node parameters using the task loss in this embodiment of the disclosure makes the reconfigurable compensation network more tolerant of manufacturing and design deviations, thus improving its robustness. Furthermore, since only two signal measurements (hardware measurements) are performed per training round, the number of measurements does not increase linearly with the dimension of the sample signal, making it ideal for closed-loop training of hardware-constrained photonic chips.
[0126] In one possible implementation, updating the node parameters of each node in the reconfigurable compensation network based on the task loss includes: determining the loss gradient of each node based on the task loss; determining a first node whose loss gradient is greater than a gradient threshold, and a second node whose loss gradient is not greater than a gradient threshold, wherein the second node does not update its node parameters but performs forward propagation; and updating the node parameters of the first node based on the loss gradient of the first node.
[0127] In each round of node updates, the loss gradient for each node can be determined based on gradient algorithms or other algorithms, such as the adjoint algorithm. This loss gradient characterizes the node's sensitivity to the sample task. A gradient threshold can be pre-set. The gradient threshold characterizes the minimum requirement for this sensitivity to handle the sample task. Nodes with loss gradients greater than the gradient threshold can be designated as the first node, and nodes with loss gradients no greater than the gradient threshold can be designated as the second node. Only the node parameters of the first node are updated, not the node parameters of the second node, but the second node participates in forward propagation. For example, the loss gradient corresponding to the second node can be set to 0. The proportion of the second node among all nodes can be 10% to 90%, preferably 60% to 90%, which can improve deployment efficiency and enable the parameters of each node to be accurately inferred for the sample task.
[0128] This disclosure provides a node-level selective parameter update method that updates only the first node, which is highly sensitive to the sample task, reducing invalid updates, improving update efficiency, and reducing energy consumption. Furthermore, it eliminates the need to update the parameters of the second node, reducing uncontrollable factors that may arise during the second node update, minimizing interference with the first node, and improving deployment efficiency and accuracy.
[0129] In one possible implementation, the multi-task adaptive configuration method of the chip includes: training the reconfigurable compensation network contained in the chip in multiple rounds for each type of sample task to obtain a set of gradient vectors corresponding to each type of sample task; the set of gradient vectors includes the gradient vectors of each node for a single type of sample task, and the gradient vectors represent the loss gradients obtained by the same node in each round of training for a single type of sample task; based on each set of gradient vectors, determining shared nodes with similar gradient vectors for each type of sample task. It should be noted that the node parameters of the shared nodes can be updated during training.
[0130] In this embodiment, the same chip can perform inference for different types of sample tasks. Based on task type, sample tasks can be: speech recognition, image recognition, character recognition, sensor classification, multimodal fusion, etc. For each type of sample task, the reconfigurable compensation network in the chip, initially set, can be trained multiple times. Each training round yields the loss gradient of each node corresponding to the current sample task. Therefore, after training the reconfigurable compensation network for a single sample task, multiple loss gradients for each node can be obtained. Based on these multiple loss gradients, the loss gradient vector for each node can be determined. For example, multiple loss gradients of the same node for the same sample task can be encoded to obtain a loss gradient vector corresponding to the sample task and the node. Alternatively, multiple loss gradients of the same node for the same sample task can be normalized to obtain a normalized loss gradient, which is then concatenated to obtain a loss gradient vector corresponding to the sample task and the node. Thus, each node can correspond to at least one loss gradient vector for each sample task. The summation of the loss gradient vectors of each node corresponding to each type of sample task yields the set of gradient vectors corresponding to each type of sample task.
[0131] In this embodiment of the disclosure, a first similarity can be determined between each pair of gradient vectors of the same node in each gradient vector set. If all first similarities of the same node are greater than a first similarity threshold, the same node is determined to be a shared node. Alternatively, if the mean squared error of the first similarities of the same node is less than a second similarity threshold, the same node is determined to be a shared node. The similarity threshold can be set according to the proportion of shared nodes.
[0132] In this embodiment of the disclosure, either the node parameters corresponding to the shared node and the sample task can be used as the shared node parameter of the shared node. Alternatively, the average value of the node parameters corresponding to the shared node and the sample task can be used as the shared node parameter of the shared node. The above are merely examples, and this embodiment of the disclosure does not limit the method for determining the shared node parameter of the shared node.
[0133] In this embodiment, the loss gradient vector can comprehensively and accurately characterize the sensitivity of each node to the sample task throughout the training process. Nodes with similar sensitivities (gradient vectors) for different types of sample tasks are designated as shared nodes, improving the rationality and accuracy of the determination of shared nodes.
[0134] In one possible implementation, the node includes: the shared node and the private node, and the node parameters include: shared node parameters and private node parameters; the method further includes: when the current target task and the previous target task are different types of tasks, obtaining the private node parameters corresponding to the current target task; updating the private node parameters corresponding to the current target task to the private node, without updating the shared node parameters of the shared node.
[0135] In this embodiment, based on the identifier of the target task, it is determined whether the current target task is the same as or different from the previous target task. If they are different, it indicates that a target task switch is required. When switching between different types of target tasks, only the private node parameters corresponding to the private node need to be updated, while the shared node parameters corresponding to the shared node are maintained. This improves the efficiency of target task switching while accurately performing inference for various types of target tasks. Furthermore, it allows the single chip in this disclosure to adaptively and dynamically adjust node parameters based on the target task to handle different types of target tasks, improving the chip's multi-dimensional reusability.
[0136] Figure 2 This is a schematic diagram of chip multitasking switching provided in an embodiment of this disclosure. Figure 2 The task-aware neurons in this disclosure share a photonic neural network, which is set within the chip. This chip can perform inference for three types of recognition tasks: digit recognition (Task 1), uppercase letter recognition (Task 2), and lowercase letter recognition (Task 3). The diagram schematically illustrates the nodes of the neural network, which correspond to the nodes of the reconfigurable compensation network in the chip. Unlabeled nodes represent shared nodes, and nodes marked with "×" represent private nodes. The parameters of the shared nodes are the same for all three tasks, while the parameters of the private nodes for each task can differ. The same chip can be used to perform inference for digit recognition, uppercase letter recognition, and lowercase letter recognition tasks.
[0137] In another example, the chip in this disclosure can perform dual-task four-class vowel recognition. For ease of description, the neural network that performs dual-task four-class vowel recognition is named the first neural network. The dual tasks include task A and task B. The nodes of the first neural network can correspond to the nodes of the reconfigurable compensation network in the chip. Each training sample contains n-dimensional acoustic features, which are first compressed into an m-dimensional first vector by the data preprocessing layer of the first neural network. Here, m and n are both positive integers, and m is less than n. The first vector is then mapped to the driving channels of multiple Mach-Zehnder modulators in the chip. One or more inverse-designed metaphotonic matrix processing units and the reconfigurable compensation network process the first vector to obtain photocurrent. After digital calibration of the photocurrent and processing by the task readout head, the vowel classification task results for the two tasks are obtained. The test accuracy for task A is 83.5%, and the test accuracy for task B is 80.9%. This result shows that the inverse-designed metaphotonic matrix processing units and the reconfigurable compensation network can not only be used for static optical interconnects, but also together with other components in the chip to form a trainable multi-task photonic computing system.
[0138] Figure 3 This is a performance illustration shown on the EMNIST dataset, provided as an embodiment of the present disclosure.
[0139] exist Figure 3 In the diagram, the top left figure shows the average test accuracy under different shared node ratios. The top right figure shows the training process after warm-up training with a 90% node-level sharing mode applied and fine-tuned. The bottom figure shows the confusion matrix for different tasks (digit recognition, uppercase letter recognition, and lowercase letter recognition) under the 90% shared node condition. This embodiment demonstrates that the node-level parameter sharing task adaptation method of this disclosure can improve the photonic hardware reuse efficiency while reducing the active control overhead in multi-task photonic neural networks.
[0140] Figure 4 This is a schematic diagram comparing the shared node distribution patterns of hierarchical sharing and node-level sharing methods under different shared node ratios in the embodiments of this disclosure.
[0141] exist Figure 4 In the diagram, light colors represent shared nodes, and dark colors represent private nodes. Compared to hierarchical sharing, node-level sharing does not simply replace nodes continuously according to network layers or cascade stages, but rather selectively retains task-sensitive nodes throughout the entire photonic network; thus, with the same proportion of shared nodes, node-level sharing can preserve more efficient task-adaptive degrees of freedom.
[0142] In one possible implementation, the chip includes: an input modulation module, one or more inverse-design metaphotonic matrix processing units, a reconfigurable compensation network, a photodetector module, and a control module; the input modulation module is used to encode the data to be processed to obtain an input light field; the inverse-design metaphotonic matrix processing unit includes a dielectric perturbation structure; the one or more inverse-design metaphotonic matrix processing units are used to perform a linear optical transformation on the input light field based on their respective dielectric perturbation structures to obtain a first light field; the reconfigurable compensation network is used to adjust the first light field based on node parameters matched with the target task to obtain a second light field, the reconfigurable compensation network including an optical interference module with a reduced number of nodes; the photodetector module is used to convert the second light field into an electrical signal; the control module is used to output an inference result for the target task based on the electrical signal.
[0143] Figure 5 This is a schematic diagram of the chip structure provided in an embodiment of this disclosure. Data to be processed is input to the input modulation module in the chip. The data to be processed can be selected according to the type of the target task. The data to be processed can be audio data, image data, or multimodal sensor signal data, etc. One or more inverse-design metaphotonic matrix processing units are connected to the input modulation module as an integral component to process the input light field output by the input modulation module. A reconfigurable compensation network is connected to this integral component, so that the integral component can perform optical linear transformation functions to reduce the area occupied by the reconfigurable compensation network. A photodetector module can be connected to the reconfigurable compensation network to convert the second light field output by the reconfigurable compensation network into an electrical signal. A control module is connected to the input modulation module, the integral component (one or more inverse-design metaphotonic matrix processing units), the reconfigurable compensation network, and the photodetector module respectively to control each module in the chip. Furthermore, the control module can output inference results for the target task based on the electrical signals.
[0144] In this embodiment of the disclosure, the inverse-design metaphotonic matrix processing unit includes a dielectric perturbation structure. The dielectric perturbation structure can perform adaptive linear optical transformations of the input optical field for the target task.
[0145] In one possible implementation, the dielectric perturbation structure is one or a combination of the following structures: shallow etched aperture array, groove structure, ridge perturbation structure, subwavelength dielectric pixel, or non-periodic dielectric pattern. This enriches the possibilities for optical transformations to adapt to multi-task switching scenarios and improve chip reusability. Furthermore, using shallow etched structures, such as shallow etching, can reduce backscattering, unwanted domain resonance, and cascade loss accumulation.
[0146] In this embodiment, the units in the multiple connected inverse-design metaphotonic matrix processing units are connected in a cascaded, parallel, cross-connected, or block-connected manner; or, the connection method is a combination of two or more of the following connection methods: cascaded, parallel, cross-connected, and block-connected. In this way, even when facing complex optical transformations, such as complex matrix combinations, accurate linear optical transformations can still be achieved, truly separating linear optical transformations from traditional optical interference modules while maintaining the accuracy of this function.
[0147] When the units in a multi-connected inverse-design metaphotonic matrix processing unit are cascaded, these multi-connected inverse-design metaphotonic matrix processing units constitute a cascaded structure. This cascaded structure may include an inter-level power equalization module to eliminate channel power imbalances between the cascaded units.
[0148] In one example, the cascaded structure can support input light fields of different wavelengths, and / or different spatial channels, and / or different matrix sub-blocks, to support chip multiplexing in the wavelength dimension, and / or spatial channel dimension, and / or time dimension, thereby improving chip throughput and multiplexing efficiency.
[0149] In one example, the reconfigurable compensation network may include: interferometer units, phase shifters, variable optical attenuators, micro-ring modulators, thermo-optic modulators, electro-optic modulators, carrier modulators, and phase change material modulation units. The variable optical attenuator can be used to adjust the optical power distribution of different channels to compensate for manufacturing errors, insertion loss differences, response deviations caused by frozen nodes (nodes whose parameters are not updated during deployment), and the output intensity requirements of different tasks.
[0150] This disclosure discloses a chip incorporating one or more inverse-design metaphotonic matrix processing units capable of performing optical linear transformations via a dielectric perturbation structure. By separating the optical linear transformation function from the optical interferometry module and performing it using an inverse-design metaphotonic matrix processing unit with a much smaller area than the optical interferometry module, the number of nodes in the optical interferometry module can be effectively reduced, thus shrinking the area of the optical interferometry module and consequently the chip area. Furthermore, the reduction in nodes lowers thermal tuning power consumption, thermal crosstalk, and calibration complexity.
[0151] Figure 6 This is another schematic diagram of a chip structure provided for an embodiment of this disclosure.
[0152] In this example, the input modulation module includes a beam splitter and a modulator. The modulator may include one or a combination of at least two of the following devices: an intensity modulator, a phase modulator, a Mach-Zehnder modulator array, a polarization modulator, and a wavelength selector. The meta-processing unit contains one or more inverse-designed meta-photonic matrix processing units (MPUs). The matrix operator photonic device library contains multiple complex matrix operators. The MPUs of the meta-processing unit are interconnected, and the connection may include cascading. The MPU contains a dielectric perturbation structure. The dielectric perturbation structure in each MPU of the meta-processing unit can be determined based on the complex matrix operators in the matrix operator photonic device library. The input modulation module may include a Mach-Zehnder modulator.
[0153] In this example, the chip includes a reconfigurable compensation network, which comprises multiple Mach-Zehnder interferometers. For ease of description, the reconfigurable compensation network in this example is referred to as a Mach-Zehnder interferometer network. The Mach-Zehnder interferometer network may also include other devices ( Figure 6 (Not shown in the image). Other devices may include one or more of the following: phase shifters, variable optical attenuators, micro-ring modulators, thermo-optic modulators, electro-optic modulators, carrier modulators, and phase change material modulation units. The chip also includes a photodetector (PD) array formed by multiple photodetectors. The chip also includes a control module (…). Figure 6 (Not shown in the image).
[0154] This disclosure also provides a method for fabricating a reverse-designed metaphotonic matrix processing chip. The method includes:
[0155] Obtain an integrated photonic substrate. The integrated photonic substrate may be a single material or at least two of the following materials: silicon-based insulator wafer, silicon nitride wafer, thin-film lithium niobate wafer, III-V group semiconductor wafer, or other photonic materials.
[0156] An input waveguide, an output waveguide, a basic dielectric perturbation structure for an inversely designed metaphotonic matrix processing unit, a reconfigurable compensation network, and a photoelectric detection region are formed on the substrate. The basic dielectric perturbation structure can be a blank region or a modifiable, general-purpose structure.
[0157] Waveguide patterns and dielectric perturbation structure patterns can be obtained using electron beam lithography, deep ultraviolet lithography, or nanoimprint lithography.
[0158] Shallow waveguide structures are obtained based on waveguide patterns using reactive ion etching, inductively coupled plasma etching, or other etching processes. Shallow dielectric perturbation structures are obtained based on dielectric perturbation structure patterns using reactive ion etching, inductively coupled plasma etching, or other etching processes.
[0159] An upper cladding material is deposited on a substrate with a shallow-etched waveguide structure and a shallow-etched dielectric perturbation structure. The upper cladding material can be one or a combination of at least two of the following materials: silicon dioxide, silicon nitride, polymer, or air cladding.
[0160] A metal electrode, thermal phase modulator, electro-optic modulator, variable optical attenuator, photodetector, and electrical interconnect structure are disposed on a substrate with an upper cladding layer to obtain an initial chip.
[0161] The initial chip is cut, end face processed, fiber coupled, packaged, and electrically led out to obtain the chip.
[0162] Figure 7 This is a schematic diagram of a chip structure determination device provided in an embodiment of the present disclosure. The chip includes one or more inverse design metaphotonic matrix processing units, and the connection method includes cascading. The device 20 includes:
[0163] The initial complex matrix operator acquisition module 21 is used to acquire the initial complex matrix operator corresponding to each of the one or more inverse design metaphotonic matrix processing units, wherein the initial complex matrix operator is used to constrain the amplitude and phase;
[0164] The inverse design objective function acquisition module 22 is used to acquire the inverse design objective function; the inverse design objective function includes at least: a matrix error term and a transmission efficiency constraint term.
[0165] The first loss determination module 23 is used to perform simulated optical linear transformation based on each of the initial complex matrix operators, and to determine the first loss between the optical signal transmission transmittance and the target transmission transmittance of the one or more inverse design metaphotonic matrix processing units based on the inverse design objective function.
[0166] The connected inverse design metaphotonic matrix processing unit determination module 24 is used to iteratively update each of the initial complex matrix operators based on the first loss until the stopping condition is met, and determine the target complex matrix operator corresponding to the one or more inverse design metaphotonic matrix processing units, so as to obtain one or more inverse design metaphotonic matrix processing units with fixed geometry of the medium perturbation structure.
[0167] In one possible implementation, the initial complex matrix operator acquisition module 21 is further configured to:
[0168] The relative phase value interval and the beam split ratio value interval are evenly divided according to the same number of sub-intervals to obtain the same number of relative phase values and beam split ratio values.
[0169] The relative phase value and the beam splitting ratio value are paired to obtain multiple relative phase-beam splitting ratio combinations, thereby determining multiple candidate complex matrix operators, and adding the multiple candidate complex matrix operators to the device library;
[0170] Candidate complex matrix operators are selected from the device library as the initial complex matrix operators;
[0171] The iterative update of each of the initial complex matrix operators based on the first loss includes:
[0172] Based on the first loss, a candidate complex matrix operator is selected from the device library to replace the initial complex matrix operator, thereby updating the initial complex matrix operator.
[0173] In one possible implementation, the chip includes a reconfigurable compensation network connected to the inverse-design metaphotonic matrix processing unit, and the device 20 includes an in-loop verification and deployment module;
[0174] The in-loop verification and deployment module is used for:
[0175] The calibration optical signal is input into the chip to obtain the first output signal output by the chip;
[0176] Based on the difference between the first output signal and the calibration optical signal, the chip parameters of the chip are calibrated. The chip parameters include: input parameters, routing parameters, attenuation parameters, and network parameters in the reconfigurable compensation network, wherein the network parameters include node parameters.
[0177] For a given sample task, the reconfigurable compensation network is trained to obtain the node parameters corresponding to the sample task.
[0178] The training includes:
[0179] Based on the sample task, the sample optical signal is input into the chip to obtain a second output signal representing the task result output by the chip;
[0180] The second output signal is compared with the target output signal to obtain the task loss;
[0181] Based on the task loss, update the node parameters of each node in the reconfigurable compensation network;
[0182] Iteratively execute the steps from input sample light signal to update node parameters until the task loss meets the preset conditions.
[0183] In one possible implementation, the in-loop verification and deployment module is further configured to:
[0184] Obtain the perturbation vector;
[0185] Based on the perturbation vector, the parameters of the current node are incrementally adjusted to obtain the first fine-tuning parameter;
[0186] Based on the first fine-tuning parameter, the sample light signal is processed to obtain the first fine-tuning output signal;
[0187] Based on the perturbation vector, the parameters of the current node are adjusted by a reduction to obtain the second fine-tuning parameter;
[0188] Based on the second fine-tuning parameter, the sample light signal is processed to obtain the second fine-tuning output signal;
[0189] The step of comparing the second output signal with the target output signal to obtain the task loss includes:
[0190] The first fine-tuned output signal is compared with the target output signal to obtain the first sub-loss.
[0191] The second fine-tuned output signal is compared with the target output signal to obtain the second sub-loss.
[0192] The task loss is obtained based on the first sub-loss and the second sub-loss.
[0193] In one possible implementation, the in-loop verification and deployment module is further configured to:
[0194] Based on the task loss, determine the loss gradient for each node;
[0195] The first node whose loss gradient is greater than the gradient threshold is identified, and the second node whose loss gradient is not greater than the gradient threshold is identified. The second node does not update its node parameters but propagates forward.
[0196] Update the node parameters of the first node based on the loss gradient of the first node.
[0197] In one possible implementation, the device 20 further includes: a multi-task adaptive configuration module;
[0198] The multi-task adaptive configuration module is used for:
[0199] The in-loop verification and deployment module is controlled to perform multiple rounds of training on the reconfigurable compensation network contained in the chip for each type of sample task, so as to obtain a set of gradient vectors corresponding to each type of sample task.
[0200] The multi-task adaptive configuration module is also used for:
[0201] The gradient vector set contains the gradient vectors of each node for a single-type sample task. The gradient vectors represent the loss gradients obtained by the same node in each round of training for a single-type sample task.
[0202] Based on the aforementioned gradient vector sets, shared nodes with similar gradient vectors are identified for each type of sample task.
[0203] In one possible implementation, the node includes: the shared node and the private node, and the node parameters include: shared node parameters and private node parameters;
[0204] The device 20 further includes:
[0205] The private node parameter acquisition module is used to acquire the private node parameters corresponding to the current target task when the current target task and the previous target task are different types of tasks.
[0206] The private node parameter update module is used to update the private node parameters corresponding to the current target task to the private node, without updating the shared node parameters of the shared node.
[0207] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.
[0208] This disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above method.
[0209] This disclosure also provides a non-volatile computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the above-described method.
[0210] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method.
[0211] Figure 8 This is a schematic diagram of an electronic device for determining chip structure, provided as an embodiment of the present disclosure. For example, the electronic device 1900 can be provided as a server or a terminal device. (Refer to...) Figure 8The electronic device 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by the processing component 1922. The application programs stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1922 is configured to execute instructions to perform the methods described above.
[0212] Electronic device 1900 may also include a power supply component 1926 configured to perform power management of electronic device 1900, a wired or wireless network interface 1950 configured to connect electronic device 1900 to a network, and an input / output interface 1958 (I / O interface). Electronic device 1900 can operate on an operating system, such as Windows Server, stored in memory 1932. TM Mac OS X TM Unix TM Linux TM FreeBSD TM Or similar.
[0213] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by a processing component 1922 of an electronic device 1900 to perform the above-described method.
[0214] Computer-readable storage media can be tangible devices capable of holding and storing programs / instructions used by instruction execution devices. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.
[0215] The computer program (or computer-readable program instructions) described herein can be downloaded from a computer-readable storage medium to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage medium in the respective computing / processing device.
[0216] The computer program (or computer program instructions) used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions to implement various aspects of this disclosure.
[0217] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.
[0218] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.
[0219] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.
[0220] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.
[0221] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.
Claims
1. A method for determining a chip structure, characterized in that, The chip includes one or more inverse-design metaphotonic matrix processing units, wherein the multiple inverse-design metaphotonic matrix processing units are connected in a cascaded manner, the method comprising: Obtain the initial complex matrix operator corresponding to each of the one or more inverse design metaphotonic matrix processing units, the initial complex matrix operator being used to constrain amplitude and phase; Obtain the inverse design objective function; the inverse design objective function includes at least: a matrix error term and a transmission efficiency constraint term; Based on each of the initial complex matrix operators, a simulated optical linear transformation is performed, and based on the inverse design objective function, the first loss between the optical signal transmission transmittance and the target transmission transmittance of the one or more inverse design metaphotonic matrix processing units is determined. Based on the first loss, each of the initial complex matrix operators is iteratively updated until the stopping condition is met, and the target complex matrix operator corresponding to the one or more inverse design metaphotonic matrix processing units is determined to obtain one or more inverse design metaphotonic matrix processing units with fixed geometry of the medium perturbation structure.
2. The method according to claim 1, characterized in that, The step of obtaining the initial complex matrix operator corresponding to each of the one or more inverse-designed metaphotonic matrix processing units includes: The relative phase value interval and the beam split ratio value interval are evenly divided according to the same number of sub-intervals to obtain the same number of relative phase values and beam split ratio values. The relative phase value and the beam splitting ratio value are paired to obtain multiple relative phase-beam splitting ratio combinations, thereby determining multiple candidate complex matrix operators, and adding the multiple candidate complex matrix operators to the device library; Candidate complex matrix operators are selected from the device library as the initial complex matrix operators; The iterative update of each of the initial complex matrix operators based on the first loss includes: Based on the first loss, candidate complex matrix operators are selected from the device library to replace the initial complex matrix operator, thereby updating the initial complex matrix operator.
3. The method according to claim 1 or 2, characterized in that, The chip includes a reconfigurable compensation network connected to the inverse-designed metaphotonic matrix processing unit, and the on-loop verification and deployment method of the chip includes: The calibration optical signal is input into the chip to obtain the first output signal output by the chip; Based on the difference between the first output signal and the calibration optical signal, the chip parameters of the chip are calibrated. The chip parameters include: input parameters, routing parameters, attenuation parameters, and network parameters in the reconfigurable compensation network, wherein the network parameters include node parameters. For a given sample task, the reconfigurable compensation network is trained to obtain the node parameters corresponding to the sample task. The training includes: Based on the sample task, the sample optical signal is input into the chip to obtain a second output signal representing the task result output by the chip; The second output signal is compared with the target output signal to obtain the task loss; Based on the task loss, update the node parameters of each node in the reconfigurable compensation network; Iteratively execute the steps from input sample light signal to update node parameters until the task loss meets the preset conditions.
4. The method according to claim 3, characterized in that, The process of inputting sample optical signals into the chip to obtain a second output signal representing the task result from the chip, based on the sample task, includes: Obtain the perturbation vector; Based on the perturbation vector, the parameters of the current node are incrementally adjusted to obtain the first fine-tuning parameter; Based on the first fine-tuning parameter, the sample light signal is processed to obtain the first fine-tuning output signal; Based on the perturbation vector, the parameters of the current node are adjusted by a reduction to obtain the second fine-tuning parameter; Based on the second fine-tuning parameter, the sample light signal is processed to obtain the second fine-tuning output signal; The step of comparing the second output signal with the target output signal to obtain the task loss includes: The first fine-tuned output signal is compared with the target output signal to obtain the first sub-loss. The second fine-tuned output signal is compared with the target output signal to obtain the second sub-loss. The task loss is obtained based on the first sub-loss and the second sub-loss.
5. The method according to claim 3, characterized in that, The step of updating the node parameters of each node in the reconfigurable compensation network based on the task loss includes: Based on the task loss, determine the loss gradient for each node; The first node whose loss gradient is greater than the gradient threshold is identified, and the second node whose loss gradient is not greater than the gradient threshold is identified. The second node does not update its node parameters but propagates forward. Update the node parameters of the first node based on the loss gradient of the first node.
6. The method according to claim 3, characterized in that, The multi-task adaptive configuration method for the chip includes: For each type of sample task, the reconfigurable compensation network contained in the chip is trained in multiple rounds to obtain a set of gradient vectors corresponding to each type of sample task. The gradient vector set contains the gradient vectors of each node for a single-type sample task. The gradient vectors represent the loss gradients obtained by the same node in each round of training for a single-type sample task. Based on the aforementioned gradient vector sets, shared nodes with similar gradient vectors are identified for each type of sample task.
7. The method according to claim 6, characterized in that, The nodes include: shared nodes and private nodes, and the node parameters include: shared node parameters and private node parameters; The method further includes: If the current target task is a different type from the previous target task, obtain the private node parameters corresponding to the current target task; Update the private node parameters corresponding to the current target task to the private node, but do not update the shared node parameters of the shared node.
8. A reverse-designed metaphotonic matrix processing chip, characterized in that, The chip includes: an input modulation module, one or more inverse-designed metaphotonic matrix processing units, a reconfigurable compensation network, a photodetector module, and a control module; The input modulation module is used to encode the data to be processed to obtain the input optical field; The inverse design metaphotonic matrix processing unit includes a medium perturbation structure; the one or more inverse design metaphotonic matrix processing units are used to perform a linear optical transformation on the input optical field based on their respective medium perturbation structures to obtain a first optical field; The reconfigurable compensation network is used to adjust the first optical field based on node parameters that match the target task to obtain a second optical field. The reconfigurable compensation network includes an optical interference module with a reduced number of nodes. The photoelectric detection module is used to convert the second light field into an electrical signal; The control module is used to output inference results for the target task based on the electrical signal.
9. The chip according to claim 8, characterized in that, The multiple inverse-designed metaphotonic matrix processing units can be connected in a cascaded, parallel, cross-connected, or block-connected manner. Alternatively, the connection method may be a combination of two or more of the following connection methods: cascading, parallel connection, cross connection, and block connection.
10. The chip according to claim 8 or 9, characterized in that, The dielectric perturbation structure is one or a combination of the following structures: shallow etched aperture array, groove structure, ridge perturbation structure, subwavelength dielectric pixel, and non-periodic dielectric pattern.
