Electro-optical mapping system and method for electronic intelligent model

Abstract

The invention provides an electronic intelligent model-oriented electro-optical mapping system and method, and the system comprises a semantic feature extraction module which is used for carrying out the semantic feature coding of an input image, text or multi-modal data, and outputting a semantic feature representation representing the calculation capability of an electronic intelligent model; the optical parameter generation module is used for generating optical calculation parameters for driving the optical calculation module according to the semantic feature representation under the condition of introducing a physical constraint condition of the target optical calculation module; and the optical calculation module is used for executing optical calculation based on the optical calculation parameters so as to reproduce the corresponding calculation capability of the electronic intelligent model in a function or calculation behavior level and realize mapping of the electronic intelligent model to the optical calculation system. According to the application, the efficient mapping and reliable deployment of the electronic intelligent model to the optical computing system can be realized without directly mapping the complete digital weight of the electronic model.

Description

Technical Field

[0001] This application relates to the fields of optical computing, electronic computing, and artificial intelligence, and specifically to an electro-optical mapping system and method for electronic intelligent models. Background Technology

[0002] In recent years, deep learning-based electronic intelligent models have made significant progress in fields such as computer vision, natural language processing, and multimodal understanding. These models typically rely on large-scale electronic computing platforms for training and inference, and their computing power is mainly supported by high-dimensional parameters and complex computational structures, which places high demands on computing power and energy consumption in practical deployment.

[0003] Meanwhile, optical computing, as an emerging computing paradigm, is increasingly recognized as an important way to break through the bottlenecks of traditional electronic computing due to its inherent advantages in parallelism, bandwidth, and energy efficiency. By rationally designing the physical processes of light propagation, interference, and modulation, optical computing systems can complete specific forms of matrix operations or signal processing tasks under extremely low latency and power consumption conditions, demonstrating potential application value in accelerating artificial intelligence inference.

[0004] However, existing optical computing systems are typically constrained by physical implementation limitations, resulting in computational structures and parameter forms that differ significantly from electronic intelligent models. The high-precision numerical weights, complex nonlinear structures, and flexible computational topologies of electronic models are difficult to directly map into the parameter space of optical devices, which are subject to phase range, modulation accuracy, and noise constraints. This cross-computational paradigm mismatch makes it difficult to efficiently and reliably deploy electronic intelligent models onto optical computing systems.

[0005] To alleviate these problems, existing research has attempted to simplify model structures or redesign optical networks for specific tasks. However, these approaches often require retraining the model or are only applicable to single application scenarios, making it difficult to reuse the general computing capabilities already established in electronic intelligent models. Furthermore, the lack of a systematic method to characterize the mapping relationship between electronic models and optical computing also restricts the further application of optical computing in complex intelligent tasks.

[0006] Therefore, a new technical solution is urgently needed to solve the cross-domain mapping and physical constraint problems in the process of mapping electronic models to optical computing systems, while making full use of the existing computing power of electronic intelligent models, so as to achieve efficient deployment and stable operation of electronic intelligent models on optical computing platforms.

[0007] A search revealed Chinese patent application number 202010255884.0, which discloses a diffraction deep neural network system based on residual networks. This system improves optical system performance by introducing residual connection modules to optimize the network structure. However, this method relies on training the optical model from scratch, which is fundamentally different from the approach used in this application—mapping an electronic model to an optical platform. Summary of the Invention

[0008] In view of the deficiencies in the prior art, the purpose of this application is to provide an electro-optic mapping system and method for electronic intelligent models.

[0009] In a first aspect, this application provides an electro-optical mapping system for electronic intelligent models, comprising an electronic side module, a cross-domain conversion module, and an optical computing module; The electronic side module is used to encode semantic features of input images, text, or multimodal data, and output semantic feature representations that characterize the semantic understanding ability of the electronic intelligent model. A cross-domain conversion module is used to generate optical computing parameters for driving the optical computing module based on the semantic feature representation, under the physical constraints of the optical computing module. The optical computing module is used to perform optical calculations based on the optical computing parameters, so as to reproduce the corresponding task processing capabilities of the electronic intelligent model at the functional or computational behavior level, and realize the mapping of the electronic intelligent model to the optical computing module.

[0010] Optionally, the electronic side module is the feature extraction part of a pre-trained electronic intelligent model, which remains frozen or undergoes only limited parameter updates during the electro-optic mapping process.

[0011] Optionally, there is no requirement for an element-by-element one-to-one correspondence between the numerical weights of the electronic intelligent model and the optical computing parameters of the optical computing module.

[0012] Optionally, the physical constraints include one or more of the following related to the optical computing module: phase modulation range, amplitude modulation range, parameter quantization accuracy, propagation loss range, or noise level range.

[0013] Optionally, the optical computing parameters include optical weighting parameters, phase parameters, amplitude parameters, or combinations thereof used to characterize optical computing behavior.

[0014] Optionally, the cross-domain conversion module includes: The mapping unit inputs the semantic feature representation into the parameter generation model or mapping function to obtain the initial parameter vector; The matching and rearrangement unit performs dimensional matching and rearrangement on the initial parameter vector to make it consistent with the adjustable parameter interface of the target light computing module in terms of dimension and structure. The constraint unit performs physical constraint mapping on the rearranged initial parameter vector to ensure that the generated optical computing parameters meet the physical constraint conditions, thereby obtaining optical computing parameters that can be used to drive the optical computing module.

[0015] Optionally, it also includes deploying an adaptation module; The deployment adaptation module is used to adjust the optical computing parameters based on the actual optical path output of the optical computing module during the system deployment phase, so as to achieve consistency between the output of the optical computing module and the downstream interface.

[0016] A second aspect of this application provides an electro-optic mapping method for electronic intelligent models, comprising: Semantic feature encoding is performed on the input images, text, or multimodal data, and semantic feature representations that characterize the semantic understanding ability of the electronic intelligent model are output. Provide an optical computing module, and under the physical constraints of the optical computing module, generate optical computing parameters for driving the optical computing module based on the semantic feature representation; Optical calculations are performed based on the optical calculation parameters to reproduce the corresponding task processing capabilities of the electronic intelligent model at the functional or computational behavior level, thereby realizing the mapping of the electronic intelligent model to the optical calculation module.

[0017] A third aspect of this application provides a terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it can be used to run the electro-optical mapping system for electronic intelligent models, or to execute the electro-optical mapping method for electronic intelligent models.

[0018] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can be used to run the electro-optical mapping system for electronic intelligent models, or to execute the electro-optical mapping method for electronic intelligent models.

[0019] The electro-optical mapping system for electronic intelligent models provided in this application uniformly describes the mapping relationship between the computing power of electronic intelligent models and optical computing parameters, enabling electronic intelligent models to be deployed to optical computing modules without large-scale retraining. It has good versatility, scalability and engineering application potential.

[0020] Other technical effects resulting from the additional features will be further illustrated in the corresponding embodiments. Attached Figure Description

[0021] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a structural diagram of an electro-optical mapping system for an electronic intelligent model, according to an exemplary embodiment. Figure 2 This is a flowchart illustrating an electro-optic mapping method for an electronic intelligent model according to an exemplary embodiment; Figure 3 This is a detailed flowchart illustrating an electro-optical mapping method for an electronic intelligent model according to an exemplary embodiment. Detailed Implementation

[0022] The present application will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any way. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application, and these all fall within the protection scope of the present application. Parts not described in detail in the following embodiments can be implemented using existing technology.

[0023] The cross-domain mapping and physical constraints encountered during the mapping process from electronic models to optical computing systems hinder the efficient deployment and stable operation of electronic intelligent models on optical computing platforms. To address these issues, this application provides an electro-optic mapping system for electronic intelligent models.

[0024] Reference Figure 1 As shown, in one embodiment of this application, an electro-optical mapping system for electronic intelligent models includes an electronic side module, a cross-domain conversion module, and an optical computing module; The electronic side module is used to encode semantic features of input images, text, or multimodal data, and output semantic feature representations that characterize the semantic understanding ability of the electronic intelligent model. The cross-domain conversion module is used to generate optical computing parameters for driving the optical computing module based on semantic feature representation, given the physical constraints of the optical computing module. Specifically, optical computing parameters refer to the weighting parameters used to configure the internal physical states (such as phase and amplitude) of the optical computing module. These parameters define the mapping logic from semantic feature representation to the final task result. This enables the optical computing module to receive the semantic features output by the electronic module and perform subsequent discrimination or regression calculations based on the optical computing parameters, thereby completing the overall task of the electronic intelligent model.

[0025] The optical computing module is used to perform optical calculations based on optical computing parameters, so as to reproduce the corresponding task processing capabilities of the electronic intelligent model at the functional or computational behavior level, and realize the mapping of the electronic intelligent model to the optical computing module.

[0026] Specifically, task processing capability refers to the electronic intelligent model's ability to generate final decision or prediction results based on input data, including but not limited to classification and recognition, object detection, numerical regression, or logical judgment. This capability focuses on the model's final functional output at the application level, distinct from the intermediate semantic feature representation output by the electronic side module.

[0027] The above embodiments, through a unified description of the mapping relationship between the electronic model's computational capabilities and optical computational parameters, enable the optical computational system to undertake the corresponding computational functions in the electronic intelligent model while maintaining physical realizability.

[0028] In one specific embodiment of this application, such as Figure 1 As shown, the electronic side module includes an electronic input interface, a feature semantic extraction unit, a feature compression unit, and a semantic feature output unit.

[0029] The system comprises several components: an electronic input interface that receives raw image, text, or multimodal data, converts it into electronic signals, and sends them to subsequent units; a feature semantic extraction unit that uses a pre-trained electronic neural network to extract high-level semantic features from the raw data (the parameters of which are typically frozen during subsequent mapping); a feature compression unit that compresses or reduces the dimensionality of the extracted feature data (which may be substantial), reducing the data bandwidth transmitted to the cross-domain conversion module and lowering latency; and a semantic feature output unit that, as the data output, sends the processed semantic features to the cross-domain conversion module.

[0030] The cross-domain conversion module includes an electronic semantic feature receiving unit and an optical feature mapping unit.

[0031] The electronic semantic feature receiving unit receives semantic feature representations. The optical feature mapping unit converts these semantic feature representations into optical computation parameters that can be used to drive the optical computing module. Semantic features can be processed through parameter generation networks, such as multilayer perceptron networks, attention networks, or other neural network structures, mapping semantic features to optical computation parameters.

[0032] The optical computing module includes an optical input interface, a linear optical modulation unit, and an optical output interface.

[0033] The optical input interface converts the electrical signal parameters generated by the cross-domain conversion module into optical signals. The linear optical modulation unit modulates the light field according to the optical calculation parameters. Phase modulation or amplitude modulation of the incident light field can be achieved through a diffraction phase modulation layer, a spatial light modulator, or other optical modulation devices. The light field propagates through free space between the modulation layers and couples with the modulation parameters during propagation, thus realizing the optical calculation process. The optical output interface converts the calculated optical signal into an electrical signal (via a photodetector) or directly outputs an optical signal for transmission to downstream devices (such as displays and decision controllers).

[0034] In the above embodiments, each unit has a clear division of labor, feature compression reduces bandwidth, mapping network adapts to constraints, and linear modulation achieves high-speed, low-power computing, thereby improving system efficiency and applicability.

[0035] In one specific embodiment, the optical feature mapping unit of the cross-domain conversion module specifically includes: The mapping subunit inputs the semantic feature representation into the parameter generation model or mapping function to obtain the initial parameter vector; The matching and rearrangement subunit performs dimensional matching and rearrangement of the initial parameter vector to make it consistent with the adjustable parameter interface of the target light computing module in terms of dimension and structure. The constraint sub-unit performs physical constraint mapping on the rearranged initial parameter vector to ensure that the generated optical computing parameters meet the physical constraint conditions, thus obtaining optical computing parameters that can be used to drive the optical computing module.

[0036] The above embodiments realize cross-domain parameter conversion from the field of electronic intelligence to the field of optics.

[0037] To reduce training complexity, in some specific embodiments of this application, the semantic feature extraction module is the feature extraction part of the pre-trained electronic intelligent model, which is kept frozen or only updated with limited parameters during the electro-optic mapping process.

[0038] For example, the image feature extraction part based on ResNet-50 The main body of the model uses a ResNet-50 convolutional neural network pre-trained on the ImageNet dataset. The image feature extraction part it uses includes an initial 7×7 convolutional layer, a max pooling layer, and four subsequent residual block groups (the first 49 of the 50 layers), which finally outputs a 2048-dimensional high-level semantic feature vector.

[0039] The above embodiments, by directly reusing the already trained feature extraction structure in the electronic intelligent model, can obtain stable semantic feature representations without changing the main structure of the electronic model, thereby reducing the training overhead in the mapping process. Since the feature extraction part of the electronic model has already been trained on large-scale data, the extracted semantic features can effectively represent the high-level semantic information in the input data. Therefore, there is no need to retrain this part to support the subsequent optical parameter generation process, thereby improving the mapping efficiency and stability of the system.

[0040] Electronic intelligent models typically employ high-dimensional numerical weights for computation, while optical computing systems are limited by physical conditions such as phase modulation range, amplitude modulation capability, and device precision, resulting in fundamental differences in their parameter representation. Therefore, in one specific embodiment of this application, it is proposed that consistency between the electronic intelligent model and the optical computing system be achieved at the computational function or behavioral level, without requiring a one-to-one correspondence between the numerical weights of the electronic model and the optical computing parameters.

[0041] Specifically, computational consistency refers to the equivalence of the input-output mapping relationship. That is, for any input x, the output f(x) of the electronic model and the output g(x) of the optical computing module are equivalent in terms of task metrics (such as consistent classification accuracy).

[0042] Consistency in computational behavior stems from similarity in internal decision-making logic. That is, the optical computing module ensures correct results because its "understanding" logic aligns with the electronic intelligence model.

[0043] The above embodiments enable the effective mapping of electronic model computation capabilities to optical computation modules without the need to establish element-by-element parameter mapping relationships.

[0044] Optical computing modules have physical limitations such as phase modulation range, parameter quantization accuracy, and noise. In some specific embodiments of this application, when generating optical computing parameters, at least one physical constraint related to the optical computing module is introduced, including phase modulation range, amplitude modulation capability, parameter quantization accuracy, propagation loss, or noise.

[0045] For example: The phase adjustment range of a silicon-based Mach-Zehnder interferometer (MZI) is typically limited to 0 to 2π radians. Passive optics cannot amplify optical signals, and their amplitude transmittance is limited to 0 to 1. The digital-to-analog converter (DAC) driving the optical modulator has 8-bit precision, meaning the voltage control signal can only take 256 discrete levels. The transmission loss of the optical waveguide is 3 dB / cm. If the total optical path length is 2 cm, the signal strength will attenuate by 6 dB. This loss must be pre-calculated when generating parameters and compensated for during the system design phase through input optical power adjustment or parameter optimization to prevent the signal from being submerged in noise. Laser sources have intensity noise, with optical power fluctuations ranging from ±1%. The generated optical parameters must be robust to ensure that the system's classification or calculation results remain accurate and do not jump within this noise fluctuation range.

[0046] The above embodiments, by constraining physical realizability during the parameter generation stage, ensure that the generated optical calculation parameters conform to the working range of actual devices, avoiding the inability of theoretical parameters to be physically implemented, thereby improving the reliability and feasibility of electro-optic mapping in actual deployment.

[0047] Different types of optical computing systems may use different forms of parameters to describe their computing process. In some specific embodiments of this application, optical computing parameters include optical weight parameters, phase parameters, amplitude parameters or combinations thereof used to characterize optical computing behavior.

[0048] Specifically, in systems based on diffractive optical networks, optical computation parameters can be represented as phase modulation distributions; in systems based on tunable optical modulators, optical computation parameters can include amplitude modulation parameters or phase modulation parameters; in some optical computation structures, more complex computational behaviors can be achieved by combining phase modulation and amplitude modulation.

[0049] The above embodiments, by unifying the various forms of describing optical computing parameters, enable the electro-optic mapping method to be applicable to different types of optical computing architectures.

[0050] During the actual deployment of optical computing systems, the actual optical path output often deviates from the ideal simulation results due to non-ideal factors such as device manufacturing errors, environmental fluctuations, and optical noise. To address this, this application's embodiments include a deployment adaptation module, used to fine-tune optical computing parameters based on the actual optical path output during system deployment. This adjustment only targets a small number of parameters related to task adaptation or device compensation, eliminating the need to retrain the semantic feature extraction module, thereby achieving consistency between the optical computing module output and the downstream interface.

[0051] Specifically, the deployment adapter module can be integrated within the optical computing module, for example, as part of a nonlinear or auxiliary modulation unit, for real-time calibration of the optical path state. The downstream interface refers to the electronic processing circuitry or decision-making unit subsequently connected to the optical computing module; consistency refers to the matching of the output signal of the optical computing module with the input requirements of the downstream interface in terms of level range, logical semantics, or data format.

[0052] For example, the output of the optical computing module is coupled with a photodetector and an analog-to-digital converter (ADC), which together form the downstream interface. The optical signal output by the optical computing module is converted into an electrical signal by the photodetector, then sent to the ADC for digital processing, and finally the downstream processor identifies and classifies the result.

[0053] During the system deployment phase, the optical computing module can be an integrated optical chip or an optical path system built from discrete optical components. In this example, the optical computing module is specifically implemented using a silicon-based optical chip.

[0054] The optical chip has been manufactured and loaded with initial optical calculation parameters. After inputting a set of standard calibration images, the adapter module was deployed and detected that the overall amplitude of the electrical signal output by the photodetector was 5% lower than the simulation expectation (due to waveguide loss error), resulting in a lower voltage signal received by the ADC, close to its lower recognition threshold.

[0055] Therefore, the adaptation module does not retrain the entire model; it only adjusts the global gain coefficient or bias voltage (i.e., some optical calculation parameters) in the optical computing module to compensate for the deviation between the actual optical path and the simulation model. After correction, the optical signal is converted to accurately match the optimal input voltage range of the ADC circuit, completing error calibration and restoring the system to normal classification accuracy.

[0056] By fine-tuning some optical computing parameters through the deployment adaptation module in the above embodiments, device errors and environmental disturbances in the actual system can be effectively compensated, the output results of the optical computing module can be stabilized, and its interface requirements with downstream processing modules can be kept consistent. At the same time, since this adjustment process involves only a small number of parameters, there is no need to retrain the electronic model or reconstruct the optical computing structure, which significantly reduces the system deployment and maintenance costs.

[0057] Based on the same technical concept, other embodiments of this application also provide an electro-optic mapping method for electronic intelligent models, such as... Figure 2 As shown, it includes the following steps: S100 encodes semantic features of input images, text, or multimodal data and outputs semantic feature representations that characterize the semantic understanding ability of the electronic intelligent model. S200 provides an optical computing module that, under the physical constraints of the optical computing module, generates optical computing parameters for driving the optical computing module based on semantic feature representation. The S300 performs optical calculations based on optical computing parameters to reproduce the corresponding task processing capabilities of the electronic intelligent model at the functional or computational behavior level, thereby realizing the mapping of the electronic intelligent model to the optical computing module.

[0058] The specific steps in the above examples of this application can be referred to the implementation technology of the module corresponding to the electro-optic mapping system for electronic intelligent models in the above embodiments, and will not be repeated here.

[0059] In some specific implementations, S200 may employ the following steps, such as Figure 3 As shown: S201, based on the semantic feature extraction model constructed in S100, establishes a numerical simulation model or physical constraint model that matches the optical computing module, to characterize the realizable parameter space of the optical computing module.

[0060] S202, construct semantic feature samples or intermediate representations for optical computation parameter generation based on semantic feature representation (i.e., construct data for training or weight generation).

[0061] Specifically, the semantic feature representation is obtained by the electronic module through semantic feature encoding of the input data, and it can be a high-dimensional feature vector or feature tensor. To adapt to the subsequent optical computation parameter generation process, the semantic feature representation can be further processed to construct semantic feature samples or intermediate representations.

[0062] Semantic feature representations can be processed through operations such as feature transformation, dimensionality compression, normalization, or feature rearrangement to obtain intermediate feature representations for optical parameter generation. For example, semantic feature representations can be converted into fixed-dimensional feature vectors through one or more mapping functions, which can then be used as input to the optical parameter generation module.

[0063] For example, when the input data is an image, the electronic intelligent model can obtain the corresponding semantic feature vector through its feature extraction network. For instance, a high-dimensional feature vector represents the semantic information in the image. Subsequently, this feature vector can be linearly mapped or nonlinearly transformed to convert it into an intermediate feature representation for generating optical computation parameters, which can then be used in the subsequent optical computation parameter generation process.

[0064] S203, under the condition of satisfying the numerical simulation model or physical constraint model, generates optical computing parameters for driving the optical computing module based on semantic feature samples or intermediate representations.

[0065] Specifically, the optical parameter generation process includes: S2031, input the semantic feature samples or intermediate representations into the parameter generation model or mapping function to obtain the initial parameter vector; S2032, perform dimension matching and rearrangement of the initial parameter vector to make it consistent with the adjustable parameter interface of the target light computing module in terms of dimension and structure; S2033, perform physical constraint mapping on the initial parameter vector to ensure that the generated optical calculation parameters meet constraints such as phase modulation range, amplitude modulation range and / or quantization accuracy, thereby obtaining optical calculation parameters that can be used to drive the optical calculation module.

[0066] For example, when the target optical computing module is a diffractive optical computing structure containing L phase modulation layers, and each layer contains H×W tunable phase units, the optical computing parameters can be represented as a phase parameter tensor of size L×H×W. In this case, the parameter generation model can map the intermediate representation to an initial vector of length L×H×W and rearrange it into an L×H×W tensor; then, it is constrained to a phase range of 0 to 2π through a bounded mapping, for example, using an element-wise constraint mapping φ=2π·sigmoid(u), and if necessary, further discretized φ according to a preset quantization bit width. The generated phase parameter φ is written into the phase modulation unit of the optical computing module, thereby driving the optical computing module.

[0067] The parameter generation model here is pre-trained, and its training process is as follows: the semantic feature representation is input into the parameter generation model, the initial parameter vector is output, and it is converted into optical computing parameters that conform to the physical range of the device through a physical constraint mapping function (such as the sigmoid function); the optical computing parameters are substituted into the numerical simulation model to simulate the predicted output of the optical computing module; the loss value between the predicted output and the standard output of the electronic intelligent model is calculated; the internal weights of the parameter generation model are updated based on the loss value through the backpropagation algorithm until the loss converges or the preset number of iterations is reached.

[0068] The above embodiments provide a conversion process from semantic feature representation to optical computing parameters, thereby realizing the mapping of electronic intelligent models to optical computing modules.

[0069] Of course, in some other implementations, the method can be divided into a preparation phase and a deployment phase. In the preparation phase, a reusable optical parameter generation mechanism is formed by uniformly modeling the computational capabilities and optical constraints of the electronic intelligent model; in the deployment phase, the core generation mechanism remains unchanged, and only minor adjustments are made to adapt to specific tasks or device conditions to complete the electro-optic mapping.

[0070] The above embodiments enable the efficient deployment and application expansion of electronic intelligent models on optical computing platforms without relying on overall retraining of the electronic intelligent model or optical computing module.

[0071] Compared with existing technologies, the electro-optical mapping method and system for electronic intelligent models proposed in this application significantly reduces the complexity of cross-computation paradigm mapping by aligning the electronic model with optical computing at the computational behavior level, and improves the versatility and adaptability of optical computing systems in complex intelligent tasks.

[0072] The preferred features in the above embodiments can be used individually in any embodiment, or in any combination thereof, provided they do not conflict with each other. Furthermore, parts not described in detail in the embodiments can be implemented using existing technologies.

[0073] The following examples and comparative examples will be used to further illustrate this application in order to better understand the above-mentioned technical solutions. It should be understood that the following are only some examples and are not intended to limit this application.

[0074] Application Example 1: Taking image classification as an example, electro-optic mapping is performed on the pre-trained electronic intelligent model, and the corresponding computational function is implemented in the optical computing module.

[0075] Specifically, a pre-trained electronic intelligent model is selected as the semantic feature extraction module, such as a deep neural network model containing multi-layer convolutional structures. This model is trained on a publicly available image dataset to extract semantic feature representations of the input image. During the electro-optic mapping process, the parameters of the feature extraction portion of this electronic model are frozen, and only its output semantic feature vector is used as the input to the subsequent optical parameter generation module.

[0076] Subsequently, the cross-domain conversion module generates corresponding optical computation parameters based on semantic feature representation. In this embodiment, the target optical computation module adopts a diffractive optical computation structure containing multiple phase modulation layers. The optical parameter generation module generates corresponding phase modulation parameters based on semantic feature representation, and maps the generated parameters to the phase modulation distribution of the optical computation module while satisfying the constraints of phase modulation range and device accuracy.

[0077] In the experimental verification, the generated optical calculation parameters were configured into the optical computing module, and the light field propagation process was simulated using a numerical optical propagation model (a software simulation tool used to simulate the propagation behavior of light in physical devices in a computer), thereby obtaining the output results of the optical computing module. By comparing the optical calculation output with the output results of the electronic intelligent model on the same input data, the computational performance after electro-optic mapping can be evaluated.

[0078] Experimental results show that in image classification tasks, the optical computing parameters generated based on the method of this application enable the optical computing module to achieve classification behavior similar to that of the electronic intelligent model. In experiments on a publicly available image dataset, the classification accuracy of the electronic intelligent model was 91.4%, while after electro-optic mapping, the optical computing module achieved a classification accuracy of approximately 89.6% under numerical simulation conditions. This indicates that the method of this application can effectively map the electronic intelligent model to the optical computing structure while maintaining the main computing capabilities. The performance degradation compared to the original electronic model is less than 2%, verifying that the electro-optic mapping method proposed in this application can effectively map the electronic intelligent model to the optical computing module while maintaining the consistency of computing functions. Furthermore, by introducing the actual optical path output as a calibration signal during the deployment phase, and adjusting only a few calibration parameters, the classification accuracy improved to 92.1%, indicating that the optoelectronic co-adaptation mechanism can effectively compensate for device errors and improve the actual operating performance of the system.

[0079] The experimental results above demonstrate that by combining semantic feature representation with optical parameter generation mechanisms, the computational power of an electronic intelligent model can be mapped to an optical computing module without directly mapping the complete numerical weights of the electronic model, thereby achieving an effective mapping of the electronic intelligent model to an optical computing system.

[0080] Based on the same technical concept, in other embodiments of this application, a terminal is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it can be used to perform the above-described method or to run the above-described system.

[0081] Based on the same technical concept, in other embodiments of this application, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, can be used to perform the above-described method or to run the above-described system.

[0082] Optionally, the memory is used to store programs; the memory may include volatile memory, such as random-access memory (RAM), such as static random-access memory (SRAM), double data rate synchronous dynamic random-access memory (DDR SDRAM), etc.; the memory may also include non-volatile memory, such as flash memory. The memory is used to store computer programs (such as application programs and functional modules that implement the above methods), computer instructions, etc., and the aforementioned computer programs and computer instructions can be partitioned and stored in one or more memories. Furthermore, the aforementioned computer programs, computer instructions, data, etc., can be accessed by the processor.

[0083] The aforementioned computer programs, computer instructions, etc., can be stored in partitions within one or more memory locations. Furthermore, the aforementioned computer programs, computer instructions, data, etc., can be accessed by a processor.

[0084] A processor is used to execute a computer program stored in memory to implement the various steps of the methods involved in the above embodiments. For details, please refer to the relevant descriptions in the preceding method embodiments.

[0085] The processor and memory can be separate structures or integrated structures. When the processor and memory are separate structures, they can be coupled together via a bus.

[0086] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0087] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0088] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0089] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0090] The foregoing has described some specific embodiments of this application. It should be understood that this application is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the substantive content of this application. The above-described preferred features can be used in any combination without conflict.

Claims

1. An electro-optic mapping system for electronic intelligent models, characterized in that, It includes an electronic side module, a cross-domain conversion module, and an optical computing module; The electronic side module is used to encode semantic features of input images, text, or multimodal data, and output semantic feature representations that characterize the semantic understanding ability of the electronic intelligent model. A cross-domain conversion module is used to generate optical computing parameters for driving the optical computing module based on the semantic feature representation, under the physical constraints of the optical computing module. The optical computing module is used to perform optical calculations based on the optical computing parameters, so as to reproduce the corresponding task processing capabilities of the electronic intelligent model at the functional or computational behavior level, and realize the mapping of the electronic intelligent model to the optical computing module.

2. The electro-optic mapping system for electronic intelligent models according to claim 1, characterized in that, The electronic side module is the feature extraction part of the pre-trained electronic intelligent model, which remains frozen or undergoes only limited parameter updates during the electro-optic mapping process.

3. The electro-optic mapping system for electronic intelligent models according to claim 1, characterized in that, There is no requirement for an element-by-element one-to-one correspondence between the numerical weights of the electronic intelligent model and the optical calculation parameters of the optical computing module.

4. The electro-optic mapping system for electronic intelligent models according to claim 1, characterized in that, The physical constraints include one or more of the following related to the optical computing module: phase modulation range, amplitude modulation range, parameter quantization accuracy, propagation loss range, or noise level range.

5. The electro-optic mapping system for electronic intelligent models according to claim 1, characterized in that, The optical computation parameters include optical weighting parameters, phase parameters, amplitude parameters, or combinations thereof used to characterize optical computation behavior.

6. The electro-optic mapping system for electronic intelligent models according to claim 1, characterized in that, The cross-domain conversion module includes: The mapping unit inputs the semantic feature representation into the parameter generation model or mapping function to obtain the initial parameter vector; The matching and rearrangement unit performs dimensional matching and rearrangement on the initial parameter vector to make it consistent with the adjustable parameter interface of the target light computing module in terms of dimension and structure. The constraint unit performs physical constraint mapping on the rearranged initial parameter vector to ensure that the generated optical computing parameters meet the physical constraint conditions, thereby obtaining optical computing parameters that can be used to drive the optical computing module.

7. The electro-optic mapping system for electronic intelligent models according to claim 1, characterized in that, It also includes a deployment adaptation module; The deployment adaptation module is used to adjust the optical computing parameters based on the actual optical path output of the optical computing module during the system deployment phase, so as to achieve consistency between the output of the optical computing module and the downstream interface.

8. An electro-optic mapping method for electronic intelligent models, characterized in that, include: Semantic feature encoding is performed on the input images, text, or multimodal data, and semantic feature representations that characterize the semantic understanding ability of the electronic intelligent model are output. Provide an optical computing module, and under the physical constraints of the optical computing module, generate optical computing parameters for driving the optical computing module based on the semantic feature representation; Optical calculations are performed based on the optical calculation parameters to reproduce the corresponding task processing capabilities of the electronic intelligent model at the functional or computational behavior level, thereby realizing the mapping of the electronic intelligent model to the optical calculation module.

9. A terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it can be used to run the system of any one of claims 1-7, or to execute the method of claim 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program can be used to run the system of any one of claims 1-7, or to perform the method of claim 8.

Images