Self-attention network system based on light computing acceleration

Abstract

The invention discloses a self-attention network system based on optical computing acceleration, and relates to the technical field of data processing, and the system comprises an input conversion module which converts a text feature vector into an optical signal, an optical domain operation module which carries out the adaptive processing of a parameter matrix of a self-attention network and completes the operation of an optical domain matrix, and a data processing module which carries out the adaptive processing of the parameter matrix. The signal processing module carries out wavelength matching, phase adjustment and mixed interference on the optical signals, and the output module generates correlation parameters. According to the architecture, natural high parallelism and low delay characteristics of photon calculation are used for replacing a traditional electronic processor to complete large-scale matrix operation, and delay and energy consumption in the data processing process are reduced. The problems of high calculation delay, high energy consumption and insufficient parallel capability caused by large-scale matrix operation when a traditional electronic processor runs a self-attention network are solved, and the purposes of improving the calculation efficiency and precision of correlation parameters in the text classification process and improving the text classification efficiency are achieved. And the real-time performance of the self-attention network in a large-scale text data scene is enhanced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and particularly relates to a self-attention network system based on optical computing acceleration. BACKGROUND

[0002] In the natural language processing task of text classification, the related art generates correlation parameters representing the correlation between sequence elements by using a self-attention network to realize deep understanding of the semantic structure of the text. These correlation parameters are the core of constructing attention weights, directly affecting the focusing ability of the model on key information, and then determining the classification accuracy. In the related art, the calculation of the correlation parameters relies on a large number of matrix multiplication operations, which needs to be completed by an electronic processor.

[0003] The related art has significant limitations in generating correlation parameters. In the classical self-attention network, the calculation of the correlation parameters involves global correlation operations between sequence elements. With the exponential growth of the data size, the amount of matrix operations increases sharply, resulting in high computational delay and high energy consumption of the electronic processor, which makes it difficult to generate accurate correlation parameters in real time. At the same time, the traditional electronic processor is limited by Moore's Law and has insufficient performance in parallel processing of large-scale matrix operations, which directly affects the calculation efficiency and accuracy of the correlation parameters and restricts the application of the self-attention network in large-scale text classification tasks. SUMMARY

[0004] The present application provides a self-attention network system based on optical computing acceleration to at least solve the problems of high computational delay, high energy consumption, and insufficient parallel performance in calculating the correlation parameters of the self-attention network in the related art, which affect the calculation efficiency and accuracy.

[0005] The present application provides a self-attention network system based on optical computing acceleration, comprising: an input conversion module, an optical domain operation module, a signal processing module, and an output module.

[0006] The input conversion module is used to receive an input vector and convert it into an optical signal, and the input vector is a feature vector obtained by processing a text sequence.

[0007] The optical domain operation module includes an adaptation unit and an operation unit. The adaptation unit is used to adaptively process the parameter matrix in the self-attention network. The operation unit is used to perform optical domain matrix operations based on the adapted parameter matrix and the optical signal of the input vector to generate an intermediate optical signal.

[0008] The signal processing module is used to perform wavelength matching, phase adjustment, and hybrid interference on the intermediate optical signal and another input vector optical signal processed by the input conversion module to generate a hybrid optical signal containing sum and difference information.

[0009] The output module is configured to detect the mixed optical signal, and output a correlation parameter representing a correlation degree between sequence elements, and the correlation parameter is used to construct a self-attention mechanism to classify the text.

[0010] By converting the text feature vector into an optical signal, performing optical matrix operation on the parameter matrix and the input vector by means of the optical domain operation module, and generating the correlation parameter by means of the signal processing module, the high parallelism and low delay characteristics of the photon computing are used to replace the operation process of the traditional electronic processor, so that the technical problems of high calculation delay, large energy consumption and insufficient parallel performance of the electronic processor in calculating the correlation parameter can be solved, and the calculation efficiency and accuracy of the correlation parameter are improved, and the applicability of the self-attention network in large-scale text classification tasks is enhanced. BRIEF DESCRIPTION OF DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present application, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0012] Figure 1 A structural schematic diagram of a self-attention network system based on optical computing acceleration is provided for the embodiments of the present application.

[0013] Figure 2 A self-attention network system based on optical computing acceleration is provided for the embodiments of the present application.

[0014] Figure 3 A silicon-based MZI unit structure diagram is provided for the embodiments of the present application.

[0015] Figure 4 A modulator network array schematic diagram is provided for the embodiments of the present application.

[0016] Figure 5 A flowchart of a self-attention network construction method based on optical computing acceleration is provided for the embodiments of the present application.

[0017] Figure 6 A structural schematic diagram of an electronic device is provided for the embodiments of the present application. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0019] It should be noted that in the description of the present application, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such a process, method, article or device. The terms "first", "second" and the like in the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0020] The present application solves the problems of low computing efficiency, high energy consumption and limited parallel processing capability of self-attention network in text classification task caused by large-scale matrix operation. The input conversion module is used to convert the text feature vector into an optical signal. The optical domain operation module is used to adaptively process the parameter matrix of the self-attention network and complete the optical domain matrix operation. The wavelength matching, phase adjustment and hybrid interference operation of the signal processing module are used to finely process the optical signal. Finally, the output module generates the correlation parameter representing the correlation degree of the sequence elements. By using the high parallelism and low delay characteristics of photonic computing, the matrix operation of the traditional electronic processor is migrated to the optical domain to complete, thereby realizing the efficiency and low energy consumption of the correlation parameter calculation in the self-attention mechanism, and improving the real-time performance and scalability of the text classification.

[0021] In order to enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0022] Figure 1 The structure diagram of a self-attention network system based on optical computing acceleration provided by the embodiment of the present application is shown in FIG. 1. As shown in FIG. 1, the embodiment of the present application provides a self-attention network system 10 based on optical computing acceleration, which specifically comprises an input conversion module 101, an optical domain operation module 102, a signal processing module 103 and an output module 104. Figure 1

[0023] The input conversion module 101 is used to receive an input vector and convert it into an optical signal. The input vector is a feature vector obtained by processing a text sequence.

[0024] Specifically, the electrical signal of the text feature vector is converted into an optical signal by a modulator, and the wave-particle duality of light is used to realize the conversion of the signal carrier. An interface is established between the electrical domain information and the optical domain operation to provide an adaptive signal form for subsequent optical domain processing. The physical limitations of electronic signals in parallel transmission and processing are broken through, and the high bandwidth characteristics of optical signals support high-density data parallel transmission, laying a foundation for large-scale matrix operation.

[0025] ​The optical domain operation module 102 comprises an adaptation unit and an operation unit, the adaptation unit is used for performing adaptation processing on a parameter matrix in a self-attention network; and the operation unit is used for performing optical domain matrix operation on the optical signal of the input vector based on the adapted parameter matrix, to generate an intermediate optical signal.

[0026] Specifically, the adaptation unit performs normalization preprocessing on the parameter matrix of the self-attention network, and performs linear transformation to constrain the element value of the parameter matrix to the interval [0, 1], so as to match the dynamic range of the optical modulator, solve the problem of nonlinear response of the optical modulator, ensure the physical feasibility of the optical domain multiplication of the parameter matrix and the input vector, improve the numerical stability of the optical domain operation, and avoid calculation error caused by modulator saturation. The signal processing module is used for wavelength matching, phase adjustment and hybrid interference of the intermediate optical signal and the input vector optical signal processed by the other input conversion module, to generate a hybrid optical signal containing sum and difference information.

[0027] The operation unit maps the adapted parameter matrix to an operation modulation network based on an optical domain matrix multiplication architecture, and realizes matrix-vector multiplication through step-by-step modulation and optical field superposition, replaces the von Neumann architecture of the traditional electronic processor, and directly completes linear transformation with the spatial parallelism of optical waves, so that the time complexity is reduced from O(n²) of electronic devices to O(1) of optical domain, and the operation efficiency is significantly improved. Meanwhile, the non-dissipative characteristic of the optical signal can effectively reduce the energy consumption.

[0028] The signal processing module 103 is used for wavelength matching, phase adjustment and hybrid interference of the intermediate optical signal and the input vector optical signal processed by the other input conversion module, to generate a hybrid optical signal containing sum and difference information.

[0029] Specifically, through wavelength division multiplexing (WDM) technology, the same wavelength channel is allocated to the intermediate optical signal and the input vector optical signal, the synchronous transmission of multi-dimensional data is realized, the strict alignment of the two optical signals in space and spectrum is ensured, the coherent condition for subsequent interference is provided, the cross-channel crosstalk is eliminated, and the accuracy of the interference result is ensured.

[0030] The contrast of the interference fringes is optimized by applying precise phase shift through a thermo-optic or electro-optic phase shifter, the phase relationship of the optical waves is controlled, the sum and difference values can be effectively distinguished from the interference result, the signal-to-noise ratio of the hybrid optical signal is improved, and the detectability of the correlation parameter is enhanced.

[0031] The mode coupling of the two optical signals is realized through a directional coupler, the orthogonal components containing the sum and difference values are generated, the result of the optical domain matrix operation is converted into separable physical quantities, and subsequent detection is facilitated. The weak signal difference is amplified through interference effect, and the resolution of the correlation parameter is improved.

[0032] The output module 104 is used for detecting the mixed optical signal, outputting a correlation parameter representing the correlation between sequence elements, and the correlation parameter is used for constructing a self-attention mechanism to classify the text.

[0033] Specifically, the two orthogonal components of the mixed optical signal are detected by the balanced photodetector, the common-mode noise is suppressed by differential amplification, the optical domain interference result is converted into an electrical signal, the correlation parameter representing the correlation between sequence elements is analyzed, high-sensitivity optical-electric conversion is realized, and the accuracy of the correlation parameter is ensured; and the differential detection mechanism significantly reduces the influence of environmental noise.

[0034] The optical computing accelerated self-attention network architecture provided by the embodiment of the application realizes efficient conversion of electronic-optical signals through the input conversion module, breaks through the data transfer limitation of the traditional computing paradigm; completes large-scale linear transformation at the speed of light through parallel matrix multiplication of the optical domain operation module; accurately extracts the correlation parameter through wavelength matching, phase control and interference measurement of the signal processing module; and realizes low-noise and high-resolution result analysis through high-sensitivity detection of the output module. The parallelism, low delay and low power consumption of photonics are fully integrated, the computing process of the self-attention mechanism is reconstructed, and the real-time performance and energy efficiency ratio of the text classification task are significantly improved.

[0035] In some optional embodiments, the input conversion module is a modulator array, the modulator array is composed of a plurality of modulator units in cascade, and the modulator units correspond to the elements of the input vector.

[0036] Specifically, each modulator unit corresponds to an element of the input vector, and the amplitude or phase of the optical carrier is real-time controlled through an electrical driving signal. When the input vector is loaded, each modulator unit independently modulates the light intensity of the corresponding optical path according to the input value, so as to encode the discrete digital feature vector into a continuous optical signal. This process follows the physical law of electro-optic effect, that is, the applied electric field changes the refractive index of the waveguide material, and then controls the propagation characteristics of the optical signal. The digital feature vector in the electronic domain is converted into an analog signal in the optical domain, providing a physical carrier for subsequent optical domain matrix operation. Through the cascaded modulator array, as shown in Figure 4 , Figure 4 The modulator cascade network array provided by the embodiment of the application can realize one-time parallel loading of multi-dimensional data, avoid the time overhead of element-by-element serial processing in traditional electronic computing, provide an optical signal dynamic range matched with the parameter matrix for the subsequent optical domain operation unit, and ensure the feasibility of linear operation.

[0037] The embodiment of the application realizes efficient conversion of electronic-optical signals and synchronous loading of multi-dimensional data through a spatial parallel architecture of a modulator array; ensures the accuracy of numerical mapping according to the linear characteristics of the electro-optic effect; and significantly improves the data input efficiency of the self-attention mechanism in combination with the low-loss transmission and high parallelism of optical signals. The high-quality input basis is provided for optical domain matrix operation, and the performance problems of data transfer and serial processing in traditional electronic calculation are broken through.

[0038] In some optional embodiments, the modulator array is a silicon-based Mach-Zehnder modulator array.

[0039] Specifically, a Mach-Zehnder interferometer (MZI) is a silicon-based optical device that can realize various functions based on the principle of optical interference, such as optical switches, modulators, etc., and has a wide range of applications in optical computing, optical communication, etc. A silicon-based MZI unit is composed of two couplers and two phase shifters, as shown in Figure 3 , Figure 3 The silicon-based MZI unit structure provided by the embodiment of the application is shown in the overall network architecture diagram as Figure 2 , Figure 2 The self-attention network system based on optical computing acceleration provided by the embodiment of the application is shown in the diagram. The silicon-based MZI unit structure allows adjustment of the optical path difference to realize interference of different phases. When the input optical signal passes through the input coupler (also known as a beam splitter), it will be divided into two beams, and each beam will pass through two different phase shifters to produce a certain phase difference. Finally, the two beams pass through the output coupler (also known as a beam combiner) again to combine into a new optical signal, forming an interference signal. According to the phase difference, it can accurately regulate the amplitude and phase of the optical signal, thereby completing complex mathematical operations.

[0040] By using a silicon-based Mach-Zehnder modulator array as the core device of the input conversion module. The silicon-based Mach-Zehnder modulator is based on the electro-optic effect of silicon photonics: when an external electrical signal acts on the two arm-shaped waveguides of the modulator, the refractive index of the waveguide is changed through the carrier depletion effect, thereby regulating the phase difference of the optical signal in the double arms. The numerical value of the input vector element is mapped to the electrical driving signal and applied to the corresponding modulator unit, thereby modulating the amplitude or phase of the optical carrier. The CMOS compatibility of the silicon-based platform allows large-scale integration of high-density modulator arrays, and each modulator unit can be independently controlled to realize accurate correspondence with the input vector element.

[0041] The embodiment of the present application realizes efficient and low-loss conversion of electronic-optical signals by adopting a silicon-based Mach-Zehnder modulator array, fully utilizing the advantages of silicon photonics, such as high integration, low power consumption and high linearity. Not only does it solve the limitations of traditional electronic interfaces in terms of bandwidth and delay, but it also provides high-quality input optical signals for subsequent optical domain matrix operations, significantly improving the computational efficiency and energy efficiency ratio of the self-attention mechanism, and laying a foundation for optical computing acceleration of text classification tasks.

[0042] The embodiment details the process of adapting the parameter matrix in the self-attention network in the above embodiment. The specific implementation of the process includes the following steps:

[0043] Step a1: Analyze the maximum and minimum values of the elements in the parameter matrix.

[0044] Specifically, by analyzing the maximum and minimum values of the parameter matrix, the numerical distribution range is quantified, providing a normalization benchmark for subsequent linear transformation, and the numerical span of the parameter matrix is defined, providing a mathematical basis for the mapping relationship between the input domain and the output domain of the linear transformation, ensuring that the value range after transformation is strictly limited to the target interval, avoiding signal truncation or nonlinear distortion problems caused by parameter values exceeding the physical dynamic range of the optical modulator, while preserving the relative proportion relationship between parameters, providing a foundation for high-precision optical domain computation.

[0045] Step a2: Perform linear transformation on the elements in the matrix according to the maximum and minimum values, so that the range of the transformed elements is between 0 and 1, and obtain the adapted parameter matrix.

[0046] Specifically, affine transformation is performed on each element through linear normalization, mapping the original parameters to the [0, 1] interval. The transformation has a preservation property, i.e., the relative size relationship of the elements in the original matrix remains unchanged after transformation, the numerical range of the parameter matrix is forcibly constrained to the linear operating area of the optical modulator, making it compatible with the physical adjustable range of the optical signal (such as extinction ratio and modulation depth), eliminating calculation errors caused by numerical overflow, realizing dynamic range adaptation of the parameter matrix and the optical domain operation unit, maximizing the use of the linear response characteristics of the optical modulator, and improving the numerical precision and stability of the optical domain matrix multiplication.

[0047] The embodiment of the present application analyzes the extreme values of the parameter matrix to obtain its numerical distribution characteristics, and combines linear normalization transformation to map the parameters to the linear operating interval of the optical modulator, realizing accurate dynamic range adaptation of the parameter matrix and the optical domain operation unit. This process not only preserves the relative weight relationship between parameters, but also avoids nonlinear distortion of optical signals caused by numerical overflow, significantly improving the numerical precision and system robustness of optical domain matrix operations, and providing a key guarantee for efficient optical computation of the self-attention mechanism.

[0048] In some optional embodiments, the linear transformation formula is:

[0049]

[0050] wherein W qk is a weight matrix; max(W qk ) is the maximum value of the weight matrix elements; min(W qk ) is the minimum value of the weight matrix elements; (W qk ) it is the element of the weight matrix W qk located at the ith row and the tth column, i is a row index, t is a column index, i = 1, 2, … d, t = 1, 2, … d.

[0051] Specifically, the linear transformation formula is based on the Min-Max normalization principle, and an affine mapping from the original numerical space to the target interval [0, 1] is constructed by analyzing the global extreme values (maximum and minimum) of the weight matrix. The mathematical essence is to eliminate the absolute scale difference of parameter values by translation and scaling operations, and only the relative proportion relationship is retained. This process is a deterministic linear transformation, which has a preservation property, that is, the relative size relationship of the elements in the original matrix remains unchanged after transformation.

[0052] The numerical values of the weight matrix elements are forced to be constrained to the linear working interval [0, 1] of the optical modulator, solving the problem of signal truncation or nonlinear distortion caused by parameter values exceeding the physical limit of optoelectronic devices. By normalizing and compressing the dynamic range of parameter values, the noise amplification effect of optical signals caused by large numerical fluctuations is suppressed, the robustness of optical domain matrix multiplication is improved, the input characteristics of the parameter matrix and the optical domain operation unit (such as a silicon-based modulator array) are completely matched, and there is no need to design a complex dynamic gain control module, simplifying the system implementation complexity.

[0053] By mapping the weight matrix to the interval [0, 1], the embodiment of the application realizes the accurate adaptation of the dynamic range of the parameter matrix and the optical domain operation unit. The relative weight relationship between parameters is preserved, avoiding nonlinear distortion of optical signals caused by numerical overflow, significantly improving the numerical accuracy and system robustness of optical domain matrix operation, and providing protection for efficient optical domain calculation of self-attention mechanisms.

[0054] The embodiment details the process of performing optical domain matrix operation on the optical signal based on the adapted parameter matrix and the input vector in the above-mentioned embodiment to generate an intermediate optical signal. The specific implementation of the process includes the following steps:

[0055] b1, map the adapted parameter matrix to the operation modulation network, the operation modulation network corresponds to the modulator units of the modulator array, and the operation units in the operation modulation network correspond to the elements in the parameter matrix.

[0056] Specifically, by matching the elements of the parameter matrix with the independent modulation units in the operation modulation network, a physical correspondence is formed, ensuring that the spatial distribution of the parameters is consistent with the propagation path of the optical signal, constructing a direct association between the parameter matrix and the optical domain operation resources, enabling the parameter elements to independently regulate the signal characteristics of the corresponding optical path, providing a physical carrier for subsequent distributed parallel computing, realizing the hardware deployment of the parameter matrix, eliminating the storage access delay in traditional electronic computing, and providing an expandable physical basis for large-scale parallel matrix operation.

[0057] Step b2, input the optical signal of the input vector into the operation modulation network, so that the optical signal passes through each operation unit in turn according to the row and column order.

[0058] Specifically, by determining the propagation direction of the optical path and the arrangement order of the modulation units through the waveguide cascade architecture, the optical signal is forced to pass through each modulation unit in turn according to the row and column priority of matrix operation, ensuring that the multiplication operation of the input vector and the parameter matrix conforms to the row and column order defined by mathematics, avoiding the timing disorder problem caused by parallel processing, and guaranteeing the correctness of the calculation logic. The element-by-element traversal of the input vector is completed at the speed of light, significantly shortening the calculation period, while the low-loss characteristics of the waveguide structure maintain the signal integrity.

[0059] Step b3, the operation unit modulates the optical signal according to the numerical value of the corresponding parameter matrix element.

[0060] Specifically, by applying external electrical / thermal excitation signals, the attenuation coefficient or phase shift of the modulation unit to the optical signal is dynamically adjusted, realizing the linear mapping of the optical signal power and the parameter value, converting the parameter value into a physically controllable optical signal modulation depth, completing the core calculation step of parameter weighting, and achieving accurate weight allocation with high linearity of optical modulation characteristics, avoiding quantization errors in electronic digital calculation, while using the high dynamic range of optical signals to improve calculation accuracy. The modulated optical signal is transmitted and superimposed at the output end of the operation modulation network to generate the intermediate optical signal corresponding to the multiplication operation of the parameter matrix and the input vector.

[0061] Step b4, transmit and superimpose the modulated optical signal at the output end of the operation modulation network to generate the intermediate optical signal corresponding to the multiplication operation of the parameter matrix and the input vector.

[0062] Specifically, by using optical couplers or waveguide cross structures to converge the output optical signals of each modulation unit in space, the physical summation of the row-column inner product in matrix multiplication is realized, the single-point product completed by distributed parallel is accumulated into a global vector output, the complete matrix operation process from parameter weighting to result aggregation is completed, and the parallel summation of massive data is completed at the speed of light without additional computing resources, significantly reducing the computational complexity and energy consumption, while avoiding the problem of bus contention in electronic systems.

[0063] The embodiment of the application establishes the physical correspondence between the parameters and the optical path resources through the accurate mapping of the parameter matrix and the operation modulation network; ensures the logical correctness of the matrix operation through the row and column order traversal of the optical signal; realizes the high linearity and low error of weight distribution through the optical power modulation driven by the parameters; and completes the efficient calculation of parallel summation through the spatial superposition of the optical signal. The high parallelism, low delay and low power consumption advantages of photonic computing are fully utilized to reconstruct the core computing paradigm of the self-attention mechanism, and the real-time performance and energy efficiency ratio of the text classification task are significantly improved.

[0064] In some optional embodiments, the modulated optical signal is:

[0065]

[0066] wherein, is the modulated optical signal, is the optical signal of the input vector, W qk is the weight matrix.

[0067] Specifically, the input vector is linearly transformed by the weight matrix to generate the modulated optical signal. Based on the continuous regulation of the power of the input optical signal according to the numerical value of the weight matrix element by each modulation unit, the inner product operation of the parameter matrix and the input vector is converted into real-time modulation of the optical intensity signal, and through the spatial layout of the optical waveguide network and the phase / amplitude control of the modulator, the optical simulation of matrix multiplication is realized.

[0068] The large-scale matrix multiplication is completed at the speed of light, avoiding the overhead of memory access and instruction transmission in electronic computing, and the single-step operation delay tends to be close to the light propagation time; only the optical path corresponding to the non-zero weight is modulated and consumed, compared with the full device power supply mode of the electronic processor, the invalid power consumption is significantly reduced; the high dynamic range and low crosstalk characteristics of the optical signal suppress the cumulative effect of quantization error and thermal noise in traditional electronic computing, maintaining numerical accuracy; the multi-channel parallel capability of the optical waveguide supports seamless expansion to higher dimensions, providing linearly growing computing resources for long sequence text processing.

[0069] The embodiment of the application realizes the speed-of-light level acceleration of the core calculation of the self-attention mechanism by completely migrating the linear transformation of the weight matrix and the input vector to the optical domain. This process not only preserves the mathematical rigor of matrix multiplication, but also fully utilizes the high parallelism and low energy consumption advantages of photonic computing, effectively reducing the generation delay and power consumption of the intermediate optical signal.

[0070] In some optional embodiments, the signal processing module comprises a wavelength division multiplexing unit, a phase shifter and a directional coupler; the specific implementation of wavelength matching, phase adjustment and hybrid interference of the intermediate optical signal and the input vector optical signal processed by the other input conversion module comprises the following steps:

[0071] Step c1, the wavelength division multiplexing unit matches the wavelengths of the intermediate optical signal and the input vector optical signal processed by the other input conversion module, and assigns the same wavelength channel.

[0072] Specifically, based on wavelength division multiplexing, the wavelength selector and the channel assigner are used to map the elements in the intermediate optical signal and the input vector optical signal that need to be associated to the same wavelength channel, establish the space-spectrum correspondence of the two optical signals, ensure that the elements that need to be calculated for correlation are in the same wavelength channel, provide a physical carrier for subsequent interference, eliminate crosstalk between different wavelengths, ensure that interference only occurs in the target element pair, improve the accuracy of correlation calculation; at the same time, wavelength multiplexing supports parallel processing of multi-dimensional data, and expands the system throughput.

[0073] Step c2, the phase shifter adjusts the phase of the two optical signals by a preset phase offset.

[0074] Specifically, an accurate phase offset (such as π / 2) is introduced through an electro-optic or thermo-optic phase shifter, the propagation constant of light wave is controlled by the electro- or thermo-induced refractive index change of the material, the phase is artificially intervened, the phase relationship of the two optical signals is calibrated, the contrast and orthogonality of the interference fringes are optimized, the sum and difference signals can be effectively separated, the signal-to-noise ratio of the interference result is significantly improved, and the measurement error caused by the randomness of the initial phase is suppressed; through dynamic phase compensation, the process deviation and environmental fluctuation are adapted, and the system robustness is enhanced. For example: the input optical signal is y, and the output of the phase shifter is wherein is the phase of the moving optical signal, is the imaginary unit.

[0075] Step c3, the directional coupler couples the two optical signals after phase adjustment to generate a hybrid optical signal including sum and difference.

[0076] Specifically, based on the evanescent field coupling effect, when two optical signals propagate in close proximity in the directional coupler, the mode field exchanges energy, resulting in an interference distribution at the output port. By controlling the coupling length and the refractive index difference, a specific power distribution and phase inversion can be achieved; the two optical signals that have completed wavelength matching and phase adjustment are coherently superimposed to generate a mixed optical signal containing the sum and difference. By directly implementing the dot product operation required by the self-attention mechanism at the physical layer, complex algorithms in the digital domain are not needed; the orthogonal components of the mixed optical signal naturally carry correlation information, simplifying the subsequent optical-electric conversion and parameter extraction process.

[0077] The directional coupler is composed of two waveguides close to each other, so that energy can be transferred between them, and the transmission matrix is wherein is the projection coefficient. For a 50:50 directional coupler, .

[0078] The embodiment of the application realizes wavelength-space mapping of multi-dimensional data through a wavelength division multiplexing unit, establishes a strict element-level correspondence relationship; through precise phase control of the phase shifter, the controllability and stability of the interference condition are optimized; through coherent coupling of the directional coupler, the optical domain matrix operation result is converted into a physical quantity that can be directly detected. Fully integrating the high parallelism, low loss and reconfigurability of photonics, the core calculation of the self-attention mechanism is completed in a single physical layer, significantly improving the energy efficiency ratio and real-time performance of the text classification task.

[0079] In some optional embodiments, the wavelength division multiplexing unit comprises:

[0080] The wavelength selector is configured to provide independent wavelengths equal in number to the dimensions of the input vector, and the independent wavelengths are matched with the channels.

[0081] Specifically, based on wavelength-selective devices such as gratings or array waveguides, a set of independent wavelengths equal in number to the dimensions of the input vector is generated, and each wavelength corresponds to a unique channel. By adjusting the device parameters (such as temperature or current), the center wavelength and channel spacing can be precisely controlled, and each input vector element is assigned a dedicated wavelength label, building an orthogonal optical frequency domain resource pool, avoiding timing conflicts and address competition in traditional electronic buses, and realizing fine-grained division of optical domain resources, with an upper limit of tens to hundreds of independent channels, providing physical layer protection for large-scale parallel computing; the low crosstalk between wavelengths improves the system reliability.

[0082] The channel allocator is configured to allocate elements at corresponding positions in the intermediate optical signal and another input vector optical signal to the same wavelength channel.

[0083] Specifically, the mapping relationship between the intermediate optical signal and the input vector optical signal elements is dynamically established according to a control signal through an integrated optical switch matrix or a thermo-optic tunable router. The optical path redirection with low insertion loss is realized through a waveguide cross structure, the element pairs that need to be calculated for correlation are forcibly associated and constrained to the same wavelength channel, so that the subsequent interference operation is only applied to the target element combination, the false interference between the unrelated elements is eliminated, and the specificity of the correlation calculation is improved; the dynamic reconfigurable characteristic supports online adjustment of the mapping strategy to adapt to different task requirements.

[0084] The waveguide array includes waveguides consistent with the number of wavelength channels, and is used to carry optical signals of different wavelengths for parallel transmission.

[0085] Specifically, by optimizing the waveguide width, height and cladding refractive index difference, efficient binding and low bending loss transmission of specific wavelengths are realized, a physically isolated transmission path is provided for multi-wavelength signals, and crosstalk between channels is suppressed; the parallel waveguide architecture supports simultaneous transmission and processing of multiple wavelength signals, ensures the amplitude and phase stability of each wavelength signal, and maintains high signal-to-noise ratio; the compact planar optical path design greatly reduces the system volume, improves the integration and scalability.

[0086] The embodiment of the application realizes wavelength coding of input vector elements by constructing an orthogonal optical frequency domain resource pool through a wavelength selector; establishes an element-level mapping relationship through a channel allocator to ensure accurate targeting of correlation calculation; and provides a low-loss parallel transmission channel through a waveguide array to ensure synchronous processing of multi-dimensional data. Through the advantages of high density, low power consumption and high parallelism of photonic integration technology, the resource contention and delay problem of large-scale matrix operation in the self-attention mechanism is solved at the physical layer, and hardware support is provided for the text classification task.

[0087] The embodiment details the process of wavelength matching of the wavelength division multiplexing unit in the above-mentioned embodiment to the input vector optical signal processed by another input conversion module, allocating channels of the same wavelength, and the specific implementation mode of the process includes the following steps:

[0088] d1, configure the wavelength channels of the wavelength division multiplexing unit, the number of wavelength channels is the same as the dimension of the input vector, and each channel corresponds to a different wavelength.

[0089] Specifically, based on the spectrum resource management of wavelength division multiplexing, an equal number of independent wavelength channels is dynamically configured according to the dimension of the input vector, the wavelength is accurately allocated through a tunable laser or a fixed filter array, the exclusive optical frequency domain resource is pre-allocated for each element of the input vector, the orthogonal optical signal transmission channel is established, the timing conflict and address competition problem in the traditional electronic bus is avoided, the fine-grained division of the optical domain resource is realized, the system can be expanded to a large number of independent channels, and the processing requirement of the high-dimensional text feature vector is met; and the low crosstalk characteristic between wavelengths guarantees the stability of the multi-channel parallel transmission.

[0090] Step d2, by identifying the elements in the corresponding positions in the intermediate optical signal and another input vector optical signal, the same wavelength channel is allocated to the elements in the corresponding positions, so that the elements of the two signals share the channel for parallel transmission.

[0091] Specifically, the element positions of the intermediate optical signal and the input vector optical signal are sequentially identified through the control signal, and the element pairs that need to be calculated for correlation are dynamically routed to the same wavelength channel according to the optical switch matrix or the thermo-optic tunable router, so that the element pairs that need to interact in the self-attention mechanism are forced to be associated and constrained to the same wavelength channel, and it is ensured that the subsequent interference operation only acts on the target element combination, so that the false interference between the unrelated elements is eliminated, and the specificity of the correlation calculation is improved; the dynamic routing capability supports online adjustment of the mapping strategy, and adapts to different task requirements and data distribution changes.

[0092] The embodiment of the present application realizes efficient management and accurate mapping of optical domain resources through pre-configuration and dynamic allocation of wavelength channels. By establishing an orthogonal optical frequency domain resource pool, physical layer protection is provided for large-scale parallel computing; and by the dynamic routing mechanism, the targeting of the correlation calculation is ensured, and irrelevant interference is suppressed.

[0093] The embodiment details the process of phase adjustment of the phase shifter on the two optical signals by the preset phase offset in the above-mentioned embodiment, and the specific implementation manner of the process comprises the following steps:

[0094] Step e1, obtaining the interference requirement of the directional coupler, and setting the preset phase offset according to the interference requirement.

[0095] Specifically, by analyzing the relationship between the light intensity distribution of the output port of the directional coupler and the phase difference of the input optical signal, the required phase difference value for realizing the target interference effect, such as the maximum extinction ratio or a specific light splitting ratio, is analyzed, an explicit calibration reference is provided for subsequent phase adjustment, and it is ensured that the phase relationship of the two optical signals meets the working conditions of the interferometer. By calculating the preset ideal phase difference, non-optimal interference results caused by blind adjustment are avoided, and the initial alignment efficiency of the system is improved.

[0096] Step e2, associate the phase shifter with the transmission waveguide of the two optical signals, change the effective refractive index of the waveguide by applying voltage or controlling temperature.

[0097] Specifically, by using the electro-optic effect (electric field induced refractive index change) or the thermo-optic effect (temperature gradient induced refractive index distribution change), the propagation constant of the waveguide is dynamically regulated by applying an electric signal or a heat source to realize precise control of the phase of the optical signal. The physical coupling between the phase shifter and the optical signal transmission path is established, and the electric / thermal excitation signal is converted into the phase change of the optical signal, realizing the phase adjustment ability with sub-wavelength level precision, supporting the dynamic correction of the phase drift caused by process deviation and environmental fluctuation.

[0098] Step e3, monitor the phase difference of the two optical signals to obtain real-time phase difference.

[0099] Specifically, by using the interferometer self-sensing technology or the external phase detector, the phase difference information is extracted in real time by analyzing the light intensity waveform or beat frequency signal after the interference of the two optical signals, which provides real-time data support for subsequent error judgment. With high sampling rate, the instantaneous change of the phase difference is captured to ensure the timeliness and accuracy of the feedback control.

[0100] Step e4, compare the real-time phase difference with the preset phase offset.

[0101] Specifically, by using the digital signal processor or related comparator circuit, the real-time phase difference is compared with the preset value to generate an error signal, which judges whether the current phase difference meets the system performance requirements and quantifies the direction and amplitude of the phase deviation, providing a basis for subsequent adjustment strategy.

[0102] Step e5, when the deviation between the real-time phase difference and the preset phase offset is within the preset tolerance range, the current driving parameter of the phase shifter is maintained to maintain a stable phase difference.

[0103] Specifically, when the error signal amplitude is less than the preset threshold, the driving parameter of the phase shifter, such as the voltage / temperature setting value, is locked to stop active adjustment, inhibit system oscillation caused by excessive adjustment, ensure the stability of the phase difference within the target range, reduce control energy consumption, prolong the service life of the device, and at the same time guarantee the stability of the interference result.

[0104] Step e6, when the deviation between the real-time phase difference and the preset phase offset is not within the preset tolerance range, repeat the step of changing the effective refractive index of the waveguide by applying voltage or controlling temperature until the real-time phase difference is within the preset tolerance range.

[0105] Specifically, the driving parameters of the phase shifter are dynamically adjusted according to the error signal, and the driving system converges to the target phase difference. The steady-state error is eliminated through iterative adjustment, and the system is forced to return to the preset operating point, which significantly improves the robustness of phase control and adapts to the long-term drift caused by environmental disturbances and device aging.

[0106] The embodiments of the present invention achieve precise adjustment of the phase difference between two optical signals using a phase shifter through phase demand modeling and closed-loop feedback control. A low-latency, high-linearity phase control link is established through efficient energy conversion via electro-optical / thermo-optical effects. Real-time monitoring and dynamic correction mechanisms ensure stability and reliability in complex environments.

[0107] This embodiment describes in detail the process of coupling the two phase-adjusted optical signals using the directional coupler in the above embodiment to generate a mixed optical signal including a sum value and a difference value. The specific implementation of this process includes the following steps:

[0108] Step m1: input the two optical signals after phase adjustment into the two input ends of the directional coupler respectively.

[0109] Specifically, the directional coupler has a dual-port input. By injecting the two optical signals to be interfered into adjacent waveguides, they interact according to the evanescent field coupling excitation mode, providing initial conditions for subsequent energy exchange and interference, ensuring that the two signals are highly overlapped in space and time, realizing collimated input of optical signals, and reducing coupling losses caused by alignment errors.

[0110] Step m2: Based on the mode coupling between the waveguides, the two optical signals are caused to exchange energy and interfere with each other in the coupler.

[0111] Specifically, when the distance between the two waveguides is less than several times the wavelength, the evanescent fields of the guided modes overlap, triggering a mode coupling effect. By controlling the coupling length and refractive index difference, partial or complete energy transfer can be achieved. By driving the amplitude and phase of the two optical signals into coherent superposition, physically meaningful sum and difference components are generated, achieving an optical-domain simulation of the inner product operation of the self-attention mechanism within a single device.

[0112] Step m3: Obtain interference results from the two output ends of the coupler respectively. The interference result of the first output end is the sum of the energy of the two signals, and the interference result of the second output end is the difference of the energy of the two signals.

[0113] Specifically, one end of the directional coupler outputs a sum signal of in-phase superposition, and the other end outputs a difference signal of anti-phase cancellation. By decomposing the optical domain interference results into independently detectable physical quantities, the complete information required for self-attention weight calculation is retained; the real and imaginary parts of matrix multiplication are directly separated through the physical layer, without the need for complex number operations in the digital domain.

[0114] Step m4: integrating the interference results of the first output end and the second output end to obtain a mixed optical signal including a sum value and a difference value.

[0115] Specifically, the two output signals are combined into a composite optical signal through an optical combiner. The light intensity distribution carries both sum and difference information, providing a multi-dimensional optical signal containing complete correlation information for subsequent photoelectric detection, simplifying the subsequent signal processing process and reducing the requirements for the bandwidth and resolution of the analog-to-digital converter.

[0116] The embodiment of the present invention directly implements the inner product operation of the self-attention mechanism at the physical layer through the mode coupling effect of the directional coupler; synchronously obtains the sum and difference signals through orthogonal separation of the dual output ends; and eliminates the problems of memory access and bus contention in traditional electronic computing through optical domain integrated design.

[0117] In some optional embodiments, the mixed optical signal is:

[0118]

[0119] in, is the optical signal, j is the imaginary unit, is the intermediate optical signal, For The intermediate optical signal of the same wavelength, Input optical signals with the same wavelength to the same location and The element at position i.

[0120] Specifically, for the input optical signal and Same position with the same wavelength , after the above phase shifter and directional coupler, the output mixed optical signal is z i The results of optical matrix operations are converted into directly detectable physical quantities, where the sum component represents the energy superposition of the two signals, and the difference component reflects their difference. Together, the two constitute the correlation measure required by the self-attention mechanism. Through imaginary units, the sum and difference information are transmitted synchronously in a single physical channel, avoiding the resource overhead of multi-channel separation in traditional solutions. The interference effect is used to amplify the effective signal while suppressing the influence of incoherent stray light, thereby improving the signal-to-noise ratio. The complex domain linear transformation is completed at the speed of light, replacing the multi-step numerical calculations of electronic processors, significantly reducing latency and energy consumption.

[0121] This embodiment of the present invention achieves efficient calculation of the correlation parameters in the self-attention mechanism directly in the optical domain by combining complex-domain matrix transformation with coherent interference. This approach leverages the high parallelism and low power consumption of photonic computing, while also effectively improving text classification accuracy and robustness through orthogonal component separation and precise phase control.

[0122] The embodiment details the process of detecting the mixed optical signal in the above embodiment and outputting the correlation parameter representing the correlation degree between sequence elements. The output module includes a balanced photodetector, and the specific implementation of the process includes the following steps:

[0123] Step n1, detecting the optical power of two outputs in the mixed optical signal through two detection ends of the balanced photodetector.

[0124] Specifically, the two orthogonal components of the mixed optical signal are synchronously detected by two symmetrically arranged photodiodes. The common-mode noise such as ambient light interference is suppressed through a differential amplification circuit, only the optical power difference information of the two signals is retained, the optical domain interference result is converted into an electrical signal, and the optical power values of the two optical signals are extracted to provide original data for subsequent correlation calculation. The differential detection significantly improves the signal-to-noise ratio and eliminates the influence of light source intensity fluctuation and background noise; the double-end synchronous detection ensures the integrity of the sum / difference value signal and avoids information loss caused by single-end detection.

[0125] Step n2, analyzing the two optical powers to obtain a power difference, and the power difference is positively correlated with the semantic correlation degree of the corresponding element pair in the text sequence.

[0126] Specifically, the power difference of the optical domain interference is mapped to the correlation parameter required by the self-attention mechanism to establish a quantitative index of the semantic correlation degree between text elements. The linear relationship of the optical power difference directly reflects the semantic similarity between elements, and the key features can be extracted without complex algorithms.

[0127] Step n3, outputting the power difference as a correlation parameter.

[0128] Specifically, the power difference is used as a self-attention weight parameter and directly input into the subsequent neural network layer for weighted summation and classification to complete the final conversion from the optical signal to the correlation parameter, provide an interpretable numerical basis for the self-attention mechanism, complete the extraction and output of the correlation parameter at the speed of light, and avoid the memory access delay of the electronic processor.

[0129] The embodiment of the application realizes efficient conversion of the optical domain interference result to the correlation parameter through differential detection of the balanced photodetector.

[0130] In some optional embodiments, the power difference is:

[0131]

[0132] wherein, is the power difference, is the optical signal code, is the correlation parameter, The optical signals of the same wavelength are encoded.

[0133] Specifically, the optical simulation of matrix multiplication is realized by the physical phenomenon of light interference, and the inner product operation of the parameter matrix and the input vector is converted into a measurable optical power difference; the matrix multiplication in traditional electronic calculation is migrated to the optical domain, the parallelism and low delay characteristics of photonics are utilized to realize the linear algebra operation of time complexity; the core calculation can be completed only by interference and detection of optical signals, avoiding a large amount of invalid power consumption of electronic processors; the dot product result is directly reflected by the continuous change of optical power, avoiding the precision loss caused by digital quantization.

[0134] The inner product calculation of the entire vector space can be completed by a single light interference operation, which is much faster than the serial execution speed of an electronic processor; the high dynamic range of the optical signal supports the parameter matrix with a large numerical value span, avoiding overflow problems in electronic calculation, without additional analog-to-digital conversion or buffering mechanism, and the optical signal is directly mapped to the final parameter, simplifying the calculation link.

[0135] In some optional embodiments, the method further comprises: dividing the modulator array according to a preset function to obtain a first array and a second array; and respectively assigning optical signals of different wavelength ranges to the modulators in the first array and the second array.

[0136] Specifically, for example, the modulator array is divided into a coarse-grained subarray and a fine-grained subarray according to the function; the coarse-grained subarray is assigned optical signals of a long wavelength range to process sentence-level features, and the fine-grained subarray is assigned optical signals of a short wavelength range to process word-level details. Spatial division can be performed in a left-right partitioning or top-bottom stacking manner. The two arrays process features of different granularities in parallel, and the wavelength assignment matches the function. At the same time, long sequence global semantics and local key vocabulary are finely modeled, solving the precision loss problem caused by a single granularity, while maintaining the parallelism of the optical domain, realizing multi-scale feature fusion.

[0137] Through the above description of the embodiments, those skilled in the art can clearly understand that the processes according to the above embodiments can be realized by means of software and the necessary general hardware platform, of course, they can also be realized by hardware, but in many cases the former is a better embodiment.

[0138] Figure 5 A flowchart of the self-attention network construction method of the optical computing acceleration provided by the embodiments of the present application is shown in FIG. Figure 5 As shown in the figure, the embodiments of the present application also provide a self-attention network construction method of optical computing acceleration, and the flowchart includes the following steps:

[0139] Step S501: receiving an input vector and converting it into an optical signal, the input vector being a feature vector obtained by processing a text sequence.

[0140] Step S502: Perform adaptive processing on the parameter matrix in the self-attention network and perform optical domain matrix operation on the optical signal of the input vector based on the adapted parameter matrix to generate an intermediate optical signal.

[0141] Step S503: Perform wavelength matching, phase adjustment and hybrid interference on the intermediate optical signal and the input vector optical signal processed by another route input conversion module to generate a hybrid optical signal containing sum and difference information.

[0142] Step S504: Detect the hybrid optical signal to output a correlation parameter representing the correlation between sequence elements, and the correlation parameter is used to construct a self-attention mechanism to classify text.

[0143] The optical computing accelerated self-attention network construction method provided by the embodiment of the present application realizes cross-domain efficient coding by converting the text feature vector into an optical signal, performs optical domain matrix operation based on the adapted parameter matrix to take advantage of the high parallelism of photonics, accurately aligns multi-dimensional data using wavelength matching and phase control technology, directly extracts sequence element correlation using hybrid interference effect, and finally quickly outputs the correlation parameter by photoelectric conversion and interference pattern analysis. The method of the embodiment of the present application reconstructs the optical domain computing of the self-attention mechanism, significantly improves the energy efficiency ratio and real-time performance of large-scale matrix operation, and solves the problem of insufficient computing power of traditional electronic computing in long sequence processing.

[0144] In some optional embodiments, the adaptive processing on the parameter matrix in the self-attention network includes:

[0145] Analyzing the maximum and minimum values of the elements in the parameter matrix;

[0146] Linearly transforming the elements in the matrix according to the maximum and minimum values, so that the range of the transformed elements is between 0 and 1, to obtain the adapted parameter matrix.

[0147] In some optional embodiments, the optical domain matrix operation based on the adapted parameter matrix and the optical signal of the input vector to generate an intermediate optical signal includes:

[0148] Mapping the adapted parameter matrix to an operation modulation network, the operation modulation network corresponds to the modulator units of the modulator array, and the operation units in the operation modulation network correspond to the elements in the parameter matrix;

[0149] Inputting the optical signal of the input vector into the operation modulation network, so that the optical signal passes through each operation unit in row and column order;

[0150] The operation unit modulates the power of the optical signal according to the numerical value of the corresponding parameter matrix element;

[0151] The modulated optical signals are transmitted and superimposed at the output end of the operation modulation network to generate an intermediate optical signal corresponding to the multiplication operation of the parameter matrix and the input vector.

[0152] In some optional embodiments, the intermediate optical signal and the input vector optical signal processed by another input conversion module are subjected to wavelength matching, phase adjustment and hybrid interference, including:

[0153] The intermediate optical signal and the input vector optical signal processed by another input conversion module are subjected to wavelength matching, and channels with the same wavelength are allocated.

[0154] The two optical signals are subjected to phase adjustment by a preset phase offset.

[0155] The two optical signals subjected to phase adjustment are coupled to generate a hybrid optical signal including a sum value and a difference value.

[0156] In some optional embodiments, the intermediate optical signal and the input vector optical signal processed by another input conversion module are subjected to wavelength matching, and channels with the same wavelength are allocated, including:

[0157] The wavelength channels of the wavelength division multiplexing unit are configured, the number of wavelength channels is the same as the dimension of the input vector, and each channel corresponds to a different wavelength.

[0158] The elements at corresponding positions in the intermediate optical signal and the input vector optical signal are identified, and the same wavelength channel is allocated to the elements at the corresponding positions, so that the elements of the two signals share the channel for parallel transmission.

[0159] In some optional embodiments, the two optical signals are subjected to phase adjustment by a preset phase offset, including:

[0160] The interference requirement of the directional coupler is obtained, and the preset phase offset is set according to the interference requirement.

[0161] The phase shifter is associated with the transmission waveguide of the two optical signals, and the effective refractive index of the waveguide is changed by applying voltage or controlling temperature.

[0162] The phase difference of the two optical signals is monitored to obtain a real-time phase difference.

[0163] The real-time phase difference is compared with the preset phase offset.

[0164] When the deviation between the real-time phase difference and the preset phase offset is within a preset tolerance range, the current driving parameter of the phase shifter is maintained to maintain a stable phase difference.

[0165] When the deviation of the real-time phase difference from the preset phase offset is not within the preset tolerance range, the step of changing the effective refractive index of the waveguide by applying a voltage or controlling temperature is repeatedly performed until the real-time phase difference is within the preset tolerance range.

[0166] In some optional embodiments, the two phase-adjusted optical signals are coupled to generate a mixed optical signal including a sum value and a difference value, including:

[0167] The two phase-adjusted optical signals are respectively input into two input ends of a directional coupler;

[0168] Based on the mode coupling between the waveguides, the two optical signals exchange energy and interfere in the coupler;

[0169] The interference results are respectively obtained from two output ends of the coupler, and the interference result of the first output end is a sum value of the energy superposition of the two signals, and the interference result of the second output end is a difference value of the energy of the two signals;

[0170] The interference results of the first output end and the second output end are integrated to obtain a mixed optical signal including a sum value and a difference value.

[0171] In some optional embodiments, the mixed optical signal is detected to output a correlation parameter representing the correlation degree between sequence elements, including:

[0172] The optical powers of the two outputs in the mixed optical signal are detected;

[0173] The two optical powers are analyzed to obtain a power difference value, and the power difference value is positively correlated with the semantic correlation degree of the corresponding element pair in the text sequence;

[0174] The power difference value is output as the correlation parameter.

[0175] The description of the features in the embodiment of the self-attention network construction method accelerated by optical computing can refer to the related description of the embodiment of the self-attention network system accelerated by optical computing, which will not be repeated here.

[0176] Figure 6 The structure schematic diagram of the electronic device provided by the embodiment of the present application is shown in FIG. 1. Figure 6 As shown in FIG. 1, the electronic device 60 provided by the embodiment of the present application includes at least one processor 601 and a memory 602. Optionally, the electronic device 60 further includes a communication component 603. The processor 601, the memory 602 and the communication component 603 are connected through a bus.

[0177] In the specific implementation process, the at least one processor 601 executes the computer execution instructions stored in the memory 602, so that the at least one processor 601 executes the above-mentioned embodiment of the self-attention network construction method accelerated by optical computing.

[0178] The specific implementation process of the processor 601 can refer to the method embodiments described above, which have similar implementation principles and technical effects, and thus will not be described here.

[0179] In the above embodiments, it should be understood that the processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the application can be directly embodied as execution completed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0180] The memory can include a random access memory (RAM) and can also include a non-volatile memory (NVM), such as at least one disk memory.

[0181] The bus can be an industry standard architecture (ISA) bus, a peripheral component (PCI) bus, an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application does not limit to only one bus or one type of bus.

[0182] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program, and the computer program is configured to execute the steps in any of the above optical computing accelerated self-attention network construction method embodiments when running.

[0183] In an example embodiment, the above computer readable storage medium can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic or optical disk, and various computer program storage media.

[0184] Embodiments of the present application also provide a computer program product, which comprises a computer program, and the computer program realizes the steps in any of the optical computing accelerated self-attention network construction method embodiments when executed by a processor.

[0185] Embodiments of the present application also provide another computer program product, which comprises a non-volatile computer readable storage medium, and the non-volatile computer readable storage medium stores a computer program, and the computer program realizes the steps in any of the optical computing accelerated self-attention network construction method embodiments when executed by a processor.

[0186] Those skilled in the art will further appreciate that the functions of the examples described herein, including any related steps of a method, can be implemented using electronic hardware, computer software, or any combination of the two. To clearly illustrate this interchangeability of hardware and software, various examples have been described herein in terms of their functionality, which has been described generally and symbolically in flow charts. Having thus described the functionality of the examples, a person of ordinary skill in the art will appreciate that these and other examples can be implemented by a variety of methods, including methods that use hardware only, software only, or a combination of both. The various examples have been described in relation to particular embodiments, which are intended to be illustrative only and changes can be made to the embodiments described without departing from the scope of the application.

[0187] The above describes in detail a self-attention network system based on optical computing acceleration provided by the present application. The principles and implementation modes of the present application are described by applying specific examples in this paper, and the above description of the examples is only applicable to help understand the method of the present application and its core idea. It should be pointed out that for ordinary skilled in the art, some improvements and modifications can be made to the present application without departing from the principles of the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

Claims

1. A self-attention network system based on optical computing acceleration, characterized in that: include: Input conversion module, optical domain operation module, signal processing module, output module; The input conversion module is used to receive an input vector and convert it into an optical signal, wherein the input vector is a feature vector obtained by processing the text sequence; The light domain operation module includes an adaptation unit and an operation unit. The adaptation unit is used to adapt the parameter matrix in the self-attention network; the operation unit is used to perform light domain matrix operation based on the adapted parameter matrix and the light signal of the input vector to generate an intermediate light signal. The signal processing module is used to perform wavelength matching, phase adjustment and mixed interference on the intermediate optical signal and the input vector optical signal processed by the input conversion module to generate a mixed optical signal containing sum and difference information; The output module is used to detect the mixed light signal and output a correlation parameter representing the correlation between sequence elements. The correlation parameter is used to construct a self-attention mechanism to classify text.

2. The self-attention network system based on optical computing acceleration according to claim 1, characterized in that The input conversion module is a modulator array, which is composed of a plurality of modulator units connected in cascade, and the modulator units correspond to the elements of the input vector.

3. The self-attention network system based on optical computing acceleration according to claim 2, characterized in that The modulator array is a silicon-based Mach-Zehnder modulator array.

4. The self-attention network system based on optical computing acceleration according to claim 1, characterized in that The adapting process of the parameter matrix in the self-attention network includes: Analyzing the maximum and minimum values ​​of the elements in the parameter matrix; The elements in the matrix are linearly transformed according to the maximum value and the minimum value, so that the transformed elements range from 0 to 1, thereby obtaining an adapted parameter matrix.

5. The self-attention network system based on optical computing acceleration according to claim 4, characterized in that The linear transformation formula is: Among them, W qk is the weight matrix; max(W qk ) is the maximum value of the weight matrix element; min(W qk ) is the minimum value of the weight matrix element; (W qk ) it is the weight matrix W qk The element at row i and column t, where i is the row index and t is the column index, i=1,2,…d, t=1,2,…d.

6. The self-attention network system based on optical computing acceleration according to claim 2, characterized in that: The performing of an optical domain matrix operation based on the adapted parameter matrix and the optical signal of the input vector to generate an intermediate optical signal includes: Mapping the adapted parameter matrix to an operational modulation network, wherein the operational modulation network corresponds to a modulator unit of a modulator array, and the operational units in the operational modulation network correspond to elements in the parameter matrix; Input the optical signal of the input vector into the operation modulation network, so that the optical signal passes through each operation unit in row and column order; The operation unit performs power modulation on the optical signal according to the value of the corresponding parameter matrix element; The modulated optical signals are transmitted and superimposed at the output end of the operational modulation network to generate an intermediate optical signal corresponding to the multiplication operation of the parameter matrix and the input vector.

7. The self-attention network system based on optical computing acceleration according to claim 6, characterized in that The modulated optical signal is: in, is the modulated optical signal, is the optical signal of the input vector, W qk is the weight matrix.

8. The self-attention network system based on optical computing acceleration according to claim 1, characterized in that The signal processing module includes a wavelength division multiplexing unit, a phase shifter and a directional coupler; The wavelength matching, phase adjustment and mixed interference of the intermediate optical signal and the input vector optical signal processed by the input conversion module include: The wavelength division multiplexing unit matches the wavelength of the intermediate optical signal with the input vector optical signal processed by the input conversion module, thereby allocating channels with the same wavelength; The phase shifter adjusts the phases of the two optical signals by a preset phase offset; The directional coupler couples the two optical signals after phase adjustment to generate a mixed optical signal including a sum value and a difference value.

9. The self-attention network system based on optical computing acceleration according to claim 8, characterized in that The wavelength division multiplexing unit includes: A wavelength selector is used to provide independent wavelengths having the same number as the dimensions of the input vector, where the independent wavelengths are matched with the channels; A channel distributor, configured to distribute the intermediate optical signal and elements corresponding to positions in another input vector optical signal to the same wavelength channel; The waveguide array contains waveguides with the same number of wavelength channels, which are used to carry optical signals of different wavelengths for parallel transmission.

10. The self-attention network system based on optical computing acceleration according to claim 8, characterized in that The wavelength division multiplexing unit allocates channels of the same wavelength by wavelength matching the intermediate optical signal with another input vector optical signal processed by the input conversion module, including: Configuring wavelength channels of the wavelength division multiplexing unit, wherein the number of the wavelength channels is the same as the dimension of the input vector, and each channel corresponds to a different wavelength; By identifying the elements at corresponding positions in the intermediate optical signal and another input vector optical signal, the same wavelength channel is allocated to the elements at corresponding positions, so that the elements of the two signals share the channel for parallel transmission.

11. The self-attention network system based on optical computing acceleration according to claim 8, characterized in that: The phase shifter adjusts the phases of the two optical signals by a preset phase offset, including: Obtaining an interference requirement of the directional coupler, and setting a preset phase offset according to the interference requirement; Associating the phase shifter with a transmission waveguide for two optical signals, and changing the effective refractive index of the waveguide by applying voltage or controlling temperature; Monitoring the phase difference between the two optical signals to obtain a real-time phase difference; Comparing the real-time phase difference with a preset phase offset; When the deviation between the real-time phase difference and the preset phase offset is within a preset tolerance range, maintaining the current driving parameters of the phase shifter to maintain a stable phase difference; When the deviation between the real-time phase difference and the preset phase offset is not within a preset tolerance range, the step of changing the effective refractive index of the waveguide by applying voltage or controlling temperature is repeated until the real-time phase difference is within the preset tolerance range.

12. The self-attention network system based on optical computing acceleration according to claim 8, characterized in that: The directional coupler couples the two phase-adjusted optical signals to generate a mixed optical signal including a sum value and a difference value, including: The two optical signals after phase adjustment are input into the two input terminals of the directional coupler respectively; Based on the mode coupling between the waveguides, the two optical signals exchange energy and interfere with each other in the coupler; Obtain interference results from the two output ends of the coupler respectively, the interference result from the first output end is the sum of the energy of the two signals, and the interference result from the second output end is the difference between the energy of the two signals; The interference results of the first output end and the second output end are integrated to obtain a mixed optical signal including a sum value and a difference value.

13. The self-attention network system based on optical computing acceleration according to claim 12, characterized in that: The mixed optical signal is: in, is the optical signal, j is the imaginary unit, is the intermediate optical signal, For The intermediate optical signal of the same wavelength, Input optical signals with the same wavelength to the same location and The element at position i.

14. The self-attention network system based on optical computing acceleration according to claim 12, characterized in that: The output module includes a balanced photodetector, which detects the mixed optical signal and outputs a correlation parameter representing the correlation between sequence elements, including: The optical power of the two output paths in the mixed optical signal is detected by the two detection ends of the balanced photodetector; The two optical powers are analyzed to obtain a power difference, which is positively correlated with the semantic relevance of the corresponding element pairs in the text sequence; The power difference is output as a correlation parameter.

15. The self-attention network system based on optical computing acceleration according to claim 14, characterized in that: The power difference is: in, is the power difference, To encode the optical signal, For Optical signals of the same wavelength are encoded.

Images