Data processing method, device, computer program product, equipment and medium

Abstract

The present application provides a data processing method, apparatus, computer program product, device and medium, including obtaining data to be processed and inputting it into a data processing model, wherein the data processing model includes an interference gating module, an interference fine-tuning module and multiple diffraction expert modules based on a sub-wavelength unit; determining one or more data flow objects matching the data to be processed according to the interference gating module in the data processing model; performing data processing on the data to be processed to generate a first processing result according to the diffraction expert module matching the data flow object in the data processing model; and processing the first processing result according to the interference fine-tuning module in the data processing model to output a target processing result. Based on the interference-diffraction model architecture, all data processing steps are completed on the optical computing device, avoiding performance loss caused by frequent photoelectric conversion / digital-to-analog conversion, thereby improving the overall data processing speed.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a data processing method, apparatus, computer program product, equipment and medium. Background Art

[0002] With the explosive growth of data, the demand for computing power is also facing explosive growth. Traditional electronic computing methods can no longer meet the current era of high computing power demands.

[0003] As an emerging technology, optical neural network uses optical devices to simulate the operation process of neural network, and completes the calculation and extracts the required information by processing the optical signal in space, time and spectrum. The hybrid expert model reduces unnecessary calculation by dynamically selecting corresponding experts to participate in the calculation, further improving the patentability and efficiency of the model in data processing.

[0004] In order to further improve the data processing efficiency, the present application proposes a method for realizing data processing by combining optical computing mode and hybrid expert model. Summary of the invention

[0005] Based on this, it is necessary to provide a data processing method, device, computer program product, equipment and medium to address the above technical problems.

[0006] In a first aspect, the present application provides a data processing method, comprising:

[0007] Acquire the data to be processed and input it into a data processing model, wherein the data processing model includes an interference gating module based on an interferometer, an interference fine-tuning module, and a plurality of diffraction expert modules based on sub-wavelength units;

[0008] Determine one or more data flow objects matching the data to be processed according to the interference gating module in the data processing model;

[0009] According to the diffraction expert module matched with the data flow object in the data processing model, data processing is performed on the data to be processed to generate a first processing result;

[0010] The first processing result is processed according to the interference fine-tuning module in the data processing model to output a target processing result.

[0011] In some embodiments, determining one or more data flow objects matching the data to be processed according to the interference gating module in the data processing model includes:

[0012] Determining a plurality of unitary rotation matrices based on first parameters and second parameters of a plurality of interferometers in an interferometric gating module;

[0013] Determine an output vector according to the unitary rotation matrix and the data to be processed;

[0014] Determine the data flow object based on the output vector and the preset threshold.

[0015] In some embodiments, determining the data flow object according to the output vector and a preset threshold includes:

[0016] Compare the output vector with a preset threshold;

[0017] In response to detecting that there is a first vector greater than a preset threshold in the output vector, it is determined that the data flow object is a diffraction expert module that matches the first vector.

[0018] In some embodiments, after comparing the output vector with a preset threshold, the method includes:

[0019] In response to the presence of a first vector greater than a preset threshold in the undetected output vector, a model abnormality alarm is triggered.

[0020] In some embodiments, the data processing model is deployed on an optical computing device, and the hardware configuration process of the data processing model before deployment includes:

[0021] Set up the data processing model architecture;

[0022] Pre-training multiple diffraction expert modules included in the data processing model architecture to determine target specification parameters of the diffraction expert modules;

[0023] Characterize the sub-wavelength units on the optical diffraction device that match the diffraction expert module according to the target specification parameters;

[0024] Determine, according to a preset gating matrix, a first parameter and a second parameter of an interferometer of an interferometric gating module included in the data processing model architecture;

[0025] adjusting an adjustable phase shifter of an interferometer constituting an interferometric gating module according to the first parameter and the second parameter;

[0026] According to a preset operation matrix, determining a third parameter and a fourth parameter matched by an interference fine-tuning module included in the data processing model architecture;

[0027] The adjustable phase shifter of the interferometer constituting the interference fine-tuning module is adjusted according to the third parameter and the fourth parameter.

[0028] In some embodiments, a plurality of diffraction expert modules included in the data processing model architecture are pre-trained to determine target specification parameters of the diffraction expert modules, including:

[0029] Inputting the training data in the expert module training set into the neural network deployed by the diffraction expert module;

[0030] Adjusting the specification parameters of the on-chip units constituting the diffraction expert module and performing simulation to obtain simulation results matching the specification parameters;

[0031] In response to detecting that the error between the simulation result and the training result in the expert module training set reaches a preset condition, the specification parameters matching the simulation result are determined as target specification parameters matching the diffraction expert module.

[0032] In some embodiments, determining the first parameter and the second parameter of the interferometer of the interference gating module included in the data processing model architecture according to the preset gating matrix includes:

[0033] Performing singular value decomposition on the gating matrix to obtain a first orthogonal matrix, a second orthogonal matrix, and a first diagonal matrix;

[0034] Determine, according to the first orthogonal matrix, first parameters and second parameters of an interferometer in a first interferometer group in an interference gating module that matches the first orthogonal matrix;

[0035] Determine, according to the second orthogonal matrix, first parameters and second parameters of an interferometer in a second interferometer group in an interference gating module that matches the second orthogonal matrix;

[0036] The first parameter and the second parameter of the interferometer in the third interferometer group matching the first diagonal matrix in the interference gating module are determined according to the first diagonal matrix.

[0037] In some embodiments, determining the third parameter and the fourth parameter matched by the interference fine-tuning module included in the data processing model architecture according to the preset operation matrix includes:

[0038] Performing singular value decomposition on the gating matrix to obtain a third orthogonal matrix, a fourth orthogonal matrix, and a second diagonal matrix;

[0039] Determine, according to the third orthogonal matrix, a third parameter and a fourth parameter of an interferometer in a fourth interferometer group in the interference gating module that matches the third orthogonal matrix;

[0040] Determine, according to the second orthogonal matrix, a third parameter and a fourth parameter of an interferometer in a fifth interferometer group in the interference gating module that matches the second orthogonal matrix;

[0041] The third parameter and the fourth parameter of the interferometer in the sixth interferometer group matching the second diagonal matrix in the interference gating module are determined according to the second diagonal matrix.

[0042] In some embodiments, determining first parameters and second parameters of an interferometer in a first interferometer group matching the first orthogonal matrix in an interference gating module according to the first orthogonal matrix includes:

[0043] Determine first parameters and second parameters of the interferometers in the first interferometer group matched by the first orthogonal matrix according to the first preset formula and the second preset formula;

[0044] Determine first parameters and second parameters of interferometers in the first interferometer group matched by the first orthogonal matrix according to a first preset formula and a second preset formula;

[0045] The first preset formula is:

[0046] ;

[0047] The second preset formula is: ;

[0048] in, , , represents the first orthogonal matrix, is a unit imaginary number; and Both represent the amplitude coefficient, represents the phase angle, Represents the first parameter, Indicates the second parameter.

[0049] In some embodiments, determining first parameters and second parameters of an interferometer in a second interferometer group matching a second orthogonal matrix in an interference gating module according to the first orthogonal matrix includes:

[0050] Determine first parameters and second parameters of the interferometers in the second interferometer group matched by the first orthogonal matrix according to the third preset formula and the fourth preset formula;

[0051] Determine first parameters and second parameters of the interferometers in the second interferometer group matched with the first orthogonal matrix according to a third preset formula and a fourth preset formula;

[0052] The third preset formula is:

[0053] ;

[0054] The fourth preset formula is: ;

[0055] in, , , represents the first orthogonal matrix, is a unit imaginary number, and represents the amplitude coefficient, represents the phase angle, Represents the first parameter, Indicates the second parameter.

[0056] In some embodiments, determining first parameters and second parameters of an interferometer in a third interferometer group matching the first diagonal matrix in an interferometric gating module according to the first diagonal matrix includes:

[0057] determining the number of interferometers in the third interferometer group according to the number of eigenvalues ​​in the first diagonal matrix;

[0058] A first parameter of an interferometer in the third interferometer group is determined according to the eigenvalues ​​in the first diagonal matrix, and a second parameter of an interferometer in the third interferometer group is determined to be zero.

[0059] In some embodiments, the data processing model architecture includes a first model architecture and a second model architecture;

[0060] The first model architecture includes an interference gating module, a plurality of diffraction expert modules and a plurality of interference fine-tuning modules, wherein the number of the interference fine-tuning modules matches the number of the diffraction expert modules;

[0061] The second model architecture includes an interferometric gating module, multiple diffraction expert modules, and an interferometric fine-tuning module.

[0062] In some embodiments, after adjusting the adjustable phase shifter of the interferometer constituting the interferometric gating module according to the first parameter and the second parameter, the method further comprises:

[0063] obtaining a plurality of linear waveguides at the output of the interferometric gating module;

[0064] Randomly determine a first linear waveguide and a second linear waveguide in the linear waveguides belonging to the same interferometer;

[0065] The first linear waveguide is connected to the tapered input waveguide.

[0066] In some embodiments, after preparing the sub-wavelength unit on the optical diffraction device that matches the diffraction expert module according to the target specification parameters, the method further includes:

[0067] Acquire the light signal intensity of each area portion of the output area of ​​the light diffraction device;

[0068] The area where the intensity of the screening light signal is higher than the preset intensity and is continuous is the target output area;

[0069] Connect the target output region to the tapered output waveguide.

[0070] In some embodiments, the training process of the data processing model before deployment includes:

[0071] Inputting historical training data into the data processing model to generate a first training result;

[0072] Calculate the loss function value according to the first training result, the historical real result and the loss function;

[0073] In response to detecting that the loss function value is greater than or equal to a preset loss value or the number of iterations is less than a preset number, adjusting the interference fine-tuning module and the interference gating module in the data processing model;

[0074] Inputting historical training data into the adjusted data processing model to generate a second training result, and calculating a second loss function value according to the second training result, the historical real result and the loss function;

[0075] Repeat the step of obtaining the second loss function value until the second loss function value is less than the preset loss value or the number of iterations is equal to the preset number.

[0076] In some embodiments, adjusting the interference fine-tuning module and the interference gating module in the data processing model includes:

[0077] calculating first and second gradients of the loss function with respect to first and second parameters of the interference gating module;

[0078] calculating a third gradient and a fourth gradient of the loss function with respect to a third parameter and a fourth parameter of the interference fine-tuning module;

[0079] The first parameter, the second parameter, the third parameter and the fourth parameter of the data processing model are adjusted according to the first gradient, the second gradient, the third gradient and the fourth gradient and a preset algorithm.

[0080] In a second aspect, a data processing device is provided, the device comprising:

[0081] An interference gating module, used for receiving the data to be processed and determining one or more data flow direction objects matching the data to be processed;

[0082] A plurality of diffraction expert modules, used for performing data processing on the data to be processed to generate a first processing result;

[0083] The interference fine-tuning module is used to process the first processing result to output a target processing result.

[0084] In some embodiments, the interference gating module is an interferometer network constructed by cascading multiple interferometers;

[0085] The interference gating module is connected to a receiving area in a surface waveguide of an optical diffraction device through a tapered gradient input waveguide, wherein the optical diffraction device includes a plurality of diffraction expert modules.

[0086] In some embodiments, the interference fine-tuning module is an interferometer network constructed by cascading multiple interferometers;

[0087] The interference fine-tuning module is connected to the output region in the surface waveguide of the optical diffraction device through a tapered gradient output waveguide.

[0088] In a third aspect, the present application also provides a data processing system, comprising:

[0089] An input module, used to obtain data to be processed and input it into a data processing model, wherein the data processing model includes an interference gating module built based on an interferometer, an interference fine-tuning module, and multiple diffraction expert modules built based on sub-wavelength units;

[0090] A gating module, used to determine one or more data flow objects matching the data to be processed according to the interference gating module in the data processing model;

[0091] A calculation module, used for performing data processing on the data to be processed to generate a first processing result according to a diffraction expert module matched with the data flow object in the data processing model;

[0092] The fine-tuning module is used to process the first processing result according to the interference fine-tuning module in the data processing model to output a target processing result.

[0093] In a fourth aspect, the present application provides a computer program product, which implements the steps of the following method when the computer program is executed by a processor:

[0094] Acquire the data to be processed and input it into a data processing model, wherein the data processing model includes an interference gating module based on an interferometer, an interference fine-tuning module, and a plurality of diffraction expert modules based on sub-wavelength units;

[0095] Determine one or more data flow objects matching the data to be processed according to the interference gating module in the data processing model;

[0096] According to the diffraction expert module matched with the data flow object in the data processing model, data processing is performed on the data to be processed to generate a first processing result;

[0097] The first processing result is processed according to the interference fine-tuning module in the data processing model to output a target processing result.

[0098] In a fifth aspect, the present application provides an electronic device, the electronic device comprising: one or more processors;

[0099] and a memory associated with one or more processors, the memory being used to store program instructions, which, when read and executed by the one or more processors, perform the following operations:

[0100] Acquire the data to be processed and input it into a data processing model, wherein the data processing model includes an interference gating module based on an interferometer, an interference fine-tuning module, and a plurality of diffraction expert modules based on sub-wavelength units;

[0101] Determine one or more data flow objects matching the data to be processed according to the interference gating module in the data processing model;

[0102] According to the diffraction expert module matched with the data flow object in the data processing model, data processing is performed on the data to be processed to generate a first processing result;

[0103] The first processing result is processed according to the interference fine-tuning module in the data processing model to output a target processing result.

[0104] In a sixth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, and the computer program enables a computer to perform the following operations:

[0105] Acquire the data to be processed and input it into a data processing model, wherein the data processing model includes an interference gating module based on an interferometer, an interference fine-tuning module, and a plurality of diffraction expert modules based on sub-wavelength units;

[0106] Determine one or more data flow objects matching the data to be processed according to the interference gating module in the data processing model;

[0107] According to the diffraction expert module matched with the data flow object in the data processing model, data processing is performed on the data to be processed to generate a first processing result;

[0108] The first processing result is processed according to the interference fine-tuning module in the data processing model to output a target processing result.

[0109] The beneficial effects achieved by this application are:

[0110] The present application provides a data processing method, including obtaining data to be processed and inputting it into a data processing model, wherein the data processing model includes an interference gating module, an interference fine-tuning module, and multiple diffraction expert modules based on a sub-wavelength unit; determining one or more data flow objects matching the data to be processed according to the interference gating module in the data processing model; performing data processing on the data to be processed to generate a first processing result according to the diffraction expert module matching the data flow object in the data processing model; and processing the first processing result according to the interference fine-tuning module in the data processing model to output a target processing result. Based on the interference-diffraction model architecture, all data processing steps are completed on the optical computing device, avoiding performance loss caused by frequent photoelectric conversion / digital-to-analog conversion, thereby improving the overall data processing speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0111] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work, among which:

[0112] Figure 1 is a flow chart of a data processing method provided in an embodiment of the present application;

[0113] Figure 2 It is a schematic diagram of the basic structure of an interferometer provided in an embodiment of the present application;

[0114] Figure 3 It is a hardware schematic diagram of a data processing model provided in an embodiment of the present application;

[0115] Figure 4 It is a schematic diagram of a data processing model hardware loading method provided in an embodiment of the present application;

[0116] Figure 5 It is a data processing model architecture topology diagram provided in an embodiment of the present application;

[0117] Figure 6 It is another data processing model architecture topology diagram provided in an embodiment of the present application;

[0118] Figure 7 is a data processing device architecture diagram provided in an embodiment of the present application;

[0119] Figure 8 It is a data processing system architecture diagram provided by an embodiment of the present application;

[0120] Fig. 9 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0121] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0122] It should be understood that in the description of the present application, unless the context clearly requires otherwise, words such as "include", "comprises", and the like in the entire specification and claims should be interpreted as inclusive rather than exclusive or exhaustive; that is, the meaning of "including but not limited to".

[0123] It should also be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of this application, unless otherwise specified, "plurality" means two or more.

[0124] It should be noted that the terms "S1", "S2", etc. are only used for the purpose of describing the steps, and do not specifically refer to the order or sequence, nor are they used to limit the present application. They are only for the convenience of describing the method of the present application, and cannot be understood as indicating the order of the steps. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in this field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present application.

[0125] As mentioned in the background technology, in the context of explosive growth in computing power demand, optical neural networks (ONN) as an emerging technology have shown their unique advantages. Optical neural networks use optical devices to simulate the computing process of neural networks, and complete calculations and extract the required information by processing optical signals in spatial, temporal and spectral dimensions. Compared with traditional electronic computing, optical neural networks have shown the advantages of high bandwidth and high connectivity, high computing performance and low power consumption when processing complex computing tasks, which makes them have great potential in the era of high computing power demand. However, the current scalability of optical neural networks is poor, and the actual computing power is difficult to compete with the most advanced electronic chips. The application of optical computing in real scenarios faces challenges, lacks computing cases that efficiently combine algorithms and hardware, and has difficulties in implementation. Therefore, it is very important to give full play to the advantages of optical computing and design an optical neural network with hardware efficiency and application value in the era of information explosion.

[0126] Mixture of Experts (MoE) is a neural network that decomposes a large neural network into multiple small expert networks, each of which is responsible for processing a subset of the input data; then the outputs of each expert network are combined to form the final prediction result. By dynamically selecting the most suitable expert to participate in the calculation, unnecessary calculations can be reduced, thereby improving the professionalism and efficiency of the model. This sparse model component allows the model to maintain efficient operation while expanding the parameter scale, which greatly alleviates the computing pressure. However, the introduction of the MoE model layer also brings additional communication overhead, and it is necessary to add a many-to-many (All-to-All) communication operation before and after the MoE model layer, which brings higher requirements for hardware interconnection topology and communication bandwidth; the communication bandwidth and other problems faced by MoE happen to be the advantages of optical computing. However, there is currently no efficient optical computing solution to implement the MoE system to achieve rapid data processing, and related technologies are still blank. Therefore, the present application provides a data processing method based on an all-optical integrated data processing model to achieve efficient data processing.

[0127] It is understandable that the optical device-based data processing method provided in this application can be packaged into a software and hardware collaborative module and integrated into the server. In some specific application scenarios, it can be used to implement multimodal language processing tasks, including but not limited to expert question and answer, code generation, and intelligent customer service; in other specific application scenarios, it can also be used to implement image recognition and computer vision tasks, including but not limited to image classification, object detection, and image generation; in some specific application scenarios, it can also be used to process personalized recommendation tasks. This application does not limit the specific application scenarios.

[0128] Embodiment 1

[0129] The present application embodiment provides a data processing method, specifically, Figure 1 As shown, the present application constructs a data processing model based on an optical device, and uses the trained data processing model to further realize efficient data processing. It should be noted that in the present application, the data processing model is mainly a data processing model constructed with a hybrid expert module architecture. Based on the interference gating module, it is determined to which diffraction expert module the data to be processed of the input model flows to perform calculation processing to generate a first processing result. The diffraction expert module is used to implement complex task calculations on the data to be processed and ensure that the model obtains higher performance. Then, the calculated first processing result is further input into the interference fine-tuning module to further process the first processing result to obtain the final output, which specifically includes:

[0130] S1. Obtain the data to be processed and input it into a data processing model. The data processing model includes an interference gating module based on an interferometer, an interference fine-tuning module, and multiple diffraction expert modules based on sub-wavelength units.

[0131] It can be understood that the data processing model can handle a variety of tasks, such as multimodal language processing tasks, image recognition tasks, computer vision tasks, and personalized recommendation tasks; the above-mentioned data to be processed can be one or more of text data, image data, and video data. For example, in the image-text matching task in the multimodal language processing task, the data to be processed can be product pictures; when processing expert question tasks, the data to be processed can be natural language text, including but not limited to written language content in the form of sentences, paragraphs, and articles.

[0132] S2. Determine one or more data flow objects that match the data to be processed according to the interference gating module in the data processing model.

[0133] Specifically, multiple unitary rotation matrices are determined based on the first parameters and second parameters of multiple interferometers in the interference gating module; the output vector is determined based on the unitary rotation matrix and the data to be processed; the data flow object is determined based on the output vector and a preset threshold, and one data flow object corresponds to one diffraction expert module.

[0134] In a specific implementation scenario, the present application can construct an interference gating module based on Mach-Zehnder interferometer (MZI), which is composed of a cascade of multiple Mach-Zehnder interferometers to form an interferometer network; MZI is a commonly used silicon photonic device and also the smallest matrix core. The matrix network constructed by cascading MZI according to specific rules can perform arbitrary matrix multiplication. It can be understood that, if Figure 2 As shown in the figure, MZI consists of two couplers and two groups of interference arms. The interference arms are waveguides in which light propagates. One of the interference arms in each group is equipped with an adjustable phase shifter to provide (i.e. the first parameter, representing the beam splitter angle) and (i.e. the second parameter, indicating the phase difference) phase shift, such as Figure 2 As shown in the figure, the black rectangle is the coupler and the white rectangle is the adjustable phase shifter. and , you can change , realizing any unitary rotation matrix. The phase modulation is realized by external electrical signals. Currently, there are two commonly used adjustable phase shifters: electro-optical phase shifter and thermo-optical phase shifter. That is, the phase is adjusted by adjusting the input voltage or the temperature of the device through electrical signals. and of adjustment.

[0135] Each MZI interferometer has two inputs and two outputs, and the input-output relationship is represented by a unitary rotation matrix. It can be expressed by the following formula:

[0136] ;

[0137] Understandably, , In1 and In2 represent two data inputs, Out1 and Out2 represent two data outputs, Represents an imaginary number.

[0138] Therefore, the data to be processed is processed by multiple interferometers after being input into the interference gating module in the data processing model, that is, it is processed by multiple unitary rotation matrices. Transformation to generate an output vector. Then, by comparing the output vector and the preset threshold, the data flow object of the data to be processed is determined; it can be understood that the output vector of the interference gating module can be expressed as G=[G1, G2…Gn], and the elements in the vector correspond one-to-one to the diffraction expert module. When the elements in the vector are greater than the preset threshold, the data to be processed is output from the interference gating module and input to the corresponding diffraction expert module. If the elements in the vector are less than or equal to the preset threshold, the corresponding data to be processed is not output to the corresponding diffraction expert module. Of course, if there is no element greater than the preset threshold in the output vector of the entire interference gating module, that is, no output is generated in the entire interference gating module, then the interference gating module can be deemed abnormal, triggering a model abnormality alarm to prompt the staff to troubleshoot the model. The specific alarm form is not limited in this application.

[0139] Among them, the above preset threshold is preferably set to 0.5. Of course, in the specific implementation scenario, the technical personnel in this field will make adaptive adjustments, and this application does not limit this. This application uses an interferometer to make decisions on data streams. Based on the strong adjustability of optical interference calculation, the adaptability of the entire data processing model is achieved, thereby enhancing the scope of application in the data processing process. Through the interference gating module, a suitable diffraction expert module is selected for calculation, further improving the efficiency of the entire data processing process.

[0140] Of course, the interference gating module of the data processing model used to construct the data processing method disclosed in this application is not limited to the Mach-Zehnder interferometer, but can also be a Michelson interferometer, a Sagnac interferometer, a Fabry-Perot interferometer, etc., which can be achieved by making adaptive changes in the specific implementation scenario. This application does not limit the type of interferometer.

[0141] The interference gating module is an interferometer network constructed by cascading multiple interferometers; the interference gating module is connected to the receiving area in the surface waveguide of the optical diffraction device through a tapered gradient input waveguide, wherein the optical diffraction device includes multiple diffraction expert modules. Specifically, Figure 3 As shown, the output optical signal of the interference gating module is connected to the end face of the optical diffraction device through a tapered gradient input waveguide. Preferably, in some implementation scenarios, the tapered gradient input waveguide is gradually extended in the form of a linear function; wherein the tapered gradient input waveguide is as shown in FIG. Figure 3 As shown in the dotted box on the left, the width of the tapered gradient input waveguide gradually widens with the transmission distance. Generally, the width of the tapered gradient input waveguide gradually narrows at a rate of change of 40nm / um. The specific rate of change can be adjusted according to the process size and requirements, and this application does not limit this. It is worth noting that the effective refractive index of the end face of the optical diffraction device matches the end face of the tapered gradient input waveguide, thereby achieving coupling with extremely low loss and reducing the construction cost. This application realizes the connection between the interference gating module and the optical diffraction device through a linear gradient input waveguide, and further realizes the data transmission of the interference gating module and the diffraction expert module. The installation cost is low and the installation is simple, which improves the generalizability of the entire data processing model to a certain extent.

[0142] S3. Process the data to be processed according to the diffraction expert module in the data processing model that matches the data flow object to generate a first processing result.

[0143] In a specific implementation scenario, this application uses a sub-wavelength unit (SWU) to construct a diffraction expert module. It can be understood that one sub-wavelength unit corresponds to one diffraction expert module, that is, one diffraction expert module is a separate neural network, which can be any of the neural networks such as a fully linked neural network (FFN, Feed Forward Network), a convolutional neural network (CNN, Convolutional Neural Network) and a recurrent neural network (RNN, Recurrent Neural Network). This application does not limit the type of neural network in the specific diffraction expert module. Multiple diffraction expert modules are integrated in an optical diffraction device to form a diffraction expert array. As a micro-nano processing structure, the common process of the sub-wavelength unit is to hollow out the silicon dioxide layer to form multiple silicon substrate strips. Its function is similar to that of an interferometer. The phase adjustment function is achieved by adjusting the specifications of the sub-wavelength unit, thereby realizing complex matrix multiplication.

[0144] The idea of ​​diffraction calculation is different from that of interference calculation. It is not limited by linear waveguide. Figure 3The optical diffraction device in the demonstration architecture includes a surface waveguide, and the optical signal can propagate on the entire surface. Therefore, the input data to be processed do not need to interact with each other through a coupler like MZI, but can interact directly through diffraction.

[0145] S4. Process the first processing result according to the interference fine-tuning module in the data processing model to output a target processing result.

[0146] It can be understood that the structure of the interference fine-tuning module is similar to that of the interference gating module, and is connected to the optical diffraction device through a tapered gradient output waveguide, wherein the width of the tapered gradient output waveguide gradually narrows with the transmission distance, and the tapered gradient output waveguide and the tapered gradient input waveguide are both tapered gradient waveguides.

[0147] Specifically, the interference fine-tuning module is an interferometer network constructed by cascading multiple interferometers; the interference fine-tuning module is connected to the output area in the surface waveguide of the optical diffraction device through a tapered gradient output waveguide. In the present application, the multiplication and addition operations required for the first processing result are determined according to the specific task, and the first processing result is converted into a corresponding operation matrix, which is finally converted into the third parameter and the fourth parameter of each interferometer in the interference fine-tuning module after singular value decomposition, so as to realize the secondary operation of the first processing result output by the diffraction expert module by the interference fine-tuning module to output the final target processing result.

[0148] It is understandable that, except for the stage of inputting data to be processed and the stage of reading target processing results, the entire data processing process can be completed entirely on an optical computing device, such as an optical computing chip, and the specific carrier type itself is not limited. Therefore, there will be no performance loss due to frequent photoelectric conversion / digital-to-analog conversion, thereby improving the overall data processing speed.

[0149] In specific implementation scenarios, such as Figure 4 As shown, the hardware configuration process of the data processing model provided by the present application before being deployed on the optical computing device specifically includes:

[0150] X1. Set the data processing model architecture.

[0151] First, the data processing model is initialized. First, the specific data processing model architecture is determined. The data processing model architecture includes two types of model architectures, namely the first model architecture and the second model architecture. Figure 5In the topological diagram shown, the dotted box represents the interference module, the solid box represents the diffraction module, the first column is the interference gating module, the second column is the diffraction expert module, and the third column is the interference fine-tuning module. The first model architecture includes an interference gating module, multiple diffraction expert modules and multiple interference fine-tuning modules. The number of interference fine-tuning modules matches the number of diffraction expert modules, that is, each diffraction expert module is connected to a separate interference fine-tuning module. Figure 6 As shown in the topology diagram, the second model architecture includes an interference gating module, multiple diffraction expert modules and an interference fine-tuning module. It should be noted that in the second model architecture, all outputs of the diffraction album module are processed by an interference fine-tuning module. When constructing a specific model, a specific data processing model architecture can be selected according to the specific application scenario. The advantage of the first model architecture is that it has strong fine-tuning performance, but it has high requirements for device integration. The advantage of the second model architecture is that it is easy to implement, but the fine-tuning performance is reduced. After determining the data processing model architecture, it is also necessary to set the number of diffraction expert modules in the model and the type of neural network integrated with the diffraction expert modules.

[0152] X2. Pre-train multiple diffraction expert modules included in the data processing model architecture to determine the target specification parameters of the diffraction expert modules.

[0153] The diffraction expert module constructed based on the sub-wavelength unit in this application cannot be modified once it is engraved on the optical diffraction device. Therefore, it is necessary to simulate and train each diffraction expert module in a simulation environment to determine the optimal target specification parameters. After determining that each diffraction expert module has completed pre-training, the next step will be entered.

[0154] Specifically, the training data in the expert module training set is input to the neural network deployed by the diffraction expert module. It can be understood that the above expert module training set is a training data set that matches the neural network on the diffraction expert module, and one diffraction expert module corresponds to one training set; in the simulation training, the specification parameters of the on-chip units constituting the diffraction expert module are adjusted and simulated to obtain simulation results that match the specification parameters; in response to detecting that the error between the simulation result and the training result in the expert module training set reaches a preset condition, the specification parameters that match the simulation result are determined to be the target specification parameters that match the diffraction expert module. The above specification parameters include the length, width, thickness and position of the sub-wavelength unit. The above preset condition can preferably be set to 0.5%, which is set by a person skilled in the art according to the actual network accuracy, and this application does not limit this.

[0155] X3. Prepare sub-wavelength units on the optical diffraction device that match the diffraction expert module according to the target specification parameters.

[0156] It can be understood that by determining the target specification parameters of the sub-wavelength unit matched by each diffraction expert module, after determining the length, width and thickness of each sub-wavelength unit during characterization, the spacing between each unit in the array composed of the entire sub-wavelength unit can also be determined according to the position of each sub-wavelength unit.

[0157] X4. Determine the first parameter and the second parameter of the interferometer of the interference gating module included in the data processing model architecture according to the preset gating matrix.

[0158] It can be understood that the gating matrix is ​​a transformation matrix corresponding to a gating module pre-trained in a simulation environment, and can be determined through simulation training, which is not elaborated in this application.

[0159] The specific method for determining the first parameter and the second parameter includes: performing singular value decomposition on the gating matrix to obtain a first orthogonal matrix, a second orthogonal matrix and a first diagonal matrix. Specifically, the gating matrix is ​​defined as matrix A, and then calculated and , and then respectively ask and The eigenvalues ​​and eigenvectors of For example, solve To get all the eigenvalues ; Then the solved eigenvalues Substitution , by solving the homogeneous linear equations to obtain each eigenvalue The corresponding eigenvector , and then form a first orthogonal matrix U; the method for obtaining the second orthogonal matrix V is the same as the first orthogonal matrix U, and this application will not be expanded again; by calculating each eigenvalue Of course, the above eigenvalues ​​can also be obtained by Schmidt orthogonalization method, or by using orthogonal triangular decomposition algorithm, Jacobi algorithm, etc. to obtain matrix eigenvalues, which will not be described in detail in this application.

[0160] Determine the first parameter and the second parameter of the interferometer in the first interferometer group that matches the first orthogonal matrix in the interference gating module according to the first orthogonal matrix; determine the first parameter and the second parameter of the interferometer in the second interferometer group that matches the second orthogonal matrix in the interference gating module according to the second orthogonal matrix; determine the first parameter and the second parameter of the interferometer in the third interferometer group that matches the first diagonal matrix in the interference gating module according to the first diagonal matrix. That is, the interferometers constituting the interference gating module in the present application can be divided into three interferometer groups according to the first orthogonal matrix, the second orthogonal matrix and the first diagonal matrix. In the present application, the first parameter and the second parameter of the interferometer in each interferometer group are directly determined according to the gating matrix obtained in the simulation training process, thereby realizing the hardware preparation of the interference gating module.

[0161] Specifically, the above-mentioned determining the first parameter and the second parameter of the interferometer in the first interferometer group matching the first orthogonal matrix in the interference gating module according to the first orthogonal matrix includes: determining the first parameter and the second parameter of the interferometer in the first interferometer group matching the first orthogonal matrix according to the first preset formula and the second preset formula, that is, performing block diagonal processing on the first orthogonal matrix; the first preset formula is:

[0162] ;

[0163] The second preset formula is: ;

[0164] in, , , represents the first orthogonal matrix, is a unit imaginary number; and Both represent the amplitude coefficient, represents the phase angle, Represents the first parameter, represents the second parameter; wherein, it can be understood that, is the element at row n and column j, is the element in the jth row and nth column, and the element in the nth row and nth column is represented as , which can also be expressed as ,Right now Equivalent to , among which it is worth mentioning that in " is used to distinguish the elements in the nth row , no actual operational meaning.

[0165] represents the first orthogonal matrix, is a unit imaginary number, represents the amplitude coefficient, represents the phase angle, Represents the first parameter, Indicates the second parameter.

[0166] In the specific calculation, for the matrix U(n), according to the formula Available and , and then substitute into the second line of the above formula Obtain and , and so on, we get all the parameters of this group of n-1 MZIs, namely The corresponding MZI parameters. Then the matrix U(n-1) can be block diagonalized to obtain all the parameters of this group of n-2 MZIs, that is, The corresponding MZI parameters, where , and so on until the matrix U(1) is calculated, thereby obtaining the parameters of all interferometers in the first interferometer group corresponding to the first orthogonal matrix.

[0167] Specifically, the first parameter and the second parameter of the interferometer in the second interferometer group matched by the second orthogonal matrix are determined according to the third preset formula and the fourth preset formula;

[0168] The third preset formula is:

[0169] ;

[0170] The fourth preset formula is: ;

[0171] in, , represents the first orthogonal matrix, is a unit imaginary number, and represents the amplitude coefficient, represents the phase angle, Represents the first parameter, Represents the second parameter, is the element at row m and column k, is the element in the kth row and mth column, and the element in the mth row and mth column is represented as , which can also be expressed as ,Right now Equivalent to , among which it is worth mentioning that in " is used to distinguish the elements in the mth row , no actual operational meaning.

[0172] Similarly, the specific calculation process of the parameters of all interferometers in the second interferometer group corresponding to the second orthogonal matrix is ​​consistent with the calculation process of the first interferometer group, and this application will not repeat it here.

[0173] The above-mentioned determining the first parameter and the second parameter of the interferometer in the third interferometer group matching the first diagonal matrix in the interference gating module according to the first diagonal matrix includes: determining the number of interferometers in the third interferometer group according to the number of eigenvalues ​​in the first diagonal matrix, that is, determining the first parameter of the interferometer in the third interferometer group according to the eigenvalues ​​in the first diagonal matrix, and determining that the second parameter of the interferometer in the third interferometer group is zero. Specifically, for the first diagonal matrix , whose diagonal elements can directly correspond to the n-1 first parameters of the n-1 MZI parameters , where the i-th MZI , , where the first diagonal matrix It can be represented as the following matrix:

[0174] .

[0175] X5. Adjust the adjustable phase shifter of the interferometer constituting the interference gating module according to the first parameter and the second parameter.

[0176] After adjusting the adjustable phase shifter of the interferometer constituting the interference gating module according to the first parameter and the second parameter, it is also necessary to obtain multiple linear waveguides at the output end of the interference gating module; randomly determine the first linear waveguide and the second linear waveguide among the linear waveguides belonging to the same interferometer, that is, randomly select a waveguide for discarding; connect the first linear waveguide with the tapered gradient input waveguide, so as to realize the connection between the interference gating module and the optical diffraction device where the diffraction expert module is located; discard the second linear waveguide, and preferably, the discarding method may be that the second linear waveguide is not physically connected subsequently.

[0177] X6. According to the preset operation matrix, determine the third parameter and the fourth parameter of the interference fine-tuning module included in the data processing model architecture.

[0178] Specifically, according to a preset operation matrix, the third parameter and the fourth parameter matching the interference fine-tuning module included in the data processing model architecture are determined, including: performing singular value decomposition on the gating matrix to obtain a third orthogonal matrix, a fourth orthogonal matrix and a second diagonal matrix; determining the third parameter and the fourth parameter of the interferometer in the fourth interferometer group that matches the third orthogonal matrix in the interference gating module according to the third orthogonal matrix; determining the third parameter and the fourth parameter of the interferometer in the fifth interferometer group that matches the second orthogonal matrix in the interference gating module according to the second orthogonal matrix; and determining the third parameter and the fourth parameter of the interferometer in the sixth interferometer group that matches the second diagonal matrix in the interference gating module according to the second diagonal matrix.

[0179] Similar to the interference gating module, the interferometer constituting the interference fine-tuning module in the present application can be divided into three interferometer groups according to the third orthogonal matrix, the fourth orthogonal matrix and the second diagonal matrix. It can be understood that the third parameter matched by the interference fine-tuning module is essentially the same as the first parameter matched by the interference gating module, and the fourth parameter matched by the interference fine-tuning module is essentially the same as the second parameter matched by the interference gating module; the calculation process of the third parameter and the fourth parameter is the same as the first parameter and the second parameter matched by the interference gating module, and the difference is only that the transformation matrix used for decomposition is different, and the matrix that needs to be decomposed by singular value for calculating the third parameter and the fourth parameter is the operation matrix, which is a matrix representation for performing mathematical operations on the output of the diffraction expert module according to the actual processing task of the model, and the matrix that needs to be decomposed by singular value for calculating the first parameter and the second parameter is the gating matrix. Therefore, the present application will not expand the determination process of the above-mentioned third parameter and fourth parameter.

[0180] X7. Adjust the adjustable phase shifter of the interferometer constituting the interference fine-tuning module according to the third parameter and the fourth parameter.

[0181] It can be understood that the interference fine-tuning module has a similar structure to the interference gating module. After adjusting the adjustable phase shifter of the interferometer constituting the interference fine-tuning module according to the third parameter and the fourth parameter, it is also necessary to obtain the optical signal intensity of each area part of the output area of ​​the optical diffraction device; select the area part with optical signal intensity higher than the preset intensity and continuous as the target output area; connect the target output area with the tapered gradient output waveguide. Among them, the preset optical signal intensity is an intensity pre-defined before model training, which can be defined according to the optical signal intensity collected in the simulation environment, such as taking the lowest intensity value of the top m areas in the simulation environment as the preset intensity. Of course, it can also be set according to the actual situation, and the present application does not limit this. Specifically, the output part of the diffraction expert module is connected to the interference fine-tuning module through the tapered gradient output waveguide for output, such as Figure 3, through the above output design, the output of the partial area during the light diffraction period where the diffraction expert module is located is transmitted to the interference fine-tuning module. Normally, the all-optical system is linear, but the expression ability of the linear system is limited and cannot achieve the effect of the traditional neural network; optical nonlinearity can be achieved through special materials in principle, but it is difficult to achieve in the current integrated process; this application achieves nonlinear effects by designing the input and output of the optical diffraction device where the diffraction expert module is located, thereby improving the stability of the data to be transmitted in the entire data processing model.

[0182] At this point, the hardware loading of the data processing model on the optical computing device has been completed. Through the combination of the above-mentioned interference gating module, diffraction expert module and interference fine-tuning module, the diffraction-interference hybrid model architecture is realized, which can further take into account the advantages of both in terms of performance. On the one hand, the data processing model can be fine-tuned online through the interference fine-tuning module and the interference gating module, and on the other hand, the pre-trained diffraction expert module can be used to perform high-dimensional and high-parameter calculations, greatly improving the performance of the data processing model.

[0183] However, due to the accuracy and noise limitations of optical computing, as well as simulation errors, the data processing model is difficult to achieve the performance during simulation. Therefore, this application also proposes to fine-tune the data processing model of this all-optical integration online, and after the training is completed, the trained data processing model is deployed on the optical computing device for data processing, specifically including:

[0184] Input historical training data to the data processing model to obtain a first training result; wherein the historical training data can be real historical data related to the task processed by the data processing model, or data obtained in a simulation environment; calculate the loss function value according to the first training result, the historical real result and the loss function; in response to detecting that the loss function value is greater than or equal to the preset loss value or the number of iterations is less than the preset number, adjust the interference fine-tuning module and the interference gating module in the data processing model; input historical training data to the adjusted data processing model to generate a second training result, and calculate the second loss function value according to the second training result, the historical real result and the loss function; repeat the step of obtaining the second loss function value until the second loss function value is less than the preset loss value or the number of iterations is equal to the preset number. The preset loss value is set according to factors such as the model processing task type, the characteristics of the training data set and the model architecture, and is set by the staff before the model training. The above-mentioned preset number of times matching the iteration coefficient is related to the scale of the training data set and the complexity of the model, and is set by the staff before the model training.

[0185] It is understandable that in the data processing model disclosed in the present application, when adjusting the model parameters based on the loss function, only the interference gating module and the interference fine-tuning module are adjusted, and the diffraction expert module is not adjusted. Among them, the loss function is selected according to the data processing model. If the problem processed by the model is a classification problem, it can be a cross entropy loss function and a Hinge loss function, etc.; if the problem processed by the model is a regression problem, it can be a mean square error loss function and a mean absolute error function, etc.

[0186] Wherein, adjusting the interference fine-tuning module and the interference gating module in the data processing model includes: calculating the first gradient and the second gradient of the loss function relative to the first parameter and the second parameter of the interference gating module; calculating the third gradient and the fourth gradient of the loss function relative to the third parameter and the fourth parameter of the interference fine-tuning module; adjusting the first parameter, the second parameter, the third parameter and the fourth parameter of the data processing model according to the first gradient, the second gradient, the third gradient and the fourth gradient and the preset algorithm. Wherein the specific gradient calculation is a conventional technical means in this field, and this application will not expand on this. It can be understood that the above-mentioned gradients respectively indicate how the corresponding parameters should be adjusted to reduce the function value of the loss function. After obtaining the gradients of each parameter, the preset algorithm can be used to update the model parameters. The above-mentioned preset algorithm includes but is not limited to the gradient descent algorithm, the stochastic gradient descent algorithm, the momentum method, etc.

[0187] Embodiment 2

[0188] Corresponding to the above embodiment, the present application also provides a data processing device, such as Figure 7 As shown, including:

[0189] An interference gating module 710 is used to receive data to be processed and determine one or more data flow objects that match the data to be processed;

[0190] A plurality of diffraction expert modules 720, configured to perform data processing on the data to be processed to generate a first processing result;

[0191] The interference fine-tuning module 730 is used to process the first processing result to output a target processing result.

[0192] In some embodiments, the interference gating module 710 is an interferometer network constructed by cascading multiple interferometers;

[0193] The interference gating module 710 is connected to the receiving area in the surface waveguide of the optical diffraction device through a tapered gradient input waveguide, wherein the optical diffraction device 740 includes a plurality of diffraction expert modules 720.

[0194] In some embodiments, the interference fine-tuning module 730 is an interferometer network constructed by cascading multiple interferometers;

[0195] The interference fine-tuning module 730 is connected to the output region in the surface waveguide of the optical diffraction device 740 through a tapered gradient output waveguide.

[0196] Embodiment 3

[0197] Corresponding to the above embodiment, the present application also provides a data processing system, such as Figure 8 As shown, including:

[0198] An input module 810, used to obtain data to be processed and input it into a data processing model, wherein the data processing model includes an interference gating module, an interference fine-tuning module, and multiple diffraction expert modules based on sub-wavelength units.

[0199] A gating module 820, used to determine one or more data flow objects matching the data to be processed according to the interference gating module in the data processing model;

[0200] A calculation module 830, configured to perform data processing on the data to be processed to generate a first processing result according to a diffraction expert module matched with the data flow object in the data processing model;

[0201] The fine-tuning module 840 is used to process the first processing result according to the interference fine-tuning module in the data processing model to output a target processing result.

[0202] In some embodiments, the gating module is also used to determine multiple unitary rotation matrices based on the first parameters and second parameters of multiple interferometers within the interference gating module; determine the output vector based on the unitary rotation matrix and the data to be processed; and determine the data flow object based on the output vector and a preset threshold.

[0203] In some embodiments, the gating module 820 is further used to compare the output vector with a preset threshold; in response to detecting that there is a first vector greater than the preset threshold in the output vector, determine that the data flow object is a diffraction expert module matching the first vector;

[0204] In some embodiments, the gating module 820 is further configured to trigger a model abnormality alarm in response to the presence of a first vector greater than a preset threshold in the undetected output vector.

[0205] In some embodiments, the system also includes a model preparation module (not shown in the figure), which is used to perform hardware configuration of the data processing model when the data processing model is deployed on the optical computing chip, including: setting the data processing model architecture; pre-training multiple diffraction expert modules included in the data processing model architecture to determine the target specification parameters of the diffraction expert module; characterizing the sub-wavelength units on the optical diffraction device that match the diffraction expert module according to the target specification parameters; determining the first parameter and the second parameter of the interferometer of the interference gating module included in the data processing model architecture according to a preset gating matrix; adjusting the adjustable phase shifter of the interferometer constituting the interference gating module according to the first parameter and the second parameter; determining the third parameter and the fourth parameter matching the interference fine-tuning module included in the data processing model architecture according to the preset operation matrix; adjusting the adjustable phase shifter of the interferometer constituting the interference fine-tuning module according to the third parameter and the fourth parameter.

[0206] In some embodiments, the model preparation module is also used to input training data in the expert module training set into the neural network deployed by the diffraction expert module; adjust the specification parameters of the on-chip units constituting the diffraction expert module and perform simulation to obtain simulation results that match the specification parameters; in response to detecting that the error between the simulation result and the training result in the expert module training set reaches a preset condition, determine that the specification parameters that match the simulation result are the target specification parameters that match the diffraction expert module.

[0207] In some embodiments, the model preparation module is also used to perform singular value decomposition on the gating matrix to obtain a first orthogonal matrix, a second orthogonal matrix and a first diagonal matrix; determine the first parameters and second parameters of the interferometer in the first interferometer group that matches the first orthogonal matrix in the interference gating module according to the first orthogonal matrix; determine the first parameters and second parameters of the interferometer in the second interferometer group that matches the second orthogonal matrix in the interference gating module according to the second orthogonal matrix; determine the first parameters and second parameters of the interferometer in the third interferometer group that matches the first diagonal matrix in the interference gating module according to the first diagonal matrix.

[0208] In some embodiments, the model preparation module is further used to perform singular value decomposition on the gating matrix to obtain a third orthogonal matrix, a fourth orthogonal matrix, and a second diagonal matrix;

[0209] Determine, according to the third orthogonal matrix, a third parameter and a fourth parameter of an interferometer in a fourth interferometer group in the interference gating module that matches the third orthogonal matrix;

[0210] Determine, according to the second orthogonal matrix, a third parameter and a fourth parameter of an interferometer in a fifth interferometer group in the interference gating module that matches the second orthogonal matrix;

[0211] The third parameter and the fourth parameter of the interferometer in the sixth interferometer group matching the second diagonal matrix in the interference gating module are determined according to the second diagonal matrix.

[0212] In some embodiments, the model preparation module is further used to determine first parameters and second parameters of interferometers in the first interferometer group matched by the first orthogonal matrix according to the first preset formula and the second preset formula;

[0213] The first preset formula is:

[0214] ;

[0215] The second preset formula is: ;

[0216] in, , , represents the first orthogonal matrix, is a unit imaginary number; and Both represent the amplitude coefficient, represents the phase angle, Represents the first parameter, Indicates the second parameter.

[0217] In some embodiments, the first parameter and the second parameter of the interferometer in the second interferometer group matched with the first orthogonal matrix are determined according to the third preset formula and the fourth preset formula;

[0218] The third preset formula is:

[0219] ;

[0220] The fourth preset formula is: ;

[0221] in, , , represents the first orthogonal matrix, is a unit imaginary number, and represents the amplitude coefficient, represents the phase angle, Represents the first parameter, Indicates the second parameter.

[0222] In some embodiments, the model preparation module is also used to determine the number of interferometers in the third interferometer group based on the number of eigenvalues ​​in the first diagonal matrix; determine the first parameters of the interferometers in the third interferometer group based on the eigenvalues ​​in the first diagonal matrix, and determine that the second parameters of the interferometers in the third interferometer group are zero.

[0223] In some embodiments, the model preparation module is also used to input historical training data into the data processing model to generate a first training result; calculate the loss function value based on the first training result, the historical real result and the loss function; in response to detecting that the loss function value is greater than or equal to a preset loss value or the number of iterations is less than a preset number, adjust the interference fine-tuning module and the interference gating module in the data processing model; input historical training data into the adjusted data processing model to generate a second training result, and calculate the second loss function value based on the second training result, the historical real result and the loss function; repeat the step of obtaining the second loss function value until the second loss function value is less than the preset loss value or the number of iterations is equal to the preset number.

[0224] In some embodiments, the model preparation module is also used to calculate the first gradient and the second gradient of the loss function relative to the first parameter and the second parameter of the interference gating module; calculate the third gradient and the fourth gradient of the loss function relative to the third parameter and the fourth parameter of the interference fine-tuning module; and adjust the first parameter, the second parameter, the third parameter and the fourth parameter of the data processing model according to the first gradient, the second gradient, the third gradient and the fourth gradient and a preset algorithm.

[0225] Embodiment 4

[0226] Corresponding to all the above embodiments, the embodiments of the present application further provide a computer program, which implements the steps of the following method when executed by a processor:

[0227] Acquire the data to be processed and input it into a data processing model, wherein the data processing model includes an interference gating module based on an interferometer, an interference fine-tuning module, and a plurality of diffraction expert modules based on sub-wavelength units;

[0228] Determine one or more data flow objects matching the data to be processed according to the interference gating module in the data processing model;

[0229] According to the diffraction expert module matched with the data flow object in the data processing model, data processing is performed on the data to be processed to generate a first processing result;

[0230] The first processing result is processed according to the interference fine-tuning module in the data processing model to output a target processing result.

[0231] Embodiment 4

[0232] Corresponding to all the above embodiments, an embodiment of the present application provides an electronic device, including: one or more processors; and a memory associated with the one or more processors, the memory is used to store program instructions, and when the program instructions are read and executed by the one or more processors, the following operations are performed:

[0233] Acquire the data to be processed and input it into a data processing model, wherein the data processing model includes an interference gating module based on an interferometer, an interference fine-tuning module, and a plurality of diffraction expert modules based on sub-wavelength units;

[0234] Determine one or more data flow objects matching the data to be processed according to the interference gating module in the data processing model;

[0235] According to the diffraction expert module matched with the data flow object in the data processing model, data processing is performed on the data to be processed to generate a first processing result;

[0236] The first processing result is processed according to the interference fine-tuning module in the data processing model to output a target processing result.

[0237] in, Fig. 9 The electronic device architecture is shown as an example, which may include a processor 910, a video display adapter 911, a disk drive 912, an input / output interface 913, a network interface 914, and a memory 920. The processor 910, the video display adapter 911, the disk drive 912, the input / output interface 913, the network interface 914, and the memory 920 may be communicatively connected via a bus 930.

[0238] Among them, the processor 910 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more circuits, etc., to execute relevant programs to implement the technical solution provided in this application.

[0239] The memory 920 can be implemented in the form of ROM (Read Only Memory, writable memory), RAM (Random Access Memory, random access memory), static storage device, dynamic storage device, etc. The memory 920 can store an operating system 921 for controlling the execution of the electronic device 900, and a basic input and output system (BIOS) 922 for controlling the low-level operations of the electronic device 900. In addition, a web browser 923, a data storage management system 924, and an icon font processing system 925, etc. can also be stored. The above-mentioned icon font processing system 925 can be an application program that specifically implements the operations of the aforementioned steps in the embodiment of the present application. In short, when the technical solution provided in the present application is implemented by software or firmware, the relevant program code is stored in the memory 920 and is called and executed by the processor 910.

[0240] The input / output interface 913 is used to connect the input / output module to realize information input and output. The input / output module can be configured as a component in the device (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0241] The network interface 914 is used to connect to a communication module (not shown) to achieve communication interaction between the device and other devices. The communication module can achieve communication through a wired method (such as USB, network cable, etc.) or a wireless method (such as mobile network, WIFI, Bluetooth, etc.).

[0242] The bus 930 comprises a pathway for transmitting information between the various components of the device (eg, the processor 910 , the video display adapter 911 , the disk drive 912 , the input / output interface 913 , the network interface 914 , and the memory 920 ).

[0243] In addition, the electronic device 900 can also obtain information on specific collection conditions from the virtual resource object collection condition information database for use in condition judgment, etc.

[0244] It should be noted that, although the above device only shows a processor 910, a video display adapter 911, a disk drive 912, an input / output interface 913, a network interface 914, a memory 920, a bus 930, etc., in the specific implementation process, the device may also include other components necessary for normal execution. In addition, it can be understood by those skilled in the art that the above device may also only include components necessary for implementing the solution of the present application, and does not necessarily include all the components shown in the figure.

[0245] It can be seen from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution of the present application can essentially or in other words, the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., including several instructions for enabling a computer device (which can be a personal computer, a cloud service end, or a network device, etc.) to execute the methods of various embodiments of the present application or certain parts of the embodiments.

[0246] Embodiment 5

[0247] Corresponding to all the above embodiments, the embodiments of the present application further provide a computer-readable storage medium storing a computer program, which enables a computer to perform the following operations:

[0248] Acquire the data to be processed and input it into a data processing model, wherein the data processing model includes an interference gating module based on an interferometer, an interference fine-tuning module, and a plurality of diffraction expert modules based on sub-wavelength units;

[0249] Determine one or more data flow objects matching the data to be processed according to the interference gating module in the data processing model;

[0250] According to the diffraction expert module matched with the data flow object in the data processing model, data processing is performed on the data to be processed to generate a first processing result;

[0251] The first processing result is processed according to the interference fine-tuning module in the data processing model to output a target processing result.

[0252] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, in which the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0253] The above are only preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A data processing method, characterized in that: The method comprises: Acquire data to be processed and input them into a data processing model, wherein the data processing model includes an interference gating module based on an interferometer, an interference fine-tuning module, and a plurality of diffraction expert modules based on sub-wavelength units; Determine one or more data flow objects matching the data to be processed according to the interference gating module in the data processing model; According to the diffraction expert module in the data processing model that matches the data flow object, data processing is performed on the data to be processed to generate a first processing result; Processing the first processing result according to the interference fine-tuning module in the data processing model to output a target processing result; The data processing model is deployed on an optical computing device, and the hardware configuration process of the data processing model before deployment includes: Setting the data processing model architecture; Pre-training a plurality of the diffraction expert modules included in the data processing model architecture to determine target specification parameters of the diffraction expert modules; The sub-wavelength units on the optical diffraction device that match the diffraction expert module are characterized according to the target specification parameters.

2. The method according to claim 1, characterized in that The determining one or more data flow objects matching the data to be processed according to the interference gating module in the data processing model includes: Determining a plurality of unitary rotation matrices according to first parameters and second parameters of a plurality of interferometers in the interferometric gating module; Determine an output vector according to the unitary rotation matrix and the data to be processed; The data flow object is determined according to the output vector and a preset threshold.

3. The method according to claim 2, characterized in that The step of determining the data flow object according to the output vector and a preset threshold value includes: comparing the output vector with a preset threshold; In response to detecting that there is a first vector greater than the preset threshold in the output vector, determining that the data flow object is a diffraction expert module matching the first vector.

4. The method according to claim 3, characterized in that After comparing the output vector with a preset threshold, the method comprises: In response to not detecting the presence of a first vector greater than the preset threshold in the output vector, a model abnormality alarm is triggered.

5. The method according to any one of claims 1 to 4, characterized in that: The hardware configuration process of the data processing model before deployment also includes: Determining, according to a preset gating matrix, a first parameter and a second parameter of an interferometer of an interferometric gating module included in the data processing model architecture; adjusting an adjustable phase shifter of an interferometer constituting the interferometric gating module according to the first parameter and the second parameter; Determine, according to a preset operation matrix, a third parameter and a fourth parameter matched by the interference fine-tuning module included in the data processing model architecture; The adjustable phase shifter of the interferometer constituting the interference fine-tuning module is adjusted according to the third parameter and the fourth parameter.

6. The method according to claim 5, characterized in that The pre-training of the plurality of diffraction expert modules included in the data processing model architecture to determine target specification parameters of the diffraction expert modules includes: Inputting training data in the expert module training set into the neural network deployed by the diffraction expert module; Adjusting the specification parameters of the on-chip units constituting the diffraction expert module and performing simulation to obtain simulation results matching the specification parameters; In response to detecting that the error between the simulation result and the training result in the expert module training set reaches a preset condition, the specification parameters matching the simulation result are determined as target specification parameters matching the diffraction expert module.

7. The method according to claim 5, characterized in that Determining the first parameter and the second parameter of the interferometer of the interference gating module included in the data processing model architecture according to the preset gating matrix includes: Performing singular value decomposition on the gating matrix to obtain a first orthogonal matrix, a second orthogonal matrix and a first diagonal matrix; Determine, according to the first orthogonal matrix, first parameters and second parameters of an interferometer in a first interferometer group in an interferometer gating module that matches the first orthogonal matrix; Determine, according to the second orthogonal matrix, first parameters and second parameters of an interferometer in a second interferometer group in an interference gating module that matches the second orthogonal matrix; Determine first parameters and second parameters of an interferometer in a third interferometer group in an interferometer gating module that matches the first diagonal matrix according to the first diagonal matrix.

8. The method according to claim 7, characterized in that The step of determining the third parameter and the fourth parameter matched by the interference fine-tuning module included in the data processing model architecture according to the preset operation matrix includes: Performing singular value decomposition on the gating matrix to obtain a third orthogonal matrix, a fourth orthogonal matrix and a second diagonal matrix; Determine, according to the third orthogonal matrix, a third parameter and a fourth parameter of an interferometer in a fourth interferometer group in an interference gating module that matches the third orthogonal matrix; Determine, according to the second orthogonal matrix, a third parameter and a fourth parameter of an interferometer in a fifth interferometer group in an interference gating module that matches the second orthogonal matrix; The third parameter and the fourth parameter of the interferometer in the sixth interferometer group matching the second diagonal matrix in the interference gating module are determined according to the second diagonal matrix.

9. The method according to claim 7, characterized in that: The determining, according to the first orthogonal matrix, first parameters and second parameters of an interferometer in a first interferometer group in an interference gating module that matches the first orthogonal matrix comprises: Determine first parameters and second parameters of the interferometers in the first interferometer group matched by the first orthogonal matrix according to the first preset formula and the second preset formula; The first preset formula is: The second preset formula is: in, j∈[1,n-1], U(n) represents the first orthogonal matrix, i is a unit imaginary number; β n and β′ n Both represent the amplitude coefficient, α n represents the phase angle, θ j Represents the first parameter, Indicates the second parameter.

10. The method according to claim 7, characterized in that The determining, according to the first orthogonal matrix, first parameters and second parameters of an interferometer in a second interferometer group in an interference gating module that matches the second orthogonal matrix comprises: Determine first parameters and second parameters of the interferometers in the second interferometer group matched with the first orthogonal matrix according to a third preset formula and a fourth preset formula; The third preset formula is: The fourth preset formula is: in, k∈[1, m-1], V(m) represents the first orthogonal matrix, i is a unit imaginary number, β m and β′ m represents the amplitude coefficient, α m represents the phase angle, θ k Represents the first parameter, Indicates the second parameter.

11. The method according to claim 7, characterized in that Determining first parameters and second parameters of an interferometer in a third interferometer group matching the first diagonal matrix in an interference gating module according to the diagonal matrix includes: Determining the number of interferometers in the third interferometer group according to the number of eigenvalues ​​in the diagonal matrix; A first parameter of an interferometer in the third interferometer group is determined according to the eigenvalues ​​in the diagonal matrix, and a second parameter of an interferometer in the third interferometer group is determined to be zero.

12. The method according to claim 5, characterized in that The data processing model architecture includes a first model architecture and a second model architecture; The first model architecture includes an interference gating module, a plurality of diffraction expert modules and a plurality of interference fine-tuning modules, and the number of the interference fine-tuning modules matches the number of the diffraction expert modules; The second model architecture includes an interference gating module, multiple diffraction expert modules and an interference fine-tuning module.

13. The method according to claim 5, characterized in that After adjusting the adjustable phase shifter of the interferometer constituting the interferometric gating module according to the first parameter and the second parameter, the method further comprises: Acquire a plurality of linear waveguides at the output end of the interference gating module; Randomly determine a first linear waveguide and a second linear waveguide in the linear waveguides belonging to the same interferometer; The first linear waveguide is connected to the tapered gradient input waveguide to achieve connection between the interference gating module and the optical diffraction device.

14. The method according to claim 13, characterized in that After preparing the sub-wavelength unit on the optical diffraction device that matches the diffraction expert module according to the target specification parameters, the method further includes: Acquire the light signal intensity of each area portion of the output area of ​​the light diffraction device; The area where the intensity of the screening light signal is higher than the preset intensity and is continuous is the target output area; The target output region is connected to the tapered gradient output waveguide to achieve connection between the interference fine-tuning module and the light diffraction device.

15. The method according to claim 5, characterized in that The training process of the data processing model before deployment includes: Inputting historical training data into the data processing model to generate a first training result; Calculate the loss function value according to the first training result, the historical real result and the loss function; In response to detecting that the loss function value is greater than or equal to a preset loss value or the number of iterations is less than a preset number, adjusting the interference fine-tuning module and the interference gating module in the data processing model; Inputting historical training data into the adjusted data processing model to generate a second training result, and calculating a second loss function value according to the second training result, the historical real result and the loss function; Repeat the step of obtaining a second loss function value until the second loss function value is less than a preset loss value or the number of iterations is equal to a preset number.

16. The method according to claim 15, characterized in that The step of adjusting the interference fine-tuning module and the interference gating module in the data processing model includes: calculating first and second gradients of the loss function with respect to first and second parameters of the interference gating module; calculating a third gradient and a fourth gradient of the loss function with respect to a third parameter and a fourth parameter of the interference fine-tuning module; The first parameter, the second parameter, the third parameter and the fourth parameter of the data processing model are adjusted according to the first gradient, the second gradient, the third gradient and the fourth gradient and a preset algorithm.

17. A data processing device, characterized in that: The device comprises: An interference gating module, used for receiving data to be processed and determining one or more data flow direction objects matching the data to be processed; A plurality of diffraction expert modules, used for performing data processing on the data to be processed to generate a first processing result; An interference fine-tuning module, used for processing the first processing result to output a target processing result; Among them, the sub-wavelength units on the optical diffraction device and the diffraction expert module are characterized according to target rule parameters, and the target specification parameters are obtained by pre-training multiple diffraction expert modules.

18. The device according to claim 17, characterized in that The interference gating module is an interferometer network constructed by cascading multiple interferometers; The interference gating module is connected to the receiving area in the surface waveguide of the optical diffraction device through a tapered gradient input waveguide, wherein the optical diffraction device includes a plurality of the diffraction expert modules.

19. The device according to claim 17, characterized in that The interference fine-tuning module is an interferometer network constructed by cascading multiple interferometers; The interference fine-tuning module is connected to the output region in the surface waveguide of the optical diffraction device through a tapered gradient output waveguide.

20. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 16 are implemented.

21. An electronic device, characterized in that: The electronic device comprises: one or more processors; And a memory associated with the one or more processors, the memory is used to store program instructions, and when the program instructions are read and executed by the one or more processors, the method of any one of claims 1-16 is executed.

22. A computer-readable storage medium, characterized in that: The computer program stores a computer program, wherein the computer program enables a computer to execute the method according to any one of claims 1 to 16.

Images