Data processing method based on photoelectric neural network, chip, equipment and medium

Abstract

The invention relates to the technical field of artificial intelligence, and discloses a data processing method based on a photoelectric neural network, a chip, equipment and a medium. The existing problems are solved by optimizing the signal processing and transmission architecture: the mode division multiplexing technology of the orthogonal mode is adopted, the optical routing module can transmit multiple groups of data in parallel in a single route, the dependence on multiple active devices is reduced, and the area proportion of the active devices in a chip is reduced; the electro-optical conversion module intensively processes electric signal light conversion data, so that area waste caused by scattered layout is avoided, and the integration level is improved; optical data is transmitted to each calculation module through branches, parallel calculation is realized, resource allocation is optimized in combination with a task scheduling rule, and the calculation power density is improved; the modular design enables the calculation module to be expanded as required, enhances the expandability of the system, and gives full play to the advantages of high bandwidth and low loss of optical calculation.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and more specifically to data processing methods, chips, devices, and media based on photoelectric neural networks. Background Technology

[0002] In the era of big data, the massive growth of data has provided a sufficient training foundation for the development of artificial neural networks (ANNs), enabling them to achieve significant success in many fields such as adaptive control, object recognition, and natural language processing. However, the operation of large-scale artificial neural networks relies on enormous computing power, and their current main applications still depend on software simulation using von Neumann electronic computers. With the exponential growth in the number of transistors in CPUs and GPUs, Moore's Law and Dennard's scaling law are gradually becoming ineffective, electronic computing is facing bottlenecks, and the limitations of the traditional von Neumann architecture are becoming increasingly prominent. In contrast, photons have inherent advantages of high speed and high parallelism, providing potential for achieving ultra-high-speed computing. Therefore, optical computing technology, which uses light as an information carrier, has attracted widespread attention. Optical computing systems have the characteristics of high bandwidth, low energy consumption, and strong anti-interference capabilities, and are expected to break through the limitations of electronic computing, becoming an important development direction in the future computing field.

[0003] In existing technologies, various architectural schemes have been proposed for the implementation of optical computing. Existing technology one uses an optoelectronic computing unit to achieve high-speed matrix operations and analog calculations, while delegating operations unsuitable for optoelectronic computing, such as time-domain delay and data storage, to a microelectronic unit. The computing, control, and storage units communicate efficiently via optical interconnects, which are key to improving the overall system performance. Existing technology two, after the electro-optical conversion module processes the data to be computed, directly connects to the optoelectronic computing module to perform mathematical operations on the optical domain data, ultimately transmitting the results to a photodetector and electronic chip for processing and storage. However, existing technology one suffers from problems such as high chip losses, low integration density, and limited computing power density due to the active devices in the electro-optical conversion unit occupying the majority of the chip area; existing technology two, due to the large area of ​​active devices such as modulators and detectors, results in low scalability of the computing module, low overall chip computing power density, and poor scalability, failing to fully leverage the advantages of optical computing. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a data processing method, chip, device and medium based on photoelectric neural networks to solve the problem that active devices occupy a large area of ​​the chip, resulting in high chip loss, low integration, limited computing power density and poor scalability, and thus failing to fully utilize the advantages of optical computing.

[0005] In a first aspect, embodiments of the present invention provide a data processing method based on a photoelectric neural network, the method comprising:

[0006] Based on the current data processing task, raw data from various application scenarios are acquired, and the raw data from various application scenarios are encoded according to the task scheduling rules to obtain electrical signals.

[0007] The electrical signal is transmitted to the electro-optical conversion module, the electrical signal is processed by the electro-optical conversion module to obtain corresponding optical data, and the optical data is transmitted to the optical routing module;

[0008] The optical routing module transmits the optical data to each computing module via a modular multiplexing method using orthogonal modes.

[0009] Obtain the calculation results output by each calculation module, and concatenate the calculation results to obtain the task processing result corresponding to the data processing task.

[0010] Furthermore, the step of processing the electrical signal through the electro-optic conversion module to obtain corresponding optical data includes:

[0011] The electro-optic conversion module converts the electrical signal into an optical signal.

[0012] The optical domain signal is loaded onto the optical carrier generated by the on-chip light source through the electro-optic conversion module to obtain the optical data.

[0013] Furthermore, the step of transmitting the optical data to each computing module via the optical routing module in a mode-division multiplexing manner according to orthogonal mode includes:

[0014] The optical routing module separates and distributes the optical data according to different orthogonal modes, so that different types of data are respectively mapped to non-interfering orthogonal modes, thus obtaining orthogonal mode optical signals;

[0015] The orthogonal mode optical signal is split based on a compact optical routing structure and transmitted to the corresponding computing module according to the task scheduling rules.

[0016] Furthermore, the step of concatenating the calculation results to obtain the task processing result corresponding to the data processing task includes:

[0017] Obtain the data relationships between the various calculation results;

[0018] Based on the task scheduling rules and the data association relationships, the calculation results are combined to obtain the complete task processing result.

[0019] Secondly, embodiments of the present invention provide a chip based on a photoelectric neural network, the chip comprising a peripheral electronic chip, an electro-optical conversion module, an optical routing module, a computing module, and a controller connected in sequence;

[0020] The peripheral electronic chip is used to acquire raw data from multiple application scenarios based on the current data processing task, encode the raw data from multiple application scenarios according to the task scheduling rules to obtain electrical signals, and transmit the electrical signals to the electro-optical conversion module.

[0021] The electro-optic conversion module is used to process the electrical signal to obtain corresponding optical data, and transmit the optical data to the optical routing module;

[0022] The optical routing module is used to transmit the optical data to each computing module in a mode-division multiplexing manner according to the orthogonal mode;

[0023] The controller is used to acquire the calculation results output by each calculation module, and to concatenate the calculation results to obtain the task processing result corresponding to the data processing task.

[0024] Furthermore, the optical routing module has a first number of arrayed waveguide gratings on its left side and a second number of arrayed waveguide gratings on its right side, the second number being less than or equal to the first number. The waveguide crossing structure between different optical paths in the optical routing module is implemented by a multimode interferometer, and each arrayed waveguide grating has multiple ports.

[0025] The port of the left-side arrayed waveguide grating receives light modulated by an electro-optic modulator, which is then transmitted through an optical path constructed by a waveguide and a multimode interferometer. By transmitting different modes of light, changing the wavelength / polarization state of the light, and adjusting the effective refractive index of the cross waveguide through thermal effects, the on / off state of the optical path is controlled, thus realizing the routing of light to any optical path on the right.

[0026] Furthermore, the structure of the cross waveguide includes at least two input waveguides and two output waveguides, with a width of 300-600 nm and a thickness of 150 nm-4000 nm; both the input waveguides and the output waveguides include a waveguide layer, a cladding layer, and a substrate material, with the waveguide layer located in the middle and the cladding layer covering its outer side in sequence, and the substrate material at the bottom layer, used to support the waveguide layer and the cladding layer.

[0027] Furthermore, the waveguide layer includes at least Si, Si3N4, and polymethyl methacrylate, and the cladding layer and substrate material include at least SiO2, Al2O3, and polymethyl methacrylate.

[0028] Thirdly, embodiments of the present invention provide a computer device, including: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the method described in the first aspect or any corresponding embodiment thereof.

[0029] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing computer instructions that cause a computer to perform the method described in the first aspect or any corresponding embodiment thereof.

[0030] This application addresses existing problems by optimizing the signal processing and transmission architecture: It employs orthogonal mode division multiplexing (EMD) technology, enabling the optical routing module to transmit multiple sets of data in parallel within a single route, reducing reliance on multiple active devices and lowering their area proportion within the chip; the electro-optical conversion module centrally processes electrical signals into optical data, avoiding area waste caused by dispersed layouts and improving integration; optical data is distributed to each computing module for parallel computing, optimizing resource allocation based on task scheduling rules and increasing computing power density; the modular design allows computing modules to be expanded as needed, enhancing system scalability and fully leveraging the high bandwidth and low loss advantages of optical computing. Attached Figure Description

[0031] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0032] Figure 1 This is a flowchart illustrating a data processing method based on an optoelectronic neural network according to some embodiments of the present invention;

[0033] Figure 2 This is a structural block diagram of a chip based on an opto-neural network according to an embodiment of the present invention;

[0034] Figure 3 This is a structural block diagram of a chip based on an opto-neural network according to an embodiment of the present invention;

[0035] Figure 4 This is a structural block diagram of a chip based on an opto-neural network according to an embodiment of the present invention;

[0036] Figure 5 This is a structural block diagram of a chip based on an opto-neural network according to an embodiment of the present invention;

[0037] Figure 6 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0039] According to embodiments of the present invention, a data processing method, chip, device, and medium based on photoelectric neural networks are provided. It should be noted that the steps shown in the flowcharts in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowcharts, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0040] This embodiment provides a data processing method based on photoelectric neural networks. Figure 1 This is a flowchart of a data processing method based on a photoelectric neural network according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps:

[0041] Step S101: Based on the current data processing task, obtain raw data from multiple application scenarios, and encode the raw data from multiple application scenarios according to the task scheduling rules to obtain electrical signals.

[0042] In this embodiment, the first step is to acquire raw data from various application scenarios based on the current data processing task (such as bird activity analysis by a smart camera, matrix operations in complex scenes, etc.). This raw data may include bird image pixel information in image recognition scenarios, environmental spatial coordinate data in 3D imaging scenarios, and bird call waveform data in speech recognition scenarios. After acquiring the raw data, the peripheral electronic chip will uniformly encode this multi-scenario data according to preset task scheduling rules.

[0043] On the one hand, different types of raw data (such as images, sounds, coordinates) are converted into digital signal formats that conform to chip processing standards to ensure that the data can be recognized and transmitted in the electrical domain;

[0044] On the other hand, based on task priority, data correlation, and the processing capacity of computing modules, data is assigned to specific computing modules to clarify which data needs to be processed first and which data should be assigned to specific computing modules.

[0045] Meanwhile, to adapt to subsequent optical domain transmission and computing needs, the peripheral electronic chip will also scale the encoded data stream, adjusting the data transmission rate and data volume to avoid reduced transmission efficiency or resource waste due to excessively large or small data volumes. After encoding and data stream scaling, an electrical signal that can be directly used for electro-optical conversion is finally formed.

[0046] Step S102: Transmit the electrical signal to the electro-optical conversion module, process the electrical signal to obtain the corresponding optical data, and then transmit the optical data to the optical routing module.

[0047] In this embodiment, the peripheral electronic chip transmits the processed electrical signal to the electro-optical conversion module. Upon receiving the electrical signal, the electro-optical conversion module first activates its internal signal conversion mechanism. Through core components such as a modulator, it converts the information carried by the electrical signal (such as encoded bird image features and environmental coordinate parameters) from the electrical domain to the optical domain, achieving a fundamental change in signal form. Simultaneously, the chip's built-in on-chip light source generates a stable optical carrier (similar to an "optical carrier" for information transmission). The electro-optical conversion module then precisely loads the converted optical domain signal onto this optical carrier, enabling the optical carrier to carry the actual task data.

[0048] This loading process must ensure that parameters such as the intensity and frequency of the optical signal strictly correspond to the information characteristics of the electrical signal to guarantee the accuracy of data transmission in the optical domain. Finally, the electro-optical conversion module outputs optical data carrying complete mission information and transmits it to the optical routing module, preparing for subsequent optical domain splitting transmission.

[0049] Specifically, the electrical signal is processed by the electro-optical conversion module to obtain the corresponding optical data, including the following steps A1-A2:

[0050] Step A1: Convert the electrical signal into an optical signal using an electro-optical conversion module.

[0051] Specifically, after receiving electrical signals from the external electronic chip, the electro-optical conversion module first activates its internal signal preprocessing unit to perform preliminary verification and adaptation of the electrical signals. These electrical signals contain encoded and scaled raw data from multiple scenes (such as image pixel information, voice waveform data, environmental coordinate parameters, etc.), and their format must conform to the input standard for electro-optical conversion. During preprocessing, the module filters noise interference in the electrical signals and adjusts the signal amplitude and frequency to ensure stable adaptation to the subsequent conversion mechanism.

[0052] The core conversion stage relies on an electro-optic modulator (such as a modulation structure based on lithium niobate or silicon-based materials) within the module to achieve the signal domain conversion. The working principle of the electro-optic modulator is based on the "electro-optic effect"—when an electrical signal passes through the electrodes of the modulator, it changes the refractive index of the optical material inside the modulator, thereby affecting the physical properties of the light passing through the material (such as intensity, phase, frequency, or polarization state). Specifically, the digital information in the electrical signal (such as the logical states of "0" and "1") is converted into changes in light intensity (strong light represents "1", weak light represents "0") or phase shifts, allowing the originally informationless optical signal to carry the data characteristics of the electrical signal.

[0053] To ensure conversion accuracy, the module monitors the correspondence between the electrical and optical signals in real time and corrects conversion deviations through a feedback adjustment mechanism. For example, if fluctuations in the amplitude of the electrical signal cause instability in the optical signal intensity, the feedback unit automatically adjusts the modulator's operating voltage to ensure that the parameters of the optical signal strictly match the information of the electrical signal. Finally, after processing by the electro-optic modulator, all the information of the electrical signal is successfully mapped into the optical signal, forming an optical signal that can be transmitted in the optical domain, preparing it for subsequent loading onto the optical carrier.

[0054] Step A2: The optical domain signal is loaded onto the optical carrier generated by the on-chip light source through the electro-optic conversion module to obtain optical data.

[0055] Specifically, after completing the optical domain signal conversion, the electro-optical conversion module enters the signal loading stage. First, the module activates the built-in on-chip light source (such as an integrated light source based on silicon-based silicon nitride or III-V compound light sources) to generate a stable optical carrier. The optical carrier is a continuous optical signal with a fixed wavelength and frequency, acting as a "carrier" for information transmission; its stability directly affects the reliability of subsequent data transmission. The on-chip light source maintains a constant operating temperature through a temperature control unit to prevent wavelength drift of the optical carrier due to temperature fluctuations, ensuring that its physical characteristics meet transmission requirements.

[0056] Subsequently, the module precisely combines the optical domain signal and the optical carrier through a coupling structure to achieve the "loading" process. This process is similar to "loading goods onto a transport vehicle": the optical carrier acts as the "transport vehicle," and the optical domain signal acts as the "goods." Through the secondary action of the modulator, the characteristics of the optical domain signal (such as intensity changes and phase shifts) are superimposed onto the optical carrier. Specifically, if the optical domain signal is generated through intensity modulation, the intensity of the optical carrier will fluctuate synchronously with the changes in the optical domain signal during loading; if it is generated through phase modulation, the phase of the optical carrier will shift with the signal characteristics, thus allowing the optical carrier to carry the actual task data.

[0057] After loading is complete, the module performs quality checks on the synthesized optical signal, monitoring its intensity, signal-to-noise ratio, and wavelength stability using an optical power meter and a spectrum analyzer to ensure the information carried by the optical carrier is complete and distortion-free. Finally, the loaded and verified optical signal is defined as "optical data," which retains the transmission characteristics of the optical carrier (such as high speed and low loss) while containing all the task information from the original electrical signal. At this point, the electro-optical conversion module transmits the optical data to the optical routing module through its output port, entering the next stage of the splitting transmission process.

[0058] In step S103, the optical data is transmitted to each computing module via the optical routing module in a mode division multiplexing manner according to the orthogonal mode.

[0059] In this embodiment, after receiving optical data transmitted by the electro-optical conversion module, the optical routing module performs splitting transmission based on orthogonal mode division multiplexing (EMD) technology. The optical routing module internally contains a compact optical routing structure based on an AWG design. Its core utilizes the physical characteristics of orthogonal modes such as TE0 and TE1—these orthogonal modes are independent and do not interfere with each other during propagation, acting like multiple parallel "optical channels." The optical routing module first parses the optical data, dividing it into multiple groups of sub-data according to the marked task allocation information, such as type and priority. Each group of sub-data corresponds to an orthogonal mode. For example, bird image feature data is assigned to the TE0 mode, and environmental coordinate data is assigned to the TE1 mode. Subsequently, through EMD technology, multiple groups of sub-data are transmitted in parallel within the same optical route, significantly improving transmission efficiency.

[0060] Meanwhile, the optical routing module dynamically optimizes its routing strategy based on real-time monitoring of input data changes (such as sudden increases in data volume or adjustments to task priorities). Through a low-insertion-loss optical routing structure, it precisely directs sub-data carried by different orthogonal modes to their corresponding computing modules 1-5, and adjusts the workload of each computing module to ensure an optimal balance between speed, computational accuracy, and energy consumption. For example, if the amount of bird image data suddenly increases, the optical routing module can temporarily increase the transmission bandwidth of the TEO mode, prioritizing the transmission of image data to computing module 1, which excels at image processing, while reducing the proportion of other non-urgent data transmission.

[0061] Specifically, the optical routing module transmits optical data to each computing module via a mode division multiplexing method using orthogonal modes, including the following steps B1-B2:

[0062] Step B1: The optical data is separated and distributed according to different orthogonal modes by the optical routing module, so that different types of data are respectively mapped to non-interfering orthogonal modes, and orthogonal mode optical signals are obtained.

[0063] After receiving the optical data transmitted by the electro-optical conversion module, the optical routing module first activates its internal signal analysis unit to identify the structure and information characteristics of the optical data. The optical data contains various types of task information, such as bird image data in a smart camera scenario, environmental model parameters generated by the 3D imaging module, and bird call audio data captured by the voice recognition module. These data exist in the optical domain as composite signals and require orthogonal mode separation for accurate allocation.

[0064] The core separation process relies on the AWG (Arrayed Waveguide Grating)-based mode beam splitting structure in the optical routing module. This structure utilizes the physical characteristics of orthogonal modes such as TE0 and TE1—different orthogonal modes have unique electromagnetic field distributions and phase characteristics when propagating in the waveguide, and they are independent and do not interfere with each other. The parsing unit calls a preset mode matching algorithm based on the data category label (e.g., image data corresponds to a specific coded identifier, and sound data corresponds to another identifier) ​​to match various types of information in the composite optical data with the corresponding orthogonal modes: for example, matching the high-frequency characteristics of image data with the transmission characteristics of the TE0 mode, and matching the spatial coordinate data of the environmental model with the phase stability of the TE1 mode.

[0065] During the separation process, the optical routing module uses evanescent wave coupling or a cone mode converter to achieve mode conversion of the optical signal, ensuring that different types of data can smoothly "enter" the corresponding orthogonal mode channel. Simultaneously, the module monitors the signal strength and interference of each orthogonal mode in real time, dynamically adjusting the refractive index distribution of the waveguides (e.g., using thermal effects to fine-tune the effective refractive index of the cross waveguides) to avoid crosstalk between different orthogonal modes. Finally, after separation and allocation, each type of data is precisely loaded into its dedicated orthogonal mode, forming multiple sets of non-interfering orthogonal mode optical signals, laying the foundation for subsequent splitting transmission.

[0066] Step B2: Based on the compact optical routing structure, the orthogonal mode optical signal is split and transmitted to the corresponding computing module according to the task scheduling rules.

[0067] After the orthogonal mode optical signals are separated, the optical routing module initiates the splitting transmission process based on the compact optical routing structure (composed of low insertion loss waveguides, MMI cross-waveguides, etc.). The core advantage of the compact optical routing structure is that it enables the parallel transmission of multiple sets of orthogonal mode optical signals in a limited space through a densely arranged waveguide network, while keeping the insertion loss at a low level of 0.2-6dB, minimizing signal attenuation.

[0068] First, the task scheduling unit in the module calls the preset task scheduling rules in step S101, and, combined with the real-time load status of the computing modules (such as the current computing power utilization rate and processing latency of each core), assigns a target computing module to each group of orthogonal mode optical signals. For example, the TE0 mode optical signal carrying bird image data will be preferentially assigned to computing module 1 (assuming it is good at image processing) because image recognition tasks have high computing power requirements; the TE1 mode optical signal carrying bird song data will be assigned to computing module 2 according to the lightweight requirements of audio processing.

[0069] During the splitting process, the MMI cross-waveguide plays a crucial role: through a specific input / output waveguide design (at least two inputs and two outputs) and multimode interference effects, it achieves interference-free switching of orthogonal mode optical signals between different waveguide paths. When the orthogonal mode optical signals are transmitted to the cross node, the MMI structure adjusts the propagation direction of the optical signals according to scheduling instructions, so that the TE0 mode optical signals are transmitted along the waveguide pointing to computing module 1, the TE1 mode optical signals are transmitted along the waveguide pointing to computing module 2, and so on. At the same time, the module monitors the polarization state and wavelength stability of the waveguides to ensure that the parameters of the split optical signals match the input requirements of the target computing module (e.g., core 3 only receives optical signals of a specific wavelength).

[0070] Finally, through the branching and guidance of the compact optical routing structure, the orthogonal mode optical signals are accurately transmitted to the corresponding computing modules 1 to 5. Each core only receives optical signals related to its task, which not only ensures the high efficiency of data transmission, but also provides accurate input support for subsequent parallel computing.

[0071] Furthermore, when introducing an adaptive modulus multiplexing mechanism into the data processing flow, the peripheral electronic chip first synchronously identifies the data type (such as high-frequency images, audio, and text) and monitors real-time bandwidth requirements (such as bandwidth fluctuations caused by frame rate changes in image data) during the raw data encoding stage. Subsequently, the mechanism dynamically adjusts the orthogonal mode allocation strategy through a preset algorithm model (combining data characteristics and historical transmission efficiency). When high-frequency image data is detected, more orthogonal mode channels such as TE0 and TE1 are automatically allocated (e.g., from 2 channels to 4) to ensure parallel transmission of large-capacity data. For text data with low bandwidth requirements, the number of orthogonal modes occupied is reduced to avoid resource waste. The optical routing module receives allocation instructions in real time and quickly completes mode switching through evanescent wave coupling or cone mode converters, ensuring that the adjusted orthogonal mode channels are precisely matched with the needs of the computing module. Ultimately, this improves the efficiency of high-frequency data transmission while achieving dynamic optimization of orthogonal mode resources.

[0072] Step S104: Obtain the calculation results output by each calculation module, and concatenate the calculation results to obtain the task processing result corresponding to the data processing task.

[0073] In this embodiment, after completing their assigned sub-tasks, each calculation module outputs corresponding calculation results: calculation module 1 may output bird species identification results, calculation module 2 may output bird flight trajectory analysis data, and calculation module 3 may output environmental obstacle (such as tree branches) location determination results, etc. These calculation results are first aggregated to the data integration unit, which performs preliminary verification on the results, removes invalid data caused by transmission or calculation errors, and marks the correlation between the data (such as the bird species corresponding to a certain flight trajectory).

[0074] Subsequently, the computer initiates the splicing program, and based on the encoding logic and task scheduling rules of the original data in step S101, reassembles the scattered calculation results according to time sequence, spatial correlation, or logical relationship. For example, the species identification result of "sparrow" is spliced ​​with the trajectory data of "flying from branch A to branch B at 8:05", and then combined with the environmental data of "the coordinates of branch A are (x1, y1, z1)" to finally form a complete "bird activity analysis report", which is the final task processing result corresponding to the current data processing task.

[0075] In this embodiment of the application, the calculation results are spliced ​​together to obtain the task processing result corresponding to the data processing task, including the following steps C1-C2:

[0076] Step C1: Obtain the data relationships between the various calculation results.

[0077] After obtaining the calculation results output by each calculation module, it is first necessary to perform structured parsing on these scattered results to extract key clues related to the data. The results output by each calculation module may contain information in different dimensions. For example, after processing image data, calculation module 1 outputs the recognition result "bird species is sparrow", with the shooting timestamp "08:05:23" and image frame number "F001"; after processing trajectory data, calculation module 2 outputs "sparrow moved from coordinates A(x1,y1,z1) to coordinates B(x2,y2,z2)", with the time interval "08:05:20-08:05:30"; after processing environmental data, calculation module 3 outputs "tree branches obstruct the view at coordinate A", with the associated spatial identifier "region C".

[0078] After parsing, a multi-dimensional matching mechanism is initiated through the data association engine: From the time dimension, the timestamps or time intervals of each calculation result are compared, and it is found that "08:05:23" of calculation module 1 falls within the interval of "08:05:20-08:05:30" of calculation module 2, indicating that they describe events in the same time period; From the spatial dimension, the coordinate mapping algorithm confirms that "coordinate A" of calculation module 2 and "region C" of calculation module 3 have spatial overlap, indicating that environmental data and trajectory data are associated; From the logical dimension, based on the task chain of "bird identification-trajectory tracking-environmental analysis" in the original task, it is confirmed that all three types of results serve the core objective of "sparrow activity monitoring" and have logical correlation.

[0079] In addition, the consistency of the verification data is checked. For example, it is checked whether the "sparrow" identified by calculation module 1 matches the "small bird" feature tracked by calculation module 2. If a conflict exists, a backtracking mechanism is initiated to retrieve the original light data for re-verification. Finally, through triple correlation verification of time, space, and logic, a correlation map with "sparrow" as the core is formed, clarifying the subordinate relationships and interaction logic of each calculation result, providing a clear correlation basis for subsequent splicing.

[0080] Step C2: Based on the task scheduling rules and the data association relationships, the calculation results are combined to obtain the complete task processing result.

[0081] After clarifying the relationships between the calculation results, the system initiates the splicing process according to the task scheduling rules determined in step S101 (such as "prioritizing the preservation of core data integrity" and "chaining events according to the timeline"). First, the priority parameters in the task scheduling rules guide the system to determine the splicing main line: for example, if the task requires "taking the bird activity timeline as the core", then the trajectory time interval "08:05:20-08:05:30" output by calculation module 2 is used as the baseline axis, and other results are embedded according to time nodes.

[0082] The specific splicing process is divided into three layers: The basic layer integrates data from the same source, merging the multi-frame image recognition results related to "sparrow" in calculation module 1 (such as "F001-F005 are all sparrows") into a comprehensive conclusion that "the target was confirmed as a sparrow during the period from 08:05 to 08:06"; The association layer integrates cross-dimensional data, combining the "tree branch obstruction at coordinate A" from calculation module 3 with the "staying at coordinate A for 0.5 seconds at 08:05:25" from calculation module 2, to supplement the detail that "the sparrow briefly stayed at coordinate A due to tree branch obstruction"; The logic layer constructs causal relationships, analyzing the correlation between "tree branch obstruction" and "staying behavior" through the preset logic of "environmental influence behavior" in the task scheduling rules, and deriving the deeper conclusion that "the sparrow may adjust its flight trajectory to avoid obstruction".

[0083] During the data stitching process, redundant information is removed through redundancy checks (e.g., "coordinate A," mentioned in multiple cores, is only retained once). A format unification module converts the output formats of different calculation modules (e.g., text descriptions, coordinate lists, image feature values) into a unified structured report format. Finally, after multi-layered integration and logical organization, a complete task processing result is formed, encompassing "target recognition, temporal trajectory, spatial environment, and behavior analysis." For example, "On [Date] 2024, from 08:05 to 08:06, a sparrow was observed flying from coordinate A to coordinate B. During this time, it briefly paused due to tree branches obstructing its path at coordinate A, and its flight trajectory exhibited an S-shaped adjustment influenced by the environment," fully meeting the requirements of the original data processing task.

[0084] This application addresses existing problems by optimizing the signal processing and transmission architecture: It employs orthogonal mode division multiplexing (EMD) technology, enabling the optical routing module to transmit multiple sets of data in parallel within a single route, reducing reliance on multiple active devices and lowering their area proportion within the chip; the electro-optical conversion module centrally processes electrical signals into optical data, avoiding area waste caused by dispersed layouts and improving integration; optical data is distributed to each computing module for parallel computing, optimizing resource allocation based on task scheduling rules and increasing computing power density; the modular design allows computing modules to be expanded as needed, enhancing system scalability and fully leveraging the high bandwidth and low loss advantages of optical computing.

[0085] Secondly, embodiments of the present invention provide a chip based on a photoelectric neural network, such as... Figure 2 As shown, the chip includes a peripheral electronic chip 100, an electro-optical conversion module 200, an optical routing module 300, a computing module 400, and a controller 500 connected in sequence.

[0086] The peripheral electronic chip 100 is used to acquire raw data from various application scenarios based on the current data processing task, encode the raw data from various application scenarios according to the task scheduling rules, obtain electrical signals, and transmit the electrical signals to the electro-optical conversion module 200.

[0087] The electro-optical conversion module 200 is used to process electrical signals to obtain corresponding optical data and transmit the optical data to the optical routing module;

[0088] The optical routing module 300 is used to transmit optical data to each computing module 400 via a splitter according to the orthogonal mode mode division multiplexing method.

[0089] The controller 500 is used to acquire the calculation results output by each calculation module, and to concatenate the calculation results to obtain the task processing result corresponding to the data processing task.

[0090] In the embodiments of this application, such as Figure 3As shown, the electro-optic conversion module is an important component of this optoelectronic device system. It is built based on Si3N4 (silicon nitride) and connected to a continuous wave (CW) light source on the left. After modulation by a related structure, the electrical signal is converted into an optical signal through multiple units within the module. It can be used in conjunction with spectral (P-λ distribution diagram) to achieve specific optical signal processing and output. It is used for electro-optic signal conversion and subsequent optical signal modulation in scenarios such as optical communication and optical sensing. Multiple parallel units can support parallel electro-optic conversion operations of multiple signals.

[0091] In this embodiment of the application, a first number of arrayed waveguide gratings are provided on the left side of the optical routing module, and a second number of arrayed waveguide gratings are provided on the right side of the optical routing module. The second number is less than or equal to the first number. The waveguide cross structure between different optical paths in the optical routing module is realized by a multimode interferometer, and each arrayed waveguide grating is provided with multiple ports.

[0092] The port of the left-side arrayed waveguide grating receives light modulated by an electro-optic modulator, which is then transmitted through an optical path constructed by a waveguide and a multimode interferometer. By transmitting different modes of light, changing the wavelength / polarization state of the light, and adjusting the effective refractive index of the cross waveguide through thermal effects, the on / off state of the optical path is controlled, thus realizing the routing of light to any optical path on the right.

[0093] It should be noted that, as Figure 4 As shown, the left side has a first number (1-10) of arrayed waveguide gratings (AWGs), while the right side has a second number (1-15). The second number strictly adheres to the design principle of being less than or equal to the first number. This ratio ensures sufficient allocation of input optical signals while avoiding idle and wasted port resources on the right side. Each AWG is equipped with 2-15 ports. The left ports (e.g., P11L1-P11LN) serve as the optical signal receivers, and the right ports (e.g., P11R1-P11RN) serve as the signal outputs. The insertion loss between the left and right ports is controlled at a low level of 0.2-6dB, ensuring minimal energy loss during optical signal transmission. Crucially, all waveguide intersections between different optical paths are connected using a multimode interferometer (MMI). The MMI, with its unique interference coupling principle, provides a low-loss, high-isolation physical channel for cross-transmission of optical signals, making it a core structural component for constructing complex optical path networks.

[0094] During the transmission and routing control of optical signals, the port of the left-side arrayed waveguide grating first receives modulated optical signals from the electro-optic modulator. These optical signals, carrying task data, enter the optical path network jointly constructed by the waveguide and MMI cross structure. To achieve precise transmission of optical signals to any optical path on the right, the optical routing module employs a multi-dimensional on / off control mechanism: First, by allowing the optical path to transmit light of different modes such as TE0 and TE1, the parallel transmission and path differentiation of signals are achieved by utilizing the non-interference characteristic of orthogonal modes; second, by changing the wavelength or polarization state of the transmitted light, optical signals of specific wavelengths or polarization directions can only conduct in a preset path; third, by adjusting the effective refractive index of the cross waveguide using thermal effects, the propagation direction of the optical signal is changed, thereby flexibly controlling the conduction or closure of a certain optical path. These three control methods work together to enable the optical routing module to dynamically adjust the transmission path of the optical signal according to task requirements, ultimately realizing the core routing function of accurately and efficiently transmitting the optical signals received on the left to any target optical path on the right, providing stable and reliable optical domain transmission support for task allocation by the subsequent computing module.

[0095] In this embodiment of the application, the structure of the cross waveguide includes, as follows: Figure 5 As shown, there are at least two input waveguides and two output waveguides, with a width of 300-600 nm and a thickness of 150 nm-4000 nm. Both the input waveguides and the output waveguides include a waveguide layer, a cladding layer, and a substrate material. The waveguide layer is located in the middle, and the cladding layer is successively covered on its outer side. The substrate material is at the bottom layer and is used to support the waveguide layer and the cladding layer.

[0096] In the embodiments of this application, the waveguide layer includes at least Si, Si3N4, and polymethyl methacrylate, and the cladding layer and substrate material include at least SiO2, Al2O3, and polymethyl methacrylate.

[0097] It should be noted that, as Figure 4 As shown, there are at least two input waveguides and two output waveguides. This multi-port design provides a structural basis for the parallel transmission and cross-scheduling of optical signals. In terms of dimensions, the waveguide width is strictly controlled within the range of 300–600 nm, while the thickness covers the range of 150 nm–4000 nm. This size selection not only adapts to the mode transmission characteristics of optical signals within the waveguide but also takes into account the feasibility of process implementation. Thicker silicon processes (greater thickness) have advantages in integrating III-V compound materials, reducing transmission loss, and enhancing coupling efficiency, while thinner thicknesses, although having slightly poorer process compatibility, can meet the miniaturization requirements of specific scenarios.

[0098] Both the input and output waveguides adopt a "three-layer composite structure": the middle layer is the waveguide layer, which is the core area for optical signal transmission; the outer side of the waveguide layer is covered with a cladding layer, which constrains the propagation path of the optical signal by the difference in refractive index with the waveguide layer; the bottom layer is the substrate material, which provides a stable physical support for the entire waveguide structure. The three layers are closely integrated to form a complete optical waveguide transmission unit.

[0099] In terms of material selection, the waveguide layer includes at least Si, Si3N4, and polymethyl methacrylate (PMMA). These materials possess excellent optical transmission performance—Si and Si3N4, as inorganic semiconductor materials, have high refractive indexes and low optical loss characteristics, making them suitable for high-performance optical transmission scenarios; PMMA, as an organic polymer material, has advantages in flexibility and low-cost processing. The cladding layer and substrate materials include at least SiO2, Al2O3, and PMMA. Among them, SiO2 and Al2O3, as inorganic oxides, have good refractive index matching with waveguide layer materials such as Si and Si3N4, effectively forming optical confinement; PMMA, as an organic material, can form a good interface bond with similar waveguide layers, reducing interlayer stress. This material combination ensures efficient transmission of optical signals within the waveguide and, through a reasonable refractive index difference design, ensures that the optical signal is stably confined within the waveguide layer, reducing leakage loss and providing a material basis for the reliable operation of the optical routing module.

[0100] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 6 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system).

[0101] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0102] The memory 20 stores instructions executable by at least one processor 10 to cause the at least one processor 10 to perform the method shown in the above embodiments.

[0103] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device as shown by a landing page for an app. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, which can be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0104] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0105] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0106] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0107] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A data processing method based on a photoelectric neural network, characterized in that, The method includes: Based on the current data processing task, raw data from various application scenarios are acquired, and the raw data from various application scenarios are encoded according to the task scheduling rules to obtain electrical signals. The electrical signal is transmitted to the electro-optical conversion module, the electrical signal is processed by the electro-optical conversion module to obtain corresponding optical data, and the optical data is transmitted to the optical routing module; The optical routing module transmits the optical data to each computing module via a modular multiplexing method using orthogonal modes. Obtain the calculation results output by each calculation module, and concatenate the calculation results to obtain the task processing result corresponding to the data processing task.

2. The method according to claim 1, characterized in that, The process of processing the electrical signal through the electro-optic conversion module to obtain corresponding optical data includes: The electro-optic conversion module converts the electrical signal into an optical signal. The optical domain signal is loaded onto the optical carrier generated by the on-chip light source through the electro-optic conversion module to obtain the optical data.

3. The method according to claim 1, characterized in that, The step of transmitting the optical data to each computing module via the optical routing module in a mode-division multiplexing manner according to orthogonal mode includes: The optical routing module separates and distributes the optical data according to different orthogonal modes, so that different types of data are respectively mapped to non-interfering orthogonal modes, thus obtaining orthogonal mode optical signals; The orthogonal mode optical signal is split based on a compact optical routing structure and transmitted to the corresponding computing module according to the task scheduling rules.

4. The method according to claim 1, characterized in that, The process of concatenating the calculation results to obtain the task processing result corresponding to the data processing task includes: Obtain the data relationships between the various calculation results; Based on the task scheduling rules and the data association relationships, the calculation results are combined to obtain the complete task processing result.

5. A chip based on a photoelectric neural network, characterized in that, The chip includes a peripheral electronic chip, an electro-optical conversion module, an optical routing module, a computing module, and a controller connected in sequence. The peripheral electronic chip is used to acquire raw data from multiple application scenarios based on the current data processing task, encode the raw data from multiple application scenarios according to the task scheduling rules to obtain electrical signals, and transmit the electrical signals to the electro-optical conversion module. The electro-optic conversion module is used to process the electrical signal to obtain corresponding optical data, and transmit the optical data to the optical routing module; The optical routing module is used to transmit the optical data to each computing module in a mode-division multiplexing manner according to the orthogonal mode; The controller is used to acquire the calculation results output by each calculation module, and to concatenate the calculation results to obtain the task processing result corresponding to the data processing task.

6. The chip according to claim 5, characterized in that, The optical routing module has a first number of arrayed waveguide gratings on its left side and a second number of arrayed waveguide gratings on its right side. The second number is less than or equal to the first number. The waveguide crossing structure between different optical paths in the optical routing module is realized by a multimode interferometer. Each arrayed waveguide grating has multiple ports. The port of the left-side arrayed waveguide grating receives light modulated by an electro-optic modulator, which is then transmitted through an optical path constructed by a waveguide and a multimode interferometer. By transmitting different modes of light, changing the wavelength / polarization state of the light, and adjusting the effective refractive index of the cross waveguide through thermal effects, the on / off state of the optical path is controlled, thus realizing the routing of light to any optical path on the right.

7. The chip according to claim 6, characterized in that, The structure of the cross waveguide includes at least two input waveguides and two output waveguides, with a width of 300-600 nm and a thickness of 150 nm-4000 nm. Both the input waveguides and the output waveguides include a waveguide layer, a cladding layer, and a substrate material. The waveguide layer is located in the middle, and the cladding layer is sequentially covered on its outer side. The substrate material is at the bottom layer and is used to support the waveguide layer and the cladding layer.

8. The chip according to claim 7, characterized in that, The waveguide layer includes at least Si, Si3N4, and polymethyl methacrylate, and the cladding layer and substrate material include at least SiO2, Al2O3, and polymethyl methacrylate.

9. A computer device, characterized in that, include: A memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, the processor executing the computer instructions to perform the method of any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the method of any one of claims 1 to 4.

Images