Space-time spectral intelligent all-optical reconfigurable representation method, architecture and system

Abstract

The present disclosure relates to the technical field of optical computing, and particularly relates to a kind of space-time spectrum intelligent full-optical reconfigurable characterization method, architecture and system.The method comprises: obtaining the multispectral light field input by dynamic light field under current time frame, and the multispectral light field corresponding spectral space information set is weighted and fused, to obtain multispectral spatial light field;The multispectral spatial light field is time-series modulated, to obtain the multispectral spatial light field after modulation;The multispectral spatial light field after modulation is merged with the multispectral time-series cache corresponding to dynamic light field, to obtain the first multispectral space-time sequence of dynamic light field under current time frame.The present disclosure can realize multidimensional information processing while maintaining the efficiency of optical computing.

Description

Technical Field

[0001] This disclosure relates to the field of optical computing technology, and in particular to a spatiotemporal spectrum intelligent all-optical reconfigurable characterization method, architecture and system. Background Technology

[0002] With the rapid development of artificial intelligence and scientific computing, the complexity and scale of computing demands are constantly increasing. However, existing electronic computing technologies are limited by Moore's Law, and their performance is gradually approaching saturation, making it difficult to effectively cope with the increasingly stringent demands of large-scale complex algorithms on computing power and power consumption. Light has natural advantages such as high throughput and low latency during propagation, and optical computing technology, which uses photons instead of electrons as the computing carrier, is considered the key to breaking through existing computing bottlenecks.

[0003] However, current optical computing architectures typically only perform calculations in a single dimension (such as the amplitude or phase of light) and rely on electronic processors as relays in most cases, failing to fully realize the potential of optical computing in high-speed, large-scale parallel processing. Summary of the Invention

[0004] This disclosure aims to at least partially address one of the technical problems in the related art.

[0005] Therefore, the first objective of this disclosure is to propose a spatiotemporal spectrum intelligent all-optical reconfigurable characterization method to achieve multi-dimensional information processing while maintaining the high efficiency of optical computing.

[0006] The second objective of this disclosure is to propose a spatiotemporal spectral intelligent all-optical reconfigurable representation architecture.

[0007] The third objective of this disclosure is to propose a spatiotemporal spectrum intelligent all-optical reconfigurable characterization system.

[0008] To achieve the above objectives, a first aspect of this disclosure proposes a spatiotemporal spectral intelligent all-optical reconfigurable characterization method, comprising:

[0009] The multispectral light field input in the current time frame is obtained from the dynamic light field, and the spectral spatial information set corresponding to the multispectral light field is weighted and fused to obtain the multispectral spatial light field.

[0010] The multispectral spatial light field is temporally modulated to obtain the modulated multispectral spatial light field;

[0011] The modulated multispectral spatial light field is merged with the multispectral temporal buffer corresponding to the dynamic light field to obtain the first multispectral spatiotemporal sequence of the dynamic light field in the current time frame.

[0012] Optionally, the weighted fusion of the spectral spatial information set corresponding to the multispectral light field includes:

[0013] The spectral spatial information set corresponding to the multispectral light field is weighted and fused by pre-sensory computation.

[0014] Optionally, the weighted fusion of the spectral spatial information set corresponding to the multispectral light field through pre-sensory computation includes:

[0015] Construct a signal weighting function, wherein the signal weighting function is obtained by combining a metasurface and / or a liquid crystal array;

[0016] The signal weighting function is used to perform weighted fusion of the spectral spatial information set corresponding to the multispectral light field.

[0017] Optionally, the signal weighting function includes at least one of the following functional forms:

[0018] Weighted join;

[0019] Point-by-point multiplication.

[0020] Optionally, the step of temporally modulating the multispectral spatial light field to obtain the modulated multispectral spatial light field includes:

[0021] The modulated multispectral spatial light field is obtained by performing time-series multiplication calculations on the multispectral spatial light field using weights.

[0022] Optionally, merging the modulated multispectral spatial light field with the multispectral temporal buffer corresponding to the dynamic light field includes:

[0023] The modulated multispectral spatial light field is injected into the optical resonator ring corresponding to the dynamic light field, so as to merge the modulated multispectral spatial light field with the multispectral temporal buffer composed of the optical resonator ring.

[0024] Optionally, merging the modulated multispectral spatial light field with the multispectral temporal buffer composed of the optical resonator ring includes:

[0025] Based on the multispectral transfer function, the modulated multispectral spatial light field is cycled once in the multispectral temporal buffer composed of the optical resonant ring.

[0026] To achieve the above objectives, a second aspect of this disclosure proposes a spatiotemporal spectral intelligent all-optical reconfigurable characterization architecture, comprising: at least one spatiotemporal spectral computation unit, wherein the spatiotemporal spectral computation unit includes a multispectral spatial computation module and a multispectral temporal computation module, wherein...

[0027] The multispectral spatial calculation module is used to acquire the multispectral light field input by the dynamic light field in the current time frame, and to perform weighted fusion on the spectral spatial information set corresponding to the multispectral light field to obtain the multispectral spatial light field.

[0028] The multispectral time calculation module is used to perform temporal modulation on the multispectral spatial light field to obtain the modulated multispectral spatial light field;

[0029] The multispectral time calculation module is further configured to merge the modulated multispectral spatial light field with the multispectral temporal buffer corresponding to the dynamic light field to obtain the first multispectral spatiotemporal sequence of the dynamic light field in the current time frame.

[0030] Optionally, the architecture includes multiple cascaded spatiotemporal spectrum computing units, wherein the preceding spatiotemporal spectrum computing unit among any two adjacent spatiotemporal spectrum computing units is the first spatiotemporal spectrum computing unit, and the following spatiotemporal spectrum computing unit among any two adjacent spatiotemporal spectrum computing units is the second spatiotemporal spectrum computing unit; wherein,

[0031] The first spatiotemporal spectrum calculation unit is used to acquire the second multispectral spatiotemporal sequence obtained at the end of the dynamic input of the dynamic light field, and to perform nonlinear fusion on the second multispectral spatiotemporal sequence to obtain the nonlinear fusion result;

[0032] The first spatiotemporal spectrum calculation unit is also used to input the nonlinear fusion result as a dynamic light field into the second spatiotemporal spectrum calculation unit.

[0033] To achieve the above objectives, a third aspect of this disclosure proposes a spatiotemporal spectrum intelligent all-optical reconfigurable characterization system, comprising: the spatiotemporal spectrum intelligent all-optical reconfigurable characterization architecture shown in any of the second aspects above.

[0034] In summary, the spatiotemporal spectral intelligent all-optical reconfigurable characterization method, architecture, and system disclosed herein, by utilizing the characteristics of light in the spectral, spatial, and temporal dimensions, jointly models the spatial, temporal, and spectral (wavelength) dimensions of light during propagation. This allows for full utilization of the spatial, temporal, and spectral (wavelength) information contained in the light field during propagation, overcoming the limitations of existing intelligent scene perception technologies. It enables the construction of matching across the three dimensions of spatiotemporal spectrum, achieving full connectivity across time and spectral dimensions. It effectively overcomes the bottleneck of digital memory read / write, achieving seamless integration of multi-dimensional optical computing, and realizing multi-dimensional information processing while maintaining the high efficiency of optical computing.

[0035] Additional aspects and advantages of this disclosure will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this disclosure. Attached Figure Description

[0036] The above and / or additional aspects and advantages of this disclosure will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, in which:

[0037] Figure 1 A schematic flowchart illustrating a spatiotemporal spectrum intelligent all-optical reconfigurable characterization method provided in an embodiment of this disclosure;

[0038] Figure 2 A schematic flowchart illustrating a spatiotemporal spectral intelligent all-optical reconfigurable characterization method provided in another embodiment of this disclosure;

[0039] Figure 3 A schematic diagram illustrating the principle of a spatiotemporal spectrum intelligent all-optical reconfigurable characterization method provided in this embodiment of the disclosure;

[0040] Figure 4 This is a schematic diagram of a spatiotemporal spectrum intelligent all-optical reconfigurable characterization architecture provided in an embodiment of this disclosure. Detailed Implementation

[0041] Embodiments of this disclosure are described in detail below. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this disclosure, and should not be construed as limiting this disclosure.

[0042] With the rapid development of Artificial Neural Network (ANN) technology, the performance and complexity of machine vision algorithms have significantly improved, leading to a continuous increase in the demand for high computing power. The bottleneck of traditional electronic computing architecture lies in its high power consumption and limited computing speed, which cannot fully meet the high parallel processing capabilities required for dynamic machine vision processing.

[0043] However, traditional optical computing architectures require sensors to sense and store multispectral spatial dynamic signals. This photoelectric conversion often becomes the speed bottleneck of the system and often results in a reduction in the diversity of light field modes. Directly coupling multispectral spatial content to multispectral single-mode time channels may lead to the loss of important information.

[0044] The present disclosure will now be described in detail with reference to specific embodiments.

[0045] In the first embodiment, such as Figure 1 As shown, Figure 1This is a flowchart illustrating a spatiotemporal spectral intelligent all-optical reconfigurable characterization method provided in an embodiment of this disclosure. This method can be implemented using a computer program and can run on an architecture for performing spatiotemporal spectral intelligent all-optical reconfigurable characterization. The computer program can be integrated into an application or run as a standalone utility application.

[0046] The spatiotemporal intelligent all-optical reconfigurable characterization method can be executed by the spatiotemporal intelligent all-optical reconfigurable characterization architecture or by a spatiotemporal intelligent all-optical reconfigurable characterization system that includes the spatiotemporal intelligent all-optical reconfigurable characterization architecture.

[0047] For example, this spatiotemporal spectral intelligent all-optical reconfigurable characterization method includes the following steps:

[0048] S101, acquire the multispectral light field input by the dynamic light field in the current time frame, and perform weighted fusion on the set of spectral spatial information corresponding to the multispectral light field to obtain the multispectral spatial light field;

[0049] According to some embodiments, a dynamic light field refers to a time-varying light field from a dynamic scene. This dynamic light field can, for example, be a multispectral light field ranging from milliseconds to nanoseconds.

[0050] In some embodiments, the dynamic input of the dynamic light field can be modeled as a high-dimensional temporal stack, where each element represents the multispectral spatial light field at a given moment. The current time frame refers to the time frame at the current moment.

[0051] According to some embodiments, spectral spatial information refers to information related to the spectrum in spatial dimensions. Spatial dimensions include, but are not limited to, amplitude, phase, polarization, etc.

[0052] In some embodiments, the spectral spatial information set refers to a set formed by the aggregation of at least one spectral spatial information.

[0053] S102, temporally modulates the multispectral spatial light field to obtain the modulated multispectral spatial light field;

[0054] According to some embodiments, temporal modulation refers to the modulation of a multispectral spatial light field in the time dimension.

[0055] It should be noted that, due to the parallel nature of light propagation in free space, the process of acquiring the multispectral spatial light field can be completed rapidly before the arrival of the multispectral light field in the next time frame. Therefore, the information capacity of the system can be increased, ensuring a seamless connection between the spatial dimension and the parallel spectral and high-speed temporal dimensions.

[0056] S103, merge the modulated multispectral spatial light field with the multispectral temporal buffer corresponding to the dynamic light field to obtain the first multispectral spatiotemporal sequence of the dynamic light field in the current time frame.

[0057] According to some embodiments, a multispectral temporal buffer refers to a multispectral spatiotemporal sequence preceding the current time frame.

[0058] In some embodiments, the first multispectral spatiotemporal sequence refers to the multispectral spatiotemporal sequence under the current time frame. After the next time frame arrives, the first multispectral spatiotemporal sequence will be merged with the modulated multispectral spatial light field of the next time frame as a new multispectral temporal buffer.

[0059] In summary, the method provided in this embodiment, by utilizing the characteristics of light in the spectral, spatial, and temporal dimensions, jointly models the spatial, temporal, and spectral (wavelength) dimensions of light during propagation. This allows for full utilization of the spatial, temporal, and spectral (wavelength) information contained in the light field during propagation, overcoming the limitations of existing intelligent scene perception technologies. It enables the construction of matching across the three dimensions of space, time, and spectrum, achieving full connectivity across time and spectral dimensions. It effectively overcomes the bottleneck of digital memory read / write operations, achieving seamless integration of multi-dimensional optical computing. While maintaining the high efficiency of optical computing, it enables multi-dimensional information processing. Compared with traditional computing methods, it can significantly improve the speed of dynamic machine vision processing, has broad application prospects, and is expected to bring new opportunities for high-performance intelligent computing in the post-Moore's Law era, as well as real-time analysis and control of transient scientific phenomena. Furthermore, it can be applied to fields such as unmanned systems, autonomous driving, and ultrafast science.

[0060] This embodiment also provides another spatiotemporal spectrum intelligent all-optical reconfigurable characterization method. This method can be executed by a spatiotemporal spectrum intelligent all-optical reconfigurable characterization architecture, or by a spatiotemporal spectrum intelligent all-optical reconfigurable characterization system including the spatiotemporal spectrum intelligent all-optical reconfigurable characterization architecture.

[0061] For example, such as Figure 2 As shown, the spatiotemporal spectral intelligent all-optical reconfigurable characterization method may include the following steps:

[0062] S201, acquire the multispectral light field input to the dynamic light field in the current time frame;

[0063] According to some embodiments, in dynamic scenes, multispectral light fields ranging from milliseconds to nanoseconds can be input through free space.

[0064] S202, by weighted fusion of the spectral spatial information set corresponding to the multispectral light field through pre-sensing computation, the multispectral spatial light field is obtained;

[0065] According to some embodiments, front-sensor computing belongs to free-space diffraction optical computing, which is a technology that uses optical components to modulate and process light information at the front end of a photoelectric sensor to achieve computational inference.

[0066] In some embodiments, when performing weighted fusion of the spectral spatial information set corresponding to a multispectral light field through pre-sensing computation, a signal weighting function can be constructed; the signal weighting function is then used to perform weighted fusion of the spectral spatial information set corresponding to the multispectral light field. Therefore, the effect of weighted fusion can be improved.

[0067] In some embodiments, the signal weighting function includes, but is not limited to, at least one of the following functional forms:

[0068] Weighted join;

[0069] Point-by-point multiplication.

[0070] According to some embodiments, the signal weighting function can be obtained by combining a metasurface and / or a liquid crystal array. Different combinations of metasurfaces and liquid crystal arrays can form different signal weighting functions. Furthermore, metasurfaces and liquid crystals can be cascaded in a system to perform information fusion, or they can be separated as individual devices for information preprocessing, depending on the task and function to be performed.

[0071] In some embodiments, when using a signal weighting function to weight and fuse the spectral spatial information set corresponding to a multispectral light field, since the metasurface responds differently to different wavelengths, it can weight the multispectral signal through pre-design. During the liquid crystal array and subsequent diffraction propagation, information from different wavelengths will be fused. Therefore, the entire system consisting of the metasurface and the liquid crystal array can complete the weighted fusion process of the multispectral light field signal.

[0072] Metasurfaces can be used to extract and fuse information from multispectral light fields in different dimensions (amplitude, phase, polarization, etc.). The fused signal can be considered to contain information from all dimensions in a certain dimension, thus ensuring the completeness of the information.

[0073] The liquid crystal array can be used to modulate the phase and amplitude of multispectral light fields. During modulation, feature signals relevant to the final task are initially extracted, while other redundant signals are compressed. Furthermore, the weights of the liquid crystal array can be adjusted to values ​​best suited for the intelligent task through joint modeling and simulation training.

[0074] In other words, by using metasurfaces and liquid crystal arrays to perform weighted fusion of the spectral spatial information set corresponding to the multispectral light field, feature extraction and channel expansion of the multispectral light field can be achieved. Specifically, within a single time frame, pre-sensing calculations can be performed in each multispectral light field. For a given frame, multiple rapidly changing pre-sensing calculation weights can be applied simultaneously, which is equivalent to expanding the temporal channel depth of the multispectral light field by N times compared to a single frame.

[0075] S203, the modulated multispectral spatial light field is obtained by performing time-series multiplication calculation on the multispectral spatial light field through weights;

[0076] According to some embodiments, by performing time-series multiplication calculations on the multispectral spatial light field using weights, weighted modulation of the multispectral spatial light field can be achieved, resulting in a modulated multispectral spatial light field.

[0077] In some embodiments, a multispectral spatial light field can be amplified and modulated in an optical fiber array using an optical amplifier. The amplification factor of the optical amplifier for different wavelengths is related to the voltage applied across its terminals.

[0078] In some embodiments, the voltage applied across the optical amplifier is synchronously adjusted with the multispectral spatial optical field, thereby enabling weighted modulation of different spectral signals.

[0079] S204, the modulated multispectral spatial light field is injected into the optical resonator ring corresponding to the dynamic light field, so as to merge the modulated multispectral spatial light field with the multispectral temporal buffer composed of the optical resonator ring, and obtain the first multispectral spatiotemporal sequence of the dynamic light field in the current time frame.

[0080] According to some embodiments, information storage and connection between different dimensions can be achieved through a constructed optical resonant ring. This optical resonant ring not only provides picosecond to nanosecond-level time buffers but also weights information from different spectra. Within the optical resonant ring, multispectral spatial information from different times will be connected.

[0081] In some embodiments, by utilizing the resonant characteristics of the optical resonator for light of different wavelengths and the buffering characteristics for light fields of different times, the summation (merging) between the modulated multispectral spatial light field and the multispectral temporal buffer can be realized, thus completing a full connection across time and spectral dimensions.

[0082] In this process, the multispectral spatial light field of a particular frame enters the optical resonant ring, which modulates the light field with wavelengths near the resonance wavelength. Light with a wavelength equal to the resonance wavelength enters the resonant ring and circulates once, then coherently cancels out the input light of the same wavelength, causing a sharp drop in amplitude. The resonance wavelength can be changed by the voltage applied across the resonant ring. By dynamically synchronizing the voltage with the input light field, the multispectral spatial light field at different times can be buffered and modulated.

[0083] According to some embodiments, the modulated multispectral spatial light field can be cycled once in a multispectral temporal buffer composed of optical resonant rings based on the multispectral transfer function, so as to realize the merging between the modulated multispectral spatial light field and the multispectral temporal buffer.

[0084] In this context, each time the multispectral spatial light field of a frame cycles once in the multispectral temporal buffer composed of optical resonant loops, the cyclic signal is converted by the multispectral transfer function. This function is closely related to the physical characteristics of the optical resonator and can fully integrate information from different spectral dimensions and the light field at the current moment with the light field at past moments.

[0085] It should be noted that during the dynamic input of the multispectral light field into the dynamic light field, the first multispectral spatiotemporal sequence will be iterated continuously in a fully connected manner until the dynamic input ends, resulting in the second multispectral spatiotemporal sequence corresponding to the dynamic light field. Subsequently, the dynamic light field buffered in the optical resonator ring can utilize its physical properties to nonlinearly fuse information from different spectral dimensions and different times. The nonlinear fusion result will be obtained from the output port of the optical resonator ring. This nonlinear fusion result can then be used as a new dynamic light field for further calculations. Ultimately, features including action type and flicker mode can be inferred from the light field.

[0086] According to some embodiments, Figure 3 This is a schematic diagram illustrating the principle of a spatiotemporal spectral intelligent all-optical reconfigurable characterization method provided in an embodiment of this disclosure. Figure 3 As shown, the propagating light field in space includes dimensions such as space, time, polarization, phase, amplitude, and frequency. Through spatiotemporal representation, optical networks can perform joint optimization and information processing in both spatial and temporal dimensions; through multispectral representation, optical networks can modulate the phase and amplitude of light waves at different frequencies. The method provided in this disclosure, by integrating the above two representations into the same system, can achieve joint spatiotemporal-spectral representation, enabling information to be transformed in the three dimensions of spectrum, space, and time, thus offering greater advantages than single-dimensional processing methods that are easily limited.

[0087] It should be noted that in advanced electronic computers, memory read and write operations are typically accompanied by latency exceeding hundreds of nanoseconds. Frequent data read and write operations limit the ability to process nanosecond-level dynamic events in real time, inhibiting the potential of artificial neural networks in ultra-high-speed applications. The method provided in this embodiment overcomes the dependence on memory read and write by jointly optimizing the calculation, docking, and buffering modules in the spatiotemporal spectrum dimension. This can significantly expand the application scope of photonic computing and lay a solid foundation for the control of ultra-high-speed machine vision processing and optical transient experiments. Simultaneously, with the advancement of optical modulator and detector technologies, their bandwidth has reached hundreds of GHz. Combined with low-loss photonic integrated circuits, the proposed architecture can achieve a temporal resolution exceeding 10 picoseconds. This high-speed computing capability not only accelerates the operation of neural networks but also supports real-time analysis of non-repeating transient visual phenomena, thereby enabling optical chip feedback control of dynamic light fields with frequencies above GHz.

[0088] In summary, the method provided in this embodiment first acquires the multispectral light field input from the dynamic light field in the current time frame; then, it performs weighted fusion of the spectral spatial information set corresponding to the multispectral light field using pre-sensing computation to obtain the multispectral spatial light field; thus, it can improve the effect of weighted fusion of the spectral spatial information set corresponding to the multispectral light field. Next, it performs temporal multiplication calculation on the multispectral spatial light field using weights to obtain the modulated multispectral spatial light field; thus, it can improve the accuracy of acquiring the modulated multispectral spatial light field. Finally, it injects the modulated multispectral spatial light field into the optical resonator ring corresponding to the dynamic light field to merge the modulated multispectral spatial light field with the multispectral temporal buffer composed of the optical resonator ring, obtaining the first multispectral spatiotemporal sequence of the dynamic light field in the current time frame. Therefore, it can complete full connectivity across time and spectral dimensions, achieving multidimensional information processing while maintaining the efficiency of optical computation.

[0089] To achieve the above embodiments, this disclosure also proposes a spatiotemporal spectral intelligent all-optical reconfigurable characterization architecture.

[0090] like Figure 4 As shown, the spatiotemporal spectral intelligent all-optical reconfigurable characterization architecture 400 includes: at least one spatiotemporal spectral computing unit 410, which includes a multispectral spatial computing module 411 and a multispectral temporal computing module 412, wherein,

[0091] The multispectral spatial calculation module 411 is used to obtain the multispectral light field input by the dynamic light field in the current time frame, and to perform weighted fusion on the spectral spatial information set corresponding to the multispectral light field to obtain the multispectral spatial light field.

[0092] The multispectral time calculation module 412 is used to perform temporal modulation on the multispectral spatial light field to obtain the modulated multispectral spatial light field.

[0093] The multispectral time calculation module 412 is also used to merge the modulated multispectral spatial light field with the multispectral temporal buffer corresponding to the dynamic light field to obtain the first multispectral spatiotemporal sequence of the dynamic light field in the current time frame.

[0094] It should be noted that by utilizing the combined operations of spectral, spatial, and temporal data, each spatiotemporal spectrum computing unit 410 can effectively extract information and derive semantic content, thereby achieving deep information fusion and processing.

[0095] Optionally, the spatiotemporal spectrum intelligent all-optical reconfigurable characterization architecture 400 includes multiple cascaded spatiotemporal spectrum computing units 410. The first spatiotemporal spectrum computing unit among any two adjacent cascaded spatiotemporal spectrum computing units is the first spatiotemporal spectrum computing unit, and the second spatiotemporal spectrum computing unit among any two adjacent cascaded spatiotemporal spectrum computing units is the second spatiotemporal spectrum computing unit; wherein,

[0096] The first spatiotemporal spectrum calculation unit is used to obtain the second multispectral spatiotemporal sequence obtained at the end of the dynamic input of the dynamic light field, and to perform nonlinear fusion on the second multispectral spatiotemporal sequence to obtain the nonlinear fusion result;

[0097] The first spatiotemporal spectrum calculation unit is also used to input the nonlinear fusion result as a dynamic light field into the second spatiotemporal spectrum calculation unit.

[0098] It should be noted that the more spatiotemporal spectrum computation units cascaded, the stronger the overall network performance will be. The required number of spatiotemporal spectrum computation units can be determined based on the actual intelligent task. For example, if the computational power of a single spatiotemporal spectrum computation unit is sufficient to handle the target intelligent task, the computation result can be output after the first spatiotemporal spectrum computation unit has completed its computation, and the light field characteristics can be inferred.

[0099] Optionally, the multispectral spatial calculation module 411 is used to perform weighted fusion of the spectral spatial information set corresponding to the multispectral light field, specifically for:

[0100] The spectral spatial information set corresponding to the multispectral light field is weighted and fused by pre-sensing computation.

[0101] Optionally, the multispectral spatial calculation module 411 is used to perform weighted fusion of the spectral spatial information set corresponding to the multispectral light field through pre-sensing calculation, specifically for:

[0102] Construct a signal weighting function, wherein the signal weighting function is obtained by combining the metasurface and / or the liquid crystal array;

[0103] A signal weighting function is used to perform weighted fusion of the spectral spatial information set corresponding to the multispectral light field.

[0104] Optionally, the signal weighting function includes at least one of the following functional forms:

[0105] Weighted join;

[0106] Point-by-point multiplication.

[0107] Optionally, the multispectral time calculation module 412 is used to perform temporal modulation on the multispectral spatial light field to obtain the modulated multispectral spatial light field, specifically for:

[0108] The modulated multispectral spatial light field is obtained by performing time-series multiplication calculations on the multispectral spatial light field using weights.

[0109] Optionally, the multispectral time calculation module 412 is used to merge the modulated multispectral spatial light field with the multispectral temporal buffer corresponding to the dynamic light field, specifically for:

[0110] The modulated multispectral spatial light field is injected into the optical resonator corresponding to the dynamic light field in order to merge the modulated multispectral spatial light field with the multispectral temporal buffer composed of the optical resonator.

[0111] Optionally, the multispectral timing calculation module 412 is used to merge the modulated multispectral spatial light field with the multispectral timing buffer composed of optical resonant rings, specifically for:

[0112] Based on the multispectral transfer function, the modulated multispectral spatial light field is cycled once in a multispectral temporal buffer composed of optical resonant rings.

[0113] It should be noted that the explanation of the above-mentioned embodiment of the spatiotemporal spectrum intelligent all-optical reconfigurable characterization method also applies to the spatiotemporal spectrum intelligent all-optical reconfigurable characterization architecture of this embodiment, and will not be repeated here.

[0114] In summary, the spatiotemporal spectral intelligent all-optical reconfigurable representation architecture provided in this disclosure, by utilizing the characteristics of light in the spectral, spatial, and temporal dimensions, jointly models the spatial, temporal, and spectral (wavelength) dimensions of light during propagation. It can fully utilize the spatial, temporal, and spectral (wavelength) information contained in the light field during propagation, break through the limitations of existing intelligent scene perception technologies, construct matching across the three dimensions of spatiotemporal spectrum, achieve full connectivity across time and spectral dimensions, effectively overcome the bottleneck of digital memory read and write, achieve seamless integration of multi-dimensional optical computing, and realize multi-dimensional information processing while maintaining the high efficiency of optical computing.

[0115] To achieve the above embodiments, this disclosure also proposes a spatiotemporal spectrum intelligent all-optical reconfigurable characterization system, including: the spatiotemporal spectrum intelligent all-optical reconfigurable characterization architecture provided in the foregoing embodiments.

[0116] To implement the above embodiments, this disclosure also proposes an electronic device, including: a processor and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method provided in the foregoing embodiments.

[0117] To implement the above embodiments, this disclosure also proposes a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the methods provided in the foregoing embodiments.

[0118] To implement the above embodiments, this disclosure also proposes a computer program product, including a computer program that, when executed by a processor, implements the methods provided in the foregoing embodiments.

[0119] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in this disclosure all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0120] It should be noted that personal information collected from users should be used for legitimate and reasonable purposes and should not be shared or sold outside of these legitimate uses. Furthermore, such collection / sharing should only be conducted after receiving the user's informed consent, including but not limited to notifying the user to read the user agreement / user notice and sign an agreement / authorization that includes authorization of relevant user information before the user uses the function. In addition, any necessary steps must be taken to protect and safeguard access to such personal information data and ensure that others with access to personal information data comply with their privacy policies and procedures.

[0121] This disclosure is intended to provide implementation schemes for users to selectively prevent the use or access to their personal information data. Specifically, this disclosure is intended to provide hardware and / or software to prevent or block access to such personal information data. Once personal information data is no longer needed, risks can be minimized by restricting data collection and deleting data. Furthermore, where applicable, such personal information is de-identified to protect user privacy.

[0122] The acquisition, transmission, storage, use, and processing of data in this disclosed technical solution all comply with the relevant provisions of national laws and regulations.

[0123] It should be noted that in the embodiments disclosed herein, certain software, components, models, and other existing solutions in the industry may be mentioned. These should be considered as exemplary and are intended only to illustrate the feasibility of implementing the technical solution of this application. However, they do not mean that the applicant has used or necessarily used such solutions.

[0124] In the foregoing descriptions of the embodiments, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0125] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0126] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of this disclosure includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of this disclosure pertain.

[0127] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0128] It should be understood that various parts of this disclosure can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0129] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware, and the program can be stored in a computer-readable storage medium. When executed, the program includes one or a combination of the steps of the method embodiments.

[0130] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0131] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of the present disclosure have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present disclosure.

Claims

1. A spatiotemporal spectral intelligent all-optical reconfigurable characterization method, characterized in that, include: A multispectral light field is acquired as input in the current time frame of a dynamic light field, and the spectral spatial information set corresponding to the multispectral light field is weighted and fused to obtain a multispectral spatial light field. The spectral spatial information set corresponding to the multispectral light field is weighted and fused by pre-sensing computation. A signal weighting function is constructed, which is obtained by combining metasurfaces and / or liquid crystal arrays. The signal weighting function is used to weight and fuse the spectral spatial information set corresponding to the multispectral light field. The modulated multispectral spatial optical field is obtained by performing time-series multiplication calculation on the multispectral spatial optical field by weighting. In the fiber array, the multispectral spatial optical field is amplified and modulated by an optical amplifier. The amplification factor of the optical amplifier for different wavelengths is related to the voltage applied at its two ends. The voltage applied at the two ends of the optical amplifier is synchronously adjusted with the multispectral spatial optical field to perform weighted modulation of different spectral signals. Based on the multispectral transfer function, the modulated multispectral spatial light field is looped once in a multispectral temporal buffer composed of an optical resonant ring to obtain the first multispectral spatiotemporal sequence of the dynamic light field in the current time frame. During the dynamic input of the multispectral light field, the first multispectral spatiotemporal sequence is continuously iterated in a fully connected manner until the dynamic input ends, resulting in the second multispectral spatiotemporal sequence corresponding to the dynamic light field. Subsequently, the dynamic light field buffered in the optical resonant ring can use its physical properties to nonlinearly fuse information from different spectral dimensions and different times. The nonlinear fusion result is obtained from the output port of the optical resonant ring. Then, the nonlinear fusion result is used as a new dynamic light field for further calculation, and finally, features including action type and flicker mode can be inferred from the light field.

2. The method according to claim 1, characterized in that, The signal weighting function includes at least one of the following functional forms: Weighted join; Point-by-point multiplication.

3. A spatiotemporal spectral intelligent all-optical reconfigurable representation architecture, characterized in that, include: At least one spatiotemporal spectrum calculation unit, wherein the spatiotemporal spectrum calculation unit includes a multispectral spatial calculation module and a multispectral temporal calculation module, wherein, The multispectral spatial calculation module is used to acquire the multispectral light field input by the dynamic light field in the current time frame, and to perform weighted fusion on the spectral spatial information set corresponding to the multispectral light field to obtain the multispectral spatial light field. The weighted fusion of the spectral spatial information set corresponding to the multispectral light field is performed by pre-sensing calculation. A signal weighting function is constructed, which is obtained by combining metasurfaces and / or liquid crystal arrays. The signal weighting function is used to perform weighted fusion on the spectral spatial information set corresponding to the multispectral light field. The multispectral time calculation module is used to perform time-series multiplication calculation on the multispectral spatial optical field through weights to obtain the modulated multispectral spatial optical field. In the fiber array, the multispectral spatial optical field is amplified and modulated by an optical amplifier. The amplification factor of the optical amplifier for different wavelengths is related to the voltage applied at its two ends. The voltage applied at the two ends of the optical amplifier is synchronously adjusted with the multispectral spatial optical field to perform weighted modulation of different spectral signals. The multispectral time calculation module is also used to perform a loop on the modulated multispectral spatial light field in a multispectral temporal buffer composed of optical resonant rings based on the multispectral transfer function, to obtain the first multispectral spatiotemporal sequence of the dynamic light field in the current time frame. During the dynamic input of the multispectral light field, the first multispectral spatiotemporal sequence will be continuously iterated in a fully connected manner until the dynamic input ends, to obtain the second multispectral spatiotemporal sequence corresponding to the dynamic light field. After that, the dynamic light field buffered in the optical resonant ring can use its physical properties to perform nonlinear fusion of information from different spectral dimensions and different times. The nonlinear fusion result will be obtained from the output port of the optical resonant ring. Subsequently, the nonlinear fusion result will be used as a new dynamic light field for further calculation, and finally, features including action type and flicker mode can be inferred from the light field.

4. The architecture according to claim 3, characterized in that, The architecture includes multiple cascaded spatiotemporal spectrum computing units. The preceding spatiotemporal spectrum computing unit among any two adjacent units is designated as the first spatiotemporal spectrum computing unit, and the following spatiotemporal spectrum computing unit among any two adjacent units is designated as the second spatiotemporal spectrum computing unit. The first spatiotemporal spectrum calculation unit is used to acquire the second multispectral spatiotemporal sequence obtained at the end of the dynamic input of the dynamic light field, and to perform nonlinear fusion on the second multispectral spatiotemporal sequence to obtain the nonlinear fusion result; The first spatiotemporal spectrum calculation unit is also used to input the nonlinear fusion result as a dynamic light field into the second spatiotemporal spectrum calculation unit.

5. A spatiotemporal spectral intelligent all-optical reconfigurable characterization system, characterized in that, include: The spatiotemporal spectrum intelligent all-optical reconfigurable characterization architecture as described in claim 3 or 4.

Images