Space-time spectrum intelligent all-optical reconfigurable characterization method, framework and system

Abstract

The invention relates to the technical field of optical computing, in particular to a space-time spectrum intelligent all-optical reconfigurable characterization method, framework and system. The method comprises the following steps: acquiring a multispectral light field input by a dynamic light field under a current time frame, and performing weighted fusion on a spectral space information set corresponding to the multispectral light field to obtain a multispectral space light field; performing time sequence modulation on the multispectral spatial light field to obtain a modulated multispectral spatial light field; and combining the modulated multispectral spatial light field with the multispectral time sequence cache corresponding to the dynamic light field to obtain a first multispectral time-space sequence of the dynamic light field under the current time frame. According to the invention, multi-dimensional information processing can be realized while the high efficiency of optical calculation is maintained.

Description

Technical Field

[0001] The present disclosure relates to the field of optical computing technology, and in particular to a space-time spectrum intelligent all-optical reconfigurable characterization method, architecture and system. Background Art

[0002] With the rapid development of artificial intelligence and scientific computing, the complexity and scale of computing needs are also increasing. However, the existing electronic computing technology is limited by Moore's Law, and its performance is gradually approaching saturation, making it difficult to effectively cope with the increasingly stringent requirements of large-scale complex algorithms on computing power and power consumption. Light has natural advantages such as high throughput and low latency in the propagation process. Optical computing technology that uses photons instead of electrons as computing carriers is seen as the key to breaking the existing computing bottleneck.

[0003] However, current optical computing architectures are typically only capable of computing in a single dimension (such as the amplitude or phase of light) and in most cases rely on electronic processors as relays, failing to fully tap the potential of optical computing in high-speed, massively parallel processing. Summary of the invention

[0004] The present disclosure aims to solve one of the technical problems in the related art at least to some extent.

[0005] To this end, the first objective of the present disclosure is to propose a space-time spectrum intelligent all-optical reconfigurable characterization method to achieve multi-dimensional information processing while maintaining the high efficiency of optical computing.

[0006] The second objective of the present disclosure is to propose a space-time-spectrum intelligent all-optical reconfigurable representation architecture.

[0007] The third objective of the present disclosure is to propose a space-time-spectrum intelligent all-optical reconfigurable characterization system.

[0008] To achieve the above-mentioned purpose, the first embodiment of the present disclosure proposes a space-time spectrum intelligent all-optical reconfigurable characterization method, including: Acquire a multispectral light field inputted in the current time frame of the dynamic light field, and perform weighted fusion on a set of spectral spatial information corresponding to the multispectral light field to obtain a multispectral spatial light field; Performing time-sequential modulation on the multi-spectral spatial light field to obtain a modulated multi-spectral spatial light field; The modulated multi-spectral spatial light field is merged with the multi-spectral temporal buffer corresponding to the dynamic light field to obtain a first multi-spectral spatiotemporal sequence of the dynamic light field in a current time frame.

[0009] Optionally, the weighted fusion of the spectral spatial information set corresponding to the multi-spectral light field includes: The spectral spatial information set corresponding to the multi-spectral light field is weightedly fused by means of pre-sensing calculation.

[0010] Optionally, the weighted fusion of the spectral spatial information set corresponding to the multi-spectral light field by means of pre-sensing calculation includes: Constructing a signal weighting function, wherein the signal weighting function is obtained by combining the metasurface and / or the liquid crystal array; The signal weighting function is used to perform weighted fusion on the spectral spatial information set corresponding to the multi-spectral light field.

[0011] Optionally, the signal weighting function includes at least one of the following function forms: Weighted connections; Pointwise multiplication.

[0012] Optionally, performing time-sequential modulation on the multi-spectral spatial light field to obtain a modulated multi-spectral spatial light field includes: The multi-spectral spatial light field is subjected to time-series multiplication calculation by weights to obtain a modulated multi-spectral spatial light field.

[0013] Optionally, merging the modulated multi-spectral spatial light field with the multi-spectral temporal buffer corresponding to the dynamic light field includes: The modulated multi-spectral spatial light field is injected into the optical resonance ring corresponding to the dynamic light field, so as to merge the modulated multi-spectral spatial light field with the multi-spectral timing buffer formed by the optical resonance ring.

[0014] Optionally, merging the modulated multi-spectral spatial light field with the multi-spectral timing buffer formed by the optical resonant ring includes: Based on the multi-spectral transfer function, the modulated multi-spectral spatial light field is circulated once in the multi-spectral timing buffer composed of the optical resonant ring.

[0015] To achieve the above-mentioned purpose, the second aspect of the present disclosure proposes a space-time spectrum intelligent all-optical reconfigurable characterization architecture, including: at least one space-time spectrum calculation unit, the space-time spectrum calculation unit includes a multi-spectral space calculation module and a multi-spectral time calculation module, wherein: The multispectral spatial calculation module is used to obtain a multispectral light field input by the dynamic light field in the current time frame, and to perform weighted fusion on a set of spectral spatial information corresponding to the multispectral light field to obtain a multispectral spatial light field; The multi-spectral time calculation module is used to perform time-series modulation on the multi-spectral spatial light field to obtain a modulated multi-spectral spatial light field; The multispectral time calculation module is further used to merge the modulated multispectral spatial light field with the multispectral time sequence cache corresponding to the dynamic light field to obtain a first multispectral space-time sequence of the dynamic light field in the current time frame.

[0016] Optionally, the architecture includes a plurality of cascaded space-time spectrum calculation units, wherein the first space-time spectrum calculation unit of any two adjacent space-time spectrum calculation units in the plurality of cascaded space-time spectrum calculation units is a first space-time spectrum calculation unit, and the second space-time spectrum calculation unit of any two adjacent space-time spectrum calculation units in the plurality of cascaded space-time spectrum calculation units is a second space-time spectrum calculation unit; wherein, The first space-time spectrum calculation unit is used to obtain a second multi-spectral space-time sequence obtained when the dynamic input of the dynamic light field ends, and perform nonlinear fusion on the second multi-spectral space-time sequence to obtain a nonlinear fusion result; The first space-time spectrum calculation unit is further used to input the nonlinear fusion result as a dynamic light field into the second space-time spectrum calculation unit.

[0017] To achieve the above-mentioned purpose, the third aspect of the present disclosure proposes a space-time-spectrum intelligent all-optical reconfigurable characterization system, including: the space-time-spectrum intelligent all-optical reconfigurable characterization architecture shown in any one of the aforementioned second aspects.

[0018] In summary, the space-time-spectrum intelligent all-optical reconfigurable characterization method, architecture and system provided by the present invention, by utilizing the characteristics of light in spectral, spatial and temporal dimensions, jointly modeling the space, time and spectrum (wavelength) dimensions of light during the propagation process, can make full use of the space, time and spectrum (wavelength) information contained in the light field during the propagation process, can break through the limitations of existing intelligent scene perception technology, can build a match across the three dimensions of space, time and spectrum, and realize full connection across time and spectrum dimensions. It can effectively break through the bottleneck of digital memory reading and writing, realize the seamless integration of multi-dimensional optical computing, and realize multi-dimensional information processing while maintaining the high efficiency of optical computing.

[0019] Additional aspects and advantages of the present disclosure will be given in part in the following description and in part will be obvious from the following description or learned through practice of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and / or additional aspects and advantages of the present disclosure will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which: Figure 1 A schematic diagram of a flow chart of a space-time spectrum intelligent all-optical reconfigurable characterization method provided in an embodiment of the present disclosure; Figure 2 A schematic flow chart of a space-time spectrum intelligent all-optical reconfigurable characterization method provided by another embodiment of the present disclosure; Figure 3 A schematic diagram of the principle of a space-time spectrum intelligent all-optical reconfigurable characterization method provided by an embodiment of the present disclosure; Figure 4 A schematic diagram of the structure of a space-time-spectrum intelligent all-optical reconfigurable characterization architecture provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0021] Embodiments of the present disclosure are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, and should not be construed as limiting the present disclosure.

[0022] With the rapid development of artificial neural network (ANN) technology, the performance and complexity of machine vision algorithms have been significantly improved, resulting in a continuous increase in the demand for high computing power. The bottleneck of traditional electronic computing architecture is its high power consumption and limited computing speed, which cannot fully meet the high parallel processing capabilities required for dynamic machine vision processing.

[0023] However, traditional optical computing architectures need to rely on sensors to perceive and store multi-spectral spatial dynamic signals. The photoelectric conversion in this step often becomes the speed bottleneck of the system, and the problem of reduced light field pattern diversity often occurs. Directly coupling multi-spectral spatial content to multi-spectral single-mode time channels may lead to the loss of important information.

[0024] The present disclosure is described in detail below with reference to specific embodiments.

[0025] In the first embodiment, if Figure 1 As shown, Figure 1 This is a flow chart of a space-time spectrum intelligent all-optical reconfigurable characterization method provided by an embodiment of the present disclosure. The method can be implemented by a computer program and can be run on an architecture for space-time spectrum intelligent all-optical reconfigurable characterization. The computer program can be integrated into an application or run as an independent tool application.

[0026] Among them, the space-time spectrum intelligent all-optical reconfigurable characterization method can be executed by a space-time spectrum intelligent all-optical reconfigurable characterization architecture, and can also be executed by a space-time spectrum intelligent all-optical reconfigurable characterization system including the space-time spectrum intelligent all-optical reconfigurable characterization architecture.

[0027] For example, the space-time spectrum intelligent all-optical reconfigurable characterization method includes the following steps: S101, obtaining a multispectral light field inputted by the dynamic light field in the current time frame, and performing weighted fusion on a spectral spatial information set corresponding to the multispectral light field to obtain a multispectral spatial light field; According to some embodiments, a dynamic light field refers to a time-varying light field from a dynamic scene, and the dynamic light field may be, for example, a multi-spectral light field at the millisecond to nanosecond level.

[0028] In some embodiments, the dynamic input of the dynamic light field can be modeled as a high-dimensional time stack, each element of which represents a multi-spectral spatial light field at a certain moment. The current time frame refers to the time frame of the current moment.

[0029] According to some embodiments, the spectral spatial information refers to the information related to the spectrum in the spatial dimension. The spatial dimension includes but is not limited to amplitude, phase, polarization, etc.

[0030] In some embodiments, the spectral spatial information set refers to a set formed by gathering at least one spectral spatial information.

[0031] S102, performing time-sequential modulation on the multi-spectral spatial light field to obtain a modulated multi-spectral spatial light field; According to some embodiments, temporal modulation refers to modulating the multi-spectral spatial light field in the time dimension.

[0032] It should be noted that due to the parallel characteristics of light propagating in free space, the process of acquiring the multi-spectral spatial light field can be completed quickly before the multi-spectral light field of the next time frame arrives. Therefore, the information capacity of the system can be increased, ensuring seamless connection between the spatial dimension and the parallel spectral and high-speed time dimensions. S103, merging the modulated multi-spectral spatial light field with the multi-spectral temporal buffer corresponding to the dynamic light field to obtain a first multi-spectral spatiotemporal sequence of the dynamic light field in the current time frame.

[0033] According to some embodiments, the multispectral time series buffer refers to a multispectral spatiotemporal sequence prior to a current time frame.

[0034] In some embodiments, the first multispectral space-time sequence refers to the multispectral space-time sequence in the current time frame. After the next time frame arrives, the first multispectral space-time sequence will be merged with the modulated multispectral spatial light field of the next time frame as a new multispectral timing buffer.

[0035] In summary, the method provided in this embodiment, by utilizing the characteristics of light in the spectral, spatial and temporal dimensions, jointly models the spatial, temporal and spectral (wavelength) dimensions of light during propagation, can make full use of the spatial, temporal and spectral (wavelength) information contained in the light field during propagation, can break through the limitations of existing intelligent scene perception technology, can build a match across the three dimensions of space, time and spectrum, and achieve full connection across time and spectrum dimensions. It can effectively break through the bottleneck of digital memory reading and writing, realize the seamless integration of multi-dimensional optical computing, and realize multi-dimensional information processing while maintaining the high efficiency of optical computing. Compared with traditional computing methods, it can significantly improve the speed of dynamic machine vision processing, has broad application prospects, and is expected to bring new opportunities for high-performance intelligent computing in the post-Moore era and real-time analysis and control of transient scientific phenomena, and be applied to unmanned systems, autonomous driving, ultrafast science and other fields.

[0036] This embodiment also provides another space-time spectrum intelligent all-optical reconfigurable characterization method. The method can be executed by a space-time spectrum intelligent all-optical reconfigurable characterization architecture, or by a space-time spectrum intelligent all-optical reconfigurable characterization system including the space-time spectrum intelligent all-optical reconfigurable characterization architecture.

[0037] For example, Figure 2 As shown, the space-time spectrum intelligent all-optical reconfigurable characterization method may include the following steps: S201, obtaining a multi-spectral light field inputted by a dynamic light field in a current time frame; According to some embodiments, in dynamic scenes, multi-spectral light fields at the millisecond to nanosecond scale can be input through free space.

[0038] S202, performing weighted fusion on the spectral spatial information set corresponding to the multi-spectral light field by means of pre-sensing calculation to obtain a multi-spectral spatial light field; According to some embodiments, pre-sensing computing belongs to free-space diffraction optical computing, which is a technology that uses optical components at the front end of a photoelectric sensor to modulate and process light information to achieve computational reasoning.

[0039] In some embodiments, when weighted fusion is performed on the spectral spatial information set corresponding to the multi-spectral light field by pre-sensing calculation, a signal weighting function can be constructed; the signal weighting function is used to weightedly fuse the spectral spatial information set corresponding to the multi-spectral light field. Therefore, the effect of weighted fusion can be improved.

[0040] In some embodiments, the signal weighting function includes but is not limited to at least one of the following functional forms: Weighted connections; Pointwise multiplication.

[0041] According to some embodiments, the signal weighting function can be obtained by combining the metasurface and / or the liquid crystal array. Different combinations of metasurfaces and liquid crystal arrays can form different signal weighting functions. Secondly, the metasurface and liquid crystal can be cascaded in a system to complete the function of information fusion, or they can be disassembled as separate devices to pre-process the information, depending on the tasks and functions to be achieved.

[0042] In some embodiments, when a signal weighting function is used to weight the spectral spatial information set corresponding to the multi-spectral light field, since the metasurface has different responses to different wavelengths, the metasurface can weight the multi-spectral signal through pre-design. In the liquid crystal array and the subsequent diffraction propagation process, information of different wavelengths will be fused. Therefore, the whole composed of the metasurface and the liquid crystal array can complete the weighted fusion process of the multi-spectral light field signal.

[0043] Among them, metasurfaces can be used to extract and fuse information of different dimensions (amplitude, phase, polarization, etc.) of multi-spectral light fields. The fused signal can be considered to contain information of all dimensions in a certain dimension, which can ensure the completeness of the information.

[0044] The liquid crystal array can be used to modulate the phase and amplitude of the multi-spectral light field. During the modulation process, the characteristic signals related to the final task will be initially extracted, while other redundant signals will be compressed. The weight of the liquid crystal array can be adjusted to the value most suitable for the intelligent task through joint modeling and simulation training.

[0045] That is to say, in the process of weighted fusion of the spectral spatial information set corresponding to the multi-spectral light field through the metasurface and the liquid crystal array, the multi-spectral spatial light field can be feature extracted and the channel expanded. Among them, in a single time frame, pre-sensing calculations can be performed in each multi-spectral light field. For a certain frame, multiple rapidly changing pre-sensing calculation weights can be simultaneously corresponded, which is equivalent to expanding the time channel depth N times that of a single-frame multi-spectral light field.

[0046] S203, performing time-series multiplication calculation on the multi-spectral spatial light field through weights to obtain a modulated multi-spectral spatial light field; According to some embodiments, by performing time-series multiplication calculation on the multi-spectral spatial light field through weights, weighted modulation of the multi-spectral spatial light field can be achieved to obtain a modulated multi-spectral spatial light field.

[0047] In some embodiments, the multi-spectral spatial light field can be amplified and modulated by an optical amplifier in the optical fiber array, and the amplification factor of the optical amplifier for different wavelengths is related to the voltage loaded at both ends thereof.

[0048] In some embodiments, the voltage loaded on both ends of the optical amplifier is adjusted synchronously with the multi-spectral spatial light field, thereby achieving weighted modulation of different spectral signals.

[0049] S204, injecting the modulated multi-spectral spatial light field into the optical resonant ring corresponding to the dynamic light field, so as to merge the modulated multi-spectral spatial light field with the multi-spectral timing buffer composed of the optical resonant ring, and obtain a first multi-spectral spatiotemporal sequence of the dynamic light field in the current time frame.

[0050] According to some embodiments, information storage and connection between dimensions can be achieved by constructing an optical resonant ring, which can not only provide a time buffer from picoseconds to nanoseconds, but also weight information of different spectra. In the optical resonant ring, multi-spectral spatial information at different times will be connected.

[0051] In some embodiments, the resonant characteristics of the optical resonant ring for light of different wavelengths and the caching characteristics for light fields of different times can be used to achieve the addition (merging) of the modulated multi-spectral spatial light field and the multi-spectral timing cache, which completes the full connection across time and spectral dimensions.

[0052] Among them, a certain frame of multi-spectral spatial light field enters the optical resonant ring, and the optical resonant ring can modulate the light field with a wavelength near the resonant wavelength. After the light with a wavelength equal to the resonant wavelength enters the resonant ring and circulates once, it will coherently cancel out the light with the same wavelength input at this time, and the amplitude will drop sharply. The resonant wavelength can be changed by the voltage applied to both ends of the resonant ring. By synchronizing the voltage with the dynamic input light field, the multi-spectral spatial light field at different times can be cached and modulated.

[0053] According to some embodiments, based on the multispectral transfer function, the modulated multispectral spatial light field may be circulated once in a multispectral timing buffer composed of an optical resonant ring to achieve merging between the modulated multispectral spatial light field and the multispectral timing buffer.

[0054] Among them, every time a frame of multispectral spatial light field circulates once in the multispectral timing buffer composed of the optical resonance ring, the cyclic signal will be multispectral transfer function This function is closely related to the physical properties of the optical resonant ring, and can fully integrate the information of different spectral dimensions and the light field at the current moment and the light field at the past moments.

[0055] It should be noted that in the process of dynamic light field dynamically inputting multispectral light field, the first multispectral spatiotemporal sequence will be continuously iterated in a fully connected manner until the dynamic input ends, and the second multispectral spatiotemporal sequence corresponding to the dynamic light field is obtained. After that, the dynamic light field cached in the optical resonant ring can use its physical properties to perform nonlinear fusion of information of different spectral dimensions and different times, and the nonlinear fusion result will be obtained from the output port of the optical resonant ring. Subsequently, the nonlinear fusion result can be used as a new dynamic light field for further calculation. Ultimately, features including action type, flickering pattern, etc. can be inferred from the light field.

[0056] According to some embodiments, Figure 3 The schematic diagram of the principle of a space-time spectrum intelligent all-optical reconfigurable characterization method provided by the embodiment of the present disclosure. Figure 3 As shown, the propagating light field in space contains dimensions such as space, time, polarization, phase, amplitude and frequency. Through space-time domain characterization, the optical network can complete the joint optimization and information processing of space and time dimensions; through multi-spectral characterization, the optical network can modulate the phase and amplitude of light waves at different frequencies. The method provided by the embodiment of the present disclosure can realize the joint characterization of space and time spectrum by combining the above two characterizations in the same system. Information can be converted in three dimensions of spectrum, space and time, which is more advantageous than the single-dimensional processing method that can easily reach its limit.

[0057] It should be noted that in advanced electronic computers, the read and write operations of the memory are usually accompanied by delays of more than 100 nanoseconds. Frequent data reading and writing limits the ability to process nanosecond dynamic events in real time, inhibiting the potential of artificial neural networks in ultra-high-speed applications. The method provided in this embodiment overcomes the dependence on memory reading and writing by jointly optimizing the calculation, docking and buffer modules of the space-time spectrum dimension, which can greatly expand the application scope of photon computing and lay a solid foundation for the control of ultra-high-speed machine vision processing and optical transient experiments. At the same time, with the advancement of optical modulator and detector technology, its bandwidth has reached hundreds of GHz. Combined with low-loss photonic integrated circuits, the proposed architecture can achieve a time resolution of more than 10 picoseconds. This high-speed computing capability not only accelerates the operation of neural networks, but also supports real-time analysis of non-repetitive instantaneous visual phenomena, thereby realizing optical chip feedback control of dynamic light fields with frequencies above GHz.

[0058] In summary, the method provided in this embodiment first obtains the multi-spectral light field input by the dynamic light field in the current time frame; performs weighted fusion of the spectral spatial information set corresponding to the multi-spectral light field by pre-sensing calculation to obtain the multi-spectral spatial light field; therefore, the effect of weighted fusion of the spectral spatial information set corresponding to the multi-spectral light field can be improved. Then, the multi-spectral spatial light field is calculated by time series multiplication through weights to obtain the modulated multi-spectral spatial light field, so that the accuracy of obtaining the modulated multi-spectral spatial light field can be improved. Finally, the modulated multi-spectral spatial light field is injected into the optical resonant ring corresponding to the dynamic light field to merge the modulated multi-spectral spatial light field with the multi-spectral time series cache composed of the optical resonant ring to obtain the first multi-spectral spatiotemporal sequence of the dynamic light field in the current time frame, so that full connection across time and across spectral dimensions can be completed, and multi-dimensional information processing can be realized while maintaining the high efficiency of optical computing.

[0059] In order to implement the above embodiments, the present disclosure also proposes a space-time-spectrum intelligent all-optical reconfigurable characterization architecture.

[0060] like Figure 4 As shown, the space-time spectrum intelligent all-optical reconfigurable characterization architecture 400 includes: at least one space-time spectrum calculation unit 410, the space-time spectrum calculation unit includes a multi-spectral space calculation module 411 and a multi-spectral time calculation module 412, wherein: The multi-spectral spatial calculation module 411 is used to obtain the multi-spectral light field input by the dynamic light field in the current time frame, and perform weighted fusion on the spectral spatial information set corresponding to the multi-spectral light field to obtain the multi-spectral spatial light field; The multi-spectral time calculation module 412 is used to perform time-series modulation on the multi-spectral spatial light field to obtain a modulated multi-spectral spatial light field; The multispectral time calculation module 412 is further used to merge the modulated multispectral spatial light field with the multispectral time sequence buffer corresponding to the dynamic light field to obtain a first multispectral space-time sequence of the dynamic light field in the current time frame.

[0061] It should be noted that, by utilizing the combined operation of spectrum, space and time, each space-time spectrum calculation unit 410 can effectively extract information and derive semantic content, thereby achieving deep fusion and processing of information.

[0062] Optionally, the space-time spectrum intelligent all-optical reconfigurable characterization architecture 400 includes a plurality of cascaded space-time spectrum computing units 410, wherein the first space-time spectrum computing unit in any two adjacent space-time spectrum computing units in the plurality of cascaded space-time spectrum computing units is a first space-time spectrum computing unit, and the second space-time spectrum computing unit in any two adjacent space-time spectrum computing units in the plurality of cascaded space-time spectrum computing units is a second space-time spectrum computing unit; wherein, A first space-time spectrum calculation unit is used to obtain a second multi-spectral space-time sequence obtained when the dynamic input of the dynamic light field ends, and perform nonlinear fusion on the second multi-spectral space-time sequence to obtain a nonlinear fusion result; The first space-time spectrum calculation unit is also used to input the nonlinear fusion result as a dynamic light field into the second space-time spectrum calculation unit.

[0063] It should be noted that the more space-time spectrum calculation units are cascaded, the stronger the overall network performance will be. The number of space-time spectrum calculation units required can be determined according to the actual intelligent task. For example, if the computing power of a single space-time spectrum calculation unit is sufficient to process the target intelligent task, the calculation result can be output after the calculation of the first space-time spectrum calculation unit is completed, and the light field characteristics can be inferred.

[0064] Optionally, when the multi-spectral spatial calculation module 411 is used to perform weighted fusion on the spectral spatial information set corresponding to the multi-spectral light field, it is specifically used to: The spectral spatial information set corresponding to the multi-spectral light field is weightedly fused by pre-sensing calculation.

[0065] Optionally, when the multi-spectral spatial calculation module 411 is used to perform weighted fusion on the spectral spatial information set corresponding to the multi-spectral light field by means of pre-sensing calculation, it is specifically used to: Constructing a signal weighting function, wherein the signal weighting function is obtained by combining the metasurface and / or the liquid crystal array; The signal weighting function is used to perform weighted fusion on the spectral spatial information set corresponding to the multi-spectral light field.

[0066] Optionally, the signal weighting function includes at least one of the following function forms: Weighted connections; Pointwise multiplication.

[0067] Optionally, the multi-spectral time calculation module 412 is used to perform time-series modulation on the multi-spectral spatial light field to obtain the modulated multi-spectral spatial light field, specifically for: The multi-spectral spatial light field is subjected to time-series multiplication calculation through weights to obtain a modulated multi-spectral spatial light field.

[0068] Optionally, when the multi-spectral time calculation module 412 is used to merge the modulated multi-spectral spatial light field with the multi-spectral time sequence buffer corresponding to the dynamic light field, it is specifically used to: The modulated multi-spectral spatial light field is injected into the optical resonance ring corresponding to the dynamic light field, so as to merge the modulated multi-spectral spatial light field with the multi-spectral timing buffer composed of the optical resonance ring.

[0069] Optionally, when the multi-spectral time calculation module 412 is used to merge the modulated multi-spectral spatial light field with the multi-spectral timing buffer composed of the optical resonance ring, it is specifically used to: Based on the multispectral transfer function, the modulated multispectral spatial light field is circulated once in the multispectral timing buffer composed of optical resonant rings.

[0070] It should be noted that the aforementioned explanation of the embodiment of the space-time-spectrum intelligent all-optical reconfigurable characterization method is also applicable to the space-time-spectrum intelligent all-optical reconfigurable characterization architecture of this embodiment, which will not be repeated here.

[0071] In summary, the space-time-spectrum intelligent all-optical reconfigurable characterization architecture provided by the embodiments of the present invention can make full use of the space, time and spectrum (wavelength) information contained in the light field during the propagation process by utilizing the characteristics of light in the spectral, spatial and temporal dimensions, and jointly modeling the space, time and spectrum (wavelength) dimensions of light during the propagation process. It can break through the limitations of existing intelligent scene perception technology, build a match across the three dimensions of space, time and spectrum, and realize full connection across time and spectrum dimensions. It can effectively break through the bottleneck of digital memory reading and writing, realize the seamless integration of multi-dimensional optical computing, and realize multi-dimensional information processing while maintaining the high efficiency of optical computing.

[0072] In order to implement the above embodiments, the present disclosure further proposes a space-time-spectrum intelligent all-optical reconfigurable characterization system, including: the space-time-spectrum intelligent all-optical reconfigurable characterization architecture provided by the above embodiments.

[0073] In order to implement the above embodiments, the present disclosure also proposes an electronic device, including: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method provided by the above embodiments.

[0074] In order to implement the above embodiments, the present disclosure further proposes a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the methods provided by the above embodiments.

[0075] In order to implement the above embodiments, the present disclosure also proposes a computer program product, including a computer program, which implements the methods provided by the above embodiments when executed by a processor.

[0076] The collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in this disclosure shall comply with the relevant laws and regulations and shall not violate public order and good morals.

[0077] It should be noted that personal information from users should be collected for legitimate and reasonable purposes and should not be shared or sold outside of these legitimate uses. In addition, such collection / sharing should be carried out after receiving the user's informed consent, including but not limited to notifying the user to read the user agreement / user notice and sign the agreement / authorization including authorization of relevant user information before the user uses the function. In addition, any necessary steps should be taken to protect and safeguard access to such personal information data and ensure that others who have access to personal information data comply with its privacy policy and procedures.

[0078] The present disclosure anticipates providing implementation schemes for users to selectively block the use or access of personal information data. That is, the present disclosure anticipates providing hardware and / or software to prevent or block access to such personal information data. Once the personal information data is no longer needed, risks can be minimized by limiting data collection and deleting the data. In addition, when applicable, such personal information is de-identified to protect the privacy of the user.

[0079] The acquisition, transmission, storage, use, and processing of data in the technical solution disclosed in this disclosure are in compliance with the relevant provisions of national laws and regulations.

[0080] It should be noted that in the embodiments of the present disclosure, certain software, components, models and other existing solutions in the industry may be mentioned, which should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0081] In the description of the aforementioned embodiments, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they contradict each other.

[0082] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present disclosure, "plurality" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0083] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present disclosure includes additional implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present disclosure belong.

[0084] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary, and then stored in a computer memory.

[0085] It should be understood that the various parts of the present disclosure can be implemented in hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0086] A person skilled in the art may understand that all or part of the steps in the above-mentioned embodiment method may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0087] In addition, each functional unit in each embodiment of the present disclosure may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0088] The storage medium mentioned above may be a read-only memory, a disk or an optical disk, etc. Although the embodiments of the present disclosure have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations of the present disclosure. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present disclosure.

Claims

1. A space-time spectrum intelligent all-optical reconfigurable characterization method, characterized in that: include: Acquire a multispectral light field inputted in the current time frame of the dynamic light field, and perform weighted fusion on a set of spectral spatial information corresponding to the multispectral light field to obtain a multispectral spatial light field; Performing time-sequential modulation on the multi-spectral spatial light field to obtain a modulated multi-spectral spatial light field; The modulated multi-spectral spatial light field is merged with the multi-spectral temporal buffer corresponding to the dynamic light field to obtain a first multi-spectral spatiotemporal sequence of the dynamic light field in a current time frame.

2. The method according to claim 1, characterized in that The weighted fusion of the spectral spatial information set corresponding to the multi-spectral light field includes: The spectral spatial information set corresponding to the multi-spectral light field is weightedly fused by means of pre-sensing calculation.

3. The method according to claim 2, characterized in that The weighted fusion of the spectral spatial information set corresponding to the multi-spectral light field by pre-sensing calculation includes: Constructing a signal weighting function, wherein the signal weighting function is obtained by combining the metasurface and / or the liquid crystal array; The signal weighting function is used to perform weighted fusion on the spectral spatial information set corresponding to the multi-spectral light field.

4. The method according to claim 3, characterized in that The signal weighting function includes at least one of the following function forms: Weighted connections; Pointwise multiplication.

5. The method according to claim 1, characterized in that The step of temporally modulating the multi-spectral spatial light field to obtain a modulated multi-spectral spatial light field comprises: The multi-spectral spatial light field is subjected to time-series multiplication calculation by weights to obtain a modulated multi-spectral spatial light field.

6. The method according to claim 1, characterized in that The step of merging the modulated multi-spectral spatial light field with the multi-spectral temporal buffer corresponding to the dynamic light field comprises: The modulated multi-spectral spatial light field is injected into the optical resonance ring corresponding to the dynamic light field, so as to merge the modulated multi-spectral spatial light field with the multi-spectral timing buffer formed by the optical resonance ring.

7. The method according to claim 6, characterized in that The step of merging the modulated multi-spectral spatial light field with the multi-spectral timing buffer formed by the optical resonant ring comprises: Based on the multi-spectral transfer function, the modulated multi-spectral spatial light field is circulated once in the multi-spectral timing buffer composed of the optical resonant ring.

8. A space-time spectrum intelligent all-optical reconfigurable representation architecture, characterized in that: include: At least one space-time spectrum calculation unit, the space-time spectrum calculation unit includes a multi-spectral space calculation module and a multi-spectral time calculation module, wherein: The multispectral spatial calculation module is used to obtain a multispectral light field input by the dynamic light field in the current time frame, and to perform weighted fusion on a set of spectral spatial information corresponding to the multispectral light field to obtain a multispectral spatial light field; The multi-spectral time calculation module is used to perform time-series modulation on the multi-spectral spatial light field to obtain a modulated multi-spectral spatial light field; The multispectral time calculation module is further used to merge the modulated multispectral spatial light field with the multispectral time sequence cache corresponding to the dynamic light field to obtain a first multispectral space-time sequence of the dynamic light field in the current time frame.

9. The architecture according to claim 8, characterized in that The architecture includes a plurality of cascaded space-time spectrum calculation units, wherein the first space-time spectrum calculation unit in any two adjacent space-time spectrum calculation units in the plurality of cascaded space-time spectrum calculation units is a first space-time spectrum calculation unit, and the second space-time spectrum calculation unit in any two adjacent space-time spectrum calculation units in the plurality of cascaded space-time spectrum calculation units is a second space-time spectrum calculation unit; wherein, The first space-time spectrum calculation unit is used to obtain a second multi-spectral space-time sequence obtained when the dynamic input of the dynamic light field ends, and perform nonlinear fusion on the second multi-spectral space-time sequence to obtain a nonlinear fusion result; The first space-time spectrum calculation unit is further used to input the nonlinear fusion result as a dynamic light field into the second space-time spectrum calculation unit.

10. A space-time spectrum intelligent all-optical reconfigurable characterization system, characterized in that: include: The space-time-spectrum intelligent all-optical reconfigurable characterization architecture as described in claim 8 or 9.

Images