Training and task processing result generation method and apparatus for optical computing system

Abstract

The disclosure provides a training method and a task processing result generation method of an optical computing system, and relates to the fields of artificial intelligence, optical computing and metasurface devices, etc., wherein the training method can comprise: obtaining a training sample, wherein the training sample comprises a training image and a true label, and the true label is a task processing result corresponding to the training image; generating a predicted label corresponding to the training image through an optical computing system according to an input light field corresponding to the training image, wherein the optical computing system is an optical fitting system of a tropical geometric neural network; determining a loss according to the true label and the predicted label, and updating the configuration parameters of the optical computing system according to the loss. By applying the scheme, the power consumption and delay can be reduced.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the fields of artificial intelligence, optical computing and metasurface devices, and particularly relates to a training method and a task processing result generation method and device of an optical computing system. BACKGROUND

[0002] The inference of a traditional neural network relies on a large number of matrix multiplication and addition operations, and both power consumption and delay are high. For this reason, a tropical geometry neural network is proposed. The tropical geometry neural network is a neural network model based on tropical geometry algebra, which uses a max-plus or min-plus operator to replace the multiplication and addition operation in the traditional neural network. The essence is a piecewise linear / affine selector, which has natural sparsity, good interpretability and high computational efficiency. However, the tropical geometry neural network still needs to be approximated by linear correction units (ReLU, Rectified Linear Unit), maximum pooling, sparse tensors and other means on an electronic chip, and the computational efficiency is limited. SUMMARY

[0003] The present disclosure provides a training method and a task processing result generation method and device of an optical computing system.

[0004] A training method of an optical computing system, comprising:

[0005] Obtaining a training sample, wherein the training sample comprises a training image and a true label, and the true label is a task processing result corresponding to the training image;

[0006] Generating a predicted label corresponding to the training image by the optical computing system according to an input light field corresponding to the training image, wherein the optical computing system is an optical fitting system of a tropical geometry neural network;

[0007] Determining a loss according to the true label and the predicted label, and updating configuration parameters of the optical computing system according to the loss.

[0008] A task processing result generation method, comprising:

[0009] Obtaining a target image to be processed;

[0010] Generating a task processing result corresponding to the target image by an optical computing system, wherein the configuration parameters of the optical computing system are obtained by training according to the above training method.

[0011] An optical computing system, comprising: an affine mapping module, a tropical selection module and a readout module.

[0012] The affine mapping module is configured to generate intensity scalars of each affine channel according to an input light field corresponding to the target image to be processed.

[0013] The hot-tropic selection module is configured to perform nonlinear selection on each affine channel according to the intensity scalars to obtain winning channels selected from each affine channel.

[0014] The readout module is configured to generate a task processing result corresponding to the target image according to the selection result.

[0015] A training device of an optical computing system includes a sample acquisition module, a label acquisition module, and a parameter update module.

[0016] The sample acquisition module is configured to acquire training samples, wherein the training samples include training images and true labels, and the true labels are task processing results corresponding to the training images.

[0017] The label acquisition module is configured to generate predicted labels corresponding to the training images by the optical computing system according to input light fields corresponding to the training images, wherein the optical computing system is an optical fitting system of a hot-tropic geometric neural network.

[0018] The parameter update module is configured to determine a loss according to the true labels and the predicted labels, and to update configuration parameters of the optical computing system according to the loss.

[0019] A task processing result generation device includes an image acquisition module and a result generation module.

[0020] The image acquisition module is configured to acquire a target image to be processed.

[0021] The result generation module is configured to generate a task processing result corresponding to the target image by using an optical computing system, wherein configuration parameters of the optical computing system are obtained by training according to the above training method.

[0022] An electronic device includes:

[0023] at least one processor; and

[0024] a memory communicatively connected to the at least one processor; wherein

[0025] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described above.

[0026] A non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the methods described above.

[0027] It should be understood that the descriptions in this section are not intended to identify key or essential features of the embodiments of this disclosure, nor are they intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0028] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure.

[0029] Figure 1 This is a flowchart of an embodiment of the training method for the optical computing system described in this disclosure;

[0030] Figure 2 This is a flowchart of an embodiment of the task processing result generation method described in this disclosure;

[0031] Figure 3 This is a schematic diagram of the composition structure of an embodiment of the optical computing system described in this disclosure;

[0032] Figure 4 This is a schematic diagram of the composition structure of an embodiment of the training device for the optical computing system described in this disclosure;

[0033] Figure 5 This is a schematic diagram of the composition structure of an embodiment of the task processing result generation device described in this disclosure;

[0034] Figure 6 A schematic block diagram of an electronic device that can be used to implement embodiments of the present disclosure is shown. Detailed Implementation

[0035] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0036] Furthermore, it should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0037] Figure 1This is a flowchart illustrating an embodiment of the training method for the optical computing system described in this disclosure. Figure 1 As shown, the specific implementation methods are as follows.

[0038] In step 101, training samples are obtained, which include training images and real labels, wherein the real labels are the task processing results corresponding to the training images.

[0039] In step 102, based on the input light field corresponding to the training image, a prediction label corresponding to the training image is generated by an optical computing system, wherein the optical computing system is an optical fitting system for a tropical geometric neural network.

[0040] In step 103, the loss is determined based on the real label and the predicted label, and the configuration parameters of the optical computing system are updated based on the loss.

[0041] The above-described approach utilizes an optical computing system as the optical fitting system for the tropical geometric neural network. This involves mapping the tropical geometric neural network to optical paths and training the optical computing system accordingly. Consequently, forward propagation of the tropical geometric neural network can be realized in the physical domain. This means that the core operators of the tropical geometric neural network are directly and natively implemented through the physical properties of light, rather than through mathematical approximation, thereby improving computational efficiency and significantly reducing power consumption and latency.

[0042] The scheme disclosed herein does not impose any restrictions on how to construct training samples. For example, the task processing results can be generated by manual annotation or automatic annotation; taking image classification as an example, the task processing result is the image classification result.

[0043] Training images can be converted into light fields and input into an optical computing system via spatial light modulators (SLMs) or micro-projection. The light field can be represented by a complex electric field, E(r), with intensity... Based on the light field, a predictive label corresponding to the training image can be generated using an optical computing system.

[0044] In some embodiments of this disclosure, the optical computing system may include an affine mapping (Metasurface-Affine) module, a tropical selection (DMD-Tropical) module, and a readout module. Accordingly, the method for generating prediction labels corresponding to training images may include: setting a first parameter and its initial value, for example, setting the initial value to 0, and determining the input light field corresponding to the training image as the target light field; performing the following first processing: based on the target light field, generating intensity scalars for each affine channel using the affine mapping module; based on the intensity scalars, performing nonlinear selection on each affine channel using the tropical selection module to obtain the winning channel selected from each affine channel; incrementing the value of the first parameter by one; in response to determining that the value of the first parameter is equal to M; generating prediction labels using the readout module based on the selection result, where M is a positive integer greater than 1; in response to determining that the value of the first parameter is less than M; generating an updated target light field based on the selection result; and repeating the first processing. An affine channel refers to an optical response unit formed on a relay plane after linear transformation and bias superposition of the input light field, which is spatially separated and independently addressable.

[0045] The affine mapping module, tropical selection module, and readout module can also be referred to as the affine mapping layer, tropical selection layer, and readout layer, respectively.

[0046] The affine mapping module can be a metasurface-based affine mapping module. Metasurfaces can precisely control the phase, amplitude, and polarization of light through subwavelength scale structures, providing a physical basis for optical parallel computing. Depending on the specific needs, other diffractive optical elements (DOEs) can also be used to replace metasurfaces.

[0047] The tropical selection module can be a digital micromirror device (DMD), or, depending on the actual needs, can be replaced by optical phase change material elements, silicon-based optical phased arrays, etc.

[0048] In practical applications, the components of an optical computing system can be realized through optical simulation or through physical entities. If physical entities are used, metasurfaces can be programmable metasurfaces, etc.

[0049] Furthermore, it can be seen that the scheme described in this disclosure can adopt a cyclic processing method, requiring only a single affine mapping module and a single tropical selection module. The effect of connecting multiple modules end to end can be achieved through an optical feedback loop, thereby effectively simulating a multi-layer network structure and saving hardware resources.

[0050] The initial value of the first parameter can be 0. After each execution of the first process, the value of the first parameter can be incremented by 1. Once the value of the first parameter is determined to be equal to M, the loop can be terminated. The specific value of M can be determined according to actual needs, such as 2.

[0051] In each execution of the first process, the intensity scalar of each affine channel can be generated first, based on the target light field, using the affine mapping module. In some embodiments of this disclosure, the affine mapping module can be used to perform an affine transformation on the target light field to obtain an intermediate light field. Then, based on the indicator function corresponding to each affine channel and the intermediate light field, the channel light field of each affine channel can be generated, and then the intensity scalar of each affine channel can be generated based on the channel light field.

[0052] Metasurface + free-space propagation = linear operator. The affine mapping module assumes the responsibility of the "linear part," and its optical equivalent is the formation of a set of spatially separated, addressable response spots on the relay plane. each Each corresponds to an affine channel, and a single metasurface supports... The affine channel space can be reused, and if necessary, it can be used in conjunction with a 4F Imaging System to isolate channel crosstalk.

[0053] When the first processing is executed for the first time, the target light field can be the input light field corresponding to the training image. The process of using the affine mapping module to perform an affine transformation on the target light field to obtain the intermediate light field can be equivalently represented as follows:

[0054] (1)

[0055] in, This indicates the nth time the first process is executed; it is 1 for the first execution. , Let these represent the linear operators for propagation and imaging in this processing, respectively. Indicates the metasurface complex transmittance (or reflectance) coefficient. This represents the intermediate optical field obtained after the affine transformation by the affine mapping module, but before it has been selected by the tropical selection module. This represents the total number of linear operators (propagation, pointwise multiplication, propagation) involved in this processing. This indicates the bias / reference light used in this processing step. This represents the target light field.

[0056] Additionally, a set of spatial channels or ports can be designed. The Region of Interest (ROI) refers to the ROI on the relay plane of the tropical selection module. This represents the number of affine channels, which equals the number of Regions of Interest (ROIs). Each affine channel corresponds to a specific ROI. In the affine mapping module, high channel consistency in phase delay and amplitude transmission can be achieved through element geometry, orientation, and supercell design. Diffraction or refocusing design is used to focus the energy of the i-th affine channel onto... .

[0057] Based on the indicator function and intermediate light field corresponding to each affine channel, the channel light field of each affine channel can be generated respectively. Furthermore, the intensity scalar of each affine channel can be generated respectively based on the channel light field.

[0058] That is:

[0059] (2)

[0060] (3)

[0061] (4)

[0062] in, This represents the channel light field of the i-th affine channel, that is, the complete light field distribution of the i-th affine channel. Indicates the indicator function used to obtain information from the complete intermediate light field. The portion belonging only to the i-th affine channel is "cut out" from the middle. The intensity scalar of the i-th affine channel is the value corresponding to affine channel i. The total light energy or integrated light intensity within the light source. Tropical Affine Function Monotonic correspondence (which can be approximated as linear after calibration) means that there is a stable and predictable relationship between the intensity scalar obtained by physical measurement and the affine function calculated by mathematics: if one increases, the other also increases, and if one decreases, the other also decreases.

[0063] By adopting the intensity scalar determination method described in this disclosure, a standardized process of "intermediate light field → channel light field → intensity scalar" can be used to generate physical quantities that monotonically correspond to the theoretical affine function for each affine channel, which can be accurately compared and selected by the tropical selection module, thus laying a good foundation for subsequent processing.

[0064] Accordingly, based on the intensity scalar, the tropical selection module can be used to perform nonlinear selection on each affine channel to obtain the winning channel selected from all affine channels. The tropical selection module can achieve nonlinear selection through pixel-level or sub-aperture-level gating (such as starting several channel windows in groups and measuring the corresponding integrated intensity), comparison, and routing.

[0065] In some embodiments of this disclosure, the affine channel with the largest intensity scalar value can be selected from each affine channel using the tropical selection module. Alternatively, the affine channel that meets the following requirement can be selected from each affine channel using the tropical selection module: the intensity scalar value of the affine channel is greater than or equal to the corresponding exclusive threshold. The selected affine channel is then determined as the winning channel.

[0066] The method of selecting the affine channel with the largest intensity scalar value directly corresponds to the typical tropical geometric max-plus operation, achieving globally optimal channel selection. The dedicated threshold method, on the other hand, supports more complex nonlinear decisions, allowing each channel to be activated independently, thereby enhancing the model's expressive power and sparsity. The specific method used can be chosen according to actual needs, offering great flexibility and convenience.

[0067] In addition, in some embodiments of this disclosure, routing masks corresponding to each affine channel can be generated separately. For any affine channel, in response to determining that the affine channel is the winning channel, the routing mask corresponding to the affine channel can be set to 1, and in response to determining that the affine channel is the non-winning channel, the routing mask corresponding to the affine channel can be set to 0.

[0068] That is:

[0069] (5)

[0070] in, The index representing the winning channel indicates that in In which of the affine channels is the intensity scalar? Take the largest value, for example, = 3 indicates that the intensity scalar value of the 3rd affine channel is the largest. The routing mask is a binary function (taking values ​​of 0 or 1), which is equivalent to generating a control signal for each affine channel i: = 1 indicates that all micromirrors corresponding to affine channel i on the tropical selection module are switched to the "on" state (+θ) in order to guide their beams to the next layer of coupling aperture, etc. = 0 indicates that all micromirrors corresponding to affine channel i on the tropical selection module are switched to the "off" state (-θ) so that their beams can be driven into the light trap or absorption layer.

[0071] Alternatively, the following threshold control design can be implemented:

[0072] (6)

[0073] in, This represents the specific threshold (activation threshold) corresponding to the i-th affine channel. As long as the intensity scalar of affine channel i is greater than or equal to its specific threshold, it will be activated regardless of other affine channels. That is, all micromirrors corresponding to affine channel i on the tropical selection module will switch to the "on" state (+θ).

[0074] After determining the winning channel, the value of the first parameter can be incremented by 1, and it can be determined whether the value of the first parameter is equal to M. For example, assuming that the value of M is 2, then after the first execution of the first process, the value of the first parameter will be 1, which is less than M. Therefore, the updated target light field can be generated, and the first process can be executed repeatedly.

[0075] In some embodiments of this disclosure, the product of each channel optical field and the corresponding routing mask can be obtained respectively, and the sum of each product can be determined as the composite optical field. A second input optical field can be generated based on the composite optical field and the second propagation operator of the tropical selection module, and the second input optical field can be determined as the updated target optical field. The second propagation operator is the propagation operator corresponding to the loop path.

[0076] That is:

[0077] (7)

[0078] in, Represents the combined light field. This represents the second propagation operator, namely the propagation operator from the tropical selection module to the affine mapping module. This represents the updated target light field.

[0079] Accordingly, the first process can be re-executed based on the updated target light field. For example, based on the target light field, the intensity scalar of each affine channel can be generated using the affine mapping module. Based on the intensity scalar, the affine channel can be nonlinearly selected using the tropical selection module to obtain the winning channel selected from each affine channel, as shown in formulas (1) to (6). The value is 1, now it is 2. Further, the value of the first parameter can be incremented by 1 again, and it can be determined whether the value of the first parameter is equal to M. For example, if the value of M is 2, then the value of the first parameter will be equal to M after being incremented by 1 again, so the loop can be terminated and the prediction label can be generated using the reading module.

[0080] As can be seen, the same set of affine mapping modules and tropical selection modules are reused multiple times in this disclosure, thereby greatly saving hardware resources. Furthermore, through the optical feedback loop, deep network computing can be equivalently achieved using a small number of physical layers. While maintaining the advantages of high parallelism and low energy consumption, the system's flexibility and reconfigurability are significantly improved, and manufacturing costs are reduced.

[0081] In some embodiments of this disclosure, the product of each channel optical field and the corresponding routing mask can be obtained respectively, and the sum of each product is determined as the composite optical field. Then, a first input optical field can be generated based on the composite optical field and the first propagation operator of the tropical selection module. The first propagation operator is the propagation operator corresponding to the readout path. Then, based on the first input optical field, the readout module can generate a predicted label through a vector readout method or a scalar readout method.

[0082] The generation method of the first input light field is similar to that of the second input light field, except that the second propagation operator in formula (7) is replaced by the first propagation operator. The first propagation operator is the propagation operator corresponding to the readout path, that is, the propagation operator from the tropical selection module to the readout module.

[0083] The readout module can be a detector or a complementary metal-oxide-semiconductor (CMOS). The readout module can generate predictive tags using either vector readout or scalar readout methods.

[0084] Assuming M is 2, the vector readout method (maintaining multiple ports) can be:

[0085] (8)

[0086] Scalar readout mode (single port) can be:

[0087] (9)

[0088] in, This indicates the ROI region of the readout module's probe surface.

[0089] The final read z represents the predicted label (such as the image classification result) corresponding to the training image determined by the current optical computing system.

[0090] Furthermore, a loss can be generated based on the real label and the predicted label, such as by generating the loss through cross-entropy, and the configuration parameters of the affine mapping module and the tropical selection module can be updated based on the loss.

[0091] The configuration parameters updated by the affine mapping module may include: the geometric parameters of the metasurface nanostructures, the arrangement period of the metasurface nanostructures, or the control signals applied to the programmable metasurface. The configuration parameters updated by the tropical selection module may include: the spatial division of ROIs, dedicated thresholds, and routing strategies.

[0092] Once the predetermined termination conditions are met, the training of the optical computing system can be terminated. Subsequently, the fabricated optical computing system can be used to generate the desired task processing results.

[0093] Accordingly, Figure 2 This is a flowchart illustrating an embodiment of the task processing result generation method described in this disclosure. Figure 2 As shown, the specific implementation methods are as follows.

[0094] In step 201, the target image to be processed is acquired.

[0095] In step 202, the task processing result corresponding to the target image is generated using an optical computing system. The configuration parameters of the optical computing system are as follows: Figure 1 The method shown is used for training.

[0096] For example, based on the input light field corresponding to the target image, the intensity scalar of each affine channel can be generated by the affine mapping module. Then, based on the intensity scalar, the affine channel can be nonlinearly selected by the tropical selection module to obtain the winning channel selected from each affine channel. Then, based on the selection result, the task processing result can be generated by the readout module. Accordingly, the generation efficiency of the task processing result can be improved.

[0097] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this disclosure is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this disclosure. Secondly, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this disclosure. Furthermore, for parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0098] The above is an introduction to the method embodiments. The following describes the solution described in this disclosure further through device embodiments.

[0099] Figure 3 This is a schematic diagram of the structural composition of an embodiment 300 of the optical computing system described in this disclosure. Figure 3 As shown, it includes: an affine mapping module 301, a tropical selection module 302, and a readout module 303.

[0100] The affine mapping module 301 is used to generate intensity scalars for each affine channel based on the input light field corresponding to the target image to be processed.

[0101] The tropical selection module 302 is used to perform nonlinear selection on each affine channel according to the intensity scalar to obtain the winning channel selected from all affine channels. The configuration parameters of both the affine mapping module 301 and the tropical selection module 302 are based on... Figure 1 The method shown is used for training.

[0102] The readout module 303 is used to generate the task processing result corresponding to the target image based on the selection result.

[0103] In some embodiments of this disclosure, the affine mapping module 301 may include: a metasurface-based affine mapping module.

[0104] Figure 4 This is a schematic diagram of the structural composition of an embodiment 400 of the optical computing system described in this disclosure. Figure 4 As shown, it includes: a sample acquisition module 401, a label acquisition module 402, and a parameter update module 403.

[0105] The sample acquisition module 401 is used to acquire training samples, which include training images and real labels, wherein the real labels are the task processing results corresponding to the training images.

[0106] The label acquisition module 402 is used to generate predicted labels corresponding to the training images based on the input light field corresponding to the training images through an optical computing system. The optical computing system is an optical fitting system for a tropical geometric neural network.

[0107] The parameter update module 403 is used to determine the loss based on the real label and the predicted label, and update the configuration parameters of the optical computing system based on the loss.

[0108] In some embodiments of this disclosure, the optical computing system may include an affine mapping module, a tropical selection module, and a readout module. Accordingly, the label acquisition module 402 may generate predicted labels corresponding to training images by setting a first parameter and its initial value, for example, assuming the initial value is 0, and determining the input light field corresponding to the training image as the target light field, and performing the following first processing: according to the target light field, the affine mapping module generates intensity scalars for each affine channel respectively; according to the intensity scalars, the tropical selection module performs nonlinear selection on each affine channel to obtain the winning channel selected from each affine channel; the value of the first parameter is incremented by 1; in response to determining that the value of the first parameter is equal to M, the readout module generates predicted labels according to the selection result, where M is a positive integer greater than 1; in response to determining that the value of the first parameter is less than M, the updated target light field is generated according to the selection result, and the first processing is repeated.

[0109] In some embodiments of this disclosure, the method by which the tag acquisition module 402 generates the intensity scalar of each affine channel may include: performing an affine transformation on the target light field using an affine mapping module to obtain an intermediate light field, generating the channel light field of each affine channel according to the indicator function corresponding to each affine channel and the intermediate light field, and generating the intensity scalar of each affine channel according to the channel light field.

[0110] In some embodiments of this disclosure, the tag acquisition module 402 uses the tropical selection module to perform nonlinear selection on each affine channel to obtain the winning channel selected from each affine channel. This method may include: using the tropical selection module to select the affine channel with the largest intensity scalar value from each affine channel, or using the tropical selection module to select the affine channel that meets the following requirement: the intensity scalar value of the affine channel is greater than or equal to the corresponding exclusive threshold, and the selected affine channel is determined as the winning channel.

[0111] In some embodiments of this disclosure, the tag acquisition module 402 may also generate routing masks corresponding to each affine channel. For any affine channel, in response to determining that the affine channel is a winning channel, the routing mask corresponding to the affine channel may be set to 1, and in response to determining that the affine channel is a non-winning channel, the routing mask corresponding to the affine channel may be set to 0.

[0112] In some embodiments of this disclosure, the method by which the tag acquisition module 402 generates predicted tags using the readout module may include: acquiring the product of the optical field of each channel and the corresponding routing mask, determining the sum of the products as the comprehensive optical field, generating a first input optical field based on the comprehensive optical field and the first propagation operator of the tropical selection module, wherein the first propagation operator is the propagation operator corresponding to the readout path, and generating predicted tags using the readout module in a vector readout mode or a scalar readout mode based on the first input optical field.

[0113] In some embodiments of this disclosure, the tag acquisition module 402 may generate the updated target light field in the following ways: acquiring the product of each channel light field and the corresponding routing mask, summing the products to determine the composite light field, generating a second input light field based on the composite light field and the second propagation operator of the tropical selection module, and determining the second input light field as the updated target light field, wherein the second propagation operator is the propagation operator corresponding to the loop path.

[0114] Figure 5 This is a schematic diagram of the structural composition of Embodiment 500 of the task processing result generation device described in this disclosure. Figure 5 As shown, it includes: an image acquisition module 501 and a result generation module 502.

[0115] Image acquisition module 501 is used to acquire the target image to be processed.

[0116] Result generation module 502 is used to generate task processing results corresponding to the target image using an optical computing system. The configuration parameters of the optical computing system are as follows: Figure 1 The method shown is used for training.

[0117] The specific workflow of each of the above device embodiments can be found in the relevant descriptions in the foregoing method embodiments, and will not be repeated here.

[0118] In summary, the scheme described in this disclosure uses an affine mapping module responsible for "affine transformation" and a tropical selection module responsible for "tropical nonlinearity." The combination of these components enables low-latency, large-scale parallel network inference, making it highly suitable for edge computing and high-speed inference scenarios. Furthermore, the affine mapping module can be a metasurface-based affine mapping module. Through metasurface technology, high integration can be achieved, reducing the system size. In addition, most of the computation (linear transformation) is performed by the passive metasurface, requiring no power consumption. The tropical selection module, acting as a switch, is also extremely energy-efficient and can support multimodal inputs, while also exhibiting good reconfigurability.

[0119] The images and task processing results in the embodiments described in this disclosure are not targeted at any specific user and are not intended to reflect the personal information of any specific user. The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solutions of this disclosure all comply with relevant laws and regulations and do not violate public order and good morals.

[0120] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0121] Figure 6 A schematic block diagram of an electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0122] like Figure 6As shown, the electronic device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. The RAM 603 may also store various programs and data required for the operation of the electronic device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0123] Multiple components in electronic device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of displays, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows electronic device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0124] The computing unit 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as those described in this disclosure. For example, in some embodiments, the methods described in this disclosure can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the methods described in this disclosure can be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the methods described herein by any other suitable means (e.g., by means of firmware).

[0125] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard parts (ASSPs), systems-on-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0126] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0127] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory (EPROM), flash memory, optical fiber, compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0128] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a cathode ray tube (CRT) or liquid crystal display (LCD) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0129] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0130] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0131] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0132] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A training method for an optical computing system, characterized in that, include: Obtain training samples, which include: training images and real labels, wherein the real labels are the task processing results corresponding to the training images; Based on the input light field corresponding to the training image, the optical computing system generates the predicted label corresponding to the training image. The optical computing system is an optical fitting system for a tropical geometric neural network. The loss is determined based on the real label and the predicted label, and the configuration parameters of the optical computing system are updated based on the loss. The optical computing system includes: an affine mapping module, a tropical selection module, and a readout module; The process of generating the predicted labels corresponding to the training images includes: The first parameter and its initial value are set, and the input light field corresponding to the training image is determined as the target light field. The following first process is performed: Based on the target light field, the intensity scalar of each affine channel is generated using the affine mapping module; Based on the intensity scalar, the affine channel is nonlinearly selected using the tropical selection module to obtain the winning channel selected from each affine channel; The value of the first parameter is incremented by 1. In response to determining that the value of the first parameter is equal to the sum of the initial value and M, the predicted label is generated using the readout module based on the selection result, where M is a positive integer greater than 1. In response to determining that the value of the first parameter is less than the sum of the initial value and M, the updated target light field is generated based on the selection result, and the first process is repeated.

2. The method according to claim 1, characterized in that, The generation of intensity scalars for each affine channel includes: The target light field is subjected to an affine transformation using the affine mapping module to obtain an intermediate light field. Based on the indicator function corresponding to each affine channel and the intermediate light field, the channel light field of each affine channel is generated respectively; Intensity scalars for each affine channel are generated based on the light field of each channel.

3. The method according to claim 2, characterized in that, The nonlinear selection of each affine channel using the tropical selection module to obtain the winning channel from each affine channel includes: The tropical selection module is used to select the affine channel with the largest intensity scalar value from each affine channel, or the tropical selection module is used to select the affine channel that meets the following requirement: the intensity scalar value of the affine channel is greater than or equal to the corresponding exclusive threshold. The selected affine channel is determined as the winning channel.

4. The method according to claim 3, characterized in that, Also includes: Generate routing masks for each affine channel; Specifically, for any affine channel, in response to determining that the affine channel is the winning channel, the routing mask corresponding to the affine channel is set to 1; in response to determining that the affine channel is a non-winning channel, the routing mask corresponding to the affine channel is set to 0.

5. The method according to claim 4, characterized in that, The step of generating the predicted label using the readout module includes: The product of the optical field of each channel and the corresponding routing mask is obtained respectively, and the sum of the products is determined as the comprehensive optical field; A first input light field is generated based on the integrated light field and the first propagation operator of the tropical selection module, wherein the first propagation operator is a propagation operator corresponding to the readout path; Based on the first input light field, the readout module generates the predicted label using either a vector readout method or a scalar readout method.

6. The method according to claim 4, characterized in that, The generation of the updated target light field includes: The product of the optical field of each channel and the corresponding routing mask is obtained respectively, and the sum of the products is determined as the comprehensive optical field; A second input light field is generated based on the integrated light field and the second propagation operator of the tropical selection module. The second input light field is then determined as the updated target light field. The second propagation operator is the propagation operator corresponding to the loop path.

7. A method for generating task processing results, characterized in that, include: Obtain the target image to be processed; The task processing result corresponding to the target image is generated using an optical computing system, wherein the configuration parameters of the optical computing system are obtained by training according to any one of claims 1-6.

8. An optical computing system, characterized in that, include: Affine mapping module, tropical selection module, and readout module; The affine mapping module is used to generate intensity scalars for each affine channel based on the input light field corresponding to the target image to be processed. The tropical selection module is used to perform nonlinear selection on each affine channel according to the intensity scalar to obtain the winning channel selected from each affine channel; wherein the configuration parameters of the affine mapping module and the tropical selection module are all trained according to the method of any one of claims 1-6; The readout module is used to generate the task processing result corresponding to the target image based on the selection result.

9. The optical computing system according to claim 8, characterized in that, The affine mapping module includes: an affine mapping module based on metasurfaces.

10. A training device for an optical computing system, characterized in that, include: Sample acquisition module, label acquisition module, and parameter update module; The sample acquisition module is used to acquire training samples, which include training images and real labels, wherein the real labels are the task processing results corresponding to the training images. The label acquisition module is used to generate a predicted label corresponding to the training image based on the input light field corresponding to the training image through the optical computing system. The optical computing system is an optical fitting system for a tropical geometric neural network. The parameter update module is used to determine the loss based on the real label and the predicted label, and update the configuration parameters of the optical computing system based on the loss. The optical computing system includes: an affine mapping module, a tropical selection module, and a readout module; The label acquisition module generates the predicted label corresponding to the training image by: setting a first parameter and its initial value, and determining the input light field corresponding to the training image as the target light field, and performing the following first processing: generating intensity scalars for each affine channel using the affine mapping module according to the target light field; performing nonlinear selection on each affine channel using the tropical selection module according to the intensity scalars to obtain the winning channel selected from each affine channel; incrementing the value of the first parameter by 1, in response to determining that the value of the first parameter is equal to the sum of the initial value and M, generating the predicted label using the readout module according to the selection result, where M is a positive integer greater than 1; in response to determining that the value of the first parameter is less than the sum of the initial value and M, generating an updated target light field according to the selection result, and repeating the first processing.

11. A task processing result generation device, characterized in that, include: Image acquisition module and result generation module; The image acquisition module is used to acquire the target image to be processed; The result generation module is used to generate the task processing result corresponding to the target image using an optical computing system, wherein the configuration parameters of the optical computing system are obtained by training according to the method of any one of claims 1-6.

12. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

13. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-7.

Images