Image generation system

Abstract

This application discloses an image generation system, particularly relating to the field of image processing technology, comprising a main control device and at least one light generator. The main control device is connected to each of the at least one light generator. The number of light generators is the same as the number of rows in the target image. Each light generator consists of at least one interferometer cascaded sequentially, with the number of interferometers set according to the input and output dimensions of the fully connected layer corresponding to each light generator. The main control device sends noise light signals corresponding to noise data to each of the at least one light generator. Each of the at least one light generator performs optical fully connected computation on the input noise light signals to obtain at least one pixel light signal, and feeds back the at least one pixel light signal to the main control device. The main control device generates the target image based on the at least one pixel light signal. This system achieves optical fully connected computation to replace traditional fully connected layer computation, solving the problems of training difficulties and mode collapse when generating images using fully connected layers, and improving image generation efficiency.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more particularly to an image generation system. Background Technology

[0002] In the field of image generation, most related technologies use fully connected layers to construct generative networks, and realize the transformation from random noise to image pixels through fully connected mapping of neurons. However, this type of method has significant limitations: On the one hand, fully connected layers need to establish full association between input and output neurons, and the number of parameters grows exponentially with the image dimension. This not only leads to high computational cost and slow convergence speed during training, but also easily causes gradient vanishing or exploding problems, exacerbating the training difficulty. On the other hand, fully connected layers will sever the spatial association between image pixels and are difficult to support stable adversarial training of generative networks. They are prone to pattern collapse (insufficient diversity of generated samples and repetitive monotony). At the same time, their globally intensive computational characteristics cannot be adapted to the advantages of hardware parallel acceleration, resulting in low image generation efficiency and difficulty in meeting the generation requirements of high resolution and real-time. Summary of the Invention

[0003] This application provides an image generation system and method to at least solve the problems of training difficulties, mode collapse, and low generation efficiency in the related art when using fully connected layers to generate images.

[0004] This application provides an image generation system, including: a main control device and at least one light generator, wherein the main control device is connected to at least one light generator respectively;

[0005] The number of light generators is the same as the number of rows in the target image;

[0006] The light generator consists of at least one interferometer cascaded in sequence, and the number of interferometers is set according to the input and output dimensions of the fully connected layer corresponding to the light generator.

[0007] The main control device sends noise light signals corresponding to the noise data to at least one light generator; at least one light generator performs optical fully connected calculations on the input noise light signals to obtain at least one pixel light signal, and feeds back the at least one pixel light signal to the main control device; the main control device generates a target image based on the at least one pixel light signal.

[0008] This application provides an image generation method, which is applied to any of the above-mentioned image generation systems, including:

[0009] The main control device sends the optical signal corresponding to the noise data to at least one optical generator;

[0010] At least one light generator performs optical fully connected calculations on the input light signal to obtain at least one pixel light signal, and then feeds the at least one pixel light signal back to the main control device.

[0011] The main control device generates a target image based on at least one pixel light signal.

[0012] The image generation system and method provided in this application work in concert with a main control device and a light generator (composed of cascaded interferometers adapted to the input and output dimensions of the fully connected layer) corresponding to the number of rows of the target image. Optical fully connected computation replaces traditional fully connected layer computation. This not only greatly improves the image generation efficiency by leveraging the parallel and high-speed characteristics of optical computation, but also enhances numerical stability by relying on the norm conservation characteristics of optical computation. It effectively solves the training difficulties and mode collapse problems that exist when generating images with fully connected layers. At the same time, it realizes parallel generation by row, taking into account the diversity and efficiency of image generation, and provides a reliable solution for high-resolution and high-stability image generation. Attached Figure Description

[0013] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a schematic diagram of an image generation system provided in an embodiment of this application;

[0015] Figure 2 A schematic diagram of a specific single MZI structure provided in an embodiment of this application;

[0016] Figure 3 A schematic diagram illustrating a computation using a 4×2 fully connected neural network, provided as an embodiment of this application;

[0017] Figure 4 This is a schematic diagram illustrating the process of a light generator generating pixels, as provided in an embodiment of this application.

[0018] Figure 5 This is a schematic diagram illustrating the generation of a 3×3 image using a light generator, as provided in an embodiment of this application.

[0019] Figure 6 This is a schematic diagram of another image system provided in an embodiment of this application;

[0020] Figure 7 This is a schematic diagram of a specific image generation system provided in an embodiment of this application. Detailed Implementation

[0021] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0022] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0023] With the rapid development of high-performance computing hardware, artificial intelligence algorithms have been widely adopted across various industries. Neural networks, as their core engine, have achieved an intelligent leap from perception to cognition thanks to their four core characteristics: nonlinear modeling, automatic feature extraction, distributed representation, and scalability. They are deeply applied in multiple fields, including image generation. Generative networks, a key branch of neural networks, focus on creating new data rather than simply analyzing existing data. Their core relies on latent space mapping (mapping random noise to a high-dimensional semantic space to learn data distribution) and adversarial training mechanisms (the generator and discriminator engage in a game of optimization), breaking through the limitations of traditional supervised learning.

[0024] In the development of generative networks, generative adversarial networks (GANs) have played an important role. However, due to their own optimization characteristics and hardware computing bottlenecks, they have significant technical defects: On the one hand, the dynamic game between the generator and the discriminator is prone to imbalance, which leads to unstable training process and difficulty in convergence. Moreover, the non-convex nature of the generator loss function makes it easy for it to get trapped in local optima, causing pattern collapse, which manifests as insufficient diversity and repetitiveness of generated samples. On the other hand, existing generative networks mostly rely on fully connected layers for construction. This structure will sever the spatial correlation between image pixels, and the number of parameters increases exponentially with the image dimension, further aggravating the training complexity and efficiency loss.

[0025] In recent years, optical computing has developed rapidly as a new computing paradigm. With its strong parallel processing capabilities and near-light-speed operation, it has shown enormous application potential in neural network scenarios. Silicon-based optoelectronics, in particular, has become a superior platform for optical computing due to its high-speed signal processing capabilities and compatibility with existing microelectronic manufacturing processes. It can construct complex optoelectronic circuits by integrating optical components to efficiently execute core neural network operations. However, current applications of optical computing are still limited to processing and accelerating simple matrix multiplications, and the complete construction of generative networks has not yet been achieved, failing to fully realize its potential in solving problems such as training instability and low efficiency. Therefore, combining optical computing with generative adversarial networks (GANs) is expected to specifically address the training difficulties, pattern collapse, and efficiency bottlenecks of existing generative networks, providing a new path for the technological upgrade of generative networks.

[0026] To address the challenges of training, pattern collapse, and low generation efficiency in generative networks, this application provides an image generation system. This system uses an interferometer-based optical generator as its core, designs an optical computational structure for the linear generator in a branched generation network, and constructs an optical branched generation network from noise input to the linear generator layer, ultimately efficiently completing the image generation task.

[0027] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0028] Figure 1 This is a schematic diagram of an image generation system provided in an embodiment of this application.

[0029] like Figure 1 As shown, the system includes:

[0030] The system includes a main control device and at least one light generator (1, 2, ... n light generators), with the main control device connected to at least one light generator.

[0031] The number of light generators n is strictly consistent with the number of rows in the target image (that is, when the target image contains n rows of pixels, n light generators are deployed accordingly), realizing a row-by-row generation architecture with one generator per row;

[0032] The light generator consists of at least one interferometer cascaded in sequence. The number of interferometers cascaded needs to be precisely configured according to the input and output dimensions of the fully connected layer corresponding to the light generator.

[0033] The main control device sends the optical signal corresponding to the noise data to at least one optical generator; at least one optical generator performs optical full-connection calculation on the input optical signal to obtain at least one pixel optical signal, and feeds back the at least one pixel optical signal to the main control device; the main control device generates the target image based on the at least one pixel optical signal.

[0034] The main control device sends noise light signals corresponding to the noise data to at least one light generator; each light generator in the at least one light generator performs optical fully connected calculations on the input noise light signals to obtain at least one pixel light signal and then feeds it back to the main control device; the main control device generates the target image based on these pixel light signals.

[0035] Specifically, to maximize image generation efficiency, the main control device employs a parallel transmission mode, synchronously transmitting noise optical signals corresponding to the noise data to all optical generators. Each optical generator can simultaneously initiate its signal reception process without waiting for other devices to complete reception, completely avoiding the time loss caused by serial transmission. In the optical signal processing stage, each optical generator, relying on its cascaded interferometer structure, performs parallel fully connected optical calculations: each generator independently maps the input noise optical signal to the pixel optical signal of the corresponding row in the target image. The calculation process is independent and synchronous, significantly reducing the total time for single-round pixel generation. After calculation, each optical generator also feeds back its generated pixel optical signal (after photoelectric conversion) to the main control device in parallel. The main control device synchronously acquires the single-row pixel optical signals output by all optical generators, and then, according to the preset correspondence between the optical generator number and the row order of the target image, quickly completes the splicing, integration, and format standardization of the pixel signals in each row, ultimately generating a complete and ordered target image.

[0036] The number of interferometers in each light generator is determined by the input and output dimensions of its corresponding fully connected layer. Therefore, after obtaining the initial calculation result through optical fully connected computation, each light generator needs to accurately extract the matching pixel light signal from the initial calculation result based on the output dimension of the fully connected layer.

[0037] For example, this application takes a light generator specifically including a first light generator and a second light generator as an example (corresponding to a target image of 2 rows of pixels, i.e., the number of light generators is consistent with the number of rows of the target image), and its workflow is as follows:

[0038] The main control device first converts the random noise data of the preset dimension into the corresponding noise optical signal, and sends the noise optical signal to the first optical generator and the second optical generator in parallel and synchronously to ensure that the two optical generators start signal reception and processing at the same time.

[0039] The first and second light generators, each based on their own cascaded interferometer structures (the number of interferometers adapted to the input or output dimension of the corresponding fully connected layer), independently perform optical fully connected computations on the received noisy light signals. Specifically, the first light generator outputs the first pixel light signal corresponding to the first row of pixels in the target image, and the second light generator outputs the second pixel light signal corresponding to the second row of pixels in the target image. Their computation processes are independent and completed synchronously.

[0040] The first light generator and the second light generator respectively transmit the calculated first pixel light signal and second pixel light signal to the main control device in parallel. The main control device receives the two pixel light signals synchronously to avoid the time loss of serial feedback.

[0041] The main control device splices and integrates the first pixel data and second pixel data corresponding to the first pixel light signal and the second pixel light signal according to the preset correspondence between the light generator and the image row order (the first light generator corresponds to the first row and the second light generator corresponds to the second row), and finally generates a complete target image containing two rows of pixels.

[0042] Since the number of interferometers in each light generator is determined based on the input and output dimensions of its corresponding fully connected layer (the larger of the two dimensions is used to meet the dimensionality adaptation requirements of optical fully connected computation), the process of generating pixel light signals by the first and second light generators must be combined with this configuration logic. The specific steps are as follows: First, the first and second light generators perform optical fully connected computation on the received noisy light signals respectively: The first light generator, relying on its interferometer structure configured according to the "maximum input / output dimension of the first fully connected layer," converts the noisy light signal into a first computation result containing multi-dimensional light signals; the second light generator, based on its interferometer structure configured according to the "maximum input / output dimension of the second fully connected layer," synchronously generates a second computation result containing multi-dimensional light signals. The dimension of the computation result in this step is consistent with the maximum dimension configured in each interferometer, ensuring that the computational requirements of the fully connected layer are covered.

[0043] Subsequently, the pixel light signal is determined based on the output dimension of the fully connected layer: for the first calculation result, the light signal matching the dimension is extracted from the first calculation result according to the output dimension of the first fully connected layer corresponding to the first light generator (i.e., the number of pixels in the first row of the target image), and determined as the first pixel light signal; for the second calculation result, the light signal matching the dimension is selected from the second calculation result according to the output dimension of the second fully connected layer corresponding to the second light generator (i.e., the number of pixels in the second row of the target image), and determined as the second pixel light signal. Through this logic of "first satisfying the maximum dimension calculation, and then matching the output dimension for selection", the integrity of the optical fully connected calculation is ensured, and the final output pixel light signal can accurately match the number of pixels in the corresponding row of the target image, providing accurate data for subsequent stitching and integration.

[0044] It is understood that the above example only illustrates the image generation logic of this application using a two-light-generator scenario, including a first light generator and a second light generator, and is not a limitation on the number of light generators. In practical applications, the specific number of light generators depends entirely on the required number of rows in the target image and can be flexibly adjusted according to the actual number of rows in the target image: for example, when the target image has 3 rows of pixels, 3 light generators are configured (responsible for generating the first, second, and third rows of pixels respectively); when the target image has 4 rows of pixels, 4 light generators are configured, and so on. Regardless of how the number of light generators is adjusted, its core working logic remains consistent with the two-light-generator scenario described above, namely, the main control device sends noisy light signals in parallel, each light generator independently performs optical fully connected computation, feeds back the corresponding row pixel light signals in parallel, and the main control device stitches and synthesizes the target image according to the row order. It is only necessary to match the number of light generators according to the number of rows, and the specific number is not limited in this embodiment of the application.

[0045] The parallelization design described above runs through the entire process of signal transmission, optical computing, data feedback, and image synthesis, giving full play to the high-speed characteristics of optical computing and the parallel advantages of multi-generator architecture, and significantly improving the overall efficiency of image generation.

[0046] This application employs a Mach-Zehnder interferometer (MZI) as the core computational unit of the light generator; specifically, the interferometer used in this application is a Mach-Zehnder interferometer. Its individual structure consists of two beam splitters and two phase shifters. The beam splitters achieve beam splitting and combining of the optical signal; the phase shifters can precisely change the phase of the light by adjusting physical conditions such as voltage and temperature. These adjustable phase parameters are precisely the core generator parameters required for the light generator to achieve fully connected optical computation. Figure 2 As shown, this application provides a schematic diagram of a specific single MZI structure. (Refer to...) Figure 2 A single MZI contains two input optical signals Ein and two output optical signals Eout. Through the interference effect of light, a linear transformation from input vector to output vector can be achieved, which is equivalent to a 2×2 matrix multiplication. Multiple MZIs can be cascaded to expand into an n×n mesh, which can meet the needs of higher-dimensional fully connected computation.

[0047] To achieve precise matching between optical fully connected computation and traditional neural network fully connected layers (ensuring consistency between optical computation results and traditional electrical computation results), this application establishes a clear logic for configuring the number of MZIs. The specific steps are as follows: First, determine the two core parameters of the fully connected layer corresponding to the current light generator: the input dimension *m* (i.e., the dimension of the data received by this fully connected layer) and the output dimension *w* (i.e., the dimension of the data output by this fully connected layer). Second, compare the values ​​of the input dimension *m* and the output dimension *w*, and define the larger of the two as the target dimension *k*. Finally, configure the number of MZIs according to the target dimension *k*. A k×k MZI mesh structure is constructed through cascading, and this mesh structure is the light generator.

[0048] From a computational perspective, the core function of a traditional fully connected layer in a neural network is to perform matrix multiplication from an m-dimensional input to a w-dimensional output. This process can be efficiently achieved using the aforementioned k×k MZI grid: At the start of the computation, the m-dimensional input optical signal is first fed into the MZI grid. If m < k (i.e., the input dimension is less than the target dimension), the input optical signal is expanded to k dimensions through zero-padding to ensure it matches the input dimension of the MZI grid. Subsequently, the input optical signal undergoes a linear transformation in the MZI grid through optical interference (this process is equivalent to matrix multiplication in traditional electrical computation). After the computation is completed, the optical signal outputs from the first w waveguide positions are extracted from the k waveguide outputs of the MZI grid, yielding the same m-input to w-output computation result as a traditional fully connected layer.

[0049] Compared to the traditional computer model that completes fully connected computation through two steps from multiplication to addition, the optical fully connected computation of this application has significant advantages: on the one hand, optical computation does not require breaking down the operation steps, and only a single optical interference process is needed to complete the equivalent computation, making the operation process simpler; on the other hand, since the propagation speed of optical signals in waveguides is close to the speed of light, the computation speed of optical fully connected computation far exceeds that of traditional electrical computation, which can greatly improve the overall operating efficiency of neural networks.

[0050] In the image generation system of this application, all light generators adopt a unified core structure design. That is, each light generator is composed of a k×k MZI grid as described above (where k is the larger value of the input dimension m and the output dimension w of the corresponding fully connected layer, and the MZI grid achieves optical fully connected computation through cascading). At the same time, the number of light generators is strictly matched with the number of rows of the target image: if a target image containing N rows of pixels needs to be generated, the system will be configured with N independent light generators, and each light generator is specifically responsible for generating a certain row of pixels in the target image (for example, the first light generator only generates the first row of pixels, the second light generator only generates the second row of pixels, and so on, ensuring that the generation process of each row of pixels is independent and accurate).

[0051] In practice, the system first transmits the input noisy light signal to each light generator. Each light generator, upon receiving the noisy light signal, immediately initiates optical fully connected computation of its internal k×k MZI grid. Through optical interference, it converts the noisy light signal into pixel light signals for the corresponding row (e.g., the light generator responsible for the 3rd row of pixels outputs the pixel light signal for the 3rd row). Subsequently, all light generators synchronously feed back their generated single-row pixel light signals to the main control device. After receiving all the pixel light signals for all rows, the main control device, according to the preset correspondence between "light generator number and image row order" (e.g., the output of the 1st light generator corresponds to the 1st row of the image, and the 2nd corresponds to the 2nd row), performs ordered splicing, format integration, and signal processing on all the single-row pixel light signals, ultimately forming a complete target image.

[0052] Compared to traditional fully connected electronic layers, MZI optical computing combines parallel high speed (near the speed of light) and norm conservation (strong numerical stability), which can significantly improve image generation efficiency and effectively solve problems such as training difficulties and mode collapse, providing an efficient and stable technical solution for image generation tasks.

[0053] Furthermore, to facilitate understanding of the equivalence between optical fully connected computing and traditional fully connected layers, such as Figure 3 As shown, this application provides a schematic diagram of computation using a 4×2 fully connected neural network. (Refer to...) Figure 3 As can be seen, the left side shows the logical process of traditional fully connected computation: inputting a 4-dimensional vector (x1, x2, x3, x4), and outputting a 2-dimensional vector (y1, y2) through matrix multiplication of the weight matrix. The right side shows the process of completing the fully connected computation using an MZI mesh structure (i.e., the light generator of this application): using a 4×4 MZI mesh (since the input dimension of 4 is greater than the output dimension of 2, the target dimension is 4), inputting a 4-dimensional light signal (x1, x2, x3, x4), and performing a linear transformation through the optical interference effect of the MZI mesh, finally extracting the output of the first two waveguide positions as the result (y1, y2), achieving a mathematical operation logic completely equivalent to the traditional fully connected layer on the left, and requiring only one optical operand to complete, with a computation speed close to the speed of light.

[0054] As can be seen from the above analysis, in the generative network structure of this application, optical fully connected computation can be used as the generator for each row of the target image, with its input dimension being the noise dimension z and its output dimension being the number of pixels i in each row of the target image. Figure 4 As shown, this application provides a schematic diagram of the process of a light generator generating pixels. (Refer to...) Figure 4The left side shows the logical process of fully connected computation. It takes 4-dimensional noise (V1, V2, V3, V4) as input and outputs 3 pixels (I1, I2, I3) after weight matrix operation. The right side shows the process of completing the computation using an MZI mesh structure. It uses a 4×4 MZI mesh (because the input dimension 4 is greater than the output dimension 3, the target dimension is 4). After the input 4-dimensional noise light signal, it completes a linear transformation through optical interference effect. Finally, it extracts the output of the first 3 waveguide positions as the light signal of 3 pixels, realizing the optical fully connected layer computation to generate pixel light signals, and the optical operation is completed at the speed of light throughout the process.

[0055] Since the number of light generators in this application is exactly the same as the number of rows in the target image, when using the row generation method, n light generators need to be configured according to the number of rows n of the target image, and each light generator independently generates the image pixels of the corresponding row.

[0056] For example, if the target image to be generated is 3×3 dimensions (i.e., 3 rows of pixels), then n=3 light generators need to be configured to complete the generation task. Figure 5 As shown, this application provides a schematic diagram of 3×3 image generation using light generators: input noise is sent in parallel to 3 light generators (each light generator is an optical MZI grid computing structure), each light generator independently generates pixel light signals for the corresponding row, and finally the main control device stitches the 3 rows of pixels into a complete 3×3 target image based on the pixel light signals.

[0057] In some embodiments, the image generation process of this application is completed through a collaborative architecture of a light generator, a control chip, and a storage device. The core hardware configuration and the overall workflow logic are as follows:

[0058] like Figure 6 As shown, this application provides a schematic diagram of another image system. (Refer to...) Figure 6 The main control unit of the image generation system consists of a control chip and a storage device, forming the core support unit for command control and data storage. The control chip establishes a bidirectional connection with at least one light generator, responsible for command issuance, data transmission, and processing; the storage device directly interfaces with the control chip and is specifically used to temporarily store key pixel data during the image generation process, ensuring data security and convenient retrieval.

[0059] Taking a scenario where the light generator includes a first light generator and a second light generator (corresponding to two rows of pixel target images) as an example, the specific workflow is as follows: First, the control chip, acting as the "command center" of the entire system, accurately sends the input noise data of a preset dimension to the first and second light generators, triggering parallel computation in both directions. After receiving the noise data, the first and second light generators perform optical fully connected computation on the noise light signals corresponding to the noise data through an internally cascaded interferometer structure, ultimately outputting the first and second pixel light signals corresponding to the first and second rows of pixels in the target image, respectively. Subsequently, the control chip synchronously receives the first pixel data of the first pixel light signal and the second pixel data of the second pixel light signal, and transmits the two sets of pixel data to the storage device for storage, avoiding data loss or transmission delays that could affect generation efficiency. When the system needs to output a complete target image, the control chip quickly reads the stored first and second pixel data from the storage device according to preset instructions, and then performs splicing, integration, and format standardization processing on the two sets of pixel data according to the preset row order of "first light generator corresponding to the first row, second light generator corresponding to the second row," ultimately generating a complete target image that meets the requirements.

[0060] Throughout the process, the control chip dominates the entire process of instruction issuance, image synthesis, and storage scheduling, the light generator focuses on efficient optical full-connection computing, and the storage device undertakes the reliable storage and fast retrieval of pixel data. The three have clear division of labor and close cooperation, which not only ensures the high speed of image generation, but also ensures the stability and accuracy of data processing.

[0061] To clearly illustrate the specific image generation logic of this application, as follows: Figure 7 As shown, this application provides a schematic diagram of a specific image generation system.

[0062] Reference Figure 7 The image generation system of this application adopts an integrated architecture of control, conversion, calculation and storage. The core components include a control chip, storage device, electro-optical conversion device, at least one photoelectric conversion device and beam splitter. The components form a complete working link through precise connection relationships.

[0063] The control chip, as the central core of the system, connects to at least one light generator through two key paths: one path connects to the light generator via an electro-optical conversion device, responsible for the conversion and transmission of noise data to optical signals; the other path connects to the light generator via at least one photoelectric conversion device, with the number of photoelectric conversion devices exactly matching the number of light generators, ensuring that the output signal of each light generator corresponds to an independent conversion channel. Simultaneously, a beam splitter is connected in series between the electro-optical conversion device and the light generator, serving as a diversion hub for noise optical signals, while the storage device is directly connected to the control chip to temporarily store pixel data during the generation process.

[0064] Taking a scenario where the light generator includes a first light generator and a second light generator (corresponding to two rows of pixel target images) as an example, the specific workflow of the image generation system is as follows: First, the control chip sends noise data of a preset dimension to the electro-optical conversion device, which converts the noise data in electrical signal form into a noise light signal that can be used for optical calculation. Then, the beam splitter acquires the noise light signal and distributes it evenly and synchronously to the first light generator and the second light generator, ensuring that the two light generators can start calculations based on the same noise source. Next, the first light generator and the second light generator perform optical fully connected calculations on the received noise light signals, respectively, and output the first pixel light signal and the second pixel light signal corresponding to the first and second rows of pixels. At this time, the first photoelectric conversion device and the second photoelectric conversion device, which correspond one-to-one with the light generator, start working synchronously, converting the first pixel light signal and the second pixel light signal into first pixel data and second pixel data that the control chip can process, respectively, and transmitting the two sets of data to the control chip. After receiving the data, the control chip stores it in a storage device to ensure data security, and can also read the data from the storage device according to the generation requirements, and stitch and integrate it into a complete target image in row order.

[0065] Reference Figure 7 In the image generation system of this application, the control chip not only undertakes the core responsibilities of issuing instructions and processing data, but also has the ability to dynamically adjust the generator parameters of each light generator. Since each light generator is composed of a k×k MZI grid (k is a larger value of the input and output dimensions of the corresponding fully connected layer), its generator parameters essentially include the key configuration parameters of the MZI grid (such as the phase offset of each MZI unit, waveguide coupling coefficient, etc.). These parameters directly determine the weight matrix of the optical fully connected calculation, thereby affecting the characteristics of the output pixel light signal (such as pixel brightness, color channel values, etc.).

[0066] The control chip can determine the direction and specific values ​​of the parameters that need to be adjusted for each light generator based on the generation requirements of the target image (such as generating images of a specific style and resolution) or model optimization feedback (such as problems such as insufficient pixel details and color deviation discovered by the image quality detection module). Subsequently, the control chip adjusts the generator parameters of the corresponding light generator through a dedicated communication link. After the parameters are adjusted, when the light generator receives noisy light signals and performs optical calculations, it can generate pixel light signals that meet the requirements based on the updated generator parameters, thereby enabling the main control device to stitch together a target image with better quality and more accurate features.

[0067] It is important to note that, due to the differences in weight parameters and input / output dimensions for each light generator, this application adopts a dedicated configuration strategy of one generator per MZI grid: if the system requires n light generators to generate a target image with n rows of pixels, then n independent MZI grids are configured simultaneously. The number of cascaded grids and phase parameters of each MZI grid are set individually according to the fully connected layer parameters of the corresponding light generator (such as the maximum dimension of the input / output dimensions and the weight matrix), ensuring that the computational power of each MZI grid is precisely matched with the needs of the corresponding light generator. These MZI grids achieve synchronous acquisition of noisy light signals through a beam splitter, and then convert the calculation results into digital data through their respective independent photoelectric conversion devices (i.e., ADC units) and transmit them to the control chip, ultimately forming a highly efficient working mode of splitting, dedicated computation, and independent conversion, which not only ensures the accuracy of the calculation but also improves the overall generation efficiency.

[0068] In summary, this application, through the core idea of ​​row-by-row generation, uses the MZI grid constructed by combining MZI elements as an independent light generator, configuring a dedicated MZI grid for each row of pixels in the target image (adapted to the parameters of the corresponding fully connected layer). Combined with the collaborative operation of the beam splitter, photoelectric conversion device, and control chip, it not only improves the image generation efficiency by leveraging the parallel and high-speed characteristics of optical computing, but also ensures generation diversity through the dedicated configuration of each MZI grid. This effectively solves the problems of pattern collapse and training difficulties in traditional generation networks, ultimately achieving more efficient, stable, and diverse image generation effects, and successfully completing various image generation tasks.

[0069] This application also provides an image generation method, applied to the above-mentioned image generation system, comprising:

[0070] The main control device sends the optical signal corresponding to the noise data to at least one optical generator;

[0071] The input optical signal is subjected to optical full-connection calculation by at least one of the light generators to obtain at least one pixel optical signal, and the at least one pixel optical signal is fed back to the main control device.

[0072] The main control device generates a target image based on at least one pixel light signal.

[0073] In this embodiment, the image generation system specifically includes a control chip, a storage device, an electro-optical conversion device, a photoelectric conversion device, a light generator, and a beam splitter.

[0074] First, the control chip sends random noise data of a preset dimension to the electro-optical conversion device, which converts the noise data in electrical signal form into a noise optical signal suitable for optical computing. Then, the beam splitter acquires the noise optical signal and distributes it evenly according to the number of optical generators (e.g., J), sending the noise optical signal synchronously and separately to each optical generator to ensure that all optical generators start parallel computing based on a unified noise source.

[0075] Next, each light generator (each an independent MZI mesh structure adapted to the parameters of the corresponding fully connected layer) receives the noisy light signal and performs optical fully connected calculations using its cascaded interferometers to convert the noisy light signal into pixel light signals for a single row of pixels in the target image. After the calculation is completed, each light generator feeds back its pixel light signal to the corresponding photoelectric conversion device, which converts the pixel light signal into digital pixel data and then transmits it to the control chip of the main control device.

[0076] Finally, after receiving all pixel data, the control chip first stores it in the storage device. When a complete image needs to be generated, the control chip reads all pixel data from the storage device and performs splicing, integration and format standardization processing on each row of pixel data according to the preset correspondence between the light generator and the image row order, and finally generates the target image that meets the requirements.

[0077] As can be understood, the specific execution flow of this image generation method can be found in the preceding text. Figures 1 to 7 The detailed descriptions of the corresponding embodiments will not be repeated here.

[0078] Embodiments of this application also provide an electronic device including the image generation system of any of the above embodiments.

[0079] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0080] The image generation system provided in this application has been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and core ideas of this application. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. An image generation system, characterized in that, include: The system includes a main control device and at least one light generator, wherein the main control device is connected to at least one of the light generators respectively. The number of light generators is the same as the number of rows in the target image; The light generator is composed of at least one interferometer cascaded in sequence, and the number of the interferometers is set according to the input dimension and output dimension of the fully connected layer corresponding to the light generator; The main control device sends noise light signals corresponding to the noise data to at least one of the light generators; the at least one light generator performs optical full-connection calculations on the input noise light signals to obtain at least one pixel light signal, and feeds back at least one pixel light signal to the main control device. The main control device converts at least one pixel light signal into at least one pixel data, and splices and integrates at least one pixel data according to the preset correspondence between the light generator and the image row order to generate a target image.

2. The system according to claim 1, characterized in that, Setting the number of interferometers according to the input and output dimensions of the fully connected layer corresponding to the light generator includes: Determine the input and output dimensions of the fully connected layer corresponding to the light generator; If the input dimension is greater than or equal to the output dimension, the input dimension is determined as the target dimension. If the input dimension is smaller than the output dimension, the output dimension is determined as the target dimension. The number of interferometers is set according to the target dimension.

3. The system according to claim 1, characterized in that, In the case where at least one of the light generators includes a first light generator and a second light generator, the main control device sends the noise light signal corresponding to the noise data to the first light generator and the second light generator; The first light generator and the second light generator perform optical fully connected calculations on the noisy light signal to obtain the first pixel light signal output by the first light generator and the second pixel light signal output by the second light generator. The first light generator and the second light generator respectively feed back the first pixel light signal and the second pixel light signal to the main control device; The main control device converts the first pixel light signal into first pixel data, converts the second pixel light signal into second pixel data, and generates the target image based on the first pixel data corresponding to the first pixel light signal and the second pixel data corresponding to the second pixel light signal.

4. The system according to claim 3, characterized in that, The first light generator and the second light generator respectively perform optical fully connected computation on the noisy light signal to obtain the first pixel light signal output by the first light generator and the second pixel light signal output by the second light generator, including: The first light generator and the second light generator perform optical fully connected calculations on the noisy light signal to obtain a first calculation result and a second calculation result. According to the output dimension of the first fully connected layer and the output dimension of the second fully connected layer, the first pixel light signal in the first calculation result and the second pixel light signal in the second calculation result are determined. The first fully connected layer is the fully connected layer corresponding to the first light generator, and the second fully connected layer is the fully connected layer corresponding to the second light generator.

5. The system according to claim 1, characterized in that, The main control device includes a control chip, which is connected to at least one of the light generators. In the case where at least one of the light generators includes a first light generator and a second light generator, the control chip sends the noise data to the first light generator and the second light generator so that the first light generator and the second light generator perform optical full-connection calculations on the noise light signal corresponding to the noise data to obtain a first pixel light signal and a second pixel light signal. The control chip acquires the first pixel data corresponding to the first pixel light signal and the second pixel data corresponding to the second pixel light signal, and generates the target image based on the first pixel data and the second pixel data.

6. The system according to claim 5, characterized in that, The main control device includes an electro-optical conversion device and at least one photoelectric conversion device; The control chip is connected to at least one of the light generators via the electro-optical conversion device and at least one of the photoelectric conversion devices; The number of photoelectric conversion devices is the same as the number of light generators; In the case where at least one of the light generators includes a first light generator and a second light generator, the electro-optical conversion device converts the noise data sent by the control chip into the noise light signal, and sends the noise light signal to the first light generator and the second light generator; The first photoelectric conversion device and the second photoelectric conversion device respectively convert the first pixel light signal and the second pixel light signal into first pixel data and second pixel data, and send the first pixel data and the second pixel data to the control chip.

7. The system according to claim 6, characterized in that, The main control device includes a beam splitter; the control chip is connected to at least one of the light generators through the beam splitter; the beam splitter is connected between the electro-optical conversion device and at least one of the light generators; The beam splitter acquires the noisy optical signal sent by the electro-optical conversion device and sends the noisy optical signal to the first optical generator and the second optical generator respectively.

8. The system according to claim 5, characterized in that, The main control device includes a storage device; the storage device is connected to the control chip. The storage device stores the first pixel data and the second pixel data acquired by the control chip; When the control chip determines that the target image needs to be generated, it reads the first pixel data and the second pixel data from the storage device.

9. The system according to claim 2, characterized in that, The number of interferometers is k is the target dimension.

10. An image generation method, characterized in that, The method is applied to the image generation system according to any one of claims 1 to 9, comprising: The main control device sends the optical signal corresponding to the noise data to at least one optical generator; The input optical signal is subjected to optical full-connection calculation by at least one of the light generators to obtain at least one pixel optical signal, and the at least one pixel optical signal is fed back to the main control device. The main control device converts at least one pixel light signal into at least one pixel data, and according to the preset correspondence between the light generator and the image row order, splices and integrates at least one pixel data to generate a target image.

Images