Optical Transmission Matrix Measurement Model Training Method, Matrix Measurement Method and Device

Abstract

The present application provides an optical transmission matrix measurement model training method, a matrix measurement method and device. The training method includes: using the scattered spot amplitudes respectively corresponding to each image after being transmitted by the fiber optic imaging system, and the plane amplitudes and plane phases respectively corresponding to each of the images, to train the connected first physical information neural network and measurement amplitude iterative calculation module to obtain an amplitude measurement model; training a second physical information neural network based on the target measurement amplitude output by the amplitude measurement model, for use in outputting a phase measurement model of the target measurement phase of the optical transmission matrix according to the measurement amplitude. The present application can reduce the amount of data required for training the optical transmission matrix measurement model, can effectively reduce the complexity requirements of the required equipment, and can thus effectively improve the reliability and effectiveness of the optical transmission matrix measurement model training process, and can ensure the accuracy of optical transmission matrix measurement using the trained measurement model.

Description

Technical Field

[0001] This application relates to the technical field of optical signal processing, and particularly to an optical transmission matrix measurement model training method, a matrix measurement method, and a device. Background Art

[0002] With the continuous development of global information technology, optical interconnection signal processing has become an important part of the field of information digitization. In an optical image transmission system, the target information carried by the incident light is output as an optical field after passing through a scattering medium, which presents a complex speckle distribution but still retains the initial information of the input light. Therefore, reconstructing the target information using scattered light has become a research hotspot. The transmission depth of light in the scattering medium directly affects the effectiveness of ballistic light imaging, and the determinacy of the scattering process also makes it possible to achieve focusing or imaging using scattered light.

[0003] The optical transmission matrix of the scattering medium contains multiple eigenchannels, and each channel corresponds to different transmission transmittances and output optical fields. Therefore, through the study of the optical transmission matrix, the regulation of the output optical field energy or the modulation of the optical field distribution can be achieved. In addition, the basic characteristics of optical speckles and their technical applications have also attracted extensive attention in the optical field. Although the formation of the speckle pattern is essentially random, it exhibits predictable characteristics, representing the commonality of coherent wave dynamics in disordered systems, making the speckle pattern a key tool in the fields of optical tweezers, imaging technology, and cold atom random potential. The optical transmission matrix is a qualitative and quantitative analysis tool for this characteristic.

[0004] However, traditional optical transmission matrix measurement methods often have difficulty in achieving accurate holographic measurement of the optical transmission matrix including amplitude and phase, which poses a major challenge to the effective reconstruction of scattered light signals. In addition, such methods usually require a large amount of complex acquisition data, and the data volume is often several times the number of pixels of the image resolution, or high-precision intensity-phase pairs are required for holographic reconstruction of the matrix. The above requirements not only increase the acquisition time but also pose higher requirements on the device complexity, restricting the feasibility of this method in practical applications. Therefore, reducing the acquisition data volume and lowering the device requirements while ensuring the measurement accuracy of the optical transmission matrix are the key research directions at present. Summary of the Invention

[0005] In view of this, the embodiments of this application provide an optical transmission matrix measurement model training method, a matrix measurement method, and a device to eliminate or improve one or more defects existing in the prior art.

[0006] One aspect of this application provides an optical transmission matrix measurement model training method, including:

[0007] Using the scattered spot amplitudes respectively corresponding to each image after being transmitted through the fiber optic imaging system, and the planar amplitude and planar phase respectively corresponding to each of the images, based on a preset iterative algorithm for phase recovery, train the connected first physical information neural network and measurement amplitude iterative calculation module, so as to train the first physical information neural network and the measurement amplitude calculation iterative module into an amplitude measurement model for outputting the target measurement amplitude of the optical transmission matrix corresponding to the scattered spot amplitude;

[0008] Based on the target measurement amplitudes corresponding to the scattered spot amplitudes of each of the images output by the amplitude measurement model, train the second physical information neural network, so as to train the second physical information neural network into a phase measurement model for outputting the target measurement phase of the optical transmission matrix according to the measurement amplitude; wherein, the amplitude measurement model and the phase measurement model constitute an optical transmission matrix measurement model.

[0009] In some embodiments of the present application, the step of using the scattered spot amplitudes respectively corresponding to each image after being transmitted through the fiber optic imaging system, and the planar amplitude and planar phase respectively corresponding to each of the images, based on a preset iterative algorithm for phase recovery, train the connected first physical information neural network and measurement amplitude iterative calculation module, so as to train the first physical information neural network and the measurement amplitude calculation iterative module into an amplitude measurement model for outputting the target measurement amplitude of the optical transmission matrix corresponding to the scattered spot amplitude, includes:

[0010] In the current iteration round, input the scattered spot amplitudes respectively corresponding to each image after being transmitted through the fiber optic imaging system into the first physical information neural network in sequence, so that the first physical information neural network respectively outputs the scattered spot phases corresponding to the scattered spot amplitudes respectively corresponding to each of the images;

[0011] In the measurement amplitude iterative calculation module, based on a preset iterative algorithm for phase recovery, based on the scattered spot amplitudes, the scattered spot phases, the pre-acquired planar amplitude and the planar phase respectively corresponding to each of the images, obtain the measurement amplitude of the optical transmission matrix corresponding to the current iteration round and correspondingly update the network parameters of the first physical information neural network;

[0012] Determine whether the first physical information neural network and the measurement amplitude iterative calculation module in the current iteration round converge; if so, use the measurement amplitude obtained in the current iteration round as the target measurement amplitude, and use the first physical information neural network and the measurement amplitude iterative calculation module corresponding to the current iteration round as an amplitude measurement model for outputting the optical transmission matrix corresponding to the scattered light spot amplitude; if not, perform training on the first physical information neural network and the measurement amplitude iterative calculation module in the next iteration round.

[0013] In some embodiments of the present application, the determining whether the first physical information neural network and the measurement amplitude iterative calculation module in the current iteration round converge includes:

[0014] Determine whether the correlation coefficient between the measurement amplitude of the optical transmission matrix corresponding to the current iteration round and the measurement amplitude of the optical transmission matrix corresponding to the historical iteration rounds separated from the current iteration round by at least one iteration round is higher than a preset threshold. If so, determine that the first physical information neural network and the measurement amplitude iterative calculation module in the current iteration round converge; if not, determine that the first physical information neural network and the measurement amplitude iterative calculation module in the current iteration round do not converge.

[0015] In some embodiments of the present application, before determining whether the first physical information neural network and the measurement amplitude iterative calculation module in the current iteration round converge, it further includes:

[0016] Optimize the learning rate corresponding to the first physical information neural network based on the cosine annealing method.

[0017] In some embodiments of the present application, before inputting the scattered light spot amplitudes respectively corresponding to each image after being transmitted through the fiber optic imaging system into the first physical information neural network in the current iteration round, it further includes:

[0018] Multiply the initialized data of the preset optical transmission matrix by the complex matrix composed of the plane amplitude and the plane phase corresponding to the pre-acquired image, and record the error between the amplitude in the obtained product result and the scattered light spot amplitude corresponding to the image as the loss function for iteratively training the first physical information neural network based on the stochastic gradient descent method.

[0019] In some embodiments of the present application, before determining whether the first physical information neural network and the measurement amplitude iterative calculation module in the current iteration round converge, it further includes:

[0020] Perform Gaussian blur processing on the scattered light spot phase output by the first physical information neural network.

[0021] In some embodiments of the present application, before training the connected first physical information neural network and the measured amplitude iterative calculation module based on the preset iterative algorithm for phase recovery, using the scattered spot amplitudes corresponding to each image after being transmitted through the fiber optic imaging system respectively, and the plane amplitude and plane phase corresponding to each of the images, further includes:

[0022] Loading each image based on a Gaussian distribution into the spatial light modulator in the fiber optic imaging system as the plane phase corresponding to each of the images, and the plane amplitude corresponding to each of the images being Gaussian light respectively;

[0023] Obtaining the scattered spot amplitude corresponding to each of the images received by the CCD camera in the fiber optic imaging system; performing image cropping and normalization processing on the plane phase, the plane amplitude, and the scattered spot amplitude corresponding to each of the images respectively.

[0024] Another aspect of the present application provides an optical transmission matrix measurement method, including:

[0025] Obtaining the plane amplitude and plane phase of a target image and the scattered spot amplitude obtained after the target image is transmitted through the fiber optic imaging system;

[0026] Inputting the plane amplitude, plane phase, and the scattered spot amplitude of the target image into an optical transmission matrix measurement model pre-trained by the optical transmission matrix measurement model training method, so that the amplitude measurement model in the optical transmission matrix measurement model outputs the target measurement amplitude of the optical transmission matrix corresponding to the target image according to the plane amplitude, plane phase, and the scattered spot amplitude of the target image, and enabling the phase measurement model in the optical transmission matrix measurement model to output the target measurement phase of the optical transmission matrix corresponding to the target image according to the target measurement amplitude corresponding to the target image;

[0027] Outputting an optical transmission matrix including the target measurement amplitude and the target measurement phase corresponding to the target image.

[0028] A third aspect of the present application provides an optical transmission matrix measurement model training device, including:

[0029] The amplitude measurement model training module is used to train the connected first physical information neural network and the measurement amplitude iterative calculation module based on the preset iterative algorithm for phase recovery, by using the scattered spot amplitudes respectively corresponding to each image after being transmitted by the fiber optic imaging system, and the plane amplitude and plane phase corresponding to each of the images, so as to train the first physical information neural network and the measurement amplitude calculation iterative module into an amplitude measurement model for outputting the optical transmission matrix corresponding to the scattered spot amplitude;

[0030] The phase measurement model training module is used to train the second physical information neural network based on the target measurement amplitude corresponding to the scattered spot amplitude of each of the images output by the amplitude measurement model, so as to train the second physical information neural network into a phase measurement model for outputting the target measurement phase of the optical transmission matrix according to the measurement amplitude; wherein, the amplitude measurement model and the phase measurement model constitute an optical transmission matrix measurement model.

[0031] The fourth aspect of the present application provides an optical transmission matrix measurement device, including:

[0032] The data acquisition module is used to acquire the plane amplitude and plane phase of the target image and the scattered spot amplitude obtained after the target image is transmitted by the fiber optic imaging system;

[0033] The model measurement module is used to input the plane amplitude, plane phase and the scattered spot amplitude of the target image into the optical transmission matrix measurement model pre-trained by the optical transmission matrix measurement model training method, so that the amplitude measurement model in the optical transmission matrix measurement model outputs the target measurement amplitude of the optical transmission matrix corresponding to the target image according to the plane amplitude, plane phase and the scattered spot amplitude of the target image, and the phase measurement model in the optical transmission matrix measurement model outputs the target measurement phase of the optical transmission matrix corresponding to the target image according to the target measurement amplitude corresponding to the target image;

[0034] The matrix output module is used to output the optical transmission matrix including the target measurement amplitude and the target measurement phase corresponding to the target image.

[0035] The fifth aspect of the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the optical transmission matrix measurement model training method, and / or, implements the optical transmission matrix measurement method.

[0036] The sixth aspect of the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the optical transmission matrix measurement model training method described above is implemented, and / or the optical transmission matrix measurement method described above is implemented.

[0037] The seventh aspect of the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, the optical transmission matrix measurement model training method described above is implemented, and / or the optical transmission matrix measurement method described above is implemented.

[0038] In the optical transmission matrix measurement model training method provided by the present application, by using the scattered spot amplitudes respectively corresponding to each image after being transmitted through the fiber optic imaging system, and the plane amplitudes and plane phases respectively corresponding to each of the images, based on a preset iterative algorithm for phase recovery, iterative calculations are performed on the connected first physical information neural network and the measurement amplitude iterative calculation module to train the first physical information neural network and the measurement amplitude calculation iterative module into an amplitude measurement model for outputting the target measurement amplitude of the optical transmission matrix corresponding to the scattered spot amplitude; based on the target measurement amplitudes corresponding to the scattered spot amplitudes of each of the images output by the amplitude measurement model, a second physical information neural network is trained to train the second physical information neural network into a phase measurement model for outputting the target measurement phase of the optical transmission matrix according to the measurement amplitude; wherein, the amplitude measurement model and the phase measurement model constitute an optical transmission matrix measurement model, which can reduce the amount of data required for training the optical transmission matrix measurement model, can effectively reduce the complexity requirements of the required equipment, and thus can effectively improve the reliability and effectiveness of the optical transmission matrix measurement model training process. The trained measurement model can realize holographic measurement of the optical transmission matrix including amplitude and phase, and can ensure the accuracy of the optical transmission matrix measurement using the trained measurement model. Furthermore, applications such as focusing the scattered spot amplitude on the receiving plane and restoring and extracting speckle information can be realized using the optical transmission matrix.

[0039] Additional advantages, objects, and features of the present application will be partially described below, and will become partially apparent to those of ordinary skill in the art after studying the following text, or can be learned from the practice of the present application. The objects and other advantages of the present application can be achieved and obtained by the structures specifically pointed out in the specification and the drawings.

[0040] Those skilled in the art will understand that the objects and advantages that can be achieved by the present application are not limited to the above specific descriptions, and the above and other objects that the present application can achieve will be more clearly understood according to the following detailed description. Description of the Drawings

[0041] The accompanying drawings described herein are used to provide a further understanding of the present application, form a part of the present application, and do not limit the present application. The components in the drawings are not drawn to scale, but are only for showing the principles of the present application. For the convenience of showing and describing some parts of the present application, the corresponding parts in the drawings may be enlarged, that is, may become larger relative to other components in the exemplary device actually manufactured according to the present application. In the drawings:

[0042] Figure 1 FIG. 5 is a first flowchart of a method for training an optical transmission matrix measurement model according to an embodiment of the present application.

[0043] Figure 2 FIG. 9 is a schematic diagram of an example of the architecture of an optical fiber imaging system.

[0044] Figure 3 FIG. 13 is a second flowchart of a method for training an optical transmission matrix measurement model according to an embodiment of the present application.

[0045] Figure 4 FIG. 17 is a schematic diagram of the overall architecture of an optical transmission matrix measurement model according to an embodiment of the present application.

[0046] Figure 5 FIG. 21 is a third flowchart of a method for training an optical transmission matrix measurement model according to an embodiment of the present application.

[0047] Figure 6 FIG. 25 is a flowchart of a method for measuring an optical transmission matrix according to an embodiment of the present application.

[0048] Figure 7 FIG. 29 is a flowchart of a method for measuring an optical transmission matrix based on a physics-informed neural network in an application example of the present application.

[0049] Figure 8 FIG. 33 is a schematic diagram of the execution logic of a physics-informed neural network-assisted optical transmission matrix iteration algorithm based on the Gerchberg-Saxton (GS) algorithm in an application example of the present application.

[0050] Figure 9 FIG. 37 is an example of the network architecture of an Attention-UNet neural network in an application example of the present application.

[0051] Figure 10 FIG. 41 is an example of the network architecture of a U-Net neural network in an application example of the present application.

[0052] Figures 11(a) to 11(c) FIG. 45 is an effect display diagram of using the measured optical transmission matrix formed by passing through a scattering channel, focusing on the measured transmission matrix, and focusing on the true transmission matrix respectively provided in an application example of the present application for focusing.

[0053] Figure 12 This is a schematic structural diagram of an optical transmission matrix measurement model training device in an embodiment of the present application.

[0054] Figure 13 This is a schematic structural diagram of an optical transmission matrix measurement device in an embodiment of the present application. Detailed implementation manners

[0055] To make the objectives, technical solutions and advantages of the present application more clear and understandable, the present application will be further described in detail below in conjunction with the implementation manners and the accompanying drawings. Herein, the illustrative implementation manners of the present application and their descriptions are used to explain the present application, but not to limit the present application.

[0056] Herein, it should also be noted that in order to avoid obscuring the present application due to unnecessary details, only the structures and / or processing steps closely related to the solution of the present application are shown in the drawings, while other details less related to the present application are omitted.

[0057] It should be emphasized that the term "including / containing" when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0058] Herein, it should also be noted that if not otherwise specified, the term "connection" in this document can not only refer to direct connection, but also represent indirect connection with an intermediate.

[0059] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0060] In order to solve the problems that the existing methods for changing the eye feature of a person in a video cannot reduce the amount of collected data and lower the device requirements while ensuring the measurement accuracy of the optical transmission matrix, the embodiments of the present application respectively provide an optical transmission matrix measurement model training method, an optical transmission matrix measurement model training device for executing the optical transmission matrix measurement model training method, an optical transmission matrix measurement method, an optical transmission matrix measurement device for executing the optical transmission matrix measurement method, an entity device, a computer-readable storage medium and a computer program product, which can reduce the amount of data required for training the optical transmission matrix measurement model and lower the complexity requirements of the required devices while ensuring the measurement accuracy of the optical transmission matrix.

[0061] Specifically, it will be described in detail through the following embodiments.

[0062] Based on this, the embodiments of the present application provide an optical transmission matrix measurement model training method that can be implemented by an optical transmission matrix measurement model training device. SeeFigure 1 , the optical transmission matrix measurement model training method specifically includes the following content:

[0063] Step 100: Using the scattered spot amplitudes respectively corresponding to each image after being transmitted through the fiber optic imaging system, and the planar amplitude and planar phase corresponding to each of the images, based on a preset iterative algorithm for phase recovery, perform iterative calculations on the connected first physical information neural network and measurement amplitude iterative calculation module to train the first physical information neural network and the measurement amplitude calculation iterative module into an amplitude measurement model for outputting the target measurement amplitude of the optical transmission matrix corresponding to the scattered spot amplitude.

[0064] In one or more embodiments of the present application, the optical transmission matrix can also be referred to as an optical interconnection transmission matrix; the planar amplitude refers to the amplitude of the image in the transmission plane of the fiber optic imaging system (i.e., the optical amplitude), and the amplitude distribution is Gaussian light; the planar phase refers to the phase of the image in the transmission plane of the fiber optic imaging system, that is, the image loaded on the spatial light modulator in the fiber optic imaging system; the scattered spot amplitude refers to the amplitude of the scattered spot collected by the CCD camera in the fiber optic imaging system after the image loaded on the spatial light modulator in the fiber optic imaging system is transmitted through the optical fiber, etc.; the scattered spot phase mentioned in the following content refers to the phase corresponding to the scattered spot amplitude output by the first physical information neural network according to the scattered spot amplitude input thereto.

[0065] It can be understood that the target measurement amplitude of the optical transmission matrix refers to the measurement result data output by the amplitude measurement model; the target measurement phase of the optical transmission matrix mentioned in the following content refers to the measurement result data output by the phase measurement model mentioned in the following content.

[0066] Among them, in one or more embodiments of the present application, the images are all image data based on Gaussian distribution as the measurement data base. The image data can specifically adopt a preset noise map, and these images are used to train the optical transmission matrix measurement model and can be generated by the upper computer of the system.

[0067] In step 100, the amplitude measurement iterative calculation module refers to a functional module that executes an iterative algorithm for phase recovery. It can be understood that the iterative algorithm for phase recovery adopted in one or more embodiments of the present application may specifically adopt the Gerchberg-Saxton (GS) algorithm, also known as the GS phase recovery algorithm. The GS phase recovery algorithm is an iterative algorithm for optical phase recovery. The GS phase recovery algorithm establishes a connection between the amplitude information of the input optical field and the intensity information of the output optical field by alternately performing inverse Fourier transform and forward Fourier transform to achieve the recovery of the unknown phase. The basic idea of the GS phase recovery algorithm is to estimate the phase using the known amplitude information during the wavefront propagation process, then convert the estimated phase to the frequency domain through Fourier transform, and then use the inverse Fourier transform to restore the phase to the spatial domain. It iterates continuously until convergence. In each iteration, by multiplying the complex amplitude of the target with the derived phase, a new wavefront can be obtained, then a new spectrum can be obtained through Fourier transform, and then the new spectrum is converted back to the spatial domain through inverse Fourier transform, thereby continuously correcting and optimizing the estimated value of the phase.

[0068] It should be noted that the fiber optic imaging system mentioned in one or more embodiments of the present application can adopt various fiber optic imaging systems equipped with a spatial light modulator and a CCD camera. For example, see Figure 2 , the fiber optic imaging system may sequentially include a half-wave lens, a polarizer, a lens system for beam expansion, a beam splitter, a spatial light modulator, a diaphragm, a fiber optic coupler and a fiber optic, an expander, a polarizer, and a CCD camera according to the optical path direction of the laser source. Among them, the CCD camera is any digital camera with a charge-coupled device image sensor. CCD is the abbreviation of charge coupled device.

[0069] Step 200: Based on the target measurement amplitudes corresponding to the scattered spot amplitudes of each of the images output by the amplitude measurement model, train a second physics-informed neural network to train the second physics-informed neural network into a phase measurement model for outputting the target measurement phase of the optical transmission matrix according to the measurement amplitudes; wherein, the amplitude measurement model and the phase measurement model constitute an optical transmission matrix measurement model.

[0070] In one or more embodiments of the present application, the first physics-informed neural network and the second physics-informed neural network may adopt physics-informed neural networks with the same or different model architectures. The physics-informed neural network is also a neural network based on physical information, and can also be simply referred to as a physical neural network.

[0071] It can be understood that, in order to obtain a measurement value of a high-precision optical transmission matrix with less collected data volume, in one or more embodiments of the present application, the architecture optimization of the amplitude measurement model corresponding to the first physics-informed neural network is achieved through the setting of the measurement amplitude iterative calculation module.

[0072] Among them, both the first physics-informed neural network and the second physics-informed neural network can adopt the U-Net network in the semantic segmentation model, and can also adopt the Attention-UNet network, etc.

[0073] In an example where the second physics-informed neural network adopts a U-Net network, the target measurement amplitudes corresponding to the scattered spot amplitudes of each of the images output by the amplitude measurement model are used as the input of the U-Net network. After a series of processes such as convolution, downsampling, normalization, activation function, skip connection and transposed convolution, upsampling and channel fusion in the U-Net network, the output obtained is used as the measurement phase of the optical transmission matrix; after combining the measurement phase with the target measurement amplitude, it is multiplied by the complex matrix of the transmission plane (i.e., the complex matrix composed of the plane phase and the plane amplitude) and the absolute value is taken to obtain an intermediate term, and then the loss function is calculated using the intermediate term and the scattered spot amplitude recorded by the CCD camera, and the network parameters of the second physics-informed neural network are updated by backpropagation; when the loss function is stable or reaches the maximum number of iterations, it stops, and the training of the second physics-informed neural network is completed, and the measurement phase output by the current second physics-informed neural network is used as the target measurement phase, and combined with the target measurement amplitude to obtain the optical transmission matrix measured by the final algorithm.

[0074] From the above description, it can be seen that the optical transmission matrix measurement model training method provided by the embodiments of the present application can reduce the data volume required for training the optical transmission matrix measurement model, can effectively reduce the complexity requirements of the required equipment, and thus can effectively improve the reliability and effectiveness of the optical transmission matrix measurement model training process. The trained measurement model can realize the holographic measurement of the optical transmission matrix including amplitude and phase, and can ensure the accuracy of the optical transmission matrix measurement using the trained measurement model. Furthermore, applications such as focusing the scattered spot amplitude of the receiving plane and restoring and extracting the scattered spot information can be realized using the optical transmission matrix.

[0075] In order to further improve the effectiveness and reliability of the amplitude measurement model training, further reduce the data volume required for training the optical transmission matrix measurement model and reduce the complexity requirements of the required equipment, in an optical transmission matrix measurement model training method provided by the embodiments of the present application, refer to Figure 3 and Figure 4 , step 100 in the optical transmission matrix measurement model training method specifically includes the following content:

[0076] Step 110: In the current iteration round, the scattered spot amplitudes respectively corresponding to each image after being transmitted by the fiber optic imaging system are sequentially input into the first physics-informed neural network, so that the first physics-informed neural network outputs the scattered spot phases respectively corresponding to the scattered spot amplitudes of each of the images.

[0077] Step 120: In the measurement amplitude iterative calculation module, based on a preset iterative algorithm for phase recovery, based on the scattered spot amplitudes, the scattered spot phases, the pre-acquired plane amplitude, and the plane phase respectively corresponding to each of the images, obtain the measured amplitude of the optical transfer matrix corresponding to the current iteration round and correspondingly update the network parameters of the first physics-informed neural network.

[0078] Step 130: Determine whether the first physics-informed neural network and the measurement amplitude iterative calculation module in the current iteration round converge; if so, use the measured amplitude obtained in the current iteration round as the target measured amplitude, and use the first physics-informed neural network and the measurement amplitude iterative calculation module corresponding to the current iteration round as an amplitude measurement model for outputting the target measured amplitude of the optical transfer matrix corresponding to the scattered spot amplitude; if not, perform training on the first physics-informed neural network and the measurement amplitude iterative calculation module in the next iteration round.

[0079] To improve the effectiveness and reliability of the convergence judgment of the amplitude measurement model, in an optical transfer matrix measurement model training method provided in an embodiment of the present application, refer to Figure 5 , step 130 in the optical transfer matrix measurement model training method specifically includes the following content:

[0080] Step 131: Determine whether the correlation coefficient between the measured amplitude of the optical transfer matrix corresponding to the current iteration round and the measured amplitudes of the optical transfer matrices corresponding to historical iteration rounds that are at least one iteration round apart from the current iteration round is higher than a preset threshold. If so, determine that the first physics-informed neural network and the measurement amplitude iterative calculation module in the current iteration round converge and execute step 123; if not, determine that the first physics-informed neural network and the measurement amplitude iterative calculation module in the current iteration round do not converge and execute step 133.

[0081] Step 132: Use the measured amplitude obtained in the current iteration round as the target measured amplitude, and use the first physics-informed neural network and the measurement amplitude iterative calculation module corresponding to the current iteration round as an amplitude measurement model for outputting the target measured amplitude of the optical transfer matrix corresponding to the scattered spot amplitude.

[0082] Step 133: Train the first physical information neural network and the measurement amplitude iterative calculation module for the next iteration round.

[0083] It can be understood that the historical iteration rounds that are at least one iteration round apart from the current iteration round refer to: If the current iteration round is the nth round, then the historical iteration round that is one iteration round apart from the current iteration round refers to the (n - 2)th round. It can also be set to (n - 3) etc. according to the actual application scenario.

[0084] Taking the current iteration round as the nth round and the historical iteration round as the (n - 2)th round as an example, if the correlation coefficient (correlation) between the optical transmission matrix obtained in the nth iteration and the optical transmission matrix obtained in the (n - 2)th iteration is higher than the preset threshold of 0.99999999, it is determined that the algorithm has reached the convergence state. The correlation coefficient is obtained by using the normalized correlation coefficient of the complex matrix. The specific steps are as follows:

[0085] The similarity of two matrices is measured by complex conjugate inner product and normalization. Take the conjugate of the result of the nth iteration and calculate the inner product with the result of the (n - 2)th iteration, and use the norms of the two matrices for normalization to measure their similarity.

[0086] It should be noted that if the current iteration round is the first or second round, then there is no historical iteration round that is at least one iteration round apart from the current iteration round at this time. Therefore, for the case where the current iteration round is the first or second round, steps 131 and 132 are not executed, and step 133 can be directly executed, that is, directly train the first physical information neural network and the measurement amplitude iterative calculation module for the next iteration round.

[0087] To help the iterative process jump out of the local optimum and converge to the global optimum, in an optical transmission matrix measurement model training method provided in an embodiment of the present application, the content between steps 120 and 130 in the optical transmission matrix measurement model training method may specifically include the following:

[0088] Optimize the learning rate corresponding to the first physical information neural network based on the cosine annealing method.

[0089] Specifically, in the iterative process with network parameter updates, the cosine annealing method is used to modify the learning rate of the model optimizer, so that the learning rate of this iterative process rises steeply and gradually decreases periodically, which helps the iterative process jump out of the local optimum and converge to the global optimum.

[0090] In order to achieve an initial setting of the transmission matrix that approximates the global optimum and accelerate the convergence in the subsequent iterative process, in an optical transmission matrix measurement model training method provided in an embodiment of the present application, before step 110 in the optical transmission matrix measurement model training method, the following content may specifically be included:

[0091] Multiply the preset optical transmission matrix initialization data by the complex matrix composed of the plane amplitude and the plane phase corresponding to the pre-acquired image, and record the error between the amplitude in the obtained product result and the amplitude of the scattering spot corresponding to the image as the loss function for iteratively training the first physical information neural network based on the stochastic gradient descent method.

[0092] In order to help the iterative process jump out of the local optimum and converge to the global optimum, in an optical transmission matrix measurement model training method provided in an embodiment of the present application, between step 120 and step 130 in the optical transmission matrix measurement model training method, the following content may specifically be included:

[0093] Perform Gaussian blur processing on the scattering spot phase output by the first physical information neural network.

[0094] Specifically, in the initial stage of iteration, after updating the optical transmission matrix, perform Gaussian blur on the phase of the transmission matrix, so that the updated transmission matrix terms are explored in a larger range, reducing the possibility of falling into the local optimum.

[0095] In order to further improve the effectiveness and reliability of the optical transmission matrix measurement model training process, in an optical transmission matrix measurement model training method provided in an embodiment of the present application, refer to Figure 5 , before step 100 in the optical transmission matrix measurement model training method, the following content is specifically included:

[0096] Step 010: Load each image based on the Gaussian distribution into the spatial light modulator in the fiber optic imaging system as the plane phase corresponding to each image, and the plane amplitude corresponding to each image is Gaussian light.

[0097] Step 020: Obtain the amplitude of the scattering spot corresponding to each image received by the CCD camera in the fiber optic imaging system.

[0098] Step 030: Perform image cropping and normalization processing on the plane phase, the plane amplitude, and the scattering spot amplitude corresponding to each image.

[0099] Based on the optical transmission matrix measurement model training method provided in the foregoing embodiment, the present application also provides an embodiment of an optical transmission matrix measurement method. Refer toFigure 6 , the optical transmission matrix measurement method specifically includes the following content:

[0100] Step 300: Obtain the planar amplitude and planar phase of the target image, and the scattered spot amplitude obtained after the target image is transmitted through the fiber optic imaging system;

[0101] Step 400: Input the planar amplitude, planar phase of the target image, and the scattered spot amplitude into the optical transmission matrix measurement model pre-trained by the optical transmission matrix measurement model training method, so that the amplitude measurement model in the optical transmission matrix measurement model outputs the target measurement amplitude of the optical transmission matrix corresponding to the target image according to the planar amplitude, planar phase of the target image, and the scattered spot amplitude, and enable the phase measurement model in the optical transmission matrix measurement model to output the target measurement phase of the optical transmission matrix corresponding to the target image according to the target measurement amplitude corresponding to the target image.

[0102] Step 500: Output the optical transmission matrix including the target measurement amplitude and the target measurement phase corresponding to the target image.

[0103] After the measurement result data of the optical transmission matrix is output in step 500, the focusing of any pixel point on the receiving plane and the recovery and extraction of the scattered spot information can be realized according to the optical transmission matrix. The high-quality and high-speed transmission process of the optical interconnection transmission system can be realized. In addition, in astronomical observations, due to the influence of atmospheric turbulence, the obtained optical signal is usually distorted in phase. This algorithm can help restore the wavefront information, thereby improving the observation quality; in adaptive optics, by obtaining the wavefront information in real time, this algorithm can help adjust the mirror shape in real time, correct the atmospheric influence, and optimize the optical imaging.

[0104] The optical transmission matrix measurement model training method mentioned in step 400 in the optical transmission matrix measurement method provided by this application can specifically be used to execute the processing flow of the embodiment of the optical transmission matrix measurement model training method mentioned in the above embodiment. Its functions will not be elaborated here, and reference can be made to the detailed description of the embodiment of the optical transmission matrix measurement model training method above.

[0105] As can be seen from the above description, the optical transmission matrix measurement method provided by the embodiment of this application can ensure the accuracy of the optical transmission matrix measurement using the trained measurement model, and further can realize applications such as focusing the scattered spot amplitude on the receiving plane and recovering and extracting the scattered spot information using the optical transmission matrix.

[0106] To further illustrate the embodiments of the above optical transmission matrix measurement model training method and optical transmission matrix measurement method, the present application also provides a specific application example of an optical transmission matrix measurement method based on a physics-informed neural network applied to an optical image signal system. Refer to Figure 7 , set up the optical path of the fiber imaging system, display the measurement signal basis to be transmitted on the spatial light modulator, and collect the speckle intensity output by the fiber at the receiving end using a CCD camera. These two are used to form a data set for measuring the optical transmission matrix; use a physics-informed neural network-assisted optical transmission matrix iterative algorithm based on the Gerchberg-Saxton (GS) algorithm to perform iterative calculations with network parameter updates until the result converges; extract the amplitude part of the optical transmission matrix iterative result as the input of a physics-based improved U-Net structure neural network, and use the recorded scattered speckle amplitude and the matrix multiplication result of the iterative obtained transmission matrix and the amplitude-phase complex matrix at the input end as the loss function for training the network. The output of the physics-based improved U-Net is used as the phase part of the optical transmission matrix, and an iterative training process with network parameter updates is carried out. When the value of the loss function is stable, the iteration ends. Combine the optical transmission matrix amplitude obtained in the first iteration with the optical transmission matrix phase obtained in the second iteration as the measured value of the final optical transmission matrix and store it; the measured optical transmission matrix is used to achieve the focusing effect of the scattered speckle amplitude on the receiving plane and the recovery and extraction of speckle information.

[0107] That is to say, the purpose of the application example of the present application is to provide an optical transmission matrix measurement method based on a physics-informed neural network, measure the optical interconnection transmission matrix based on the physics-informed neural network, and be able to effectively achieve high-precision measurement of the optical transmission matrix with a relatively low amount of collected data, and achieve focusing at any position on the receiving plane. Compared with the traditional optical transmission matrix measurement method, the application example of the present application can reduce the requirement for data acquisition volume and the complexity of the acquisition system.

[0108] Based on this, an optical transmission matrix measurement method based on a physics-informed neural network disclosed in the application example of the present application includes the following steps:

[0109] Step 1: Set up the reading Figure 2The fiber optic imaging system shown, the fiber optic imaging system sequentially includes a half-wave lens, a polarizer, a lens system for beam expansion, a beam splitter, a spatial light modulator, a diaphragm, a fiber optic coupler and a fiber optic, a beam expander, a polarizer, and a CCD camera according to the optical path direction of a 1550 nm laser source. The upper computer in the system generates 96 8×8 noise maps based on the Gaussian distribution as the measurement data basis; a single data basis is loaded onto the spatial light modulator as the phase of the transmission plane, and the light amplitude distribution of the transmission plane is Gaussian light; the CCD camera records the scattered light spot amplitude after passing through the system at the receiving plane, forms a measurement dataset of the optical transmission matrix for transmission and reception corresponding to the information at the transmitting end, and then performs image clipping processing to a 64×64 image and then performs normalization processing. Multiple data bases form the dataset used in the subsequent iterative training process after transmission - recording - processing.

[0110] Among them, by using various image clipping and processing sizes, an end-to-end optical transmission matrix with arbitrary resolution can be obtained.

[0111] Step 2: Construct a physical information neural network-assisted optical transmission matrix iterative algorithm based on the Gerchberg-Saxton (GS) algorithm, and perform iterative calculations with network parameter updates.

[0112] The method for constructing a physical information neural network-assisted optical transmission matrix iterative algorithm based on the Gerchberg-Saxton (GS) algorithm is as follows:

[0113] See Figure 8 , in order to obtain a high-precision measurement value of the optical transmission matrix with less acquisition data volume, the application example of this application relies on the Attention-UNet neural network based on physical information to generate its phase from the scattered spot amplitude collected by the CCD camera, combines it with the recorded spot amplitude to obtain the receiving plane term in the iteration; both the transmitting plane term and the receiving plane term are complex matrices of amplitude and phase at this time, and the iterative value of the optical transmission matrix can be calculated; repeat this process until the correlation coefficient (correlation) between the optical transmission matrix obtained in the nth iteration and the optical transmission matrix obtained in the n - 2 iteration is higher than the preset threshold of 0.99999999, then it is judged that the algorithm reaches the convergence state. Output the optical transmission matrix obtained in the nth iteration.

[0114] In Figure 8 , L represents the number of images as the measurement basis, which can be 96 in this application example; I represents the intensity of the transmitting plane; represents the phase of the transmitting plane; P represents the complex term of the transmitting plane; represents the complex form of the phase of the transmitting plane; represents the amplitude of the transmitting plane; represents the optical transmission matrix; Represents the phase of the scattered light spot; Represents the amplitude of the scattered light spot; Represents the measured amplitude; Represents the intermediate term output by the iterative algorithm; Represents the intermediate term input to the iterative algorithm.

[0115] In addition, Figure 8 the neural network in Figure 9 can adopt the network architecture of the Attention-UNet neural network as shown in

[0116] Among them, in this iterative process with network parameter updates, the cosine annealing method is used to modify the learning rate of the model optimizer, so that the learning rate of this iterative process rises steeply and gradually decreases periodically, which helps the iterative process to jump out of the local optimum and converge to the global optimum.

[0117] Among them, in the initialization process of this iterative algorithm, the amplitude of the result of multiplying the initially initialized transmission matrix by the known amplitude-phase complex matrix of the sending plane is used to perform error analysis with the amplitude recorded by the CCD camera, and it is recorded as the loss function; the transmission matrix is iterated by using the method of stochastic gradient descent to realize the initial setting of the transmission matrix that approximates the global optimum, and it speeds up the convergence in the subsequent iterative process.

[0118] Among them, in the initial stage of iteration, after updating the optical transmission matrix, Gaussian blur is performed on the phase of the transmission matrix, so that the updated transmission matrix terms are explored in a larger range, reducing the possibility of it falling into the local optimum.

[0119] Step 3: According to the research and verification, the optical transmission matrix obtained by the iterative algorithm with network parameter updates in Step 2 has high accuracy in amplitude, but there are unstable deviations in phase. Step 3 is based on the amplitude of the transmission matrix obtained in Step 2, and uses the U-Net network structure and the iterative process with physical information to achieve the convergence of the phase part of the transmission matrix. The iterative process of Step 3 does not require an additional data set, and uses the data set used in Step 2 as the training pair for supervised learning.

[0120] The specific process of Step 3 is as follows: Using the amplitude of the transmission matrix as Figure 10The input of the U-Net neural network shown is passed through a series of processes such as convolution, downsampling, normalization, activation function, skip connection, transposed convolution, upsampling, and channel fusion, and the obtained output is used as the phase of the transmission matrix; after combining the phase and amplitude, it is multiplied by the complex matrix of the transmission plane and the absolute value is taken to obtain an intermediate term. The loss function is calculated using the intermediate term and the amplitude of the scattered light spot recorded by the CCD camera, and the network parameters are updated by backpropagation; when the loss function is stable or the maximum number of iterations is reached, it stops. The combined phase of the output and the amplitude of the transmission matrix obtained in step two gives the optical transmission matrix measured by the final algorithm.

[0121] Step 4: Combine the amplitude of the optical transmission matrix obtained by training in step two and the phase value of the optical transmission matrix obtained by training in step three to obtain the final measured optical transmission matrix. Use this matrix to achieve the focusing of any pixel point on the receiving plane and the restoration and extraction of the scattered light spot information, and realize the high-quality and high-speed transmission process of the optical interconnection transmission system. Among them, the measured optical transmission matrix is used as a diagram showing the focusing effect as Figures 11(a) to 11(c) shown.

[0122] Take the values of any column of the transmission matrix, reshape it into the input size of 8×8, take the conjugate of the matrix, and use it as the input of the fiber optic imaging system as Figure 2 shown to achieve the focusing of any pixel point and the restoration and extraction of the scattered light spot information, and realize the high-quality and high-speed transmission process of the optical interconnection transmission system.

[0123] In summary, the application example of the present application has the following beneficial effects:

[0124] 1. A method for measuring an optical transmission matrix based on a physics-informed neural network disclosed in the application example of the present application. Build the optical path of the fiber optic imaging system, display the measurement signal basis to be transmitted on the spatial light modulator, and collect the speckle intensity output by the fiber optic at the receiving end with a CCD camera. These two are used to form a data set for measuring the optical transmission matrix; use an iterative algorithm for the optical transmission matrix assisted by a physics-informed neural network based on the Gerchberg-Saxton (GS) algorithm to perform iterative calculations with network parameter updates until the result converges; extract the amplitude part of the iterative result of the optical transmission matrix as the input of a physics-informed improved U-Net structure neural network, and use the amplitude of the recorded scattered light spot and the matrix multiplication result of the iterative obtained transmission matrix and the amplitude-phase complex matrix at the input end as the loss function for training the network. The output of the physics-informed improved U-Net is used as the phase part of the optical transmission matrix, and an iterative training process with network parameter updates is carried out. When the value of the loss function is stable, the iteration ends. This method effectively reduces the amount of collected data for accurately measuring the optical transmission matrix and reduces the complexity of the measurement system.

[0125] 2. A method for measuring an optical transmission matrix based on a physics-informed neural network disclosed in the application example of the present application. In the iterative process with network parameter update in step two, the cosine annealing method is used to modify the learning rate of the model optimizer, so that the learning rate in this iterative process rises steeply and gradually decreases periodically, which helps the iterative process to jump out of the local optimum and converge to the global optimum, realizing the accelerated convergence and accuracy improvement of the training process.

[0126] 3. A method for measuring an optical transmission matrix based on a physics-informed neural network disclosed in the application example of the present application. In the initialization process of the iterative algorithm in step two, an error analysis is performed on the amplitude of the result of multiplying the initially initialized transmission matrix by the known amplitude-phase complex matrix of the sending plane and the amplitude recorded by the CCD camera, and it is recorded as the loss function. The transmission matrix is iterated by using the method of stochastic gradient descent to realize the initial setting of the transmission matrix approaching the global optimum, accelerating the convergence in the subsequent iterative process, and realizing the accelerated convergence and accuracy improvement of the training process.

[0127] 4. A method for measuring an optical transmission matrix based on a physics-informed neural network disclosed in the application example of the present application. In the initial stage of iteration, after updating the optical transmission matrix, Gaussian blur is performed on the phase of the transmission matrix, so that the updated transmission matrix terms are explored in a larger range, reducing the possibility of falling into the local optimum.

[0128] 5. A method for measuring an optical transmission matrix based on a physics-informed neural network disclosed in the application example of the present application. According to the amplitude of the optical transmission matrix obtained by training in step two and the phase value of the optical transmission matrix obtained by training in step three, the final measured optical transmission matrix is combined, and this matrix is used to realize the focusing and the recovery and extraction of the scattered spot information of any pixel point on the receiving plane, realizing the high-quality and high-speed transmission process of the optical interconnection transmission system.

[0129] At the software level, the present application also provides an optical transmission matrix measurement model training device for executing all or part of the content in the optical transmission matrix measurement model training method, see Figure 12 The optical transmission matrix measurement model training device specifically includes the following content:

[0130] The amplitude measurement model training module 10 is used to train the connected first physics-informed neural network and the measurement amplitude iterative calculation module based on the scattered spot amplitudes respectively corresponding to each image after being transmitted through the fiber imaging system and the plane amplitudes and plane phases corresponding to each of the images, based on a preset iterative algorithm for phase recovery, so as to train the first physics-informed neural network and the measurement amplitude calculation iterative module into an amplitude measurement model for outputting the target measurement amplitude of the optical transmission matrix corresponding to the scattered spot amplitude;

[0131] A phase measurement model training module 20 is configured to train a second physical information neural network based on the target measurement amplitudes corresponding to the scattering spot amplitudes of each of the images output by the amplitude measurement model, so as to train the second physical information neural network into a phase measurement model for outputting a target measurement phase of the optical transmission matrix according to the measurement amplitudes; wherein, the amplitude measurement model and the phase measurement model constitute an optical transmission matrix measurement model.

[0132] The embodiment of the optical transmission matrix measurement model training device provided in this application can specifically be used to execute the processing flow of the embodiment of the optical transmission matrix measurement model training method in the above embodiment, and its functions will not be elaborated here. Reference can be made to the detailed description of the embodiment of the optical transmission matrix measurement model training method above.

[0133] The part of the optical transmission matrix measurement model training device for training the optical transmission matrix measurement model can be executed in a server or completed in a client device. Specifically, it can be selected according to the processing capacity of the client device and the limitations of the user usage scenario. This application does not make any limitations in this regard. If all operations are completed in the client device, the client device may further include a processor for specific processing of optical transmission matrix measurement model training.

[0134] The above-mentioned client device may have a communication module (i.e., a communication unit) and can communicate with a remote server to achieve data transmission with the server. The server may include a server on the task scheduling center side, and in other implementation scenarios, it may also include a server of an intermediate platform, such as a server of a third-party server platform having a communication link with the task scheduling center server. The server may include a single computer device, or a server cluster composed of multiple servers, or a server structure of a distributed device.

[0135] Any suitable network protocol can be used for communication between the above-mentioned server and the client device side, including network protocols not yet developed on the filing date of this application. The network protocol may, for example, include TCP / IP protocol, UDP / IP protocol, HTTP protocol, HTTPS protocol, etc. Of course, the network protocol may also, for example, include RPC protocol (Remote Procedure Call Protocol) and REST protocol (Representational State Transfer) used on top of the above protocols.

[0136] As can be seen from the above description, the optical transmission matrix measurement model training device provided by the embodiments of the present application can reduce the amount of data required for training the optical transmission matrix measurement model, effectively reduce the complexity requirements of the required equipment, and thus effectively improve the reliability and effectiveness of the optical transmission matrix measurement model training process. The trained measurement model can achieve holographic measurement of the optical transmission matrix including amplitude and phase, and can ensure the accuracy of the optical transmission matrix measurement using the trained measurement model. Furthermore, applications such as focusing the speckle amplitude at the receiving plane and restoring and extracting speckle information can be realized using the optical transmission matrix.

[0137] From a software perspective, the present application also provides an optical transmission matrix measurement device for executing all or part of the optical transmission matrix measurement method described above. Refer to Figure 13 , the optical transmission matrix measurement device specifically includes the following: a data acquisition module 30 for acquiring the plane amplitude and plane phase of the target image and the scattered speckle amplitude obtained after the target image is transmitted through the fiber optic imaging system.

[0138] A model measurement module 40 for inputting the plane amplitude, plane phase of the target image, and the scattered speckle amplitude into the optical transmission matrix measurement model pre-trained by the optical transmission matrix measurement model training method, so that the amplitude measurement model in the optical transmission matrix measurement model outputs the target measurement amplitude of the optical transmission matrix corresponding to the target image according to the plane amplitude, plane phase of the target image, and the scattered speckle amplitude, and the phase measurement model in the optical transmission matrix measurement model outputs the target measurement phase of the optical transmission matrix corresponding to the target image according to the target measurement amplitude corresponding to the target image.

[0139] A matrix output module 50 for outputting the optical transmission matrix including the target measurement amplitude and the target measurement phase corresponding to the target image.

[0140] The optical transmission matrix measurement model training method mentioned in the model measurement module 40 of the optical transmission matrix measurement device provided by the present application can specifically be used to execute the processing flow of the embodiments of the optical transmission matrix measurement model training method mentioned in the above embodiments. Its functions will not be elaborated here and can be referred to the detailed description of the embodiments of the optical transmission matrix measurement model training method above.

[0141] The part of the optical transmission matrix measurement model training device for performing optical transmission matrix measurement can be executed in the client device or completed in the server. Specifically, it can be selected according to the processing capacity of the client device and the limitations of the user usage scenario, etc. This application does not make any limitations in this regard. If all operations are completed in the client device, the client device may further include a processor for specific processing of optical transmission matrix measurement model training.

[0142] As can be seen from the above description, the optical transmission matrix measurement device provided by the embodiments of this application can ensure the accuracy of optical transmission matrix measurement using the trained measurement model, and thus can be used for applications such as focusing the received plane speckle amplitude and restoring and extracting speckle information using the optical transmission matrix.

[0143] The embodiments of this application further provide an electronic device, which may include a processor, a memory, a receiver, and a transmitter. The processor is used to execute the optical transmission matrix measurement model training method and / or the optical transmission matrix measurement method mentioned in the above embodiments. Among them, the processor and the memory can be connected through a bus or other means. Taking the bus connection as an example, the receiver can be connected to the processor and the memory in a wired or wireless manner.

[0144] The processor can be a Central Processing Unit (CPU). The processor can also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., or a combination of the above types of chips.

[0145] As a non-transitory computer-readable storage medium, the memory can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the optical transmission matrix measurement model training method and / or the optical transmission matrix measurement method in the embodiments of this application. The processor runs the non-transitory software programs, instructions, and modules stored in the memory to execute various functional applications and data processing of the processor, that is, to implement the optical transmission matrix measurement model training method and / or the optical transmission matrix measurement method in the above method embodiments.

[0146] The memory may include a program storage area and a data storage area. The program storage area may store an operating system and application programs required for at least one function. The data storage area may store data created by the processor and the like. In addition, the memory may include high-speed random access memory and may also include non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0147] The one or more modules are stored in the memory and, when executed by the processor, perform the optical transmission matrix measurement model training method and / or the optical transmission matrix measurement method in the embodiments.

[0148] In some embodiments of the present application, the user equipment may include a processor, a memory, and a transceiver unit. The transceiver unit may include a receiver and a transmitter. The processor, the memory, the receiver, and the transmitter may be connected through a bus system. The memory is used to store computer instructions, and the processor is used to execute the computer instructions stored in the memory to control the transceiver unit to transmit and receive signals.

[0149] As an implementation manner, the functions of the receiver and the transmitter in the present application may be considered to be implemented through a transceiver circuit or a dedicated transceiver chip, and the processor may be considered to be implemented through a dedicated processing chip, a processing circuit, or a general-purpose chip.

[0150] As another implementation manner, it may be considered to use a general-purpose computer to implement the server provided in the embodiments of the present application. That is, the program codes for implementing the functions of the processor, the receiver, and the transmitter are stored in the memory, and the general-purpose processor implements the functions of the processor, the receiver, and the transmitter by executing the codes in the memory.

[0151] The embodiments of the present application further provide a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the foregoing optical transmission matrix measurement model training method and / or the optical transmission matrix measurement method are implemented. The computer-readable storage medium may be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium well known in the technical field.

[0152] The embodiments of the present application further provide a computer program product, including a computer program which, when executed by a processor, implements the steps of the foregoing optical transmission matrix measurement model training method and / or optical transmission matrix measurement method.

[0153] Those of ordinary skill in the art should understand that the various exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Specifically, whether to implement in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a functional card, etc. When implemented in software, the elements of the present application are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium or transmitted through a data signal carried in a carrier wave on a transmission medium or a communication link.

[0154] It should be clear that the present application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order between steps after understanding the spirit of the present application.

[0155] In the present application, the features described and / or illustrated for one embodiment can be used in the same or a similar manner in one or more other embodiments, and / or combined with the features of other embodiments or replace the features of other embodiments.

[0156] The above are only the preferred embodiments of the present application and are not used to limit the present application. For those skilled in the art, the embodiments of the present application can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for training an optical transmission matrix measurement model, characterized in that: include: Using the scattered light spot amplitudes corresponding to each image after being transmitted by the optical fiber imaging system and the plane amplitudes and plane phases corresponding to each of the images, based on a preset iterative algorithm for phase recovery, the connected first physical information neural network and the measurement amplitude iterative calculation module are trained to train the first physical information neural network and the measurement amplitude calculation iterative module into an amplitude measurement model for outputting the target measurement amplitude of the optical transmission matrix corresponding to the scattered light spot amplitude; Based on the target measurement amplitude corresponding to the scattered light spot amplitude of each of the images output by the amplitude measurement model, a second physical information neural network is trained to train the second physical information neural network into a phase measurement model for outputting the target measurement phase of the optical transmission matrix according to the measurement amplitude; wherein the amplitude measurement model and the phase measurement model constitute an optical transmission matrix measurement model.

2. The optical transmission matrix measurement model training method according to claim 1, characterized in that: The method adopts the scattered light spot amplitudes corresponding to each image after being transmitted by the optical fiber imaging system and the plane amplitudes and plane phases corresponding to each image, and trains the connected first physical information neural network and the measurement amplitude iterative calculation module based on a preset iterative algorithm for phase recovery, so as to train the first physical information neural network and the measurement amplitude calculation iterative module into an amplitude measurement model for outputting the target measurement amplitude of the optical transmission matrix corresponding to the scattered light spot amplitude, including: In the current iteration round, the scattered light spot amplitudes corresponding to the images after being transmitted by the optical fiber imaging system are sequentially input into the first physical information neural network, so that the first physical information neural network outputs the scattered light spot phases corresponding to the scattered light spot amplitudes corresponding to the images respectively; In the measurement amplitude iteration calculation module, based on a preset iterative algorithm for phase recovery, based on the scattered light spot amplitude, the scattered light spot phase, the pre-acquired plane amplitude and the plane phase corresponding to each of the images, the measurement amplitude of the optical transmission matrix corresponding to the current iteration round is obtained and the network parameters of the first physical information neural network are updated accordingly; Determine whether the first physical information neural network and the measurement amplitude iterative calculation module of the current iteration round have converged; if so, use the measurement amplitude obtained in the current iteration round as the target measurement amplitude, and use the first physical information neural network and the measurement amplitude iterative calculation module corresponding to the current iteration round as an amplitude measurement model for outputting the target measurement amplitude of the optical transmission matrix corresponding to the scattered light spot amplitude; if not, perform training for the next iteration round on the first physical information neural network and the measurement amplitude iterative calculation module.

3. The optical transmission matrix measurement model training method according to claim 2, characterized in that: The determining whether the first physical information neural network and the measurement amplitude iterative calculation module of the current iteration round have converged includes: Determine whether the correlation coefficient between the measured amplitude of the optical transmission matrix corresponding to the current iteration round and the measured amplitude of the optical transmission matrix corresponding to the historical iteration round that is at least one iteration round apart from the current iteration round is higher than a preset threshold value; if so, determine that the first physical information neural network and the measured amplitude iterative calculation module of the current iteration round have converged; if not, determine that the first physical information neural network and the measured amplitude iterative calculation module of the current iteration round have not converged.

4. The optical transmission matrix measurement model training method according to claim 2, characterized in that: Before judging whether the first physical information neural network and the measured amplitude iterative calculation module of the current iteration round have converged, it also includes: optimizing the learning rate corresponding to the first physical information neural network based on the cosine annealing method.

5. The optical transmission matrix measurement model training method according to claim 2, characterized in that: Before sequentially inputting the scattered light spot amplitudes corresponding to the images after being transmitted by the optical fiber imaging system into the first physical information neural network in the current iteration round, the method further includes: The preset optical transmission matrix initialization data is multiplied by the complex matrix composed of the plane amplitude and the plane phase corresponding to the pre-acquired image, and the error between the amplitude in the product result and the scattered light spot amplitude corresponding to the image is recorded as a loss function for iterative training of the first physical information neural network based on the stochastic gradient descent method.

6. The optical transmission matrix measurement model training method according to claim 2, characterized in that: Before judging whether the first physical information neural network and the measurement amplitude iterative calculation module of the current iteration round have converged, it also includes: performing Gaussian blur processing on the scattered light spot phase output by the first physical information neural network.

7. The optical transmission matrix measurement model training method according to any one of claims 1 to 6, characterized in that: The method further comprises: using the scattered light spot amplitudes corresponding to the images after being transmitted by the optical fiber imaging system and the plane amplitudes and plane phases corresponding to the images, and training the connected first physical information neural network and the measurement amplitude iterative calculation module based on a preset iterative algorithm for phase recovery; Loading each image based on Gaussian distribution to a spatial light modulator in a fiber imaging system as a plane phase corresponding to each image, and a plane amplitude corresponding to each image is Gaussian light; Acquire the scattered light spot amplitude corresponding to each of the images received by the CCD camera in the optical fiber imaging system; The plane phase, the plane amplitude and the scattered light spot amplitude corresponding to each of the images are respectively subjected to image cropping and normalization processing.

8. An optical transmission matrix measurement method, characterized in that: include: Acquire the plane amplitude and plane phase of the target image and the scattered light spot amplitude obtained after the target image is transmitted through the optical fiber imaging system; Input the plane amplitude, plane phase and scattered light spot amplitude of the target image into the optical transmission matrix measurement model pre-trained by the optical transmission matrix measurement model training method according to any one of claims 1 to 7, so that the amplitude measurement model in the optical transmission matrix measurement model outputs the target measurement amplitude of the optical transmission matrix corresponding to the target image according to the plane amplitude, plane phase and scattered light spot amplitude of the target image, and the phase measurement model in the optical transmission matrix measurement model outputs the target measurement phase of the optical transmission matrix corresponding to the target image according to the target measurement amplitude corresponding to the target image; The output includes an optical transmission matrix including the target measurement amplitude and the target measurement phase corresponding to the target image.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the optical transmission matrix measurement model training method as described in any one of claims 1 to 7 is implemented, and / or the optical transmission matrix measurement method as described in claim 8 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the optical transmission matrix measurement model training method as described in any one of claims 1 to 7, and / or implements the optical transmission matrix measurement method as described in claim 8.

Images