Optical transmission matrix measurement model training method, matrix measurement method and equipment

Abstract

The invention provides an optical transmission matrix measurement model training method, a matrix measurement method and equipment, and the training method comprises the steps: employing the scattered light spot amplitude corresponding to each image after the image is transmitted through an optical fiber imaging system, and the plane amplitude and plane phase corresponding to each image; training the connected first physical information neural network and the measurement amplitude iterative calculation module to obtain an amplitude measurement model; and training a second physical information neural network based on a target measurement amplitude output by the amplitude measurement model so as to correspondingly output a phase measurement model of a target measurement phase of the optical transmission matrix according to the measurement amplitude. According to the invention, the data volume required for training the optical transmission matrix measurement model can be reduced, and the complexity requirement of required equipment can be effectively reduced, so that the reliability and effectiveness of the training process of the optical transmission matrix measurement model can be effectively improved; and the precision of optical transmission matrix measurement by adopting the measurement model obtained by training can be ensured.

Description

Technical Field

[0001] The present application relates to the field of optical signal processing technology, and in particular to an optical transmission matrix measurement model training method, a matrix measurement method and a device. Background Art

[0002] With the continuous development of global information technology, optical interconnect signal processing has become an important part of the field of information digitization. In the optical image transmission system, the target information carried by the incident light is output as a light field after passing through the scattering medium. It presents a complex speckle distribution, but still retains the initial information of the input light. Therefore, reconstructing target information using scattered light has become a research hotspot. The transmission depth of light in the scattering medium directly affects the effectiveness of ballistic light imaging, and the determinism of the scattering process also makes it possible to achieve focusing or imaging using scattered light.

[0003] The optical transmission matrix of the scattering medium contains multiple intrinsic channels, each corresponding to a different transmission transmittance and output light field. Therefore, through the study of the optical transmission matrix, the output light field energy can be regulated or the light field distribution can be modulated. In addition, the basic properties of optical speckle and its technical applications have also attracted widespread attention in the field of optics. Although the formation of speckle patterns is essentially random, they exhibit predictable properties, representing the commonality of coherent wave dynamics in disordered systems, making speckle patterns a key tool in the field of optical tweezers, imaging technology, and cold atomic random potentials. The optical transmission matrix is ​​a qualitative and quantitative analysis tool for this property.

[0004] However, traditional optical transfer matrix measurement methods often find it difficult to achieve accurate holographic measurement of the optical transfer matrix containing amplitude and phase, which poses a major challenge to the effective reconstruction of scattered light signals. In addition, such methods usually require a large amount of complex acquisition data, the amount of data is often several times the number of pixels of the image resolution, or high-precision intensity-phase pairs are required for holographic reconstruction of the matrix. The above requirements not only increase the acquisition time, but also put forward higher requirements on the complexity of the equipment, limiting the feasibility of this method in practical applications. Therefore, under the premise of ensuring the measurement accuracy of the optical transfer matrix, reducing the amount of acquisition data and lowering the equipment requirements are the key directions of current research. Summary of the invention

[0005] In view of this, embodiments of the present application provide an optical transmission matrix measurement model training method, a matrix measurement method and a device to eliminate or improve one or more defects existing in the prior art.

[0006] One aspect of the present application provides an optical transmission matrix measurement model training method, comprising: Using the scattered light spot amplitudes corresponding to each image after being transmitted by the optical fiber imaging system and the plane amplitudes and plane phases corresponding to each of the images, based on a preset iterative algorithm for phase recovery, the connected first physical information neural network and the measurement amplitude iterative calculation module are trained to train the first physical information neural network and the measurement amplitude calculation iterative module into an amplitude measurement model for outputting the target measurement amplitude of the optical transmission matrix corresponding to the scattered light spot amplitude; Based on the target measurement amplitude corresponding to the scattered light spot amplitude of each of the images output by the amplitude measurement model, a second physical information neural network is trained to train the second physical information neural network into a phase measurement model for outputting the target measurement phase of the optical transmission matrix according to the measurement amplitude; wherein the amplitude measurement model and the phase measurement model constitute an optical transmission matrix measurement model.

[0007] In some embodiments of the present application, the scattered light spot amplitudes corresponding to each image after being transmitted by the optical fiber imaging system and the plane amplitudes and plane phases corresponding to each of the images are used to train the connected first physical information neural network and the measurement amplitude iterative calculation module based on a preset iterative algorithm for phase recovery, so as to train the first physical information neural network and the measurement amplitude calculation iterative module into an amplitude measurement model for outputting the target measurement amplitude of the optical transmission matrix corresponding to the scattered light spot amplitude, including: In the current iteration round, the scattered light spot amplitudes corresponding to the images after being transmitted by the optical fiber imaging system are sequentially input into the first physical information neural network, so that the first physical information neural network outputs the scattered light spot phases corresponding to the scattered light spot amplitudes corresponding to the images respectively; In the measurement amplitude iteration calculation module, based on a preset iterative algorithm for phase recovery, based on the scattered light spot amplitude, the scattered light spot phase, the pre-acquired plane amplitude and the plane phase corresponding to each of the images, the measurement amplitude of the optical transmission matrix corresponding to the current iteration round is obtained and the network parameters of the first physical information neural network are updated accordingly; Determine whether the first physical information neural network and the measurement amplitude iterative calculation module of the current iteration round have converged; if so, use the measurement amplitude obtained in the current iteration round as the target measurement amplitude, and use the first physical information neural network and the measurement amplitude iterative calculation module corresponding to the current iteration round as an amplitude measurement model for outputting the target measurement amplitude of the optical transmission matrix corresponding to the scattered light spot amplitude; if not, perform training for the next iteration round on the first physical information neural network and the measurement amplitude iterative calculation module.

[0008] In some embodiments of the present application, the determining whether the first physical information neural network and the measurement amplitude iterative calculation module of the current iteration round have converged includes: Determine whether the correlation coefficient between the measured amplitude of the optical transmission matrix corresponding to the current iteration round and the measured amplitude of the optical transmission matrix corresponding to the historical iteration round that is at least one iteration round apart from the current iteration round is higher than a preset threshold value; if so, determine that the first physical information neural network and the measured amplitude iterative calculation module of the current iteration round have converged; if not, determine that the first physical information neural network and the measured amplitude iterative calculation module of the current iteration round have not converged.

[0009] In some embodiments of the present application, before determining whether the first physical information neural network and the measurement amplitude iterative calculation module of the current iteration round have converged, the method further includes: The learning rate corresponding to the first physical information neural network is optimized based on the cosine annealing method.

[0010] In some embodiments of the present application, before sequentially inputting the scattered light spot amplitudes corresponding to each image after being transmitted by the optical fiber imaging system into the first physical information neural network in the current iteration round, the method further includes: The preset optical transmission matrix initialization data is multiplied by the complex matrix composed of the plane amplitude and the plane phase corresponding to the pre-acquired image, and the error between the amplitude in the product result and the scattered light spot amplitude corresponding to the image is recorded as a loss function for iterative training of the first physical information neural network based on the stochastic gradient descent method.

[0011] In some embodiments of the present application, before determining whether the first physical information neural network and the measurement amplitude iterative calculation module of the current iteration round have converged, the method further includes: Gaussian blur processing is performed on the phase of the scattered light spot output by the first physical information neural network.

[0012] In some embodiments of the present application, the use of the scattered light spot amplitudes corresponding to each image after being transmitted by the optical fiber imaging system and the plane amplitudes and plane phases corresponding to each of the images, based on a preset iterative algorithm for phase recovery, before training the connected first physical information neural network and the measurement amplitude iterative calculation module, further includes: Loading each image based on Gaussian distribution to a spatial light modulator in a fiber imaging system as a plane phase corresponding to each image, and a plane amplitude corresponding to each image is Gaussian light; The scattered light spot amplitude corresponding to each of the images received by the CCD camera in the optical fiber imaging system is obtained; and the plane phase, the plane amplitude and the scattered light spot amplitude corresponding to each of the images are respectively subjected to image cropping and normalization processing.

[0013] Another aspect of the present application provides an optical transmission matrix measurement method, comprising: Acquire the plane amplitude and plane phase of the target image and the scattered light spot amplitude obtained after the target image is transmitted through the optical fiber imaging system; Input the plane amplitude, plane phase and scattered light spot amplitude of the target image into the optical transfer matrix measurement model pre-trained by the optical transfer matrix measurement model training method, so that the amplitude measurement model in the optical transfer matrix measurement model outputs the target measurement amplitude of the optical transfer matrix corresponding to the target image according to the plane amplitude, plane phase and scattered light spot amplitude of the target image, and the phase measurement model in the optical transfer matrix measurement model outputs the target measurement phase of the optical transfer matrix corresponding to the target image according to the target measurement amplitude corresponding to the target image; The output includes an optical transmission matrix including the target measurement amplitude and the target measurement phase corresponding to the target image.

[0014] A third aspect of the present application provides an optical transmission matrix measurement model training device, comprising: an amplitude measurement model training module, for using the scattered light spot amplitudes corresponding to each image after being transmitted by the optical fiber imaging system and the plane amplitudes and plane phases corresponding to each of the images, and based on a preset iterative algorithm for phase recovery, training the connected first physical information neural network and the measurement amplitude iterative calculation module, so as to train the first physical information neural network and the measurement amplitude iterative calculation module into an amplitude measurement model for outputting a target measurement amplitude of an optical transmission matrix corresponding to the scattered light spot amplitude; A phase measurement model training module is used to train a second physical information neural network based on the target measurement amplitude corresponding to the scattered light spot amplitude of each of the images output by the amplitude measurement model, so as to train the second physical information neural network into a phase measurement model for outputting the target measurement phase of the optical transmission matrix according to the measurement amplitude; wherein the amplitude measurement model and the phase measurement model constitute an optical transmission matrix measurement model.

[0015] A fourth aspect of the present application provides an optical transmission matrix measurement device, comprising: A data acquisition module, used to acquire the plane amplitude and plane phase of the target image and the scattered light spot amplitude obtained after the target image is transmitted through the optical fiber imaging system; A model measurement module, used for inputting the plane amplitude, plane phase and scattered light spot amplitude of the target image into the optical transfer matrix measurement model pre-trained by the optical transfer matrix measurement model training method, so that the amplitude measurement model in the optical transfer matrix measurement model outputs the target measurement amplitude of the optical transfer matrix corresponding to the target image according to the plane amplitude, plane phase and scattered light spot amplitude of the target image, and the phase measurement model in the optical transfer matrix measurement model outputs the target measurement phase of the optical transfer matrix corresponding to the target image according to the target measurement amplitude corresponding to the target image; The matrix output module is used to output an optical transmission matrix including the target measurement amplitude and the target measurement phase corresponding to the target image.

[0016] The fifth aspect of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the optical transmission matrix measurement model training method and / or implements the optical transmission matrix measurement method.

[0017] The sixth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the optical transmission matrix measurement model training method and / or the optical transmission matrix measurement method.

[0018] The seventh aspect of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the optical transmission matrix measurement model training method and / or implements the optical transmission matrix measurement method.

[0019] The optical transmission matrix measurement model training method provided in the present application adopts the scattered light spot amplitudes corresponding to each image after being transmitted by the optical fiber imaging system and the plane amplitudes and plane phases corresponding to each of the images, and based on a preset iterative algorithm for phase recovery, trains a first physical information neural network and a measurement amplitude iterative calculation module connected to each other, so as to train the first physical information neural network and the measurement amplitude calculation iterative module into an amplitude measurement model for outputting a target measurement amplitude of the optical transmission matrix corresponding to the scattered light spot amplitude; trains a second physical information neural network based on the target measurement amplitude corresponding to the scattered light spot amplitude of each of the images output by the amplitude measurement model, so as to train the second physical information neural network The network training is a phase measurement model for outputting a target measurement phase of the optical transmission matrix according to the measurement amplitude; wherein the amplitude measurement model and the phase measurement model constitute an optical transmission matrix measurement model, which can reduce the amount of data required for training the optical transmission matrix measurement model, can effectively reduce the complexity requirements of the required equipment, and thus can effectively improve the reliability and effectiveness of the optical transmission matrix measurement model training process, the measurement model obtained by training can realize holographic measurement of the optical transmission matrix including amplitude and phase, and can ensure the accuracy of optical transmission matrix measurement using the trained measurement model, and thus can use the optical transmission matrix to realize the focusing of the amplitude of the scattered light spot on the receiving plane and the recovery and extraction of speckle information and other applications.

[0020] Additional advantages, purposes, and features of the present application will be partially described in the following description, and will become partially apparent to those skilled in the art after studying the following, or may be learned from the practice of the present application. The purposes and other advantages of the present application can be achieved and obtained by the structures specifically pointed out in the specification and the drawings.

[0021] Those skilled in the art will understand that the purposes and advantages that can be achieved by the present application are not limited to the above specific description, and the above and other purposes that can be achieved by the present application will be more clearly understood based on the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings described herein are used to provide a further understanding of the present application, constitute a part of the present application, and do not constitute a limitation of the present application. The components in the drawings are not drawn to scale, but are only for the purpose of illustrating the principles of the present application. In order to facilitate the illustration and description of some parts of the present application, the corresponding parts in the drawings may be enlarged, that is, they may become larger relative to other components in the exemplary device actually manufactured according to the present application. In the drawings: Figure 1 Schematic diagram of a first flow chart of an optical transmission matrix measurement model training method in an embodiment of the present application.

[0023] Figure 2 A schematic diagram showing an example of the architecture of a fiber optic imaging system.

[0024] Figure 3 Schematic diagram of a second flow chart of the optical transmission matrix measurement model training method in one embodiment of the present application.

[0025] Figure 4 Schematic diagram of the overall architecture of an optical transmission matrix measurement model in an embodiment of the present application.

[0026] Figure 5 Schematic diagram of a third flow chart of the optical transmission matrix measurement model training method in one embodiment of the present application.

[0027] Figure 6 Schematic diagram of the process of measuring an optical transmission matrix in one embodiment of the present application.

[0028] Figure 7 This is a flow chart of an optical transmission matrix measurement method based on a physical information neural network in an application example of this application.

[0029] Figure 8 This is a schematic diagram of the execution logic of the optical transmission matrix iteration algorithm assisted by the physical information neural network based on the Gerchberg-Saxton (GS) algorithm in the application example of this application.

[0030] Fig. 9 This is an example of the network architecture of the Attention-UNet neural network in the application example of this application.

[0031] Fig.10 This is an example of the network architecture of the U-Net neural network in the application example of this application.

[0032] Figure 11(a) to Figure 11(c) The measured optical transmission matrix formed by focusing through a scattering channel, focusing through a measured transmission matrix, and focusing through a real transmission matrix provided for the application examples of this application is used as a focusing effect demonstration diagram.

[0033] Fig.12 Schematic diagram of the structure of an optical transmission matrix measurement model training device in one embodiment of the present application.

[0034] Fig.13 Schematic diagram of the structure of an optical transmission matrix measurement device in one embodiment of the present application. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the implementation modes and the accompanying drawings. Here, the illustrative implementation modes and descriptions of the present application are used to explain the present application, but are not intended to limit the present application.

[0036] It should also be noted here that in order to avoid obscuring the present application due to unnecessary details, only the structures and / or processing steps closely related to the scheme according to the present application are shown in the accompanying drawings, while other details that are not very relevant to the present application are omitted.

[0037] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0038] It should also be noted that, unless otherwise specified, the term “connection” herein may refer not only to a direct connection but also to an indirect connection involving an intermediate.

[0039] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0040] In order to solve the problem that the existing methods for changing the eye features of characters in videos cannot reduce the amount of collected data and lower the equipment requirements while ensuring the measurement accuracy of the optical transmission matrix, the embodiments of the present application respectively provide an optical transmission matrix measurement model training method, an optical transmission matrix measurement model training device for executing the optical transmission matrix measurement model training method, an optical transmission matrix measurement method, an optical transmission matrix measurement device for executing the optical transmission matrix measurement method, a physical device, a computer-readable storage medium and a computer program product, which can reduce the amount of data required for training the optical transmission matrix measurement model and reduce the complexity requirements of the required equipment while ensuring the measurement accuracy of the optical transmission matrix.

[0041] The details are described in detail through the following examples.

[0042] Based on this, the embodiment of the present application provides an optical transmission matrix measurement model training method that can be implemented by an optical transmission matrix measurement model training device, see Figure 1 , the optical transmission matrix measurement model training method specifically includes the following contents: Step 100: Using the scattered light spot amplitudes corresponding to each image after being transmitted through the optical fiber imaging system and the plane amplitudes and plane phases corresponding to each of the images, based on a preset iterative algorithm for phase recovery, the connected first physical information neural network and the measurement amplitude iterative calculation module are trained to train the first physical information neural network and the measurement amplitude calculation iterative module into an amplitude measurement model for outputting the target measurement amplitude of the optical transmission matrix corresponding to the scattered light spot amplitude.

[0043] In one or more embodiments of the present application, the optical transmission matrix may also be referred to as an optical interconnection transmission matrix; the plane amplitude refers to the amplitude of the image in the transmission plane of the fiber optic imaging system (that is, the light amplitude), and the amplitude distribution is Gaussian light; the plane phase refers to the phase of the image in the transmission plane of the fiber optic imaging system, that is, the image loaded on the spatial light modulator in the fiber optic imaging system; the scattered light spot amplitude refers to the amplitude of the scattered light spot collected by the CCD camera after the image on the spatial light modulator loaded on the fiber optic imaging system is transmitted to the CCD camera in the fiber optic imaging system via optical fiber, etc.; the scattered light spot phase mentioned in the following content refers to the phase corresponding to the scattered light spot amplitude output by the first physical information neural network based on the scattered light spot amplitude input therein.

[0044] It can be understood that the target measurement amplitude of the optical transmission matrix refers to the measurement result data output by the amplitude measurement model; the target measurement phase of the optical transmission matrix mentioned below refers to the measurement result data output by the phase measurement model mentioned below.

[0045] Among them, the images described in one or more embodiments of the present application all refer to image data based on Gaussian distribution as the measurement data basis. The image data can specifically adopt a preset noise map. These images are used to train the optical transmission matrix measurement model and can be generated by the system host computer.

[0046] In step 100, the measured amplitude iteration calculation module refers to a functional module that executes an iterative algorithm for phase recovery. It is understood that the iterative algorithm for phase recovery used in one or more embodiments of the present application can specifically adopt the Gerchberg-Saxton (GS) algorithm, also known as the GS phase recovery algorithm. The GS phase recovery algorithm is an iterative algorithm for optical phase recovery. The GS phase recovery algorithm establishes a connection between the amplitude information of the input light field and the intensity information of the output light field by alternating inverse Fourier transform and forward Fourier transform to achieve the recovery of the unknown phase. The basic idea of ​​the GS phase recovery algorithm is to estimate the phase using the known amplitude information during the wavefront propagation process, and then convert the estimated phase into the frequency domain by Fourier transform, and then use the inverse Fourier transform to restore the phase to the spatial domain, and iterate continuously until convergence. In each iteration, a new wavefront can be obtained by multiplying the complex amplitude of the target with the derived phase, and then a new spectrum is obtained by Fourier transform, and then the new spectrum is converted back to the spatial domain by inverse Fourier transform, so as to continuously correct and optimize the estimated value of the phase.

[0047] It should be noted that the fiber optic imaging system mentioned in one or more embodiments of the present application may adopt various fiber optic imaging systems equipped with a spatial light modulator and a CCD camera, for example, see Figure 2 The fiber optic imaging system may include a half-wave lens, a polarizer, a lens system for beam expansion, a beam splitter, a spatial light modulator, an aperture, a fiber coupler and an optical fiber, a beam expander, a polarizer, and a CCD camera in sequence according to the optical path of the laser source. Among them, the CCD camera is any digital camera with a charge coupled device image sensor. CCD is the abbreviation of charge coupled device.

[0048] Step 200: Based on the target measurement amplitude corresponding to the scattered light spot amplitude of each of the images output by the amplitude measurement model, train a second physical information neural network to train the second physical information neural network into a phase measurement model for outputting a target measurement phase of the optical transmission matrix according to the measurement amplitude; wherein the amplitude measurement model and the phase measurement model constitute an optical transmission matrix measurement model.

[0049] In one or more embodiments of the present application, the first physical information neural network and the second physical information neural network may adopt physical information neural networks with the same or different model architectures. A physical information neural network is a neural network based on physical information and may also be referred to as a physical neural network.

[0050] It can be understood that in order to obtain high-precision optical transmission matrix measurement values ​​with less collected data, in one or more embodiments of the present application, the architecture of the amplitude measurement model corresponding to the first physical information neural network is optimized by setting a measurement amplitude iterative calculation module.

[0051] Among them, the first physical information neural network and the second physical information neural network can both adopt the U-Net network in the semantic segmentation model, and can also adopt the Attention-UNet network, etc.

[0052] In the example where the second physical information neural network adopts a U-Net network, the target measurement amplitude corresponding to the scattered light spot amplitude of each of the images output by the amplitude measurement model is used as the input of the U-Net network, and after a series of processes such as convolution, downsampling, standardization, activation function, jump connection and transposed convolution, upsampling and channel fusion in the U-Net network, the output is used as the measurement phase of the optical transmission matrix; after the measurement phase is combined with the target measurement amplitude, it is multiplied by the complex matrix of the sending plane (that is, the complex matrix composed of the plane phase and the plane amplitude) and the absolute value is taken to obtain the intermediate term, and then the intermediate term and the scattered light spot amplitude recorded by the CCD camera are used to calculate the loss function, and the network parameters of the second physical information neural network are updated by back propagation; when the loss function is stable or the maximum number of iterations is reached, it stops, the training of the second physical information neural network is completed, and the measurement phase currently output by the second physical information neural network is used as the target measurement phase, and combined with the target measurement amplitude to obtain the optical transmission matrix measured by the final algorithm.

[0053] From the above description, it can be seen that the optical transmission matrix measurement model training method provided in the embodiment of the present application can reduce the amount of data required for training the optical transmission matrix measurement model, can effectively reduce the complexity requirements of the required equipment, and thus can effectively improve the reliability and effectiveness of the optical transmission matrix measurement model training process. The trained measurement model can realize holographic measurement of the optical transmission matrix containing amplitude and phase, and can ensure the accuracy of optical transmission matrix measurement using the trained measurement model, and thus can use the optical transmission matrix to realize the focusing of the speckle amplitude of the receiving plane and the recovery and extraction of speckle information and other applications.

[0054] In order to further improve the effectiveness and reliability of the amplitude measurement model training, further reduce the amount of data required for training the optical transmission matrix measurement model and reduce the complexity of the required equipment, in an optical transmission matrix measurement model training method provided in an embodiment of the present application, see Figure 3 and Figure 4 , step 100 in the optical transmission matrix measurement model training method specifically includes the following contents: Step 110: In the current iteration round, the scattered light spot amplitudes corresponding to each image after being transmitted through the optical fiber imaging system are sequentially input into the first physical information neural network, so that the first physical information neural network outputs the scattered light spot phases corresponding to each of the scattered light spot amplitudes corresponding to each of the images.

[0055] Step 120: In the measurement amplitude iterative calculation module, based on the preset iterative algorithm for phase recovery, based on the scattered light spot amplitude, the scattered light spot phase, the pre-acquired plane amplitude and the plane phase corresponding to each of the images, the measurement amplitude of the optical transmission matrix corresponding to the current iteration round is obtained and the network parameters of the first physical information neural network are updated accordingly.

[0056] Step 130: Determine whether the first physical information neural network and the measurement amplitude iterative calculation module of the current iteration round have converged; if so, use the measurement amplitude obtained in the current iteration round as the target measurement amplitude, and use the first physical information neural network and the measurement amplitude iterative calculation module corresponding to the current iteration round as an amplitude measurement model for outputting the target measurement amplitude of the optical transmission matrix corresponding to the scattered light spot amplitude; if not, perform training for the next iteration round on the first physical information neural network and the measurement amplitude iterative calculation module.

[0057] In order to improve the effectiveness and reliability of the convergence judgment of the amplitude measurement model, in an optical transmission matrix measurement model training method provided in an embodiment of the present application, see Figure 5 , step 130 in the optical transmission matrix measurement model training method specifically includes the following contents: Step 131: Determine whether the correlation coefficient between the measured amplitude of the optical transmission matrix corresponding to the current iteration round and the measured amplitude of the optical transmission matrix corresponding to the historical iteration round that is at least one iteration round apart from the current iteration round is higher than a preset threshold value; if so, determine that the first physical information neural network and the measured amplitude iterative calculation module of the current iteration round have converged and execute step 123; if not, determine that the first physical information neural network and the measured amplitude iterative calculation module of the current iteration round have not converged and execute step 133.

[0058] Step 132: Use the measured amplitude obtained in the current iteration round as the target measured amplitude, and use the first physical information neural network and the measured amplitude iterative calculation module corresponding to the current iteration round as an amplitude measurement model for outputting the target measured amplitude of the optical transmission matrix corresponding to the scattered light spot amplitude.

[0059] Step 133: training the first physical information neural network and the measurement amplitude iterative calculation module for the next iteration round.

[0060] It can be understood that the historical iteration round that is separated from the current iteration round by at least one iteration round refers to: if the current iteration round is the nth round, then the historical iteration round that is separated from the current iteration round by one iteration round refers to the n-2th round, and n-2 can also be set to n-3 and so on according to the actual application scenario.

[0061] Taking the current iteration round as the nth round and the historical iteration round as the n-2th round as an example, if the correlation coefficient (correlation) between the optical transmission matrix obtained in the nth iteration and the optical transmission matrix obtained in the n-2th iteration is higher than the preset threshold 0.99999999, the algorithm is judged to have reached a convergence state. The correlation coefficient is obtained by using the normalized correlation coefficient of the complex matrix, and the specific steps are as follows: The similarity of two matrices is measured by complex conjugate inner product and normalization. The result of the nth iteration is conjugated and then the inner product is calculated with the result of the n-2th iteration, and the norms of the two matrices are used for normalization to measure the similarity between the two.

[0062] It should be noted that if the current iteration round is the first or second round, there does not exist a historical iteration round with at least one iteration round between the current iteration round. Therefore, for the case where the current iteration round is the first or second round, step 131 and step 132 are not executed, and step 133 can be directly executed, that is, the first physical information neural network and the measurement amplitude iterative calculation module are directly trained for the next iteration round.

[0063] In order to help the iterative process escape from the local optimum and converge to the global optimum, in an optical transmission matrix measurement model training method provided in an embodiment of the present application, the following contents may be specifically included between step 120 and step 130 in the optical transmission matrix measurement model training method: The learning rate corresponding to the first physical information neural network is optimized based on the cosine annealing method.

[0064] Specifically, in the iterative process with network parameter updates, the cosine annealing method is used to modify the learning rate of the model optimizer, so that the learning rate of this iterative process periodically rises sharply and gradually decreases, which helps the iterative process to escape from the local optimum and converge to the global optimum.

[0065] In order to achieve an initial setting of the transmission matrix close to the global optimum and accelerate the convergence in the subsequent iteration process, in an optical transmission matrix measurement model training method provided in an embodiment of the present application, the optical transmission matrix measurement model training method may further specifically include the following contents before step 110: The preset optical transmission matrix initialization data is multiplied by the complex matrix composed of the plane amplitude and the plane phase corresponding to the pre-acquired image, and the error between the amplitude in the product result and the scattered light spot amplitude corresponding to the image is recorded as a loss function for iterative training of the first physical information neural network based on the stochastic gradient descent method.

[0066] In order to help the iterative process escape from the local optimum and converge to the global optimum, in an optical transmission matrix measurement model training method provided in an embodiment of the present application, the following contents may be specifically included between step 120 and step 130 in the optical transmission matrix measurement model training method: Gaussian blur processing is performed on the phase of the scattered light spot output by the first physical information neural network.

[0067] Specifically, in the early stage of iteration, after updating the optical transmission matrix, the phase of the transmission matrix is ​​Gaussian blurred, so that the updated transmission matrix items are explored in a larger range, reducing the possibility of falling into the local optimum.

[0068] In order to further improve the effectiveness and reliability of the optical transmission matrix measurement model training process, in an optical transmission matrix measurement model training method provided in an embodiment of the present application, see Figure 5 , the optical transmission matrix measurement model training method further specifically includes the following contents before step 100: Step 010: Load each image based on Gaussian distribution into the spatial light modulator in the fiber imaging system as the plane phase corresponding to each image, and the plane amplitude corresponding to each image is Gaussian light.

[0069] Step 020: Obtain the scattered light spot amplitude corresponding to each of the images received by the CCD camera in the optical fiber imaging system.

[0070] Step 030: performing image cropping and normalization processing on the plane phase, the plane amplitude and the scattered light spot amplitude corresponding to each of the images.

[0071] Based on the optical transmission matrix measurement model training method provided in the above embodiment, the present application also provides an embodiment of an optical transmission matrix measurement method, see Figure 6 , the optical transmission matrix measurement method specifically includes the following contents: Step 300: Acquire the plane amplitude and plane phase of the target image and the scattered light spot amplitude obtained after the target image is transmitted through the optical fiber imaging system; Step 400: Input the plane amplitude, plane phase and scattered light spot amplitude of the target image into the optical transfer matrix measurement model pre-trained by the optical transfer matrix measurement model training method, so that the amplitude measurement model in the optical transfer matrix measurement model outputs the target measurement amplitude of the optical transfer matrix corresponding to the target image according to the plane amplitude, plane phase and scattered light spot amplitude of the target image, and the phase measurement model in the optical transfer matrix measurement model outputs the target measurement phase of the optical transfer matrix corresponding to the target image according to the target measurement amplitude corresponding to the target image.

[0072] Step 500: Output an optical transmission matrix including the target measurement amplitude and the target measurement phase corresponding to the target image.

[0073] After the measurement result data of the optical transmission matrix is ​​outputted in step 500, the focusing and scattered light spot information of any pixel point on the receiving plane can be restored and extracted according to the optical transmission matrix. The high-quality and high-speed transmission process of the optical interconnection transmission system is realized. In addition, in astronomical observations, due to the influence of atmospheric turbulence, the obtained optical signal is usually phase-distorted. The algorithm can help restore the wavefront information, thereby improving the observation quality; in adaptive optics, by acquiring the wavefront information in real time, the algorithm can help adjust the mirror shape in real time, correct the atmospheric influence, and optimize the optical imaging.

[0074] The optical transmission matrix measurement model training method mentioned in step 400 of the optical transmission matrix measurement method provided in the present application can be specifically used to execute the processing flow of the embodiment of the optical transmission matrix measurement model training method mentioned in the above embodiment. Its function will not be repeated here, and reference can be made to the detailed description of the above optical transmission matrix measurement model training method embodiment.

[0075] From the above description, it can be seen that the optical transmission matrix measurement method provided in the embodiment of the present application can ensure the accuracy of optical transmission matrix measurement using the trained measurement model, and then the optical transmission matrix can be used to achieve applications such as focusing of the speckle amplitude on the receiving plane and recovery and extraction of speckle information.

[0076] In order to further illustrate the embodiments of the above optical transmission matrix measurement model training method and the optical transmission matrix measurement method, the present application also provides a specific application example of the optical transmission matrix measurement method based on physical information neural network applied to the optical image signal system, see Figure 7, build the optical path of the fiber optic imaging system, display the measurement signal base to be transmitted on the spatial light modulator, and use a CCD camera at the receiving end to collect the speckle intensity output by the optical fiber. The two are used to form a data set for measuring the optical transmission matrix; an optical transmission matrix iteration algorithm assisted by a physical information neural network based on the Gerchberg-Saxton (GS) algorithm is used to perform iterative calculations with network parameter updates until the results converge; the amplitude part of the iterative result of the optical transmission matrix is ​​extracted as the input of the improved U-Net structure neural network based on physical information, and the matrix multiplication result of the recorded scattered light spot amplitude and the iteratively obtained transmission matrix and the amplitude-phase complex matrix at the input end is used as the loss function of the training network, and the output of the improved U-Net based on physical information is used as the phase part of the optical transmission matrix. An iterative training process with network parameter updates is performed, and the iteration is terminated when the value of the loss function is stable. The optical transmission matrix amplitude obtained by the first iteration and the phase of the optical transmission matrix obtained by the second iteration are combined as the final optical transmission matrix measurement value and stored; the measured optical transmission matrix is ​​used to achieve the focusing effect of the scattered light spot amplitude on the receiving plane and the recovery and extraction of speckle information.

[0077] That is to say, the purpose of the application example of the present application is to provide an optical transmission matrix measurement method based on a physical information neural network, to measure the optical interconnection transmission matrix based on a physical information neural network, and to effectively achieve high-precision measurement of the optical transmission matrix with a low amount of collected data, and to achieve focusing at any position on the receiving plane. Compared with the traditional optical transmission matrix measurement method, the application example of the present application can reduce the demand for data collection and reduce the complexity of the collection system.

[0078] Based on this, an optical transmission matrix measurement method based on a physical information neural network disclosed in the application example of this application includes the following steps: Step 1: Build the Read Figure 2 The fiber optic imaging system shown in the figure includes a half-wave lens, a polarizer, a lens system for beam expansion, a beam splitter, a spatial light modulator, an aperture, a fiber coupler and an optical fiber, a beam expander, a polarizer and a CCD camera in sequence according to the optical path of the 1550nm laser source. The host computer of the system generates 96 8×8 noise images based on Gaussian distribution as the measurement data base; a single data base is loaded on the spatial light modulator as the phase of the transmission plane, and the light amplitude distribution of the transmission plane is Gaussian light; the CCD camera records the scattered light spot amplitude after transmission through the system on the receiving plane, and forms a measurement data set of the optical transmission matrix corresponding to the transmitting end information, and then performs image cropping processing to a 64×64 image and normalization processing. After transmission-recording-processing, multiple data bases form a data set used in the subsequent iterative training process.

[0079] Among them, an end-to-end optical transmission matrix of arbitrary resolution can be obtained by using multiple image cropping and processing sizes.

[0080] Step 2: Construct an optical transmission matrix iterative algorithm assisted by a physical information neural network based on the Gerchberg-Saxton (GS) algorithm, and perform iterative calculations with network parameter updates.

[0081] The method for constructing an optical transmission matrix iterative algorithm assisted by a physical information neural network based on the Gerchberg-Saxton (GS) algorithm is as follows: See also Figure 8 In order to obtain high-precision optical transmission matrix measurements with less data collection, the application example of this application relies on the Attention-UNet neural network based on physical information to generate its phase from the scattered light spot amplitude collected by the CCD camera, and combines it with the recorded light spot amplitude to obtain the receiving plane item in the iteration; the sending plane item and the receiving plane item are both complex matrices of amplitude and phase at this time, and the iterative value of the optical transmission matrix can be calculated; repeat this process until the correlation coefficient (correlation) between the optical transmission matrix obtained by the nth iteration and the optical transmission matrix obtained by the n-2th iteration is higher than the preset threshold 0.99999999, then the algorithm is judged to have reached the convergence state. Output the optical transmission matrix obtained by the nth iteration.

[0082] exist Figure 8 In the figure, L represents the number of images used as the measurement basis, which can be 96 in this application example; I represents the intensity of the transmission plane; represents the phase of the transmission plane; P represents the complex term of the transmission plane; The complex form representing the transmit plane phase; Indicates the amplitude of the sending plane; represents the optical transmission matrix; Indicates the phase of scattered light; Indicates the scattered light spot amplitude; Indicates the measured amplitude; Represents the intermediate items output by the iterative algorithm; Represents the intermediate terms of the input iterative algorithm.

[0083] in addition, Figure 8 The neural network in can be used as follows Fig. 9 The network architecture of the Attention-UNet neural network shown.

[0084] Among them, in this iterative process with network parameter update, the cosine annealing method is used to modify the learning rate of the model optimizer, so that the learning rate of this iterative process periodically rises sharply and gradually decreases, which helps the iterative process to jump out of the local optimum and converge to the global optimum.

[0085] Among them, during the initialization process of this iterative algorithm, the amplitude of the result of multiplying the initially initialized transmission matrix with the known transmission plane amplitude-phase complex matrix is ​​compared with the amplitude recorded by the CCD camera for error analysis and recorded as the loss function; the random gradient descent method is used to iterate the transmission matrix to achieve an initial setting of the transmission matrix that is close to the global optimal, which accelerates the convergence in subsequent iterative processes.

[0086] Among them, in the early stage of iteration, after updating the optical transmission matrix, the phase of the transmission matrix is ​​Gaussian blurred, so that the updated transmission matrix items are explored in a larger range, reducing the possibility of falling into the local optimum.

[0087] Step 3: According to research and verification, the optical transmission matrix obtained by the iterative algorithm with network parameter update in step 2 has high accuracy in amplitude, but there is an unstable deviation in phase. Step 3 is based on the amplitude of the transmission matrix obtained in step 2, and uses the U-Net network structure and the iterative process with physical information to achieve the convergence of the phase part of the transmission matrix. The iterative process of step 3 does not require additional data sets, and the data set used in step 2 is used as the training pair for supervised learning.

[0088] The specific process of step three is as follows: Take the amplitude of the transmission matrix as Fig.10 The input of the U-Net neural network shown in the figure undergoes a series of processes such as convolution, downsampling, standardization, activation function, skip connection and transposed convolution, upsampling and channel fusion, and the output is used as the phase of the transmission matrix; the phase and amplitude are combined and multiplied by the complex matrix of the transmission plane and the absolute value is taken to obtain the intermediate term, and the intermediate term is used to calculate the loss function with the amplitude of the scattered light spot recorded by the CCD camera, and the network parameters are updated by back propagation; when the loss function is stable or the maximum number of iterations is reached, it stops, and the output phase is combined with the amplitude of the transmission matrix obtained in step 2 to obtain the optical transmission matrix measured by the final algorithm.

[0089] Step 4: The final measured optical transmission matrix is ​​obtained by combining the amplitude of the optical transmission matrix obtained by training in step 2 and the phase value of the optical transmission matrix obtained by training in step 3. This matrix is ​​used to restore and extract the focusing and scattered light spot information of any pixel point on the receiving plane. This realizes the high-quality and high-speed transmission process of the optical interconnection transmission system. The effect of the measured optical transmission matrix used for focusing is shown in the figure below. Figure 11(a) to Figure 11(c) shown.

[0090] Take the value of any column of the transfer matrix, reshape it to the input size 8×8, and then take the matrix conjugate and use it as Figure 2The input of the optical fiber imaging system shown realizes the recovery and extraction of the focusing and scattering spot information of any pixel point, and realizes the high-quality and high-speed transmission process of the optical interconnection transmission system.

[0091] In summary, the application example of this application has the following beneficial effects: 1. The present application discloses a method for measuring an optical transmission matrix based on a physical information neural network. The optical path of the optical fiber imaging system is constructed, the measurement signal base to be transmitted is displayed on the spatial light modulator, and the speckle intensity output by the optical fiber is collected by a CCD camera at the receiving end. The two are used to form a data set for measuring the optical transmission matrix; an optical transmission matrix iteration algorithm assisted by a physical information neural network based on the Gerchberg-Saxton (GS) algorithm is used to perform iterative calculations with network parameter updates until the results converge; the amplitude part of the iterative result of the optical transmission matrix is ​​extracted as the input of the improved U-Net structure neural network based on physical information, and the matrix multiplication result of the recorded scattered light spot amplitude and the iteratively obtained transmission matrix and the amplitude-phase complex matrix at the input end is used as the loss function of the training network, and the output of the improved U-Net based on physical information is used as the phase part of the optical transmission matrix, and an iterative training process with network parameter updates is performed, and the iteration ends when the value of the loss function is stable. This method effectively reduces the amount of data collected for accurate measurement of the optical transmission matrix and reduces the complexity of the measurement system.

[0092] 2. In the optical transmission matrix measurement method based on physical information neural network disclosed in the application example of this application, in the iterative process with network parameter update in step 2, the learning rate of the model optimizer is modified by the cosine annealing method, so that the learning rate of this iterative process rises sharply and gradually decreases periodically, which helps the iterative process to jump out of the local optimum and converge to the global optimum. Accelerated convergence and improved accuracy of the training process are achieved.

[0093] 3. The application example of this application discloses an optical transmission matrix measurement method based on a physical information neural network. In the initialization process of the iterative algorithm in step 2, the amplitude of the result of multiplying the initially initialized transmission matrix with the known transmission plane amplitude-phase complex matrix is ​​compared with the amplitude recorded by the CCD camera for error analysis, and the result is recorded as a loss function. The transmission matrix is ​​iterated using the random gradient descent method to achieve an initial setting of the transmission matrix close to the global optimum, which accelerates the convergence in the subsequent iteration process. Accelerated convergence and improved accuracy of the training process are achieved.

[0094] 4. The application example of the present application discloses an optical transmission matrix measurement method based on a physical information neural network. In the initial iteration, after the optical transmission matrix is ​​updated, the phase of the transmission matrix is ​​Gaussian blurred, so that the updated transmission matrix items are explored in a larger range, reducing the possibility of falling into the local optimum.

[0095] 5. The application example of this application discloses an optical transmission matrix measurement method based on a physical information neural network. The final measured optical transmission matrix is ​​obtained by combining the amplitude of the optical transmission matrix obtained by training in step 2 and the phase value of the optical transmission matrix obtained by training in step 3. This matrix is ​​used to restore and extract the focusing and scattered light spot information of any pixel point on the receiving plane. The high-quality and high-speed transmission process of the optical interconnection transmission system is realized.

[0096] From the software level, the present application also provides an optical transmission matrix measurement model training device for executing all or part of the optical transmission matrix measurement model training method, see Fig.12 , the optical transmission matrix measurement model training device specifically includes the following contents: The amplitude measurement model training module 10 is used to use the scattered light spot amplitudes corresponding to each image after being transmitted by the optical fiber imaging system and the plane amplitudes and plane phases corresponding to each of the images, and based on a preset iterative algorithm for phase recovery, train the connected first physical information neural network and the measurement amplitude iterative calculation module to train the first physical information neural network and the measurement amplitude iterative calculation module into an amplitude measurement model for outputting the target measurement amplitude of the optical transmission matrix corresponding to the scattered light spot amplitude; The phase measurement model training module 20 is used to train a second physical information neural network based on the target measurement amplitude corresponding to the scattered light spot amplitude of each of the images output by the amplitude measurement model, so as to train the second physical information neural network into a phase measurement model for outputting the target measurement phase of the optical transmission matrix according to the measurement amplitude; wherein the amplitude measurement model and the phase measurement model constitute an optical transmission matrix measurement model.

[0097] The embodiment of the optical transmission matrix measurement model training device provided in the present application can be specifically used to execute the processing flow of the embodiment of the optical transmission matrix measurement model training method in the above-mentioned embodiment. Its functions will not be repeated here, and reference can be made to the detailed description of the above-mentioned optical transmission matrix measurement model training method embodiment.

[0098] The part of the optical transmission matrix measurement model training device that performs optical transmission matrix measurement model training can be executed in the server or completed in the client device. The specific selection can be based on the processing capability of the client device and the limitations of the user's usage scenario. This application does not limit this. If all operations are completed in the client device, the client device may also include a processor for specific processing of the optical transmission matrix measurement model training.

[0099] The client device may have a communication module (i.e., a communication unit) that can communicate with a remote server to achieve data transmission with the server. The server may include a server on the task scheduling center side, and other implementation scenarios may also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, or a server cluster consisting of multiple servers, or a server structure of a distributed device.

[0100] The server and the client device may communicate with each other using any suitable network protocol, including network protocols that have not yet been developed on the date of filing this application. The network protocols may include, for example, TCP / IP, UDP / IP, HTTP, HTTPS, etc. Of course, the network protocols may also include, for example, RPC (Remote Procedure Call Protocol) and REST (Representational State Transfer) protocols used on top of the above protocols.

[0101] From the above description, it can be seen that the optical transmission matrix measurement model training device provided in the embodiment of the present application can reduce the amount of data required for training the optical transmission matrix measurement model, can effectively reduce the complexity requirements of the required equipment, and thus can effectively improve the reliability and effectiveness of the optical transmission matrix measurement model training process. The trained measurement model can realize holographic measurement of the optical transmission matrix containing amplitude and phase, and can ensure the accuracy of optical transmission matrix measurement using the trained measurement model, and thus can use the optical transmission matrix to realize the focusing of the speckle amplitude of the receiving plane and the recovery and extraction of speckle information and other applications.

[0102] From the software level, the present application also provides an optical transmission matrix measurement device for executing all or part of the optical transmission matrix measurement method, see Fig.13 The optical transmission matrix measurement device specifically includes the following contents: a data acquisition module 30, which is used to obtain the plane amplitude and plane phase of the target image and the scattered light spot amplitude obtained after the target image is transmitted through the optical fiber imaging system.

[0103] The model measurement module 40 is used to input the plane amplitude, plane phase and scattered light spot amplitude of the target image into the optical transmission matrix measurement model pre-trained by the optical transmission matrix measurement model training method, so that the amplitude measurement model in the optical transmission matrix measurement model outputs the target measurement amplitude of the optical transmission matrix corresponding to the target image according to the plane amplitude, plane phase and scattered light spot amplitude of the target image, and the phase measurement model in the optical transmission matrix measurement model outputs the target measurement phase of the optical transmission matrix corresponding to the target image according to the target measurement amplitude corresponding to the target image.

[0104] The matrix output module 50 is used to output an optical transmission matrix including the target measurement amplitude and the target measurement phase corresponding to the target image.

[0105] The optical transmission matrix measurement model training method mentioned in the model measurement module 40 in the optical transmission matrix measurement device provided in the present application can be specifically used to execute the processing flow of the embodiment of the optical transmission matrix measurement model training method mentioned in the above-mentioned embodiment. Its function will not be repeated here, and reference can be made to the detailed description of the above-mentioned optical transmission matrix measurement model training method embodiment.

[0106] The part of the optical transmission matrix measurement model training device that performs optical transmission matrix measurement can be executed in the client device or in the server. The specific selection can be based on the processing capability of the client device and the limitations of the user's usage scenario. This application does not limit this. If all operations are completed in the client device, the client device may also include a processor for specific processing of the optical transmission matrix measurement model training.

[0107] From the above description, it can be seen that the optical transmission matrix measurement device provided in the embodiment of the present application can ensure the accuracy of optical transmission matrix measurement using the trained measurement model, and then the optical transmission matrix can be used to achieve applications such as focusing of the speckle amplitude on the receiving plane and recovery and extraction of speckle information.

[0108] The embodiment of the present application further provides an electronic device, which may include a processor, a memory, a receiver and a transmitter, wherein the processor is used to execute the optical transmission matrix measurement model training method and / or the optical transmission matrix measurement method mentioned in the above embodiment, wherein the processor and the memory may be connected via a bus or other means, such as by bus connection. The receiver may be connected to the processor and the memory via a wired or wireless manner.

[0109] The processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0110] As a non-transient computer-readable storage medium, the memory can be used to store non-transient software programs, non-transient computer executable programs and modules, such as the program instructions / modules corresponding to the optical transmission matrix measurement model training method and / or the optical transmission matrix measurement method in the embodiments of the present application. The processor executes various functional applications and data processing of the processor by running the non-transient software programs, instructions and modules stored in the memory, that is, implementing the optical transmission matrix measurement model training method and / or the optical transmission matrix measurement method in the above method embodiments.

[0111] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0112] The one or more modules are stored in the memory, and when executed by the processor, perform the optical transmission matrix measurement model training method and / or the optical transmission matrix measurement method in the embodiment.

[0113] In some embodiments of the present application, the user equipment may include a processor, a memory, and a transceiver unit, which may include a receiver and a transmitter. The processor, memory, receiver, and transmitter may be connected through a bus system. The memory is used to store computer instructions, and the processor is used to execute the computer instructions stored in the memory to control the transceiver unit to send and receive signals.

[0114] As an implementation method, the functions of the receiver and the transmitter in the present application can be considered to be implemented through a transceiver circuit or a dedicated chip for transceiver, and the processor can be considered to be implemented through a dedicated processing chip, a processing circuit or a general chip.

[0115] As another implementation method, it is possible to use a general-purpose computer to implement the server provided in the embodiment of the present application, that is, to store the program code for implementing the functions of the processor, receiver, and transmitter in a memory, and the general-purpose processor implements the functions of the processor, receiver, and transmitter by executing the code in the memory.

[0116] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the aforementioned optical transmission matrix measurement model training method and / or optical transmission matrix measurement method are implemented. The computer-readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0117] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the aforementioned optical transmission matrix measurement model training method and / or optical transmission matrix measurement method.

[0118] It should be understood by those skilled in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is specifically performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0119] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0120] In the present application, features described and / or illustrated for one embodiment may be used in the same manner or in a similar manner in one or more other embodiments, and / or combined with features of other embodiments or replace features of other embodiments.

[0121] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the embodiments of the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for training an optical transmission matrix measurement model, characterized in that: include: Using the scattered light spot amplitudes corresponding to each image after being transmitted by the optical fiber imaging system and the plane amplitudes and plane phases corresponding to each of the images, based on a preset iterative algorithm for phase recovery, the connected first physical information neural network and the measurement amplitude iterative calculation module are trained to train the first physical information neural network and the measurement amplitude calculation iterative module into an amplitude measurement model for outputting the target measurement amplitude of the optical transmission matrix corresponding to the scattered light spot amplitude; Based on the target measurement amplitude corresponding to the scattered light spot amplitude of each of the images output by the amplitude measurement model, a second physical information neural network is trained to train the second physical information neural network into a phase measurement model for outputting the target measurement phase of the optical transmission matrix according to the measurement amplitude; wherein the amplitude measurement model and the phase measurement model constitute an optical transmission matrix measurement model.

2. The optical transmission matrix measurement model training method according to claim 1, characterized in that: The method adopts the scattered light spot amplitudes corresponding to each image after being transmitted by the optical fiber imaging system and the plane amplitudes and plane phases corresponding to each image, and trains the connected first physical information neural network and the measurement amplitude iterative calculation module based on a preset iterative algorithm for phase recovery, so as to train the first physical information neural network and the measurement amplitude calculation iterative module into an amplitude measurement model for outputting the target measurement amplitude of the optical transmission matrix corresponding to the scattered light spot amplitude, including: In the current iteration round, the scattered light spot amplitudes corresponding to the images after being transmitted by the optical fiber imaging system are sequentially input into the first physical information neural network, so that the first physical information neural network outputs the scattered light spot phases corresponding to the scattered light spot amplitudes corresponding to the images respectively; In the measurement amplitude iteration calculation module, based on a preset iterative algorithm for phase recovery, based on the scattered light spot amplitude, the scattered light spot phase, the pre-acquired plane amplitude and the plane phase corresponding to each of the images, the measurement amplitude of the optical transmission matrix corresponding to the current iteration round is obtained and the network parameters of the first physical information neural network are updated accordingly; Determine whether the first physical information neural network and the measurement amplitude iterative calculation module of the current iteration round have converged; if so, use the measurement amplitude obtained in the current iteration round as the target measurement amplitude, and use the first physical information neural network and the measurement amplitude iterative calculation module corresponding to the current iteration round as an amplitude measurement model for outputting the target measurement amplitude of the optical transmission matrix corresponding to the scattered light spot amplitude; if not, perform training for the next iteration round on the first physical information neural network and the measurement amplitude iterative calculation module.

3. The optical transmission matrix measurement model training method according to claim 2, characterized in that: The determining whether the first physical information neural network and the measurement amplitude iterative calculation module of the current iteration round have converged includes: Determine whether the correlation coefficient between the measured amplitude of the optical transmission matrix corresponding to the current iteration round and the measured amplitude of the optical transmission matrix corresponding to the historical iteration round that is at least one iteration round apart from the current iteration round is higher than a preset threshold value; if so, determine that the first physical information neural network and the measured amplitude iterative calculation module of the current iteration round have converged; if not, determine that the first physical information neural network and the measured amplitude iterative calculation module of the current iteration round have not converged.

4. The optical transmission matrix measurement model training method according to claim 2, characterized in that: Before judging whether the first physical information neural network and the measured amplitude iterative calculation module of the current iteration round have converged, it also includes: optimizing the learning rate corresponding to the first physical information neural network based on the cosine annealing method.

5. The optical transmission matrix measurement model training method according to claim 2, characterized in that: Before sequentially inputting the scattered light spot amplitudes corresponding to the images after being transmitted by the optical fiber imaging system into the first physical information neural network in the current iteration round, the method further includes: The preset optical transmission matrix initialization data is multiplied by the complex matrix composed of the plane amplitude and the plane phase corresponding to the pre-acquired image, and the error between the amplitude in the product result and the scattered light spot amplitude corresponding to the image is recorded as a loss function for iterative training of the first physical information neural network based on the stochastic gradient descent method.

6. The optical transmission matrix measurement model training method according to claim 2, characterized in that: Before judging whether the first physical information neural network and the measurement amplitude iterative calculation module of the current iteration round have converged, it also includes: performing Gaussian blur processing on the scattered light spot phase output by the first physical information neural network.

7. The optical transmission matrix measurement model training method according to any one of claims 1 to 6, characterized in that: The method further comprises: using the scattered light spot amplitudes corresponding to the images after being transmitted by the optical fiber imaging system and the plane amplitudes and plane phases corresponding to the images, and training the connected first physical information neural network and the measurement amplitude iterative calculation module based on a preset iterative algorithm for phase recovery; Loading each image based on Gaussian distribution to a spatial light modulator in a fiber imaging system as a plane phase corresponding to each image, and a plane amplitude corresponding to each image is Gaussian light; Acquire the scattered light spot amplitude corresponding to each of the images received by the CCD camera in the optical fiber imaging system; The plane phase, the plane amplitude and the scattered light spot amplitude corresponding to each of the images are respectively subjected to image cropping and normalization processing.

8. An optical transmission matrix measurement method, characterized in that: include: Acquire the plane amplitude and plane phase of the target image and the scattered light spot amplitude obtained after the target image is transmitted through the optical fiber imaging system; Input the plane amplitude, plane phase and scattered light spot amplitude of the target image into the optical transmission matrix measurement model pre-trained by the optical transmission matrix measurement model training method according to any one of claims 1 to 7, so that the amplitude measurement model in the optical transmission matrix measurement model outputs the target measurement amplitude of the optical transmission matrix corresponding to the target image according to the plane amplitude, plane phase and scattered light spot amplitude of the target image, and the phase measurement model in the optical transmission matrix measurement model outputs the target measurement phase of the optical transmission matrix corresponding to the target image according to the target measurement amplitude corresponding to the target image; The output includes an optical transmission matrix including the target measurement amplitude and the target measurement phase corresponding to the target image.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the optical transmission matrix measurement model training method as described in any one of claims 1 to 7 is implemented, and / or the optical transmission matrix measurement method as described in claim 8 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the optical transmission matrix measurement model training method as described in any one of claims 1 to 7, and / or implements the optical transmission matrix measurement method as described in claim 8.

Images