A multimode optical fiber data transmission method and system based on long-short time model integration

Abstract

The present invention relates to a multimode optical fiber data transmission method and system based on long-short time model integration, the method comprising: S1, continuously loading different pictures into a laser input multimode optical fiber in time sequence through a spatial light modulator, and obtaining a corresponding speckle image at the output end; S2, constructing a long-short time training model with multiple neural networks embedded therein, and training the multiple neural networks separately; wherein the multiple embedded neural network models are divided into a short-time neural network for reconstruction according to different time intervals, and a long-time neural network for fine-tuning; S3, weighting the confidence of the predicted value output by the neural network in the long-short time training model to obtain a corresponding pseudo-label image, and forming a new training group with the pseudo-label image and the corresponding speckle image to fine-tune all neural networks in S2; S4, using the long-short time training model after regular training to perform multimode optical fiber data transmission. Compared with the prior art, the present invention can perform long-term continuous and accurate information transmission.

Description

Technical Field

[0001] The present invention relates to the field of optical communication, and in particular to a multimode optical fiber data transmission method and system based on long-short time model integration. Background Art

[0002] With the development of the big data era, the requirements for network transmission speed are getting higher and higher. The optical fiber communication system based on single-mode optical fiber constitutes the backbone of the modern Internet. The spatial freedom of multimode optical fiber allows it to transmit more complex spatial information. Through high-dimensional coding and compatibility with traditional multiplexing methods such as mode division multiplexing and wavelength division multiplexing, it is expected to become an important resource for improving communication speed. The transmission speed of multimode optical fiber will be an order of magnitude higher than that of traditional single-mode optical fiber, and it is an important direction for the development of ultra-high-speed communication systems in the future.

[0003] However, due to the inevitable mode mixing and mode dispersion, the input image of the multimode fiber will be converted into a complex speckle pattern at the output. At present, the deep learning transmission method has been developed on a large scale and applied to the information transmission of multimode fiber. However, due to the time-varying nature of multimode fiber, these transmission models will soon become invalid. How to continuously transmit information for a long time is still a difficult problem in the application of multimode fiber in the field of communication. Summary of the invention

[0004] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a multimode optical fiber data transmission method and system based on long- and short-time model integration, which can perform long-term continuous and accurate information transmission.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] According to a first aspect of the present invention, a multimode optical fiber data transmission method based on long-short time model integration is provided, the method comprising the following steps:

[0007] Step S1, continuously loading different images into the laser input multimode optical fiber in time sequence through a spatial light modulator, and obtaining corresponding speckle images at the output end of the multimode optical fiber;

[0008] Step S2, constructing a long-short-time training model with multiple neural networks embedded therein, taking the output speckle image as input and the corresponding image as output, and training the multiple neural networks separately; wherein the embedded multiple neural network models are divided into short-time neural networks for reconstruction at different time intervals, and long-time neural networks for fine-tuning;

[0009] Step S3, weighting the predicted values ​​output by the neural network in the long-short time training model according to the confidence level to obtain the corresponding pseudo-label images, and forming a new training group with the pseudo-label images and the corresponding speckle images to perform fine-tuning training on all the neural networks in step S2;

[0010] Step S4: Use the long-short time training model after regular training to perform multimode optical fiber data transmission.

[0011] Preferably, the number of the short-term neural networks is at least 2.

[0012] Preferably, the embedded neural network is a convolutional neural network or a fully connected neural network.

[0013] Preferably, the loss function of the long-short time training model is a mean square error (MSE) image metric, a structural similarity (SSIM) image metric, or a weighted sum of the two.

[0014] Preferably, the output speckle patterns and the corresponding true image values ​​for an initial period of time are used to perform initial training on multiple embedded neural networks, and subsequent transmission will only provide real-time speckle images.

[0015] Preferably, for the long-term neural network, the pseudo-label images of the latest period of time are taken as true value images and combined with the corresponding speckles to form the latest data set, which is provided to the embedded neural network for fine-tuning training.

[0016] Some things, the prediction value output by the neural network in the long-short time training model is weighted by confidence to obtain the corresponding pseudo-label image, specifically: the prediction value output by the neural network in the long-short time training model is soft-voted for all embedded neural networks according to the confidence weight and the corresponding pseudo-label image is obtained.

[0017] According to a second aspect of the present invention, a multimode optical fiber communication system is provided, the system comprising a multimode optical fiber data transmission module integrated with any one of the multimode optical fiber data transmission methods based on long-short time model integration.

[0018] According to a second aspect of the present invention, there is provided an electronic device, comprising a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the program, any one of the methods described above is implemented.

[0019] According to a third aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, any one of the methods described above is implemented.

[0020] Compared with the prior art, the present invention has the following advantages:

[0021] The present invention adopts a long-short-time training model with multiple neural networks embedded in it. The short-time neural network is reconstructed and trained at fixed intervals, while the long-time neural network only participates in fine-tuning training. This can effectively cope with the time-varying nature of multimode optical fiber, thereby realizing long-term continuous and accurate information transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a flow chart of the method of the present invention;

[0023] Figure 2 An optical system for transmitting images using multimode optical fiber.

[0024] Figure 3 This is the initial working mode of the long-short time training model with multiple neural networks embedded.

[0025] Figure 4 This is a self-supervised flowchart. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0027] Example

[0028] like Figure 1 As shown, this embodiment provides a multimode optical fiber data transmission method based on long-short time model integration, and the method includes the following steps:

[0029] Step S1, continuously loading different images into laser in time sequence through a spatial light modulator, inputting into a multimode optical fiber, and obtaining a corresponding speckle image at the output end of the multimode optical fiber;

[0030] Step S2, constructing a long-short-time training model with multiple neural networks embedded therein, taking the output speckle image as input and the corresponding image as output, and training the multiple neural networks separately; wherein the embedded multiple neural network models are divided into short-time neural networks for reconstruction at different time intervals, and long-time neural networks for fine-tuning;

[0031] Step S3, soft voting is performed on all embedded neural networks according to the confidence weights of the predicted values ​​output by the neural network in the long-short time training model to obtain corresponding pseudo-label images, and the pseudo-label images and the corresponding speckle images are combined into a new training group to perform fine-tuning training on all neural networks in step S2;

[0032] Step S4: Use the long-short time training model after regular training to perform multimode optical fiber data transmission.

[0033] The number of short-term neural networks is at least 2. In this embodiment, the embedded neural network is a convolutional neural network or a fully connected neural network.

[0034] In addition, the transmission objects of the present invention include but are not limited to pictures, coding information and two-dimensional space information.

[0035] The loss function of the long-short time training model is a mean square error (MSE) image metric, a structural similarity (SSIM) image metric, or a weighted sum of the two, and is selected and trained in any suitable manner known to those skilled in the art.

[0036] The output speckle patterns and the corresponding true image values ​​for an initial period of time are used to perform initial training on multiple embedded neural networks, and subsequent transmission will only provide real-time speckle images.

[0037] For long-term neural networks, the latest pseudo-label images are taken as true-value images and combined with the corresponding speckles to form the latest data set, which is provided to the neural network embedded in this model for fine-tuning training.

[0038] Next, the method of this embodiment is described in detail with reference to numerical examples.

[0039] For a long-short-term training model consisting of two short-term neural networks a and b and one long-term neural network c, the specific training process is as follows:

[0040] 1) First, 4000 images with local truth values ​​are transmitted, and the corresponding 4000 speckle images are obtained at the output end of the multimode optical fiber.

[0041] 2) The speckle images are used as the input of the neural network and the corresponding 4000 images are used as the output of the neural network to train the three neural networks.

[0042] 3) After the initial training is completed, the normal transmission mode is entered, and the latest 500 real-time speckles are input into the model. The three networks input these speckles respectively and output the predicted values. The three groups of predicted values ​​will be soft-voted according to the confidence and 500 pseudo-label images will be output. These 500 pseudo-label images and the corresponding speckles will form the latest training group to fine-tune the three neural networks. After the 5000th picture is transmitted, the neural network a is recreated and trained with the past 4000 speckles and their corresponding pseudo-labels. After the 10000th picture is transmitted, the neural network b is recreated and trained with the past 4000 speckles and their corresponding pseudo-labels. After the 15000th picture is transmitted, the neural network a is reconstructed again, and after the 20000th picture, the neural network b is reconstructed again. The two short-term models are reconstructed and trained at fixed intervals, while the long-term neural network c only participates in fine-tuning training.

[0043] This embodiment also provides a multimode optical fiber communication system, which includes a multimode optical fiber data transmission module integrated with any of the multimode optical fiber data transmission methods based on long-short time model integration, specifically as follows Figure 2 As shown:

[0044] The system includes an input image 100 and transmits it to a multimode fiber transmission optical path 200, and a speckle image 300 obtained at the multimode fiber output end. The images to be transmitted are loaded into the input end 100 in time sequence and the corresponding output end speckle image 300 is captured.

[0045] Figure 3 It is simply shown that a certain amount of images 100 and corresponding output speckle images 300 are provided in the initial stage, and three separate neural networks M1, M2, and M3 are trained as output and input respectively. Modules M1, M2, and M3 together constitute module 20 by a confidence weighting method.

[0046] Figure 4 The flowchart of the multimode optical fiber data transmission method using the long-short time integration model is briefly shown. In module 22, the real-time speckle image 300 obtained in the continuous transmission is provided to the Figure 2 After the initialization training, the module 20 will restore the input speckle image 100 according to the provided speckle image 300. The speckle image 300 and the corresponding restored speckle image 100 together constitute the training data 21, which will only retain the latest transmission image for a period of time and provide it to the embedded model in the module 20 for fine-tuning training. The speckle image 300_T passed into the module 22 refers to the speckle image 300 continuously transmitted in real time by the multimode optical fiber system 200 within a period of time, that is, the module 20 will continuously obtain the latest speckle image 300 and use it to update and fine-tune itself.

[0047] After fine-tuning training for a specific length of time in module 24, module 20 will enter module 30. In module 30, the specific network M1 in module 20 will be initialized and trained using the latest data set 21, while other networks in other modules 20 such as M2 and M3 will urgently need to use only the latest data set 21 for fine-tuning training. After module 30, module 20 will enter module 24 again for fine-tuning training for a specific length of time. The next time it enters module 30, network M2 will be initialized and trained, while other networks will only undergo fine-tuning training. Next time, M1 will continue to be initialized, so that networks M1 and M2 will only use the latest data for training, and network M3 will not be initialized, so that network M3 has experienced all the past data. This method allows our network modules to always have different perspectives for long and short periods of time, greatly improving the prediction accuracy of the integrated module 20.

[0048] The electronic device of the present invention includes a central processing unit (CPU), which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) or loaded from a storage unit into a random access memory (RAM). In the RAM, various programs and data required for device operation can also be stored. The CPU, ROM and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0049] Multiple components in the device are connected to the I / O interface, including: input units, such as keyboards, mice, etc.; output units, such as various types of displays, speakers, etc.; storage units, such as disks, optical disks, etc.; and communication units, such as network cards, modems, wireless communication transceivers, etc. The communication unit allows the device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunication networks.

[0050] The processing unit performs the various methods and processes described above, such as methods S1 to S4. For example, in some embodiments, methods S1 to S4 may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on the device via a ROM and / or a communication unit. When the computer program is loaded into the RAM and executed by the CPU, one or more steps of methods S1 to S4 described above may be performed. Alternatively, in other embodiments, the CPU may be configured to execute methods S1 to S4 in any other appropriate manner (e.g., by means of firmware).

[0051] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip systems (SOCs), load programmable logic devices (CPLDs), and the like.

[0052] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0053] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0054] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A multimode optical fiber data transmission method based on long-short time model integration, characterized in that: The method comprises the following steps: Step S1, continuously loading different images into the laser input multimode optical fiber in time sequence through a spatial light modulator, and obtaining corresponding speckle images at the output end of the multimode optical fiber; Step S2, constructing a long-short-time training model with multiple neural networks embedded therein, taking the output speckle image as input and the corresponding image as output, and training the multiple neural networks separately; wherein the embedded multiple neural network models are divided into short-time neural networks for reconstruction at different time intervals, and long-time neural networks for fine-tuning; Step S3, weighting the predicted values ​​output by the neural network in the long-short time training model according to the confidence level to obtain the corresponding pseudo-label images, and forming a new training group with the pseudo-label images and the corresponding speckle images to perform fine-tuning training on all the neural networks in step S2; Step S4: Use the long-short time training model after regular training to perform multimode optical fiber data transmission.

2. The multimode optical fiber data transmission method based on long-short time model integration according to claim 1, characterized in that: The number of the short-term neural networks is at least 2.

3. The multimode optical fiber data transmission method based on long-short time model integration according to claim 1, characterized in that: The embedded neural network is a convolutional neural network or a fully connected neural network.

4. The multimode optical fiber data transmission method based on long-short time model integration according to claim 1, characterized in that: The loss function of the long-short time training model is a mean square error (MSE) image metric, a structural similarity (SSIM) image metric, or a weighted sum of the two.

5. The multimode optical fiber data transmission method based on long-short time model integration according to claim 1, characterized in that: The output speckle patterns and the corresponding true image values ​​for an initial period of time are used to perform initial training on multiple embedded neural networks, and subsequent transmission will only provide real-time speckle images.

6. The multimode optical fiber data transmission method based on long-short time model integration according to claim 1, characterized in that: For long-term neural networks, the pseudo-label images of the latest period of time are taken as true-value images and combined with the corresponding speckles to form the latest data set, which is provided to the embedded neural network for fine-tuning training.

7. The multimode optical fiber data transmission method based on long-short time model integration according to claim 1, characterized in that: The predicted value output by the neural network in the long-short time training model is weighted by confidence to obtain the corresponding pseudo-label image, specifically: the predicted value output by the neural network in the long-short time training model is soft-voted for all embedded neural networks according to the confidence weight and the corresponding pseudo-label image is obtained.

8. A multimode optical fiber communication system, characterized in that: The system comprises a multimode optical fiber data transmission module integrated with the multimode optical fiber data transmission method based on long-short time model integration according to any one of claims 1 to 7.

9. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images