Real-time video stream optimized transmission method and system based on optical communication

Through network state prediction and photon encoding technology, the problems of high latency and poor reliability of optical communication technology in large-scale network environments are solved, efficient and stable video streaming is achieved, and user experience is improved.

CN118540552BActive Publication Date: 2024-11-26SUZHOU AIXIONGSI COMM TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411004916.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-25
Publication Date
2024-11-26
Estimated Expiration
2044-07-25

AI Technical Summary

Technical Problem

The existing optical communication technology has high transmission delay, poor transmission reliability and weak anti-network fluctuation capabilities in large-scale network environments, which affects the real-time and quality of video streams.

Method used

By obtaining historical network status data and video stream data, using network status prediction model to predict future network status, when congestion is detected, photon encoding equipment is used to encode the video stream, and divide it into multiple modules for parallel transmission, using adjustable The laser and modulator adaptively adjust the output power and modulation depth to select the optimal transmission path.

Benefits of technology

It significantly reduces transmission delay, improves transmission efficiency and quality, enhances the system's ability to adapt to network fluctuations, and avoids video lag and quality degradation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118540552B_ABST
    Figure CN118540552B_ABST
Patent Text Reader

Abstract

The present application provides a method and system for optimizing transmission of real-time video streams based on optical communication, which relates to the field of video optical communication transmission. The method includes: obtaining first network status data and first transmission video stream data corresponding to a past historical time period; obtaining second transmission video stream data corresponding to a target video stream to be transmitted in a preset time window in the future; predicting the second network status data corresponding to the preset time window based on a network status prediction model; when it is detected that the second network status data meets the preset transmission congestion state condition, using a photon encoding device to encode the photon encoding video stream corresponding to the target video stream; dividing the photon encoding video stream to obtain multiple photon encoding modules, and using multiple target transmission paths to transmit each photon encoding module in parallel. Thus, the optimized transmission of real-time video streams is achieved in a complex network environment, and the system's adaptability to network fluctuations is enhanced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of video optical communication transmission technology, and in particular to a real-time video stream optimized transmission method and system based on optical communication. Background Art

[0002] With the rapid development of the Internet and the increasing demand for high-definition video, real-time video streaming transmission has become a crucial technical field. Optical communication technology has gradually been applied to the transmission of video streaming due to its ultra-high bandwidth, low latency and anti-electromagnetic interference characteristics.

[0003] Optical communication technology itself has the characteristics of low latency, but in large-scale network environments, the problem of transmission delay still exists. Network congestion and inefficient routing on the transmission path may cause data packets to experience a long waiting time during transmission. In addition, when the network load is too high, packet loss and retransmission will also increase the overall transmission delay. This is obviously not conducive to video streaming that requires real-time response.

[0004] In addition, traditional optical communication technology is weak in dealing with network fluctuations. When network conditions change, the transmission quality of video streams will be significantly affected, and problems such as video freezes, resolution drops, and image blur may occur. This is more obvious when transmitting high-quality videos, seriously affecting the user's viewing experience.

[0005] In response to the above problems, the industry has not yet proposed a better technical solution. Summary of the invention

[0006] The present application provides a real-time video stream optimized transmission method, system, storage medium, computer program product and electronic device based on optical communication, which are used to at least solve the problems of high transmission delay, poor transmission reliability and weak resistance to network fluctuations of current traditional optical communication technology in large-scale network environments.

[0007] In a first aspect, the present application provides a real-time video stream optimized transmission method based on optical communication, comprising: obtaining first network status data and first transmission video stream data corresponding to a past historical time period; the data type of the first network status data includes network bandwidth, network delay and packet loss rate, and the data type of the first transmission video stream data includes video encoding bit rate, video resolution and video frame rate; obtaining second transmission video stream data corresponding to a target video stream to be transmitted in a preset time window in the future; inputting the first network status data, the first transmission video stream data and the second transmission video stream data into a network status prediction model to predict the second network status data corresponding to the preset time window; when it is detected that the second network status data meets a preset transmission congestion state condition, using a photon encoding device to encode the optical signal corresponding to the target video stream. sub-encoded video stream; dividing the photon-encoded video stream to obtain corresponding M photon encoding modules, selecting M target transmission paths from the available transmission path set, and transmitting the M photon encoding modules in parallel based on the M target transmission paths; the photon encoding device comprises a light source module, a modulator module and a photon encoding processing module; the light source module adopts a laser with adjustable output power to provide a photon source; the modulator module comprises an electro-optic modulator and an acousto-optic modulator to modulate the video signal into an optical signal; the photon encoding processing module comprises a beam splitter, a phase modulator and an interferometer to encode the optical signal; the output power of the light source module and the modulation depth of the modulator module are adaptively adjusted according to the network state prediction index; the network state prediction index is determined according to the predicted second network state data;

[0008] The network status estimation index is calculated by the following formula:

[0009] ,

[0010] In the formula, represents the predicted network status evaluation index, , and Respectively represent the predicted network bandwidth, network delay and packet loss rate, , and They represent the adjustment coefficients of network bandwidth, network delay and packet loss rate respectively; Indicates the preset upper limit of delay;

[0011] The output power of the light source module is adaptively adjusted by the following formula:

[0012] ,

[0013] ,

[0014] In the formula, Indicates the output power of the light source module. represents the real-time injected current, Indicates the injection current calibration value, , They represent the characteristic coefficients of the laser respectively;

[0015] The modulation depth of the modulator module is adaptively adjusted by the following formula:

[0016] ,

[0017] ,

[0018] In the formula, Indicates the modulation depth of the modulator module, Indicates the real-time driving voltage, Indicates the drive voltage calibration value, Indicates half-wave voltage.

[0019] In a second aspect, the present application provides a real-time video stream optimization transmission system based on optical communication, comprising: a historical data acquisition unit, used to acquire first network status data and first transmission video stream data corresponding to a past historical time period; the data type of the network status data includes network bandwidth, network delay and packet loss rate, and the data type of the transmission video stream data includes video encoding bit rate, video resolution and video frame rate; a data acquisition unit to be transmitted, used to acquire second transmission video stream data corresponding to a target video stream to be transmitted in a preset time window in the future; a network status prediction unit, used to input the first network status data, the first transmission video stream data and the second transmission video stream data into a network status prediction model to predict the second network status data corresponding to the preset time window; a photon video encoding unit, used to encode the photon-encoded video stream corresponding to the target video stream using a photon encoding device when it is detected that the second network status data meets the preset transmission congestion status condition; a photon video transmission unit, used to segment the photon-encoded video stream to obtain corresponding M Photon encoding modules are selected from the set of available transmission paths M target transmission path, based on the M The target transmission paths transmit the M Photon encoding module;

[0020] The photon encoding device comprises a light source module, a modulator module and a photon encoding processing module; the light source module adopts a laser with adjustable output power to provide a photon source; the modulator module comprises an electro-optic modulator and an acousto-optic modulator to modulate a video signal into an optical signal; the photon encoding processing module comprises a beam splitter, a phase modulator and an interferometer to encode the optical signal; the output power of the light source module and the modulation depth of the modulator module are adaptively adjusted according to a network state prediction index; the network state prediction index is determined according to the predicted second network state data;

[0021] The network status estimation index is calculated by the following formula:

[0022] ,

[0023] In the formula, represents the predicted network status evaluation index, , and Respectively represent the predicted network bandwidth, network delay and packet loss rate, , and They represent the adjustment coefficients of network bandwidth, network delay and packet loss rate respectively; Indicates the preset upper limit of delay;

[0024] The output power of the light source module is adaptively adjusted by the following formula:

[0025] ,

[0026] ,

[0027] In the formula, Indicates the output power of the light source module. represents the real-time injected current, Indicates the injection current calibration value, , They represent the characteristic coefficients of the laser respectively;

[0028] The modulation depth of the modulator module is adaptively adjusted by the following formula:

[0029] ,

[0030] ,

[0031] In the formula, Indicates the modulation depth of the modulator module, Indicates the real-time driving voltage, Indicates the drive voltage calibration value, Indicates half-wave voltage.

[0032] According to a third aspect, an electronic device is provided, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the steps of the real-time video stream optimized transmission method based on optical communication of any embodiment of the present application.

[0033] In a fourth aspect, the present application provides a storage medium having a computer program stored thereon, characterized in that when the program is executed by a processor, the steps of the real-time video stream optimization transmission method based on optical communication of any embodiment of the present application are implemented.

[0034] In a fifth aspect, the present application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the real-time video stream optimization transmission method based on optical communication of any embodiment of the present application.

[0035] The real-time video stream optimization transmission method based on optical communication provided by this application can produce at least the following technical effects:

[0036] (1) By collecting network status data and transmission video stream data in historical time periods, and obtaining transmission video stream data to be transmitted in the future, and inputting these data into the network status prediction model, the future network status can be accurately predicted, so that the system can identify possible network congestion in advance and take preventive measures. When transmission congestion is detected, the target video stream is encoded using a photon coding device and divided into multiple photon coding modules, which are transmitted in parallel through multiple transmission paths, greatly reducing the load of a single transmission path, significantly reducing transmission delay, and reducing the risk of data packet loss due to single path failure or congestion.

[0037] (2) When a potential network congestion risk is predicted, the system automatically switches to photon coding processing, which can take advantage of the low latency and high reliability of photon coding to ensure high-quality video transmission in network congestion. In addition, since photon coding equipment and processing are relatively complex and consume a lot of resources, the system only switches from ordinary coding processing to photon coding processing when network congestion conditions are detected, which can significantly reduce the overall resource occupancy and energy consumption of the system and extend the life of the equipment.

[0038] (3) By predicting the network status during the future transmission window, the system can adjust the output power of the light source module and the modulation depth of the modulation module in the photonic coding device in real time, and optimize the video coding strategy according to the potential future network status, so that when the network conditions change, it can still maintain a high transmission quality and transmission efficiency, reducing video transmission interruption or quality degradation caused by network fluctuations.

[0039] (4) By dividing the photon-encoded video stream into multiple modules and transmitting them in parallel based on multiple target transmission paths, the network load is effectively dispersed, the pressure on a single transmission path is reduced, and the probability of packet loss and retransmission is reduced. Combined with the efficient encoding processing of the photon encoding device, high-quality transmission of the video stream under different network conditions is ensured, avoiding problems such as video freeze, resolution degradation and image blur, and significantly improving the user's viewing experience.

[0040] Through this technical solution, a network status prediction model, an adaptive adjustment mechanism for photonic coding devices, and a multi-path parallel transmission strategy are comprehensively introduced to achieve optimized transmission of real-time video streams in complex network environments. This not only significantly reduces transmission delays, improves transmission efficiency and quality, but also enhances the system's ability to adapt to network fluctuations. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0042] Figure 1 A flowchart showing an example of a real-time video stream optimized transmission method based on optical communication according to an embodiment of the present application is shown;

[0043] Figure 2 A schematic diagram showing a structural connection of an example of a photon encoding device according to an embodiment of the present application is shown;

[0044] Figure 3 An operation flow chart of an example of content-aware segmentation of a photon-encoded video stream according to an embodiment of the present application is shown;

[0045] Figure 4 A schematic diagram showing a structural connection of an example of a multimodal spatiotemporal network according to an embodiment of the present application is shown;

[0046] Figure 5 An operation flow chart of an example of an online learning optimization network status prediction model according to an embodiment of the present application is shown;

[0047] Figure 6 A structural block diagram of an example of a real-time video stream optimization transmission system based on optical communication according to an embodiment of the present application is shown;

[0048] Figure 7 It is a schematic structural diagram of an embodiment of an electronic device of the present application. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0050] In the technical solution of this application, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved shall comply with the provisions of relevant laws and regulations and shall not violate public order and good morals.

[0051] Figure 1 A flowchart of an example of a real-time video stream optimized transmission method based on optical communication according to an embodiment of the present application is shown.

[0052] Regarding the executor of the method of the embodiment of the present application, it can be any controller or processor with computing or processing capabilities. By comprehensively introducing a network status prediction model, an adaptive adjustment mechanism of a photonic coding device, and a multi-path parallel transmission strategy, optimized transmission of real-time video streams is achieved in a complex network environment, which not only significantly reduces transmission delays, improves transmission efficiency and quality, but also enhances the system's adaptability to network fluctuations.

[0053] In some examples, it can be a video transmission optimization platform, and can be integrated and configured in an electronic device or terminal through software, hardware, or a combination of software and hardware, and the type of terminal or electronic device can be diverse, such as an optical communication transmitter, a mobile phone, a tablet computer or a desktop computer, etc.

[0054] like Figure 1 As shown, in step S110, first network status data and first transmission video stream data corresponding to a past historical time period are obtained.

[0055] Here, the data type of the first network status data includes network bandwidth, network delay and packet loss rate, and the data type of the first transmission video stream data includes video encoding bit rate, video resolution and video frame rate.

[0056] In some implementations, network status data such as network bandwidth, network delay, and packet loss rate in a historical time period are obtained through a network monitoring tool (such as SNMP protocol or traffic analyzer). In addition, video transmission monitoring tools (such as FFmpeg or other video monitoring tools) are used to record and obtain transmission video stream data in a historical time period, including video encoding bit rate, video resolution, video frame rate, etc. Thus, distributed data collection and storage technology is used to store these data in a database, ensuring that a large amount of historical data can be efficiently processed and stored in a large-scale network environment, providing a data basis for subsequent network status prediction.

[0057] In step S120, second transmission video stream data corresponding to a target video stream to be transmitted in a future preset time window is obtained.

[0058] In some embodiments, the target video stream to be transmitted in the future is determined by analyzing the video playback plan or by interacting with the video server. Further, the target video stream to be played is preloaded and the relevant transmission video stream data is extracted, for example, by preprocessing with a video content analysis tool. In addition, by preloading and obtaining data in advance, transmission planning can be done in advance to avoid the decline in network transmission quality caused by unknown parameters, and it can also ensure that the system is fully prepared before the video stream transmission starts, reducing the transmission start delay.

[0059] In step S130, the first network status data, the first transmission video stream data and the second transmission video stream data are input into a network status prediction model to predict the second network status data corresponding to a preset time window.

[0060] Here, the network status prediction model can adopt various deep learning models without restriction, such as LSTM (Long Short-Term Memory) or CNN (Convolutional Neural Network), which should not be restricted for the time being. Preferably, the network status prediction model can adopt MMSTN (Multi-Modal Spatio-Temporal Network), and more details will be expanded in conjunction with other examples below. Therefore, by predicting the network transmission quality within the future transmission window period, the system can adjust the video encoding transmission strategy in advance to ensure the smoothness of real-time video transmission (for example, live video transmission).

[0061] In step S140, when it is detected that the second network status data meets the preset transmission congestion status condition, a photon encoding device is used to encode a photon encoding video stream corresponding to the target video stream.

[0062] On the other hand, when it is detected that the second network status data does not meet the transmission congestion state condition, the control system maintains the normal video encoding mode, such as H.264 / AVC. Therefore, only when a potential future network transmission congestion condition is detected, the system uses the photon encoding device to perform photon encoding processing on the video.

[0063] It should be noted that common coding methods usually consume less computing resources and are suitable for efficient transmission under good network conditions. In addition, the low latency and high reliability characteristics of photon coding can ensure high-quality video transmission under network congestion. In this way, by combining common coding with photon coding, through network status detection and intelligent switching, while meeting the resource demand balance conditions, it also enhances the adaptability to various network environments and improves the robustness and stability of the system.

[0064] It should be understood that the encoding technology of the photon encoding device can use a variety of encoding methods, such as quantum dot encoding or photon-based efficient encoding methods, so as to ensure that high video quality can be maintained under bandwidth constraints. Preferably, an adaptive adjustment mechanism based on predicted network status can also be integrated in the photon encoding device, and more details will be expanded in conjunction with other examples below.

[0065] In step S150, the photon-encoded video stream is segmented to obtain corresponding M Photon encoding modules are selected from the set of available transmission paths M target transmission path, based on M target transmission paths for parallel transmission M Photon encoding module.

[0066] here, M is a natural number greater than 1. In one example of the embodiment of the present application, according to the system preset number M (i.e., fixed segmentation mechanism) divides the photon-encoded video stream into multiple photon-encoded modules, and then transmits them accordingly according to the multi-path transmission path with better real-time transmission status, ensuring that each module is transmitted through an independent path to improve the overall transmission speed and reliability. In another example of the embodiment of the present application, the system can use content-aware segmentation technology to perform self-use segmentation by analyzing the content of the photon-encoded video stream. M The value of is floating, and then the transmission is carried out according to the multi-path transmission path with better current real-time or predicted transmission status. More details will be developed in conjunction with other examples below. Therefore, through multi-path parallel transmission, the load of each transmission path is balanced, the congestion risk of a single path is reduced, the real-time and smoothness of the video stream is ensured, and the fault tolerance of the system is increased. For example, when a problem occurs on a path, other paths can continue to transmit, ensuring the continuity of the video stream.

[0067] Through the embodiments of the present application, by intelligently predicting the network status and dynamically selecting the encoding method, for example, triggering the use of photon encoding equipment when network congestion is detected, the efficiency and quality of video stream transmission are guaranteed under the premise of taking into account the consumption of system resources when the network condition fluctuates. Through video module segmentation and multi-path parallel transmission, transmission delay and packet loss are reduced, and the stability of the system and its adaptability to network fluctuations are improved.

[0068] Figure 2 A schematic diagram of the structural connection of an example of a photon encoding device according to an embodiment of the present application is shown.

[0069] like Figure 2 As shown, the photon encoding device 200 includes a light source module 210, a modulator module 220 and a photon encoding processing module 230. The light source module 210 uses a laser with adjustable output power to provide a photon source, which generally needs to provide a high-intensity, coherent photon source.

[0070] The modulator module 220 includes an electro-optic modulator (EOM) 221 and an acousto-optic modulator (AOM) 222, which are used to modulate the video signal into an optical signal. It should be understood that the electro-optic modulator uses the electro-optic effect (also known as the Pockels effect) to change the refractive index of the material by applying an electric field to the electro-optic material (such as lithium niobate LiNbO3 or barium titanate BaTiO3), thereby modulating the optical signal passing through the material. The acousto-optic modulator uses the acousto-optic effect to introduce sound waves into the acousto-optic material (such as quartz or potassium niobium rubidium oxide RKNbO3), causing periodic changes in the refractive index of the material, thereby modulating the optical signal passing through the material. Specifically, the video signal is input into the electro-optic modulator in the form of an electrical signal, and the modulation of the optical signal is achieved by the change of the driving voltage; in addition, the video signal is input into the acousto-optic modulator through the radio frequency driver, and the radio frequency signal drives the piezoelectric transducer to generate sound waves, thereby modulating the optical signal.

[0071] The photon coding processing module 230 includes a beam splitter 231, a phase modulator 232 and an interferometer 233, which are used to encode the optical signal. In some embodiments, the phase modulator 232 is used to modulate the phase of the optical signal to encode the video data. Through phase modulation, efficient photon coding can be achieved and transmission delay can be reduced. The optical signal is divided into multiple sub-signals by the optical beam splitter 231 for parallel processing and encoding. The interferometer 233 is used to perform interference processing on the phase-modulated optical signal to improve the stability and anti-interference ability of the signal, so that the encoded optical signal can be transmitted to the receiving end through the optical fiber, and can ensure extremely low attenuation and high stability during the transmission process to ensure that the signal can be fully transmitted.

[0072] The output power of the light source module 210 and the modulation depth of the modulator module 220 are adaptively adjusted according to the network status prediction index. According to the network status in the next transmission window predicted by machine learning, the modulation parameters of the photon encoder are adaptively adjusted to optimize the signal quality and transmission efficiency by dynamically adjusting the laser output power and modulation depth.

[0073] Here, the network state estimation index is determined according to the predicted second network state data. More specifically, the network state estimation index is calculated by the following formula:

[0074] , formula (1),

[0075] In the formula, represents the predicted network status evaluation index, , and Respectively represent the predicted network bandwidth, network delay and packet loss rate, , and They represent the adjustment coefficients of network bandwidth, network delay and packet loss rate respectively; Indicates the preset upper limit of delay.

[0076] Here, in order to more flexibly respond to changes in network conditions, a weight function is introduced To comprehensively evaluate the changes in bandwidth, delay, and packet loss rate in the transmission network.

[0077] Accordingly, the output power of the light source module 210 is adaptively adjusted by the following formula:

[0078] , formula (2),

[0079] , formula (3),

[0080] In the formula, Indicates the output power of the light source module. represents the real-time injected current, Indicates the injection current calibration value, , They represent the characteristic coefficients of the laser respectively.

[0081] Specifically, the adaptive mechanism of output power regulation is implemented based on the predicted network status parameters. When the network status changes, the output power is adjusted. To optimize the transmission effect. reduce, Increase or When increasing, Correspondingly reduce, at this time by reducing to reduce interference; on the contrary, increase To enhance signal strength.

[0082] Therefore, the network status evaluation indicators are estimated based on the network status (bandwidth, delay, packet loss rate, etc.) predicted in the future transmission window, and the output power of the laser is dynamically adjusted to ensure that the power is increased to enhance the signal strength when the bandwidth is sufficient. In addition, the power is reduced to reduce interference when the network is congested, reducing the network burden and signal interference, thereby optimizing transmission efficiency.

[0083] Accordingly, the modulation depth of the modulator module 220 is adaptively adjusted by the following formula:

[0084] , formula (4),

[0085] , formula (5),

[0086] In the formula, Indicates the modulation depth of the modulator module, Indicates the real-time driving voltage, Indicates the drive voltage calibration value, Represents half-wave voltage, that is, the modulator phase change reaches The required voltage.

[0087] Specifically, when a change in network status is predicted, the network status is adjusted based on the real-time monitoring data. To optimize the modulation depth Dynamically adjust the drive voltage based on real-time monitoring of network status and video stream characteristics , optimize the modulation depth of the electro-optic modulator For example, when the network condition is good, increase Increase modulation depth , enhance the signal transmission quality; conversely, reduce To reduce interference.

[0088] It should be noted that the modulation depth It is a unitless quantity used to describe the intensity of the modulation of the optical signal by the electro-optic modulator (such as lithium niobate modulator). It indicates the degree to which the phase, frequency or amplitude of the optical signal changes during the modulation process. The greater the modulation depth, the more significant the change of the optical signal, the higher the signal quality, and the stronger the anti-interference ability. When the bandwidth is sufficient and the delay and packet loss rate are low, the modulation depth can be increased to improve the signal quality. In addition, when the network is congested or the transmission conditions are poor, reducing the modulation depth can reduce signal interference and distortion, ensuring stable data transmission.

[0089] In this embodiment, by adopting an adaptive photon modulation strategy, the output power of the laser and the modulation depth of the electro-optic modulator can be dynamically adjusted to adapt to the real-time network conditions and video stream characteristics, thereby improving the transmission efficiency and quality of the video stream.

[0090] In some examples of the embodiments of the present application, the photon-encoded video stream is adaptively segmented by content-aware segmentation. Content-aware segmentation dynamically segments the video by analyzing the complexity and motion information of the video content. For example, high-motion scenes and key frames are segmented into smaller modules, while low-motion scenes are segmented into larger modules, thereby optimizing transmission efficiency and quality.

[0091] Figure 3 An operational flow chart showing an example of content-aware segmentation of a photon-encoded video stream according to an embodiment of the present application is shown.

[0092] like Figure 3 As shown, in step S310, for each photon coded video frame in the photon coded video stream, the number of edge pixels and the range of motion vectors in the photon coded video frame are obtained.

[0093] In some embodiments, the photon-encoded video frame is processed using an edge detection algorithm (such as a Canny edge detection algorithm, a Sobel operator, etc.) to calculate the number of edge pixels in each frame. In addition, the motion estimation algorithm in video encoding (such as a block matching algorithm, an optical flow method, etc.) is used to calculate the motion vector of each block in the photon-encoded video frame and to count the range of the motion vector.

[0094] In step S320, the target coding module size is calculated according to the number of edge pixels and the range of motion vectors.

[0095] Here, the complexity of the photon coding video frame is calculated based on the number of edge pixels and the size of the motion vector in the video frame, and the size of the photon coding module is comprehensively calculated in combination with the maximum complexity value in the video frame stream. More specifically, the target coding module size is calculated by the following formula:

[0096] , formula (6),

[0097] , formula (7),

[0098] , formula (8),

[0099] In the formula, Indicates The complexity of encoding video frames by frame photons, represents the complexity sequence of video frames, Indicates the maximum complexity value among all video frames; Indicates The number of edge pixels in the frame, Represents the total number of pixels in each video frame. Indicates The average magnitude of motion vectors of frame photon-encoded video frames; and Represent edge pixel weight and motion vector weight respectively.

[0100] In this way, by dynamically adjusting the module size according to the complexity of each video frame and the range of motion vectors, redundant data can be reduced and transmission efficiency can be improved. Smaller modules are generated for high-complexity frames, and larger modules are generated for low-complexity frames. This can reduce the delay when transmitting high-complexity frames and improve the overall transmission rate. By dynamically adjusting the size of the encoding module, the network bandwidth is ensured to be optimally utilized and the data transmission efficiency is improved.

[0101] In this embodiment, by calculating the complexity of each video frame and dynamically adjusting the size of the encoding module, frames with high complexity and high motion can be transmitted first, ensuring the priority of key content and improving the overall transmission efficiency. In the video stream, high-motion scenes and key frames are divided into smaller modules for transmission, ensuring that these important contents can be quickly transmitted to the receiving end, improving the user experience. In addition, low-motion scenes and non-key frames are divided into larger modules, which effectively reduces the bandwidth and time required for transmission and improves transmission efficiency.

[0102] In step S330, the photon coding video frame is divided into a plurality of photon coding modules according to the target coding module size.

[0103] In some embodiments, the initial segmentation position can be determined according to the target coding module size, and then the segmentation boundary can be adjusted according to the complexity of the video frame content to ensure that the complexity of each module is balanced. Finally, the boundary of each coding module is determined to generate a photon coding module. Here, during the segmentation process, the boundaries of adjacent modules are handled well to ensure seamless connection between modules and avoid image quality degradation caused by segmentation. Thus, through the adaptive video module segmentation strategy, the complexity of each coding module is ensured to be balanced and the overall coding efficiency is improved. In addition, it can ensure that the size of the segmented coding module is moderate, which is convenient for multi-path parallel transmission and reduces transmission delay and video transmission quality.

[0104] As a further preferred implementation, by preferentially transmitting smaller high-complexity frame modules, it is ensured that the key parts of the video stream can be quickly transmitted to the receiving end, thereby improving the overall clarity and smoothness of the video.

[0105] Through this embodiment, the module size and number of each video frame are dynamically adjusted, so that the network bandwidth is optimally utilized, redundant data and bandwidth waste during transmission are reduced, and efficient bandwidth utilization is achieved. In addition, smaller data blocks are more likely to be successfully transmitted during transmission, reducing delays and packet loss caused by network congestion, and improving the quality of video transmission and user experience.

[0106] In some examples of the embodiments of the present application, the optical transmitter can efficiently transmit data through multiple channels or paths, and accordingly, the predicted second network status data can include multiple path network status data, which can reflect the network status of each path. Then, the load factor of the corresponding transmission path can be calculated or predicted based on the path network status data, and multiple transmission paths with the best load factors can be selected for transmission, thereby realizing dynamic scheduling selection of the transmission path.

[0107] Figure 3 An operational flow chart showing an example of dynamically selecting a transmission path according to an embodiment of the present application is shown.

[0108] like Figure 3 As shown, in step S310, for each available transmission path in the available transmission path set, the path load factor corresponding to the available transmission path is calculated according to the path network status data corresponding to the available transmission path.

[0109] In some implementations, a nonlinear mathematical model is used to calculate the load factor to more accurately reflect the load condition of each path.

[0110] More specifically, the path loading factor is calculated using the regression analysis model:

[0111] , formula (9),

[0112] In the formula, Indicates the transmission path The load factor, , and Represent the predicted paths network bandwidth, network delay and packet loss rate, represents the model coefficients of the regression learning model, represents the natural exponential function.

[0113] Here, the logarithmic transformation of bandwidth can reduce the impact of extreme values ​​on the calculation results, while better reflecting the relative changes in bandwidth. The square term of delay is used to highlight the significant impact of larger delays on the load, ensuring that high-delay paths have a larger weight in the load factor. The square root transformation of packet loss rate is used to smooth the impact of small packet loss rate values ​​while maintaining sensitivity to larger packet loss rates. Finally, the above items are linearly combined and then exponentially calculated to ensure that the load factor is always positive and can better reflect the overall load situation.

[0114] In step S320, the path with the smallest load factor is selected from each available transmission path. M available transmission paths as the corresponding target transmission paths.

[0115] It should be noted that when segmenting videos through content awareness, the system dynamically adjusts the size of the target coding module according to the complexity and motion vector range of different video frames, which will result in the total number of photon coding modules generated for each video frame. There are differences, and the corresponding number of available transmission paths will also fluctuate.

[0116] Through this embodiment, the load factor of each transmission path is dynamically calculated using a regression analysis model, which can accurately reflect the load conditions of different paths under the network status during the transmission window period, and then select the transmission path according to the load factor to ensure that the selected transmission path has the lowest load, thereby improving the stability and quality of video stream transmission, avoiding congested paths, and improving transmission efficiency.

[0117] In some examples of the embodiments of the present application, the network status prediction model adopts a multimodal spatiotemporal network. Figure 4 A schematic diagram of the structural connection of an example of a multimodal spatiotemporal network according to an embodiment of the present application is shown.

[0118] like Figure 4 As shown, the network state prediction model 400 includes an input layer 410, a spatiotemporal convolution module 420, a Transformer module 430, a fusion layer 440 and an output layer 450.

[0119] The input layer 410 is used to receive first network status data, first transmission video stream data and second transmission video stream data.

[0120] The spatiotemporal convolution module 420 is used to extract spatiotemporal features corresponding to the first transmission video stream data and the second transmission video stream data.

[0121] It should be noted that the spatiotemporal convolution module can simultaneously capture the temporal and spatial characteristics of video data, which is particularly important for the processing of video streams. It can effectively extract the dependencies between video frames and the local features of the video content.

[0122] Here, spatiotemporal features are data features that contain both temporal and spatial information. In video stream processing, spatiotemporal features refer to the motion, changes, and spatial relationships of objects captured in video frames. Specifically, the spatial features in spatiotemporal features can reflect the image features within the video frame, such as edges, textures, and color distribution, and the temporal features in spatiotemporal features can reflect the dynamic changes between video frames, such as the object's motion trajectory, speed, and acceleration. Therefore, the spatiotemporal features in the video stream can reflect the dynamic changes of objects at different times. By capturing these changes, the future network status can be predicted more accurately.

[0123] In this embodiment, the spatiotemporal features of the first transmission video stream data and the second transmission video stream data are extracted through the spatiotemporal convolution module, which can capture the dynamic changes and spatial information in the video stream, so that the model can better learn and understand the impact of the video content on the network status.

[0124] The Transformer module 430 is used to process the spatiotemporal features and the first network state data to output corresponding global dependency features.

[0125] It should be noted that traditional recurrent neural networks (RNN) and long short-term memory networks (LSTM) have problems such as low information transmission efficiency, gradient disappearance or explosion when processing long-distance time dependencies. In this embodiment, by adopting the self-attention mechanism of the Transformer module, the dependencies between input features can be captured globally. Compared with RNN and LSTM, Transformer does not rely on sequential calculations and can process all positions in the sequence in parallel, thereby effectively solving the long-distance dependency problem. As a result, the dependencies between different positions can be flexibly calculated, generating more diverse and flexible feature representations, which can better capture the complex relationships in video streams and network status data.

[0126] In this embodiment, the Transformer module is used to process the spatiotemporal features and the first network state data, capturing long-distance temporal dependencies and global features, which helps the model understand the complex relationship between historical data and current video content, thereby improving the accuracy of prediction. In addition, the spatiotemporal convolution module and the Transformer module have strong parallel computing capabilities, can make full use of the computing resources of modern GPUs, and can complete the processing of large-scale video stream data in a shorter time, thereby improving the real-time nature of the prediction.

[0127] The fusion layer 440 is used to fuse the spatiotemporal features and the global dependency features to determine corresponding fused features.

[0128] Here, the spatiotemporal features and global dependency features are fused through the fusion layer. Based on the residual connection structure design, a more comprehensive feature representation is provided compared to the single global dependency feature. The important information in the spatiotemporal features is further utilized, avoiding the limitations of a single feature representation. This enables the model to maintain high prediction performance and robustness when facing different types of video content and network status.

[0129] The output layer 450 is used to process the fused features to predict the second network status data corresponding to a future preset time window.

[0130] In this embodiment, the model can process network status data and video stream data simultaneously, and fuse spatiotemporal features and global dependency features through a fusion layer, integrating multimodal data processing capabilities, so that the model can maintain high prediction performance and robustness when facing different types of video content and network status.

[0131] In the practice of this application, combined with the characteristics of the real-time network prediction scenario of video transmission, we also made a series of enhanced designs for the spatiotemporal convolution module and the Transformer module to improve their advantages in this scenario.

[0132] In some examples of the embodiments of the present application, the spatiotemporal convolution module adopts a multi-scale spatiotemporal convolution module to capture the features of the video stream at different time scales and spatial scales, thereby enhancing the richness of the feature representation.

[0133] Specifically, multi-scale spatiotemporal convolution is used to capture the features of video streams at different time and space scales, which can capture richer and more diverse feature information. In addition, multi-scale convolution kernels can handle different changes in time and space dimensions, thereby enhancing the ability to represent video stream features and providing more accurate and comprehensive feature representation.

[0134] The structure of the multi-scale spatiotemporal convolution module is as follows:

[0135] , formula (10),

[0136] In the formula, Representation scale The spatiotemporal convolution output of , and Represent the row index, column index and time index of the input feature map respectively; Representation scale The convolution kernel, , and Represent the row index, column index and time index of the convolution kernel respectively; R and ARespectively represent the height and width of the convolution kernel; Representation scale The temporal convolution kernel size; represents the value of the input feature map, Indicates that the input feature map is at position The value of Representation scale The bias term, Represents the activation function.

[0137] In this embodiment, convolution kernels of different sizes are used to capture dynamic changes and spatial information in the video stream. By capturing video stream features at different time and spatial scales, the richness and diversity of feature representation are enhanced, so that the model can more accurately reflect the impact of video content on the network state and also improve the model's adaptability to various complex video scenes.

[0138] In some examples of the embodiments of the present application, the Transformer module adopts a multi-head self-attention mechanism.

[0139] Specifically, the Transformer module uses a multi-head self-attention mechanism, which can focus on different parts of the input features at the same time, thereby capturing richer feature representations. By calculating multiple attention heads in parallel, the multi-head self-attention mechanism can better capture the complex dependencies between video streams and network state data. In addition, the multi-head self-attention mechanism can capture features in different subspaces, improving the model's ability to capture global and local features, allowing the model to make more accurate predictions when faced with complex network state changes.

[0140] In some examples of the embodiments of the present application, in order to ensure the prediction performance of the network state prediction model, special attention should be paid to the design details of the loss function of the network state prediction model. Here, by adopting a comprehensive loss function, the prediction error loss (Mean Squared Error, MSE), network state feature smoothing loss (Smoothness Loss) and adaptive weight loss (Adaptive Weighted Loss) are comprehensively considered. The difference between the predicted second network state data and the true value is measured through the prediction error loss term. Through the network state feature smoothing loss term, the smoothness of the network state prediction is encouraged to avoid drastic changes in the prediction results. Through the adaptive weight loss, the influence weight is adaptively adjusted according to the size of the prediction error, and the sensitivity of the model to the prediction error at different time points is enhanced.

[0141] More specifically, the loss function of the network state prediction model is for:

[0142] , formula (11),

[0143] , formula (12),

[0144] , formula (13),

[0145] , formula (14),

[0146] , formula (15),

[0147] In the formula, represents the prediction error loss term, represents the network state feature smoothing loss term, represents the adaptive weight loss term, and Represent the corresponding loss items and The adjustment factor of represents the number of samples, Indicates The network status data predicted at a time point, Indicates Real network status data at a point in time; Indicates Adaptive weights at each time point; μ Represents the tuning parameter of the adaptive weight.

[0148] In this embodiment, the difference between the predicted value and the true value is directly measured through the prediction error loss to ensure the basic prediction accuracy of the model. Through the adaptive weight mechanism, the weight is dynamically adjusted according to the size of the prediction error, so that the model is more sensitive to the time points with large errors, thereby focusing on optimizing the prediction accuracy of these key points, so that the model can better cope with the drastic changes in network status and improve the overall prediction accuracy. Through the network status feature smoothing loss, the smoothness of the prediction results is encouraged, the drastic changes in the network status prediction results are avoided, the continuity and consistency of the prediction curve are ensured, and the model is more stable when dealing with sudden changes in network status. In addition, by weighted fusion of each loss term in the comprehensive loss function, the advantages of each loss term can be effectively integrated to achieve more accurate network status prediction.

[0149] Figure 5 An operational flowchart of an example of an online learning optimization network status prediction model according to an embodiment of the present application is shown.

[0150] like Figure 5 As shown, in step S510, based on M target transmission paths for parallel transmissionM When a photon coding module is installed, the resource usage indicators of the optical communication network are monitored to see if the preset resource alarm conditions are triggered. The resource usage indicators include network bandwidth utilization, network delay, packet loss rate and signal strength.

[0151] Here, when the indicator triggers the preset alarm condition, such as the preset threshold of the current bandwidth utilization is 80%, the preset threshold of the network delay is 100 milliseconds, etc., an alarm notification can be issued in time to prevent network failures, ensure the stability and security of transmission, and to a certain extent verify the previous model prediction results.

[0152] In step S520, when the resource alarm condition is triggered, an alarm notification is generated according to the target resource usage indicator that triggers the resource limit excess condition, and the third network status data and the third transmission video stream data corresponding to the adjacent preset time period are collected.

[0153] In step S530, a data sample set of a network state prediction model is updated based on the third network state data and the third transmission video stream data, so that the network state prediction model can be optimized through online learning.

[0154] In this embodiment, when a real network congestion or failure event is detected, it can generally indicate that there is a deviation in the prediction result output by the previous model. At this time, by collecting the third network status data and the third transmission video stream data within a preset time period and updating the data sample set of the network status prediction model, the model can continue to learn and adapt to the new network status, continuously optimize and improve the prediction accuracy, and ensure the video transmission quality under different network conditions.

[0155] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of actions combined, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application. In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0156] Figure 6 A structural block diagram of an example of a real-time video stream optimization transmission system based on optical communication according to an embodiment of the present application is shown.

[0157] like Figure 6As shown, the real-time video stream optimization transmission system 600 based on optical communication includes a historical data acquisition unit 610, a data to be transmitted acquisition unit 620, a network status prediction unit 630, a photon video encoding unit 640 and a photon video transmission unit 650.

[0158] The historical data acquisition unit 610 is used to obtain the first network status data and the first transmission video stream data corresponding to the past historical time period; the data type of the network status data includes network bandwidth, network delay and packet loss rate, and the data type of the transmission video stream data includes video encoding bit rate, video resolution and video frame rate.

[0159] The to-be-transmitted data acquisition unit 620 is used to acquire second transmission video stream data corresponding to a target video stream to be transmitted in a future preset time window.

[0160] The network status prediction unit 630 is used to input the first network status data, the first transmission video stream data and the second transmission video stream data into a network status prediction model to predict the second network status data corresponding to the preset time window.

[0161] The photon video encoding unit 640 is used to encode the photon encoded video stream corresponding to the target video stream using a photon encoding device when it is detected that the second network status data meets a preset transmission congestion status condition.

[0162] The photon video transmission unit 650 is used to segment the photon coded video stream to obtain corresponding M Photon encoding modules are selected from the set of available transmission paths M target transmission path, based on the M The target transmission paths transmit the M Photon encoding module.

[0163] The photon encoding device comprises a light source module, a modulator module and a photon encoding processing module; the light source module adopts a laser with adjustable output power to provide a photon source; the modulator module comprises an electro-optic modulator and an acousto-optic modulator to modulate the video signal into an optical signal; the photon encoding processing module comprises a beam splitter, a phase modulator and an interferometer to encode the optical signal;

[0164] The output power of the light source module and the modulation depth of the modulator module are adaptively adjusted according to the network state prediction index; the network state prediction index is determined according to the predicted second network state data;

[0165] The network status estimation index is calculated by the following formula:

[0166] ,

[0167] In the formula, represents the predicted network status evaluation index, , and Respectively represent the predicted network bandwidth, network delay and packet loss rate, , and They represent the adjustment coefficients of network bandwidth, network delay and packet loss rate respectively; Indicates the preset upper limit of delay;

[0168] The output power of the light source module is adaptively adjusted by the following formula:

[0169] ,

[0170] ,

[0171] In the formula, Indicates the output power of the light source module. represents the real-time injected current, Indicates the injection current calibration value, , They represent the characteristic coefficients of the laser respectively;

[0172] The modulation depth of the modulator module is adaptively adjusted by the following formula:

[0173] ,

[0174] ,

[0175] In the formula, Indicates the modulation depth of the modulator module, Indicates the real-time driving voltage, Indicates the drive voltage calibration value, Indicates half-wave voltage.

[0176] In some embodiments, an embodiment of the present application provides a non-volatile computer-readable storage medium, in which one or more programs including execution instructions are stored, and the execution instructions can be read and executed by an electronic device (including but not limited to a computer, a server, or a network device, etc.) to execute the steps of any of the above-mentioned real-time video stream optimization transmission methods based on optical communication in the present application.

[0177] In some embodiments, the embodiments of the present application also provide a computer program product, which includes a computer program stored on a non-volatile computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer performs any step of the above-mentioned real-time video stream optimization transmission method based on optical communication.

[0178] In some embodiments, the embodiments of the present application also provide an electronic device, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the real-time video stream optimized transmission method based on optical communication.

[0179] Figure 7 is a schematic diagram of the hardware structure of an electronic device for executing a real-time video stream optimization transmission method based on optical communication provided by another embodiment of the present application, such as Figure 7 As shown, the device includes:

[0180] One or more processors 710 and memory 720, Figure 7 A processor 710 is taken as an example.

[0181] The device for executing the real-time video stream optimized transmission method based on optical communication may further include: an input device 730 and an output device 740 .

[0182] The processor 710, the memory 720, the input device 730 and the output device 740 may be connected via a bus or other means. Figure 7 The example of connecting through bus is taken in the following.

[0183] The memory 720, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions / modules corresponding to the real-time video stream optimization transmission method based on optical communication in the embodiment of the present application. The processor 710 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions and modules stored in the memory 720, that is, the real-time video stream optimization transmission method based on optical communication in the above method embodiment is realized.

[0184] The memory 720 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 720 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 720 may optionally include a memory remotely arranged relative to the processor 710, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0185] The input device 730 may receive input digital or character information and generate signals related to user settings and function control of the electronic device. The output device 740 may include a display device such as a display screen.

[0186] The one or more modules are stored in the memory 720, and when executed by the one or more processors 710, perform the real-time video stream optimized transmission method based on optical communication in any of the above method embodiments.

[0187] The above-mentioned product can execute the method provided in the embodiment of the present application, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not fully described in this embodiment, please refer to the method provided in the embodiment of the present application.

[0188] The electronic device of the embodiment of the present application exists in various forms, including but not limited to:

[0189] (1) Mobile communication equipment: This type of equipment is characterized by having mobile communication functions and its main purpose is to provide voice and data communications. This type of terminal includes: smart phones, multimedia phones, functional phones, and low-end phones.

[0190] (2) Ultra-mobile personal computer devices: These devices fall into the category of personal computers, have computing and processing functions, and generally also have mobile Internet access features. These terminals include: PDA, MID and UMPC devices, etc.

[0191] (3) Portable entertainment devices: These devices can display and play multimedia content. They include audio and video players, handheld game consoles, e-books, smart toys, and portable car navigation devices.

[0192] (4) Other onboard electronic devices with data interaction functions, such as on-board devices installed in vehicles.

[0193] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0194] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the relevant technology, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0195] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A real-time video stream optimized transmission method based on optical communication, characterized in that: include: Acquire first network status data and first transmission video stream data corresponding to a past historical time period; The data type of the first network status data includes network bandwidth, network delay and packet loss rate, and the data type of the first transmission video stream data includes video encoding bit rate, video resolution and video frame rate; Acquire second transmission video stream data corresponding to a target video stream to be transmitted in a future preset time window; Inputting the first network status data, the first transmission video stream data and the second transmission video stream data into a network status prediction model to predict second network status data corresponding to the preset time window; When it is detected that the second network status data satisfies a preset transmission congestion status condition, encoding the photon-encoded video stream corresponding to the target video stream using a photon encoding device; The photon-encoded video stream is segmented to obtain corresponding M Photon encoding modules are selected from the set of available transmission paths M target transmission path, based on the M The target transmission paths transmit the M Photon encoding module; The photon encoding device comprises a light source module, a modulator module and a photon encoding processing module; the light source module adopts a laser with adjustable output power to provide a photon source; the modulator module comprises an electro-optic modulator and an acousto-optic modulator to modulate the video signal into an optical signal; the photon encoding processing module comprises a beam splitter, a phase modulator and an interferometer to encode the optical signal; The output power of the light source module and the modulation depth of the modulator module are adaptively adjusted according to the network state prediction index; the network state prediction index is determined according to the predicted second network state data; The network status estimation index is calculated by the following formula: , In the formula, represents the predicted network status evaluation index, , and Respectively represent the predicted network bandwidth, network delay and packet loss rate, , and They represent the adjustment coefficients of network bandwidth, network delay and packet loss rate respectively; Indicates the preset upper limit of delay; The output power of the light source module is adaptively adjusted by the following formula: , , In the formula, Indicates the output power of the light source module. represents the real-time injected current, Indicates the injection current calibration value, , They represent the characteristic coefficients of the laser respectively; The modulation depth of the modulator module is adaptively adjusted by the following formula: , , In the formula, Indicates the modulation depth of the modulator module, Indicates the real-time driving voltage, Indicates the drive voltage calibration value, Indicates half-wave voltage.

2. The method for optimizing transmission of real-time video stream based on optical communication according to claim 1, characterized in that: The photon-encoded video stream is segmented to obtain corresponding M Photon encoding module, including: For each photon-encoded video frame in the photon-encoded video stream: Obtaining the number of edge pixels and the range of motion vectors in the photon-encoded video frame, Calculate the target coding module size according to the number of edge pixels and the range of motion vectors, According to the target coding module size, the photon coding video frame is divided into a plurality of photon coding modules; The target encoding module size is calculated by the following formula: , , , In the formula, Indicates The complexity of encoding video frames by frame photons, represents the complexity sequence of video frames, Indicates the maximum complexity value among all video frames; Indicates The number of edge pixels in the frame, Represents the total number of pixels in each video frame. Indicates The average magnitude of motion vectors of frame photon-encoded video frames; and Represent edge pixel weight and motion vector weight respectively.

3. The method for optimizing transmission of real-time video stream based on optical communication according to claim 2, characterized in that: The second network status data includes a plurality of path network status data, The selection from the set of available transmission paths M target transmission paths, including: For each available transmission path in the available transmission path set, calculating a path load factor corresponding to the available transmission path according to the path network state data corresponding to the available transmission path; Select the path with the smallest load factor from each of the available transmission paths. M available transmission paths as corresponding target transmission paths; The path load factor is calculated by the regression analysis model: , In the formula, Indicates the transmission path The load factor, , and Represent the predicted paths network bandwidth, network delay and packet loss rate, represents the model coefficients of the regression learning model, represents the natural exponential function.

4. The method for optimizing transmission of real-time video stream based on optical communication according to any one of claims 1 to 3, characterized in that: The network state prediction model adopts a multimodal spatiotemporal network, which includes an input layer, a spatiotemporal convolution module, a Transformer module, a fusion layer and an output layer; The input layer is used to receive first network status data, first transmission video stream data and second transmission video stream data; The spatiotemporal convolution module is used to extract the spatiotemporal features corresponding to the first transmission video stream data and the second transmission video stream data; The Transformer module is used to process the spatiotemporal features and the first network state data to output corresponding global dependency features; The fusion layer is used to fuse the spatiotemporal features and the global dependency features to determine corresponding fusion features; The output layer is used to process the fused features to predict second network status data corresponding to a future preset time window.

5. The real-time video stream optimized transmission method based on optical communication according to claim 4 is characterized in that: The spatiotemporal convolution module adopts a multi-scale spatiotemporal convolution module to capture the features of the video stream at different time scales and spatial scales, thereby enhancing the richness of feature representation: , In the formula, Representation scale The spatiotemporal convolution output of , and Represent the row index, column index and time index of the input feature map respectively; Representation scale The convolution kernel, , and Represent the row index, column index and time index of the convolution kernel respectively; R and A Respectively represent the height and width of the convolution kernel; Representation scale The temporal convolution kernel size; represents the value of the input feature map, Indicates that the input feature map is at position The value of Representation scale The bias term, Represents the activation function.

6. The method for optimizing transmission of real-time video stream based on optical communication according to claim 4, characterized in that: The Transformer module adopts a multi-head self-attention mechanism.

7. The method for optimizing transmission of real-time video stream based on optical communication according to claim 4, characterized in that: The loss function of the network state prediction model is for: , , , , , In the formula, represents the prediction error loss term, represents the network state feature smoothing loss term, represents the adaptive weight loss term, and Represent the corresponding loss items and The adjustment factor of represents the number of samples, Indicates The network status data predicted at a time point, Indicates Real network status data at a point in time; Indicates Adaptive weights at each time point; μ Represents the tuning parameter of the adaptive weight.

8. The method for optimizing transmission of real-time video stream based on optical communication according to claim 7, characterized in that: The method further comprises: Based on the M The target transmission paths transmit the M When a photon coding module is installed, the resource usage index of the optical communication network is monitored to determine whether a preset resource alarm condition is triggered; the resource usage index includes network bandwidth utilization, network delay, packet loss rate and signal strength; When the resource alarm condition is triggered, an alarm notification is generated according to the target resource usage indicator that triggers the resource overlimit condition, and the third network status data and the third transmission video stream data corresponding to the adjacent preset time period are collected; The data sample set of the network state prediction model is updated based on the third network state data and the third transmission video stream data, so that the network state prediction model can be optimized by online learning.

9. A real-time video stream optimized transmission system based on optical communication, characterized in that: include: A historical data acquisition unit, used to acquire first network status data and first transmission video stream data corresponding to a past historical time period; The data type of the first network status data includes network bandwidth, network delay and packet loss rate, and the data type of the first transmission video stream data includes video encoding bit rate, video resolution and video frame rate; A to-be-transmitted data acquisition unit, used for acquiring second transmission video stream data corresponding to a target video stream to be transmitted in a future preset time window; A network state prediction unit, configured to input the first network state data, the first transmission video stream data, and the second transmission video stream data into a network state prediction model to predict the second network state data corresponding to the preset time window; A photon video encoding unit, configured to encode a photon encoded video stream corresponding to the target video stream using a photon encoding device when detecting that the second network status data satisfies a preset transmission congestion status condition; A photon video transmission unit is used to segment the photon coded video stream to obtain corresponding M Photon encoding modules are selected from the set of available transmission paths M target transmission path, based on the M The target transmission paths transmit the M Photon encoding module; The photon encoding device comprises a light source module, a modulator module and a photon encoding processing module; the light source module adopts a laser with adjustable output power to provide a photon source; the modulator module comprises an electro-optic modulator and an acousto-optic modulator to modulate the video signal into an optical signal; the photon encoding processing module comprises a beam splitter, a phase modulator and an interferometer to encode the optical signal; The output power of the light source module and the modulation depth of the modulator module are adaptively adjusted according to the network state prediction index; the network state prediction index is determined according to the predicted second network state data; The network status estimation index is calculated by the following formula: , In the formula, represents the predicted network status evaluation index, , and Respectively represent the predicted network bandwidth, network delay and packet loss rate, , and They represent the adjustment coefficients of network bandwidth, network delay and packet loss rate respectively; Indicates the preset upper limit of delay; The output power of the light source module is adaptively adjusted by the following formula: , , In the formula, Indicates the output power of the light source module. represents the real-time injected current, Indicates the injection current calibration value, , They represent the characteristic coefficients of the laser respectively; The modulation depth of the modulator module is adaptively adjusted by the following formula: , , In the formula, Indicates the modulation depth of the modulator module, Indicates the real-time driving voltage, Indicates the drive voltage calibration value, Indicates half-wave voltage.