Network transmission optimization method, device, electronic device, and computer program

Abstract

An embodiment of the present application provides a network transmission optimization method, an apparatus, a computer-readable medium, an electronic device, and a program product, which includes the steps of receiving a data stream sent by a data sending side and which needs to be transmitted to a data receiving side, identifying key frames contained in the data stream, calculating statistics on the key frames to obtain key frame statistics, and, when determining that a network optimization process is required for the data receiving side based on the key frame statistics, performing a network optimization operation for the data receiving side.

Description

[Technical Field]

[0001] This application claims priority to a Chinese patent application bearing application number 202310621502.5 and entitled "Network Transmission Optimization Method, Apparatus, Computer-Readable Medium, and Electronic Device," filed with the State Intellectual Property Office of the People's Republic of China on May 29, 2023.

[0002] The present application relates to the technical fields of computers and communications, and more particularly to a network transmission optimization method, apparatus, storage medium, electronic device, and program product. [Background technology]

[0003] Currently, live streaming and streaming media applications have become an indispensable part of daily life, and these applications have relatively high real-time requirements and need to occupy relatively large bandwidths. However, due to the impact of network quality, the stability and user experience of these applications may also be significantly affected. Specifically, users' needs for live streaming and streaming media applications are becoming increasingly higher, requiring smoother, clearer, and more stable viewing experiences, and the content of live streaming and streaming media applications is becoming more diverse, requiring faster and lower-latency network support.

[0004] Therefore, how to improve network quality by optimizing the network in a timely and rational manner is a technical issue that needs to be resolved immediately. Summary of the Invention [Means for solving the problem]

[0005] The embodiments of the present application provide a network transmission optimization method, apparatus, computer-readable medium, electronic device, and program product, which can timely and reasonably determine whether a network optimization process is needed, and when a network optimization process is needed, take appropriate network optimization measures in a timely manner, which is beneficial to improving network transmission quality.

[0006] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.

[0007] According to one aspect of an embodiment of the present application, there is provided a network transmission optimization method, comprising: receiving a data stream sent by a data sender and which needs to be transmitted to a data receiver; identifying key frames contained in the data stream; Statisticizing the key frames to obtain key frame statistics; and performing a network optimization operation for the data sink when it is determined that a network optimization process for the data sink is necessary based on the key frame statistical information.

[0008] According to one aspect of an embodiment of the present application, there is provided a network transmission optimization device, comprising: a receiving unit configured to receive a data stream sent by a data sender and which needs to be transmitted to a data receiver; an identification unit configured to identify key frames contained in said data stream; a statistics unit configured to perform statistics on the key frames to obtain key frame statistics information; a processing unit configured to perform a network optimization operation for the data sink when determining that a network optimization operation for the data sink is necessary based on the key frame statistical information.

[0009] According to one aspect of an embodiment of the present application, a computer-readable medium is provided having a computer program stored thereon, the computer program implementing the network transmission optimization method described in the embodiment when executed by a processor.

[0010] According to one aspect of an embodiment of the present application, an electronic device is provided, comprising one or more processors and a storage device used to store one or more computer programs, which, when executed by the one or more processors, cause the electronic device to realize the network transmission optimization method described in the embodiment.

[0011] According to one aspect of the present invention, there is provided a computer program product, the computer program product including a computer program stored in a computer-readable storage medium, the computer program being read by a processor of an electronic device from the computer-readable storage medium and executed to cause the electronic device to perform the network transmission optimization method provided in the various optional embodiments. [Brief explanation of the drawings]

[0012] [Figure 1] 1 shows a schematic diagram of an exemplary application scenario in which the technical solutions of the embodiments of the present application can be applied; [Figure 2] 1 shows a schematic diagram of an exemplary application scenario in which the technical solutions of the embodiments of the present application can be applied; [Figure 3] 1 shows a schematic diagram of an exemplary application scenario in which the technical solutions of the embodiments of the present application can be applied; [Figure 4] 2 shows a flowchart of a network transmission optimization method according to an embodiment of the present application; [Figure 5] 2 shows a flowchart of a network transmission optimization method according to an embodiment of the present application; [Figure 6] 1 illustrates a schematic diagram of a network optimization process according to an embodiment of the present application; [Figure 7] 1 shows a schematic diagram of functions included in a UPF according to an embodiment of the present application. [Figure 8] 1 is a schematic diagram showing the functions of a UPF according to an embodiment of the present application; [Figure 9] 1 shows a schematic diagram of a network test environment according to one embodiment of the present application. [Figure 10] 1 illustrates a schematic diagram of a network testing process according to an embodiment of the present application; [Figure 11] 1 shows a block diagram of a network transmission optimization device according to an embodiment of the present application. [Figure 12] 1 shows a structural schematic diagram of a computer system of an electronic device suitable for implementing an embodiment of the present application; DETAILED DESCRIPTION OF THE INVENTION

[0013]

[0033] Exemplary embodiments will now be more fully described with reference to the drawings. However, exemplary embodiments may be embodied in various forms and should not be construed as being limited to these examples. On the contrary, the purpose of providing these embodiments is to make this application more thorough and complete and to fully convey the concept of exemplary embodiments to those skilled in the art.

[0014] Furthermore, the features, structures, or characteristics described herein may be incorporated into one or more embodiments in any suitable manner. In the following description, many specific details are provided to enable a thorough understanding of the embodiments. However, those skilled in the art should be aware that, when implementing the technical solutions of the present application, it is not necessary to use all the detailed features in the embodiments, and one or more specific details may be omitted, or other methods, elements, devices, steps, etc. may be adopted.

[0015] The block diagrams shown in the drawings only represent functional entities that do not necessarily correspond to physically separate entities, i.e., they may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different network and / or processor and / or microcontroller devices.

[0016] The flowcharts shown in the drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the order described. For example, some operations / steps may be further decomposed, while some operations / steps may be merged or partially merged, and therefore the order of actual execution may vary based on the actual situation.

[0017] As should be clarified, "plurality" as referred to herein refers to two or more than two. "And / or" describes a relationship between related objects and indicates that three relationships can exist; for example, A and / or B can represent three cases: A exists alone, A and B exist simultaneously, or B exists alone. The character " / " generally indicates that the related objects before and after it are in an "or" relationship.

[0018] First, it should be noted that the present application may display a prompt interface or pop-up window before collecting relevant data (e.g., a data stream transmitted from a data sender to a data receiver) and during the process of collecting the relevant data. The prompt interface or pop-up window prompts the user to confirm that relevant data is currently being collected, and the present application begins to perform the relevant step of acquiring the relevant data only after the user confirms the prompt interface or pop-up window. Otherwise (i.e., when the user does not confirm the prompt interface or pop-up window), the step of acquiring the relevant data is terminated, i.e., the relevant data is not acquired. In other words, in the embodiments of the present application, with regard to related data such as data streams, when the above embodiments of the present application are applied to specific products or technologies, user permission or consent must be obtained, and the collection, use, and processing of the related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0019] Next, technical terms used in the examples of the present application will be explained.

[0020] 5GC, or 5G Core (5G core network), is the core part of the 5G communication system, responsible for data transmission and control processing in the mobile communication network, providing higher data transmission speeds, lower latency, and higher network capacity to support the connection of more users and devices. The 5G core network architecture adopts a hierarchical design and includes two parts: the user plane and the control plane. The user plane is responsible for data transmission processing, including data encryption, decryption, and transmission, while the control plane is responsible for network control and management processing, including user identity authentication, conversation management, and resource allocation.

[0021] Regarding video push-pull stream, push stream refers to the process of transmitting video from a source side (usually a camera or video capture device) to a streaming media server, and pull stream refers to the process of obtaining video from a streaming media server and displaying it on a terminal device (usually a computer or mobile device).

[0022] UPF, or User Plane Function, refers to the network function responsible for handling user data transmission in 5G networks. UPF distributes user data traffic to different network nodes, thereby enabling the network to meet various business needs, such as providing low-latency and high-bandwidth network services, enabling flexible network configuration and efficient data transmission and management, and delivering higher network quality and user experience for various business scenarios. UPF application scenarios include enhanced mobile broadband (eMBB), the Internet of Things (IoT), and smart manufacturing (Industry 4.0).

[0023] Here, eMBB is the most basic business need in 5G networks, including application scenarios such as high-definition video, virtual reality (VR), and augmented reality (AR). The UPF in SDN (Software Defined Network) transmits user data to the appropriate network node, thereby quickly responding to the business needs of mobile users and achieving low-latency and high-bandwidth data transmission.

[0024] IoT refers to the connection of various devices and objects via the Internet to collect and transmit data. UPF in SDN realizes flexible network configuration, transmits and allocates data from various devices and objects to different network nodes, and realizes efficient data transmission and management.

[0025] Smart manufacturing refers to the use of the Internet and digital technology to digitalize and smarten up the manufacturing industry, including data collection and transmission from various devices and systems. UPF in SDN can realize network management and optimization of various devices and systems in smart manufacturing, improving manufacturing efficiency and product quality.

[0026] RTP, or Real-Time Transport Control Protocol, can be used to transmit audio, video, and other multimedia data, and the RTP protocol is highly suitable for real-time multimedia applications, such as live video streaming, network telephony, and video conferencing.

[0027] Regarding the H.264 protocol, H.264, also known as the AVC (Advanced Video Coding) protocol, is a video codec protocol and a standard used to compress digital video. The H.264 codec protocol employs advanced compression technology to compress video into relatively small file sizes while maintaining high quality. During the H.264 encoding process, a video frame is divided into multiple blocks, and each block is coded into an independent data unit called a NALU (Network Abstraction Layer Unit). A NALU is the basic unit of an H.264 codestream, containing both video data header information and the actual video data. An H.264 codestream is divided into two layers: the Video Coding Layer (VCL) and the Network Abstraction Layer (NAL). The VCL layer is responsible for compressing and encoding the video, while the NAL layer is responsible for packaging the data generated by the VCL layer into NALUs and adding header information, thereby facilitating transmission and decoding.

[0028] A video frame is a common concept in video coding and is mainly divided into I-frames, B-frames, and P-frames. Here, an I-frame is also called an independent frame or key frame, and can be coded independently without relying on information from other frames. Therefore, the I-frame has the largest amount of data, can provide relatively good picture quality, and helps restore the complete structure of a video picture in video coding. A B-frame is called a bidirectional frame, and it is decoded using the previous I-frame or P-frame and the following P-frame as reference information. The amount of data of a B-frame is smaller than both an I-frame and a P-frame, and can improve the video compression effect. A P-frame is called a predicted frame, and it is decoded based on information from a previous key frame. The main function of a P-frame is to improve video compression efficiency and reduce the volume of transmission data.

[0029] The VPP framework, or Vector Packet Processing framework, is an extensible open-source framework that provides production-quality switch / router functionality. VPP has two main functions: an extensible framework and mature switching / routing functionality. The purpose of the VPP platform is to solve the high latency problem that exists in single-packet processing using traditional scalar processing methods. To improve network scalability, VPP adopts a vector processing method, processing multiple packets simultaneously and reducing latency. VPP can function as a network switch or router, and typical usage scenarios include data centers, cloud computing, edge computing, and the Internet of Things. Furthermore, VPP's flexibility and scalability offer potential in areas such as network security and virtualized networks.

[0030] DPDK, or Data Plane Development Kit, is an open-source data plane development kit that provides a set of libraries and drivers to help developers quickly build high-performance data plane application programs. DPDK is mainly used in fields such as network functions virtualization (NFV) and software-defined networking (SDN), and can accelerate packet processing and forwarding, improving the performance and throughput of network application programs.

[0031] A rate adaptation algorithm dynamically adjusts the video rate based on factors such as the current network conditions and device performance, ensuring smooth video playback. The ABR (Adaptive Bitrate Streaming) algorithm is a common rate adaptation algorithm. It is a video streaming media transmission algorithm based on rate adaptation. It dynamically adjusts the video rate by constantly monitoring and measuring factors such as network conditions and device performance, ensuring smooth video playback. The core idea of ​​the ABR algorithm is to adapt to different network conditions by switching between different rates. Specifically, when the network conditions are relatively good, the ABR algorithm selects a relatively high rate for transmission to improve video resolution. When the network conditions are relatively poor, the ABR algorithm selects a relatively low rate for transmission to ensure smooth video playback. In practical applications, ABR algorithms usually employ several heuristic policies to select the rate, such as buffer-based ABR and rate-based ABR. Here, a buffer-based ABR algorithm may avoid buffer overflow or severe degradation by selecting a rate based on the current buffer filling status, while a rate-based ABR algorithm may ensure smooth video playback by selecting a rate based on factors such as the current network conditions and device performance.

[0032] Further, the application environment of the embodiments of the present application is described, and the technical solutions of the embodiments of the present application can be applied to scenes with relatively high requirements for real-timeness and bandwidth, such as live streaming and streaming media applications, but due to the impact of network quality, the stability and user experience of these applications will also be relatively greatly affected. Specifically, users' needs for live streaming and streaming media applications are becoming increasingly higher, requiring a smoother, clearer, and more stable viewing experience, and the content of live streaming and streaming media applications is becoming more diversified, requiring faster and lower-latency network support.

[0033] To address the above-mentioned problems, the embodiments of the present application propose a new network transmission optimization solution, which identifies and statistics the data stream in the data transmission channel between the data sender and the data receiver, thereby determining the transmission status of key frames that play a crucial role in recovering the data stream, and further determining in a timely manner whether network optimization processing is required, ensuring that the data receiver can smoothly receive and recover the data stream transmitted by the data sender, which is beneficial to improving network transmission quality.

[0034] Specifically, one application scenario of the present application is a remote driving system based on a 5G network, as shown in Figure 1, in which a camera at a vehicle end 101 transmits video stream data through a MEC (Mobile Edge Computing) gateway 1 and a 5G dedicated network 102, and after reaching the cloud side 103, the corresponding data is processed by a signaling server, a media server, and a web server, and transmitted via a data channel and a media channel to a user terminal 104 such as a simulated driving cabin, a WPF (Windows Presentation Foundation) interface, a video presentation window, and a large web screen, for presentation and playback.

[0035] Based on the technical solution of the embodiment of the present application, in the 5G dedicated network 5GC 105, the UPF can identify specific data (e.g., key frame data) in the video stream through open capability technology, and determine the current network quality through data caching and analysis. When it determines that the network quality is relatively poor, it can combine user bandwidth allocation or rate adaptation algorithms to perform network-linked optimization processing, further optimizing the data transmission process and ensuring that the video playback of the user terminal does not experience delays, static, and other phenomena.

[0036] In addition, UPF can also perform data packet discard processing using capability open technology, such as discarding key frames in a video stream, and can also easily verify the ability of user terminals to adapt to weak network environments by simulating cases where the network is relatively poor.

[0037] 2 shows a schematic diagram of the application of the technical solution of the embodiment of the present application to a vehicle-to-road cooperation system. Roadside devices 201, such as millimeter-wave radar, laser radar, and video cameras, are used to collect data such as roadside images. The collected data undergoes roadside processing 202 and is then transmitted to a V2X (Vehicle to Everything) server 203, and then transmitted to a terminal user 205 (such as a mini program, map application, in-vehicle device, and APP) by a vehicle-to-road cooperation platform 204 via a 5G core network.

[0038] Similarly, in this application scenario, a network-linked optimization process can be used to ensure that transmission data is effectively delivered to the terminal user. That is, the UPF in the 5G system can identify specific data (such as key frame data) in the transmitted data through capability open technology, and determine the current network quality through data caching and analysis. When it determines that the network quality is relatively poor, it can combine user bandwidth allocation or rate adaptation algorithms to perform network-linked optimization process.

[0039] FIG. 3 shows a schematic diagram of the application of the technical solution of the embodiment of the present application to a cloud gaming scenario. Cloud gaming, also known as gaming on demand, is an online gaming technology based on cloud computing technology. Cloud gaming technology enables high-quality games to be run on thin clients with relatively limited graphics processing and data computing capabilities. In a cloud gaming scenario, games are executed on cloud servers rather than on player game terminals, and the cloud servers render game scenes as video and audio streams and transmit them to player game terminals via a network. Player game terminals do not need to have powerful graphics computing and data processing capabilities; they only need basic streaming media playback capabilities and the ability to receive player input commands and send them to a cloud server.

[0040] Referring to FIG. 3 , in a cloud gaming scenario, a cloud gaming client terminal 301 communicates with an edge cloud gaming server side 303 in an edge cloud platform 302 via LTE (Long Term Evolution) / NR (New Radio, e.g., a radio access network of a 5G system). The edge cloud platform 302 accesses EPC (Evolved Packet Core, e.g., a 4G core network) / 5GC, and then communicates with an internet application on the cloud side. By applying the technical solutions of the embodiments of the present application, business data transmission in a cloud gaming scenario can be optimized under scenarios where network quality is degraded or data volume increases in a short period of time, thereby ensuring that terminal users do not experience problems such as delays and data corruption when receiving and rendering data.

[0041] It can be seen from this that the technical solution of the embodiment of the present application can perform optimization processing on the network under any scenario, thereby timely and reasonably determining whether network optimization processing is necessary, and further, when network optimization processing is necessary, taking corresponding network optimization measures in a timely manner, which is beneficial to improving network transmission quality.

[0042] It should be noted that in the embodiments of the present application, the device used by the terminal user may be, but is not limited to, a smartphone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, an in-vehicle terminal, a smart TV, an airplane, etc. The server referred to in the embodiments of the present application may be an independent physical server, a server cluster or a distributed system consisting of multiple physical servers, or even a cloud-side server that provides cloud computing services.

[0043] The implementation details of the technical solutions of the embodiments of the present application are discussed in detail below.

[0044] 4 shows a flowchart of a network transmission optimization method according to an embodiment of the present application, which can be performed by a network device, such as an access network device (e.g., a base station), a core network device (e.g., a UPF), or an MEC server connected to a UPF, etc. Referring to FIG. 4, the network transmission optimization method includes at least steps S410 to S440, and a detailed introduction is provided below.

[0045] In step S410, a data stream sent by a data sender and required to be transmitted to a data receiver is received.

[0046] In some optional embodiments, the data sender and the data receiver are a sender and a receiver of data, for example, the data sender may be a terminal device running a game client terminal, and the data receiver may be a game server, in which case the game data sent by the terminal device is transmitted to the game server via the access network and the core network. Optionally, the data sender may be a game server, and the data receiver may be a terminal device running a game client terminal, in which case the game data sent by the game server is transmitted to the terminal device via the core network and the access network.

[0047] Since data transmission between a data sender and a data receiver needs to pass through an access network and a core network, a network device on the transmission route between the data sender and the data receiver can receive the data stream sent by the data sender and that needs to be sent to the data receiver, for example, a UPF can receive the data stream sent by the data sender and that needs to be sent to the data receiver, or a base station can receive the data stream sent by the data sender and that needs to be sent to the data receiver. In other embodiments of the present application, other devices can receive the data stream sent by the data sender and that needs to be sent to the data receiver, for example, an MEC server connected to the UPF can receive the data stream sent by the data sender and that needs to be sent to the data receiver.

[0048] In step S420, key frames contained in the data stream are identified.

[0049] In some optional embodiments, based on the characteristics of the data frame, it is possible to identify the type of data frame contained in the data stream, i.e. whether it belongs to a key frame or not.

[0050] Specifically, when identifying a key frame, a packet in the data stream is captured, and characteristics of the captured packet are extracted, and then the characteristics of the captured packet are compared with the characteristics of the key frame packet, thereby identifying the key frame contained in the data stream.

[0051] Optionally, key frames contained in the data stream can be identified by comparing at least one of the following packet features: a protocol field in an Internet Protocol (IP) header, a version field in a Real-Time Transmission Control Protocol (RTP) layer (i.e., Version field), a padding field in the RTP layer (i.e., Padding field), an extension field in the RTP layer (i.e., Extension field), a marker field in the RTP layer (i.e., Maker field), and a network abstraction layer unit type field (i.e., Nal_unit_type field).

[0052] In the embodiment of the present application, the key frame contained in the data stream can be identified by the above fields mainly because, when transmitting the key frame based on the UDP (User Datagram Protocol) protocol, the protocol field value in the IP header of the key frame is 17, the Version field in the RTP layer is RFC 1889 Version, and the Padding field, Extension field, and Maker field are all False, and the Nal_unit_type in the H.264 layer is equal to 5, so the key frame contained in the data stream can be identified based on the above fields.

[0053] In some optional embodiments, when a key frame is divided into multiple packets for transmission, packets belonging to the same key frame can be identified based on values ​​of designated fields contained in the captured packets, where the designated field includes a start bit (e.g., start_bit) and an end bit (e.g., end_bit) in a coding protocol layer, the start bit of a first packet among the multiple packets has a first value and the end bit has a second value, the start bit of a last packet among the multiple packets has a second value and the end bit has the first value, and the start bit and the end bit of packets other than the first packet and the last packet among the multiple packets are all the second value. Optionally, the first value may be 1 and the second value may be 0.

[0054] In some optional embodiments, after identifying the key frames contained in the data stream, the identified key frames can be stored, where when the key frames are divided into multiple packets for transmission, the RTP timestamp (RTP.Timestamp) of the key frame is used as the key, the multiple packets are used as the value, and the identified key frames are stored in the form of a key-value pair.

[0055] Optionally, the key frames identified by the hash table can be stored. In this case, a predetermined storage space size is received for the hash table, and then the most recently received key frames can be stored in a queue based on the storage space size. That is, in this embodiment, the newly received key frames are stored in a queue, and because the storage space size is limited, previously received key frames are deleted to ensure that the number of key frames stored in the queue matches the predetermined storage space size.

[0056] In step S430, the key frames contained in the data stream are statistically analyzed to obtain key frame statistical information.

[0057] In some optional embodiments, the key frame statistics information may be obtained by stating the number of key frames currently to be transmitted or by stating the rate of increase of key frames to be transmitted within a unit time.

[0058] In step S440, when it is determined based on the key frame statistical information that network optimization processing is required for the data sink, a network optimization operation is performed for the data sink.

[0059] In some optional embodiments, if it is determined based on the key frame statistics that the number of key frames currently to be transmitted is greater than or equal to a first number threshold, it is determined that a network optimization process is required for the data receiver, because if the number of key frames to be transmitted is large, it may indicate that the network is congested, and therefore it may be determined that a network optimization process is required.

[0060] In some optional embodiments, if it is determined based on the key frame statistical information that the increase rate of key frames to be transmitted within a unit time is greater than or equal to a predetermined increase rate threshold, it is determined that a network optimization process is required for the data receiving side, because if the increase rate of key frames to be transmitted within a unit time is relatively large, it indicates that the video rate corresponding to the data stream is relatively high or the frequency of video screen switching is relatively high, which causes an increase in transmission pressure, and therefore it can be determined that a network optimization process is required.

[0061] In some optional embodiments, if it is determined based on the key frame statistical information that the increase rate of the key frames to be transmitted within a unit time is equal to or greater than a predetermined increase rate threshold, and the length of the group of pictures of the data stream is less than or equal to a predetermined length threshold, it is determined that a network optimization process is required for the data receiving side. This is because if the increase rate of the key frames to be transmitted within a unit time is relatively large and the group of pictures (abbreviated as GOP) is relatively long, it indicates that the video rate corresponding to the data stream is relatively high, which will cause an increase in transmission pressure, and therefore it can be determined that a network optimization process is required.

[0062] In some optional embodiments, if the number of key frame retransmission requests received within a unit time is greater than or equal to a second number threshold, it can be determined that network optimization processing is required for the data receiver, as this may indicate that the network is congested, causing relatively high packet loss.

[0063] In some optional embodiments, when performing network optimization for a data sink, the transmission rate of the data stream may be reduced and network bandwidth for the data sink may be increased.

[0064] In some optional embodiments, the network optimization process may be adaptively performed using a machine learning method. For example, when it is determined that a network optimization process is required for a data receiving side, the network features between the data sending side and the data receiving side and the business features corresponding to the data stream are obtained, and then input parameters of a pre-trained network optimization model are generated based on the network features and the business features. The pre-trained network optimization model is obtained by training using the network feature samples and the business feature samples, and the input parameters are further input into the pre-trained network optimization model to output network optimization parameters, thereby performing the network optimization process for the data receiving side based on the network optimization parameters.

[0065] Optionally, the network characteristics between the data sender and the data sink may be network state parameters, such as packet loss rate, RSSI (Received Signal Strength Indicator), and RSRP (Reference Signal Receiving Power), etc. The service characteristics corresponding to the data stream may include service bandwidth needs, service delay requirements, etc. The network optimization model may adopt a CNN (Convolutional Neural Network) model, an RNN (Recurrent Neural Network) model, etc. during training.

[0066] Optionally, when generating input parameters for the pre-trained network optimization model based on the network features and the business features, the input parameters for the network optimization model can be obtained by cascading the network features and the business features.

[0067] In some optional embodiments, the technical solutions of the embodiments of the present application may identify key frames contained in the data stream, discard a predetermined number of key frames according to a predetermined discarding policy, and then collect statistics on the number of received key frame retransmission requests, and determine the simulated network quality by discarding key frames based on the number of key frame retransmission requests obtained by the statistics, that is, in the embodiments, the network quality may be simulated by discarding key frames.

[0068] Optionally, a network simulation configuration item can be set for the network simulation function, and then the network simulation function can be turned on or off based on the network simulation configuration item. Specifically, a trigger operation for the network simulation configuration item can be received, and if the trigger operation is used to turn on the network simulation function, a step of discarding a predetermined number of key frames according to a predetermined discard policy is performed, and if the trigger operation is used to turn off the network simulation function, the process of discarding key frames is stopped.

[0069] As can be seen from the technical solution of the embodiment shown in Figure 4, the present application can timely and reasonably determine whether network optimization processing is required by identifying and statistically analyzing key frames in the data stream, and when network optimization processing is required, take corresponding network optimization measures in a timely manner, which is beneficial to improving network transmission quality.

[0070] The following describes in detail the implementation details of the technical solutions of the embodiments of the present application by taking the network optimization process based on the capability opening technology of UPF as an example.

[0071] As shown in FIG. 5, in 501, it is assumed that the current network congestion causes a degradation of network quality, resulting in packet loss in the transmission data stream, which in turn causes a degradation of the video playback quality of the client terminal, resulting in phenomena such as delays and static.

[0072] In 502, after introducing the capability opening function of the UPF in the embodiment of the present application, the UPF may identify video key frames and count the number of key frames and the number of key frame retransmission requests, and then determine whether a network quality degradation problem has occurred based on the number of key frames and the number of key frame retransmission requests obtained by the statistics.

[0073] When making a decision, two cases can be distinguished.

[0074] 1. If the number of key frame re-requests within a unit time increases, the network quality will definitely deteriorate, which will cause relatively frequent network packet loss. 2. If the number of keyframes per unit time rises sharply and the GOP is relatively small, it indicates that the user's bandwidth needs are increasing, and at this time, further decision-making is needed to optimize the network and avoid network congestion.

[0075] In 503, after determining that a network optimization process is required, the network optimization process can be performed through a fast interlocking mechanism, and in the embodiment of the present application, the following two optimization methods can be adopted:

[0076] 1. When the user's bandwidth is increased by the UPF or the base station, even if the current user's transmission needs for key frame data increase sharply, the network bandwidth can be adjusted to ensure smooth data transmission. 2. By combining and optimizing a rate adaptation algorithm, such as the ABR algorithm, even if network congestion or other quality degradation problems occur, the rate can be reduced and transmission can be performed, thereby avoiding delays in playback at the user terminal.

[0077] After performing the network optimization process, the optimization can be fed back to the user terminal at 504 to improve the video playback quality of the terminal.

[0078] It should be noted that the key frame in the embodiments of the present application may be an I-frame or an IDR (Instantaneous Decoder Refresh) frame. In the following, we will continue to use the example of the key frame being an I-frame.

[0079] FIG. 6 is a schematic diagram of a network optimization process according to an embodiment of the present application, specifically, In 601, by developing relevant functions in the UPF network element in the 5G core network, it is possible to identify, count, and cache I-frames, analyze the data amount of I-frames within a unit time, and analyze the cases of I-frame loss, and further determine whether the current network quality is good or bad in 602. For example, if the number of I-frame re-requests is high, it indicates that the I-frame loss is serious, and if the GOP of the streaming media data is relatively small, it indicates that the frame rate is relatively high, which will cause serious video delays and static.

[0080] In 603, when the UPF detects that the current network quality is degraded and is not sufficient to handle the rapidly increasing data transmission volume of I-frames or to mitigate serious I-frame loss, it can immediately initiate optimization measures in 604, for example, when mitigating serious I-frame loss, it can reduce the rate based on the ABR algorithm, i.e., increase the GOP length or increase the allocated network bandwidth, and further optimize the video playback quality of the client terminal.

[0081] Based on the technical solutions of the embodiments of the present application, the UPF in the embodiments of the present application may include the following functional modules:

[0082] 1.I frame data detection function, 2.I frame data statistics function, 3.I frame data storage function, 4. I-frame data packet loss function, used to simulate network quality through packet loss.

[0083] Optionally, the development of the UPF can be aided by the DPDK technology and the VPP framework, so that these functional modules can be centralized in the nodes in the VPP. In the present embodiment, these functions can be integrated into both the upstream and downstream nodes of the UPF, so that I-frame data can be identified, analyzed, and cached in all directions (i.e., upstream and downstream). Specifically, the functions realized by the UPF functional module are as shown in Figure 7, that is, they include the four functions mentioned above. Below, in conjunction with Figure 8, the functions limited to the UPF in the present embodiment will be described in detail.

[0084] The I-frame data detection function is used to identify I-frames in the data stream, ie, it includes an I-frame identification function 801 shown in FIG.

[0085] Here, the I frame identification function 801 can obtain the characteristics of an I frame packet by using a packet capture tool (e.g., Wireshark) to capture packets and analyze messages. After sufficient packet capture in comparative experiments and reference to RFC standards, this embodiment determines that the characteristics of an I frame packet transmitted according to the UDP protocol are as follows: the protocol field value in the IP header is 17, the version field in the RTP layer is RFC 1889 Version, and the padding field, extension field, and maker field are all False. Furthermore, the Nal_unit_type value in the H.264 layer of the I frame is equal to 5, indicating that the message is an encoded slice of an IDR image. Therefore, based on the above characteristics, the characteristics of the I frame data are obtained, and identification functions corresponding to the upstream and downstream nodes of the UPF are developed to implement the code, and the IDR data that has passed through the UPF is stored in a hash table.

[0086] It should be noted that the I-frame data detection function may further include an I-frame header / footer packet determination function 802 shown in FIG. 8. This is because an I-frame may be divided into multiple IP packets during network transmission. Therefore, it is necessary to identify the IP packets of the frame header and frame footer to determine an I-frame. Similarly, in an embodiment of the present application, this function can be developed based on Wireshark packet capture analysis and RFC standard documents. An I-frame header message is characterized by the start_bit field of the H.264 layer being 1, an I-frame footer message is characterized by the end_bit field of the H.264 layer being 1, and an I-frame intermediate message is characterized by the start_bit and end_bit of the H.264 layer being both 0.

[0087] The I-frame data statistics function is to collect statistics on the amount of I-frame data in a unit time and the number of I-frame retransmission requests received, and then subsequently analyze whether network optimization processing is required.

[0088] The I-frame data storage function 803 caches and stores I-frames identified by the UPF in a hash table. The RTP.Timestamp of the I-frame is used as the key for the hash table, and the message array (i.e., the multiple IP packets obtained by I-frame segmentation) is used as the value for the hash table. At the same time, a queue-based deletion function for the oldest I-frame data can be implemented to reduce memory overhead. The global variable I-Frame_hash_size, i.e., the size of the hash table used to store I-frames, can be set using the vat command and API (Application Programming Interface). In the code implementation, hash_get() is used to read I-frame data from the hash table, hash_set() is used to store data in the hash table, and hash_unset_mem_free() is used to free older I-frame data stored in the hash table.

[0089] The purpose of the I-frame discard function 804 is to simulate a weak network environment, i.e., the UPF actively discards packets to simulate packet loss in a real network environment. For example, the value of DROP_NUM (this parameter is a predetermined parameter) determines the number of I-frames from which packets should be discarded. After the packets are discarded, the video playback of the client terminal may be seriously affected, resulting in static and lag. After that, the client terminal may actively send an I-frame data re-request message to re-request the I-frame data and restore the video playback. Therefore, the UPF can further verify the quality of the network by counting these I-frame re-request messages. Optionally, this function can be turned on or off by configuration.

[0090] Based on the functional modules in the UPF capability opening shown in Figures 7 and 8, the technical solution of the embodiment of the present application can realize real-time statistics and analysis of I-frame data, and at the same time, after obtaining the current network quality degradation through analysis, can perform synchronized network optimization, such as increasing bandwidth, or performing optimization and feedback adjustment through the ABR rate adaptation algorithm, to ensure the smoothness of video playback on the client terminal.

[0091] 9 shows a schematic diagram of a network test environment in an embodiment of the present application. To simulate a streaming media transmission scene such as live streaming, in an embodiment of the present application, ffmpeg (an open source computer program that can record and convert digital audio and video and convert it into a stream) and ffplay (an audio-video media player provided by ffmpeg) tools can be used to perform push-pull streaming of video.

[0092] Specifically, as shown in Figure 9, a push stream side 901 pushes streaming media data based on an (R)AN (Radio Access Network), and the transmission of the streaming media data passes through a 5G core network 902, so all transmitted I information is statistically analyzed in real time by a capability open function in the UPF. The streaming media data then flows out of the N6 interface of the UPF and is sent to a streaming media server 903, after which a pull stream side 904 locally pulls the streaming media data from the streaming media server 903 for playback.

[0093] Based on the test environment shown in Figure 9, as shown in Figure 10, the network environment test scene in the embodiment of the present application includes a normal network scene 1001, a weak network simulation scene 1002, and a linked optimization mechanism scene 1003 in the embodiment of the present application.

[0094] Here, under a normal network scenario 1001, streaming media data can be transmitted based on UDP and TCP (Transmission Control Protocol) respectively, that is, the UPF real-time statistical analysis and interlocking optimization mechanism in the embodiment of this application is not introduced, and then the case of pull-stream video playback by a client terminal is observed.

[0095] The weak network simulation scene 1002 simulates the transmission of streaming media data based on the UDP protocol in an environment where the network is relatively congested and the packet loss rate is relatively high, but does not incorporate the UPF real-time statistical analysis and interlocking optimization mechanism of the embodiment of the present application. To create a weak network environment, the embodiment of the present application utilizes the I-frame discard function of the UPF's capability opening technology to simulate network packet loss, preventing the client terminal from successfully receiving the transmitted streaming media data.

[0096] In the cooperative optimization mechanism scenario 1003, the UPF real-time statistical analysis and cooperative optimization mechanism proposed in the embodiment of the present application are implemented in a weak network environment, and then streaming media data is transmitted based on the UDP protocol. During data transmission, the UPF will perform statistics, analysis, and caching of all I-frame data passing through the 5G core network. When the current network determines that the I-frame transmission needs are relatively large or the number of I-frame re-request messages sent from client terminals is increasing, that is, the network is unable to meet the needs, the cooperative optimization mechanism is initiated to optimize data transmission.

[0097] The optimization method is divided into two aspects.

[0098] 1. Bandwidth improvement: Specifically, UPF detects an increase in the current I-frame data transmission volume or an increase in the number of I-frame re-request messages sent from a client terminal, which indicates the possibility of network packet loss due to network congestion, and can ensure data transmission quality by increasing the bandwidth allocated to the user. 2. If the UPF statistics detect that the number of I frames per unit time is relatively large, it may be that the video rate played by the client terminal is relatively high or the screen switching frequency is relatively high. Therefore, the amount of I frame data transmitted can be reduced by using a method to reduce the video rate.

[0099] After testing, it was found that under normal network environment, when streaming media data is transmitted using TCP protocol, the playback effect at the client terminal is relatively good, but when streaming media data is transmitted using UDP protocol, static and delay may occur at the client terminal playback, and based on the experiment, this phenomenon is caused by network packet loss due to the reliability of the UDP protocol itself. In most applications, considering that UDP's transmission efficiency and delay performance are better than TCP protocol, UDP protocol is used for data transmission.

[0100] In order to ensure the applicability of the technical solutions of the embodiments of the present application, the test experiments are also mainly based on the UDP protocol, and in a weak network environment, network packet loss occurs constantly in I-frames, resulting in very poor playback conditions at client terminals and almost no complete screens. However, under the third type of test environment, that is, after introducing the UPF real-time statistical analysis and linked optimization mechanism, data transmission has obvious adaptive capabilities, and even in a weak network environment, it can adjust the network by analyzing the amount of I-frame data transmitted in the network within a unit time in real time, thereby ensuring the quality of video playback at client terminals.

[0101] In summary, the technical solution of the embodiments of this application is aimed at the scenario of multimedia data transmission based on 5G core networks, and proposes a data transmission solution based on UPF capability open technology and network-linked feedback mechanism, with the purpose of ensuring smooth transmission of streaming media data and improving the video playback quality of client terminals, while more effectively supporting product projects such as real-time digital twins, 5G remote operation, and industrial Internet. Compared with conventional technologies, the advantages of the embodiments of this application are that they reduce hardware cost overhead and combine UPF capability open technology with I-frame data statistics, identification, and cache functions, and the entire proposed mechanism has certain reference and application value in the field of 5G communication networks.

[0102] At the same time, the architecture proposed in the embodiments of the present application is divided into two modules: statistical analysis and optimization feedback, both of which have high scalability. The statistical analysis function module of UPF mainly performs identification, statistical analysis, and caching on I-frames of streaming media data, and the module can be further extended to statistical analysis on other data (e.g., B-frames or P-frames).

[0103] Similarly, the technical solution of the embodiment of the present application adopts a solution of increasing the user bandwidth in the optimization feedback step and optimizing it based on a rate adaptation algorithm, but in other embodiments of the present application, reinforcement learning or dynamic adaptation methods can be introduced for further optimization.In addition, in other embodiments of the present application, congestion avoidance and control algorithms can be introduced to reduce the probability of network congestion, and at the same time, more robust routing mechanisms can be introduced to ensure that user terminals can send and receive data smoothly.

[0104] The following describes an apparatus embodiment of the present application, which may be used to implement the network transmission optimization method in the above embodiment of the present application. For details not disclosed in the apparatus embodiment of the present application, please refer to the above embodiment of the network transmission optimization method in the present application.

[0105] FIG. 11 shows a block diagram of a network transmission optimization device according to one embodiment of the present application. The network transmission optimization device can be applied to a network device, which may be, for example, an access network device (e.g., a base station), a core network device (e.g., a UPF), or an MEC server connected to a UPF.

[0106] Referring to FIG. 11, a network transmission optimization device 1100 according to an embodiment of the present application includes a receiving unit 1102, an identifying unit 1104, a statistics unit 1106, and a processing unit 1108.

[0107] The receiving unit 1102 is configured to receive a data stream sent by a data sender and needing to be transmitted to a data receiver; the identification unit 1104 is configured to identify key frames contained in said data stream; The statistics unit 1106 is configured to perform statistics on the key frames to obtain key frame statistics information; and The processing unit 1108 is configured to perform a network optimization operation for the data sink when determining, based on the key frame statistical information, that a network optimization operation for the data sink is required.

[0108] In some embodiments of the present application, based on the above solution, the processing unit 1108 is configured to determine that a network optimization process is required for the data sink based on at least one of the following methods:

[0109] The method further comprises determining that a network optimization process is required for the data receiving side when it is determined based on the key frame statistical information that the number of key frames currently to be transmitted is greater than or equal to a first number threshold; determining that a network optimization process is required for the data receiving side when it is determined based on the key frame statistical information that an increase rate of key frames to be transmitted within a unit time is greater than or equal to a predetermined increase rate threshold; When the key frame statistical information determines that the rate of increase of key frames to be transmitted within a unit time is equal to or greater than a predetermined threshold value of increase rate, and the length of the screen group of the data stream is less than or equal to a predetermined threshold value of length, it is determined that network optimization processing is necessary for the data receiving side.

[0110] In some embodiments of the present application, based on the above solution means, the processing unit 1108 is further configured to determine that a network optimization process is required for the data receiving side when the number of key frame retransmission requests received within a unit time is greater than or equal to a second number threshold.

[0111] In some embodiments of the present application, based on the above solution, the identification unit 1104 is configured to capture packets in the data stream, extract features of the captured packets, and compare the features of the captured packets with features of key frame packets to identify key frames contained in the data stream.

[0112] In some embodiments of the present application, based on the above-mentioned solution, the identification unit 1104 is configured to identify key frames contained in the data stream by comparing at least one of the following packet features: a protocol field in an Internet Protocol (IP) header, a version field in a Real-Time Transmission Control Protocol (RTP) layer, a padding field in the RTP layer, an extension field in the RTP layer, a marker field in the RTP layer, and a network abstraction layer unit type field.

[0113] In some embodiments of the present application, based on the above-mentioned solution, the identification unit 1104 is further configured to, when the key frame is divided into multiple packets for transmission, identify packets belonging to the same key frame according to values ​​of designated fields contained in the captured packets; wherein the designation field includes a start bit and an end bit in a coding protocol layer; a start bit of a first packet of the plurality of packets having a first value and an end bit of a second value; a start bit of a last packet of the plurality of packets is the second value and an end bit of the last packet is the first value; Among the plurality of packets, the start bit and the end bit of the packets other than the first packet and the last packet are all the second value.

[0114] In some embodiments of the present application, based on the above-mentioned solution, the network transmission optimization device 1100 further includes a storage unit configured to identify a key frame contained in the data stream and then store the identified key frame, where when the key frame is divided into multiple packets for transmission, the storage unit stores the identified key frame in a key-value pair manner, with the RTP timestamp of the key frame as a key and the multiple packets as values.

[0115] In some embodiments of the present application, based on the above solution, the storage unit is configured to store key frames identified by a hash table, and is configured to receive a predetermined storage space size for the hash table, and store the most recently received key frames in a queue manner based on the storage space size.

[0116] In some embodiments of the present application, based on the above-mentioned solution, the network transmission optimization device 1100 further includes a discarding unit configured to discard a predetermined number of key frames according to a predetermined discarding policy after identifying key frames contained in the data stream, the statistics unit 1106 is further configured to collect statistics on the number of received key frame retransmission requests, and the processing unit 1108 is further configured to determine a simulated network quality by discarding key frames based on the number of key frame retransmission requests obtained by the statistics.

[0117] In some embodiments of the present application, based on the above solution, the receiving unit 1102 is further configured to perform the step of discarding a predetermined number of key frames according to a predetermined discard policy when the trigger operation is used to turn on the network simulation function, and to stop the process of discarding the key frames when the trigger operation is used to turn off the network simulation function.

[0118] In some embodiments of the present application, based on the above solution, the step of the processing unit 1108 performing a network optimization operation for the data sink includes at least one of the following network optimization methods: reducing the transmission rate of the data stream and increasing the network bandwidth for the data sink.

[0119] In some embodiments of the present application, based on the above-mentioned solution, the processing unit 1108 is further configured, when determining that network optimization processing is required for the data receiving side, to obtain network characteristics between the data sending side and the data receiving side and business characteristics corresponding to the data stream, generate input parameters for a pre-trained network optimization model based on the network characteristics and the business characteristics, the pre-trained network optimization model is obtained by training using network feature samples and business feature samples, input the input parameters into the pre-trained network optimization model, output network optimization parameters, and perform network optimization processing for the data receiving side based on the network optimization parameters.

[0120] FIG. 12 shows a structural diagram of a computer system of an electronic device suitable for implementing an embodiment of the present application.

[0121] It should be noted that the electronic device computer system 1200 shown in FIG. 12 is merely an example and does not limit the functionality and scope of use of the embodiments of the present application.

[0122] As shown in Figure 12, a computer system 1200 may include a central processing unit (CPU) 1201, which performs various appropriate operations and processes according to a program stored in a read-only memory (ROM) 1202 or a program loaded from a storage portion 1208 into a random access memory (RAM) 1203, thereby executing, for example, the methods described above in the above embodiments. The RAM 1203 further stores various programs and data necessary for system operation. The CPU 1201, ROM 1202, and RAM 1203 are connected to one another via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0123] The following components can be connected to the I / O interface 1205: an input section 1206 such as a keyboard and a mouse; an output section 1207 such as a cathode ray tube (CRT) and a liquid crystal display (LCD); a memory section 1208 such as a hard disk; and a communication section 1209 such as a network interface card such as a LAN (Local Area Network) card and a modem. The communication section 1209 performs communication processing via a network, such as the Internet. A driver 1210 is also connected to the I / O interface 1205 as needed. A removable medium 1211 such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory is implemented in the driver 1210 as needed, and a computer program read from the removable medium 1211 is installed in the memory section 1208 as needed.

[0124] In particular, according to an embodiment of the present application, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program is used to perform the method illustrated in the flowchart. In such an embodiment, the computer program is downloaded and installed from a network via the communication unit 1209 and / or installed from removable media 1211. When executed by the central processing unit (CPU) 1201, the computer program performs various functions specific to the system of the present application.

[0125] It should be noted that the computer-readable medium in the embodiments of the present application may be a computer-readable signal medium, a computer-readable storage medium, or any combination of the above. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to, an electrical connection having one or more conductors, a portable computer magnetic disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact magnetic disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may also be a tangible medium that contains or stores a computer program, which may be used by or in combination with an instruction execution system, apparatus, or device. As used herein, a computer-readable signal medium may include a data signal propagating in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. A computer program contained on a computer-readable medium may be transmitted by any suitable medium, including, but not limited to, wireless, wired, etc., or any suitable combination of the above.

[0126] The flowcharts and block diagrams in the figures illustrate possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. Each block in a flowchart or block diagram may represent a module, program segment, or code portion, which includes one or more executable instructions used to implement a given logical function. It should also be noted that in some implementations, the functions depicted in the blocks may occur in a different order than depicted in the figures. For example, two successively shown blocks may actually be executed essentially in parallel, or they may be executed in the reverse order, depending on the functionality involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented by a dedicated hardware-based system that performs a given function or operation, or by a combination of dedicated hardware and a computer program.

[0127] The units described in the embodiments of the present application may be implemented in a software manner or in a hardware manner, and the described units may be installed in a processor, where the names of these units do not constitute limitations on the units themselves in some cases.

[0128] In another aspect, the present application further provides a computer-readable medium, which may be included in the electronic device described in the above embodiment, or may exist independently and not be assembled to the electronic device, and which carries one or more computer programs, which, when executed by the electronic device, cause the electronic device to implement the method in the above embodiment.

[0129] It should be noted that although the foregoing detailed description refers to multiple modules or units of equipment used to perform operations, such division is not mandatory. Indeed, according to embodiments of the present application, the features and functionality of two or more modules or units described above may be embodied in a single module or unit. Conversely, the features and functionality of a single module or unit described above may be further divided and embodied by multiple modules or units.

[0130] From the description of the above embodiments, those skilled in the art can easily understand that the exemplary embodiments described herein can be realized by software, or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, a USB memory, a mobile hard disk, etc.) or a network, and contains multiple instructions to enable an electronic device (such as a personal computer, a server, a touch control terminal, or a network device, etc.) to perform the method according to the embodiments of the present application.

[0131] Other embodiments of the present application will be readily apparent to those skilled in the art after considering the specification and practicing the embodiments disclosed herein. The purpose of this application is to cover any variations, uses, or adaptations of the present application that comply with the general principles of the present application and include common general knowledge or commonly used technical means in the art that are not disclosed herein.

[0132] It should be understood that the present application is not limited to the exact construction described above and illustrated in the drawings, and that various modifications and variations can be made without departing from the scope thereof, which is limited only by the appended claims. [Explanation of symbols]

[0133] 101 Car end 102G Dedicated Network 103 Cloud side 104 User terminal 105 GC 201 Roadside equipment 202 Roadside Processing 204 Rear Road Linkage Platform 205 Terminal Users 301 Cloud gaming client terminal 302 Edge Cloud Platform 303 Edge Cloud Game Server Side 801 I frame identification function 802 I frame header footer packet detection function 803 I frame data storage function 804 I-frame discard function 901 Pushstream side 902G Core Network 903 Streaming Media Server 904 Pull stream side 1001 Normal Network Scene 1002 Weak Net Simulation Scene 1003 Interlocking optimization mechanism scene 1100 Network Transmission Optimizer 1102 receiving unit 1104 Identification Unit 1106 Statistics Unit 1108 Processing Unit 1200 Computer System 1201 Central Processing Unit (CPU) 1202 Dedicated Memory (Read-Only Memory, ROM) 1203 RAM 1204 Bus 1205 Output (Input / Output, I / O) Interface 1206 Input section 1207 Output part 1208 Memory part 1209 Communication section 1210 driver 1211 Possible medium

Claims

1. 1. A network transmission optimization method, the method being performed by an electronic device, comprising: receiving a data stream sent by a data sender and which needs to be transmitted to a data receiver; identifying key frames contained in the data stream; Statisticizing the key frames to obtain key frame statistics; and when it is determined based on the key frame statistical information that a network optimization process is necessary for the data sink, performing a network optimization operation for the data sink.

2. determining that a network optimization process is required for the data sink based on at least one of the following methods: The method further comprises determining that a network optimization process is required for the data receiving side when determining that the number of key frames currently to be transmitted is greater than or equal to a first threshold number based on the key frame statistical information; determining that a network optimization process is required for the data receiving side when it is determined based on the key frame statistical information that an increase rate of key frames to be transmitted within a unit time is greater than or equal to a predetermined increase rate threshold; and 2. The network transmission optimization method of claim 1, wherein when it is determined based on the key frame statistical information that the increase rate of key frames to be transmitted within a unit time is equal to or greater than a predetermined increase rate threshold, and the length of a picture group of the data stream is less than or equal to a predetermined length threshold, it is determined that network optimization processing is necessary for the data receiving side.

3. 2. The network transmission optimization method of claim 1, further comprising: determining that a network optimization process for the data sink is necessary when the number of key frame retransmission requests received within a unit time is greater than or equal to a second number threshold.

4. The step of identifying key frames contained in the data stream comprises: capturing packets in the data stream and extracting characteristics of the captured packets; 2. The method of claim 1, further comprising: identifying key frames contained in the data stream by comparing characteristics of captured packets with characteristics of key frame packets.

5. said step of identifying key frames contained in said data stream by comparing characteristics of captured packets with characteristics of key frame packets, identifying key frames contained in the data stream by comparing at least one of the following packet characteristics:

5. The network transmission optimization method of claim 4, wherein the packet features are a protocol field in an Internet Protocol (IP) header, a version field in a Real-time Transmission Control Protocol (RTP) layer, a padding field in the RTP layer, an extension field in the RTP layer, a marker field in the RTP layer, and a network abstraction layer unit type field.

6. When the key frame is divided into a plurality of packets and transmitted, the method includes: further comprising identifying packets belonging to the same key frame based on values ​​of designated fields contained in the captured packets; the designation field includes a start bit and an end bit in a coding protocol layer; a start bit of a first packet of the plurality of packets having a first value and an end bit of a second value; a start bit of a last packet of the plurality of packets is the second value and an end bit of the last packet is the first value; 2. The network transmission optimization method according to claim 1, wherein the start bit and the end bit of the packets other than the first packet and the last packet among the plurality of packets are both the second value.

7. after identifying key frames contained in the data stream, storing the identified key frames; 2. The network transmission optimization method of claim 1, wherein when the key frame is divided into multiple packets for transmission, the key frame is identified by a key-value pair scheme using a real-time transmission control protocol (RTP) timestamp of the key frame as a key and the multiple packets as values, and the key frame is stored.

8. storing the keyframes identified by the hash table; The network transmission optimization method includes: receiving a predetermined storage space size for the hash table; 8. The network transmission optimization method of claim 7, further comprising: storing the most recently received key frames in a queue manner based on the size of the storage space.

9. after identifying key frames contained in the data stream, discarding a predetermined number of key frames according to a predetermined discarding policy; collecting statistics on the number of received key frame retransmission requests; 2. The network transmission optimization method of claim 1, further comprising: determining a simulated network quality by discarding key frames based on the number of key frame retransmission requests obtained by statistics.

10. receiving a trigger operation on a network simulation configuration item; When the trigger operation is used to turn on a network simulation function, discarding a predetermined number of key frames according to a predetermined discard policy is performed; 10. The network transmission optimization method of claim 9, further comprising: stopping the process of discarding the key frames when the trigger operation is used to turn off a network simulation function.

11. The step of performing a network optimization operation on the data sink includes at least one of the following network optimization methods:

11. The network transmission optimization method according to claim 1, wherein the network optimization scheme is to reduce the transmission rate of the data stream and to increase the network bandwidth to the data receiving side.

12. When it is determined that a network optimization process for the data receiving side is necessary, acquiring network characteristics between the data sending side and the data receiving side and business characteristics corresponding to the data stream; generating input parameters of a pre-trained network optimization model based on the network features and the business features, the pre-trained network optimization model being obtained by training with network feature samples and business feature samples; inputting the input parameters into the pre-trained network optimization model and outputting network optimization parameters; The network transmission optimization method according to any one of claims 1 to 10, further comprising: performing a network optimization process for the data receiving side based on the network optimization parameters.

13. A network transmission optimization device, comprising: a receiving unit configured to receive a data stream sent by a data sender and which needs to be transmitted to a data receiver; an identification unit configured to identify key frames contained in said data stream; a statistics unit configured to perform statistics on the key frames to obtain key frame statistics information; a processing unit configured to perform a network optimization operation on the data sink when determining that a network optimization process is necessary for the data sink based on the key frame statistical information.

14. The processing unit is further configured to determine that a network optimization process is required for the data sink based on at least one of the following methods: The method further comprises determining that a network optimization process is required for the data receiving side when determining that the number of key frames currently to be transmitted is greater than or equal to a first threshold number based on the key frame statistical information; determining that a network optimization process is required for the data receiving side when it is determined based on the key frame statistical information that an increase rate of key frames to be transmitted within a unit time is greater than or equal to a predetermined increase rate threshold; and 14. The network transmission optimization device according to claim 13, wherein when it is determined based on the key frame statistical information that the increase rate of key frames to be transmitted within a unit time is equal to or greater than a predetermined increase rate threshold, and the length of a picture group of the data stream is less than or equal to a predetermined length threshold, it determines that network optimization processing is necessary for the data receiving side.

15. 14. The network transmission optimization device of claim 13, wherein the processing unit is further configured to determine that a network optimization process for the data sink is necessary when the number of key frame retransmission requests received within a unit time is greater than or equal to a second number threshold.

16. 14. The network transmission optimization device of claim 13, wherein the identification unit is configured to capture packets in the data stream, extract features of the captured packets, and compare the features of the captured packets with features of key frame packets to identify key frames contained in the data stream.

17. When the key frame is divided into a plurality of packets for transmission, the identification unit is further configured to identify packets belonging to the same key frame based on values ​​of designated fields contained in the captured packets; the designation field includes a start bit and an end bit in a coding protocol layer; a start bit of a first packet of the plurality of packets having a first value and an end bit of a second value; a start bit of a last packet of the plurality of packets is the second value and an end bit of the last packet is the first value; 14. The network transmission optimization device according to claim 13, wherein the start bit and the end bit of the packets other than the first packet and the last packet of the plurality of packets are all the second value.

18. 13. A computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the network transmission optimization method of any one of claims 1 to 12.

19. An electronic device, one or more processors; and a memory adapted to store one or more computer programs which, when executed by the one or more processors, cause the electronic device to implement the network transmission optimization method of any one of claims 1 to 12.

20. 13. A computer program product, the computer program product including a computer program, the computer program being stored in a computer-readable storage medium, the computer program being read by a processor of an electronic device and executed by the processor to cause the electronic device to perform the network transmission optimization method of any one of claims 1 to 12.

Images