Video transmission method and device based on neural network coding, and communication equipment

Abstract

The invention discloses a video transmission method and device based on neural network coding and communication equipment, and the method comprises the steps: obtaining a transmission bandwidth, the transmission bandwidth is determined based on a first parameter, a second parameter, a third parameter and a fourth parameter, the first parameter represents the data size of a neural network, and the second parameter represents the data size of the neural network; the second parameter represents the video coding frame number of the previous time sequence neural network, the third parameter represents the frame rate of the previous time sequence video, and the fourth parameter represents the network one-way transmission delay; and allocating bandwidth resources based on the transmission bandwidth, and transmitting a neural network by using the bandwidth resources.

Description

Technical Field

[0001] The present application relates to video transmission technology in the wireless field, and in particular to a video transmission method and device based on neural network coding, and a communication device. Background Art

[0002] Video coding based on neural networks is a cutting-edge video coding technology. In video coding technology based on neural networks, each frame in the video is encoded into a neural network according to a certain unit (i.e., the number of video coding frames), and a neural network is used to represent a section of content in the video. The service server transmits the neural network to the terminal, and the terminal can decode a section of content in the corresponding video from the neural network. How to ensure that the transmission of the neural network has a high quality is a technical problem that needs to be solved at present. Summary of the invention

[0003] In order to solve the above technical problems, the embodiments of the present application provide a video transmission method and device based on neural network coding, a communication device, a chip, and a computer-readable storage medium.

[0004] The video transmission method based on neural network coding provided in the embodiment of the present application includes:

[0005] The wireless access network acquires a transmission bandwidth, where the transmission bandwidth is determined based on a first parameter, a second parameter, a third parameter, and a fourth parameter, wherein the first parameter represents the amount of data of the neural network, the second parameter represents the number of video encoding frames of the previous time-series neural network, the third parameter represents the frame rate of the previous time-series video, and the fourth parameter represents the network unidirectional transmission delay;

[0006] The wireless access network allocates bandwidth resources based on the transmission bandwidth and uses the bandwidth resources to transmit the neural network.

[0007] The video transmission method based on neural network coding provided in the embodiment of the present application includes:

[0008] The service server determines the transmission bandwidth based on the first parameter, the second parameter, the third parameter and the fourth parameter, wherein the first parameter represents the data volume of the neural network, the second parameter represents the number of video encoding frames of the previous time sequence neural network, the third parameter represents the frame rate of the previous time sequence video, and the fourth parameter represents the network one-way transmission delay;

[0009] The service server indicates the transmission bandwidth to the wireless access network, and the transmission bandwidth is used by the wireless access network to allocate bandwidth resources for transmitting the transmission neural network.

[0010] The video transmission device based on neural network coding provided in the embodiment of the present application is applied to a wireless access network, and the device includes:

[0011] An acquisition unit, used to acquire a transmission bandwidth, wherein the transmission bandwidth is determined based on a first parameter, a second parameter, a third parameter, and a fourth parameter, wherein the first parameter represents the amount of data of a neural network, the second parameter represents the number of video encoding frames of a previous time-series neural network, the third parameter represents the frame rate of a previous time-series video, and the fourth parameter represents a one-way transmission delay of the network;

[0012] an allocating unit, configured to allocate bandwidth resources based on the transmission bandwidth;

[0013] A sending unit is used to transmit the neural network using the bandwidth resources.

[0014] The video transmission device based on neural network coding provided in the embodiment of the present application is applied to a service server, and the device includes:

[0015] A determination unit, configured to determine a transmission bandwidth based on a first parameter, a second parameter, a third parameter, and a fourth parameter, wherein the first parameter represents the amount of data of a neural network, the second parameter represents the number of video encoding frames of a previous time-series neural network, the third parameter represents the frame rate of a previous time-series video, and the fourth parameter represents a one-way transmission delay of a network;

[0016] A sending unit is used to indicate the transmission bandwidth to the wireless access network, and the transmission bandwidth is used by the wireless access network to allocate bandwidth resources for transmitting the transmission neural network.

[0017] The communication device provided in an embodiment of the present application includes: a processor and a memory, the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute any one of the above-mentioned video transmission methods based on neural network coding.

[0018] The chip provided in an embodiment of the present application includes: a processor, which is used to call and run a computer program from a memory, so that a device equipped with the chip executes any of the above-mentioned video transmission methods based on neural network coding.

[0019] The computer-readable storage medium provided in an embodiment of the present application is used to store a computer program, and the computer program enables a computer to execute any one of the above-mentioned video transmission methods based on neural network coding.

[0020] The technical solution of the embodiment of the present application proposes a video transmission method based on neural network coding, and clarifies that the transmission bandwidth is determined based on the first parameter, the second parameter, the third parameter and the fourth parameter, wherein the first parameter represents the amount of data of the neural network, the second parameter represents the number of video coding frames of the previous time-series neural network, the third parameter represents the frame rate of the previous time-series video, and the fourth parameter represents the network unidirectional transmission delay, so that the wireless access network can more accurately allocate bandwidth resources for the transmission of the neural network based on the determined transmission bandwidth, improve resource utilization, and ensure stable transmission of the neural network. In addition, due to the use of video coding technology based on neural networks, the advantages of stable bandwidth during neural network transmission can be fully utilized to better realize the scheduling of bandwidth resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1-1 It is a schematic diagram of the coding algorithm included in the video coding standard;

[0022] Figure 1-2 It is a schematic diagram of the playback order of each frame in the video and the corresponding encoding order;

[0023] Figure 2 This is a schematic diagram of the sizes of I frames, P frames, and B frames;

[0024] Figure 3 is a schematic diagram of a neural network used for video encoding;

[0025] Figure 4 It is a schematic diagram of the change of the amount of data of the neural network over time;

[0026] Figure 5 It is a flowchart of a video transmission method based on neural network coding provided in an embodiment of the present application;

[0027] Figure 6 is a flow chart of calculating the value of B by a wireless access network provided in an embodiment of the present application;

[0028] Figure 7 This is a flow chart of a wireless access network identifying a first timing sequence provided by an embodiment of the present application;

[0029] Figure 8 Schematic diagram 1 of the structure of a video transmission device based on neural network coding provided in an embodiment of the present application;

[0030] Fig. 9 The structure of the video transmission device based on neural network coding provided by the embodiment of the present application is shown in FIG. Figure 2 ;

[0031] Fig.10 is a schematic structural diagram of a communication device provided in an embodiment of the present application;

[0032] Fig.11 It is a schematic structural diagram of the chip of an embodiment of the present application. DETAILED DESCRIPTION

[0033] The following will describe the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0034] It should be noted that in the embodiments of the present application, the term "and / or" is only a description of the association relationship of the associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the embodiments of the present application, the character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0035] In the description of the embodiments of the present application, the term "corresponding" may indicate that there is a direct or indirect correspondence relationship between two things, or may indicate that there is an associative relationship between the two things.

[0036] Video is the carrier of business content and is composed of a series of frames in sequence. At present, video coding is usually implemented based on video coding standards. International video coding standards mainly include H.264, H.265, H.266, etc. Domestic video coding standards include AVS2, AVS3, etc. Figure 1-1 As shown in the figure, the video coding standard includes prediction coding, transform coding, entropy coding and other algorithms. Based on these algorithms, encoding is performed in frames to obtain a series of I frames, P frames and B frames. Among them, I frames are key frames that can be decoded independently, P frames are prediction frames that need to rely on the previous I frame or P frame for decoding, and B frames are bidirectional prediction frames that need to rely on both the previous and the following I frame or P frame for decoding. Figure 1-2 As shown, Figure 1-2 It shows the playback order of each frame in the video and the corresponding encoding order. Corresponding to the encoding order, Figure 1-2 The relationship between I-frames, P-frames, and B-frames is also shown (the arrowed curves in the figure indicate this relationship), which indicates the coding relationship between frames. Taking the P-frame (play number 4) as an example, the coding of this frame depends on its associated I-frame (play number 1).

[0037] When video encoding is performed based on the current video encoding standard, the resulting I-frame, P-frame, and B-frame have different sizes. Figure 2 As shown, Figure 2The size of I frames, P frames, and B frames is shown. I frames are larger, followed by P frames, and then B frames. When I frames, P frames, and B frames are transmitted, the traffic in the network will be unstable. Since I frames are larger and P / B frames are smaller, there will be burst traffic near I frames, causing network resource congestion and reduced service experience, while the traffic near P / B frames is smaller.

[0038] With the widespread application of deep learning in the field of image processing, video coding can be implemented based on neural networks. Video coding based on neural networks (abbreviated as neural network coding) can also achieve coding efficiency based on video coding standards or even higher. Neural network coding can be encoded in units of pixels or frames. For example, encoding in units of frames can be used to encode each frame in the video into a neural network according to a certain unit (i.e., the number of video encoding frames). A neural network is used to represent a section of content in the video. The service server transmits the neural network to the terminal, and the terminal can decode a section of content in the corresponding video from the neural network. Figure 3 As shown, Figure 3 A neural network for video coding is shown. The neural network consists of an embedding layer (Embedding), a multilayer perceptron (Multilayer Perceptron, MLP), a convolutional layer (Conv), etc. The input of the neural network can be a time sequence t, and the output is a group of frames in the video corresponding to the time sequence. This group of frames is a section of content in the video represented by the neural network. In some embodiments, the input of the neural network is a pixel coordinate, denoted as (x, y, t), where x and y represent the horizontal and vertical coordinates of the pixel, respectively, and t represents the time sequence. The output of the neural network is the pixel value corresponding to the pixel with coordinates (x, y) at the time sequence t. In other embodiments, the input of the neural network is a frame index, denoted as t, t can represent both a frame index and a time sequence, and the output of the neural network is a group of frame images corresponding to the frame index / time sequence t.

[0039] For video encoding based on neural networks, after the structure and parameters of the neural network are determined, the number of video encoding frames of the neural network is set, that is, how many frames of content in the video the neural network represents. Optionally, the neural network can be compressed (compression methods include but are not limited to pruning, quantization, etc.), and the neural network (or the compressed neural network) is transmitted to the terminal, and the terminal can decode a section of content in the video from the neural network. When the neural network (or the compressed neural network) is transmitted, the amount of data of the neural network (or the compressed neural network) is relatively stable, such as Figure 4 As shown, there is no burst traffic, which is more suitable for video services such as on-demand. Therefore, video encoding based on neural networks is beneficial. However, how to ensure that the transmission of neural networks has high quality needs to be optimized. To this end, the following technical solutions of the embodiments of the present application are proposed.

[0040] According to the technical solution of the embodiment of the present application, the Radio Access Network (RAN) can obtain the transmission bandwidth required for the transmission of the neural network, so that bandwidth resources can be allocated for the transmission of the neural network more accurately according to the acquired transmission bandwidth, thereby improving resource utilization and ensuring stable transmission of the neural network.

[0041] Figure 5 is a flow chart of a video transmission method based on neural network coding provided in an embodiment of the present application, such as Figure 5 As shown, the method comprises the following steps:

[0042] Step 501: The wireless access network obtains a transmission bandwidth, which is determined based on a first parameter, a second parameter, a third parameter and a fourth parameter, wherein the first parameter represents the amount of data of the neural network, the second parameter represents the number of video encoding frames of the previous time-series neural network, the third parameter represents the frame rate of the previous time-series video, and the fourth parameter represents the network one-way transmission delay.

[0043] Here, the transmission bandwidth refers to the bandwidth used to transmit the neural network, and the relevant understanding of the neural network can refer to the above description.

[0044] For video services, after receiving N1 (N1≥1) frames of video data sent by the network (usually wireless access network), the terminal caches the N1 frames of video data and starts playing the N1 frames of video data. At the same time, the terminal will also receive N2 (N2≥1) frames of video data sent by the network in the next sequence in real time. If the terminal receives and decodes the N2 frames of video data in the next sequence after the last frame of video data in the N1 frames of video data is played, it will not affect the normal playback of the terminal. The values ​​of N1 and N2 can be the same or different.

[0045] Here, N1 frame video data and N2 frame video data correspond to different timings. The sending of N1 frame video data is achieved by sending a neural network in one timing. Similarly, the sending of N2 frame video data is achieved by sending a neural network in the next timing. The neural networks sent at different timings can be different. For example, the values ​​of the parameters of the neural network are different. Different neural networks represent the encoding results of different video data. For example, neural network 1 represents the encoding result of N1 frame video data, and neural network 2 represents the encoding result of N2 frame video data. N1 frame video data can be decoded from neural network 1, and N2 frame video data can be decoded from neural network 2.

[0046] For example, the neural network encodes the video in units of 150 frames, that is, the number of video encoding frames of the neural network is 150 frames, or the neural network represents the encoding result of 150 frames of video data, where the frame rate of the video is 30fps, and 150 frames require 150÷30=5s to play. After the terminal receives the 150 frames of video data of the first time sequence (corresponding to the neural network of the first time sequence), it starts to play these 150 frames of video data, and the transmission delay left for the 150 frames of video data of the next time sequence (i.e., the neural network of the next time sequence) is T+5s, where T is the one-way network delay, T=RTT / 2, and RTT is the two-way network delay. Here, the one-way network delay refers to the end-to-end delay, which can be the delay between the service server and the terminal, or the delay between the wireless access network and the terminal. Therefore, the transmission delay requirement of the video data depends on the duration of the video time playback of the previous time sequence. After the transmission delay of the neural network is determined, since the structure and number of parameters of the neural network are determined, the data volume of the neural network is determined, and the transmission bandwidth of the next time series neural network can be predicted based on the transmission delay of the neural network and the data volume of the neural network.

[0047] It should be noted that the first time sequence is the time sequence corresponding to the first transmission of the neural network.

[0048] The transmission bandwidth is determined based on a first parameter, a second parameter, a third parameter and a fourth parameter, wherein the first parameter represents the amount of data of the neural network (i.e., the amount of data of the parameters of the neural network), the second parameter represents the number of video encoding frames of the previous time series neural network, the third parameter represents the frame rate of the previous time series video, and the fourth parameter represents the network one-way transmission delay.

[0049] Specifically, the transmission bandwidth is determined by the following formula (1):

[0050] B=D nn ÷[(N s-1 ÷f s-1 )+T] (1)

[0051] Among them, B represents the transmission bandwidth, D nn Represents the first parameter (indicating the amount of data in the neural network), N s-1 represents the second parameter (indicating the number of video encoding frames of the previous temporal neural network), f s-1 represents the third parameter (indicating the frame rate of the previous time-series video), and T represents the fourth parameter (indicating the network one-way transmission delay).

[0052] If the user updates the video source or changes the encoding algorithm, then the first parameter (D nn ), the second parameter (N s-1 ), the third parameter (f s-1) will be updated. For example, when encoding a video with a higher peak signal-to-noise ratio (PSNR) and higher quality, the number of parameters of the neural network will increase, so the first parameter (D nn ) will become larger. For another example, when encoding smoother videos, the third parameter (f s-1 ) will become larger. For example, if the video delay required by the user becomes smaller, the second parameter (N s-1 ) will become smaller. nn ), the second parameter (N s-1 ), the third parameter (f s-1 ) is updated, the transmission bandwidth (B) needs to be recalculated according to the above formula (1).

[0053] Before sending the neural network to the terminal, the wireless access network needs to know the transmission bandwidth, which is used for allocating bandwidth resources. The wireless access network can know the transmission bandwidth in any of the following ways.

[0054] Mode 1: The wireless access network obtains the transmission bandwidth from the service server; the transmission bandwidth is determined by the service server based on a first parameter, a second parameter, a third parameter, and a fourth parameter, wherein the first parameter, the second parameter, and the third parameter are obtained at the application layer of the service server, and the fourth parameter is measured by the service server. Here, when the transmission bandwidth is updated, the service server indicates the updated transmission bandwidth to the updated transmission bandwidth, and the wireless access network obtains the updated transmission bandwidth from the service server.

[0055] Here, the service server determines the transmission bandwidth based on the first parameter, the second parameter, the third parameter and the fourth parameter, and the service server indicates the transmission bandwidth to the wireless access network, so that the wireless access network allocates bandwidth resources for transmitting the transmission neural network.

[0056] Specifically, the service server can calculate the value of the transmission bandwidth (B) according to the above formula (1), and then notify the wireless access network for the wireless access network to allocate bandwidth resources. Wherein, T is the end-to-end delay, such as the transmission delay between the service server and the terminal or the transmission delay between the wireless access network and the terminal. The service server can measure the network two-way transmission delay (RTT), and T=RTT / 2 can be calculated based on RTT. The technical solution of the embodiment of the present application does not limit the way in which the service server measures RTT. For example, the RTT can be measured in the following way: the service server sends a downlink data packet and adds a sending timestamp t1 to the downlink data packet; after receiving the downlink data packet, the user plane function (UPF) sends the downlink data packet to the UE; after receiving the downlink data packet, the terminal sends the corresponding uplink data packet to the UPF; the UPF sends the uplink data packet to the service server; after receiving the uplink data packet, the service server records the receiving timestamp t2 of the uplink data packet, and the RTT can be calculated based on t2 and t1. For D nn , N s-1 , f s-1 The value of can be obtained at the application layer of the service server. When the value of B calculated by the service server is updated, it is notified to the wireless access network side. The service server can determine whether the value of B is updated by comparing the value of B in the current sequence with the value of B in the previous sequence. For example, if the value of B in the previous sequence is 200M and the value of B in the current sequence is 190M, then the value of B in the current sequence is updated compared to the previous sequence, and it needs to be notified to the wireless access network side.

[0057] It should be noted that usually the N of the first time series (initial time series) is s-1 is 0 or close to 0. According to the above formula (1), this will lead to a larger value of B. In the subsequent timing, the value of B will tend to be stable.

[0058] Mode 2: The wireless access network determines the transmission bandwidth based on the first parameter, the second parameter, the third parameter and the fourth parameter, wherein the first parameter, the second parameter and the third parameter are sent to the wireless access network by the service server, and the fourth parameter is measured by the wireless access network. Here, when at least one of the first parameter, the second parameter, the third parameter and the fourth parameter is updated, the wireless access network re-determines the transmission bandwidth based on the updated parameter. Here, the network one-way transmission delay is the one-way transmission delay between the wireless access network and the terminal.

[0059] Here, the wireless access network can calculate the value of the transmission bandwidth (B) according to the above formula (1) for the allocation of bandwidth resources. Wherein, T is the end-to-end delay, such as the transmission delay between the wireless access network and the terminal. The wireless access network can measure the downlink air interface delay as T. The downlink air interface delay refers to the delay from the top layer of the wireless access network to the top layer of the terminal. The top layer here refers to the top layer of the user plane protocol stack. Taking 5G as an example, the top layer is the Service Data Adaptation Protocol (SDAP) layer. The technical solution of the embodiment of the present application does not limit the way in which the terminal measures the downlink air interface delay. For example, the downlink air interface delay can be measured in the following way: the wireless access network (which can also be described as a base station) sends a downlink data packet to the terminal and stamps the downlink data packet with a sending timestamp t1; after the terminal receives the downlink data packet, it uses the same QoS parameters as the downlink data packet to send an uplink data packet to the wireless access network; after the wireless access network receives the uplink data packet, it records the receiving timestamp t2, and the downlink air interface delay can be calculated based on t2 and t1. For D nn , N s-1 , f s-1 The value of can be sent to the terminal by the service server. nn , N s-1 , f s-1 When the value of any parameter is updated, the value of B will be recalculated.

[0060] like Figure 6 As shown, Figure 6 The flowchart of calculating the value of B by the wireless access network is shown, including the following steps:

[0061] 1. Data packets are exchanged between the wireless access network and the terminal to measure the downlink air interface delay T.

[0062] 2. The service server sends parameter D to the wireless access network nn , N s-1 , f s-1 .

[0063] 3. Wireless access network based on T, D nn 、N s-1 and f s-1 , and calculate the value of B by combining the above formula (1).

[0064] Step 502: The wireless access network allocates bandwidth resources based on the transmission bandwidth, and uses the bandwidth resources to transmit the neural network.

[0065] It should be noted that the transmission neural network refers to the parameters of the transmission neural network.

[0066] Here, the bandwidth of the allocated bandwidth resources needs to satisfy the transmission bandwidth obtained in the above step 501 as much as possible. In some implementations, the bandwidth of the allocated bandwidth resources is not less than the transmission bandwidth obtained in the above step 501.

[0067] In order to avoid users having to wait too long when the service is connected, the wireless access network needs to identify the video data of the first sequence of the video service and assign it a higher priority transmission guarantee, such as allocating larger bandwidth resources in a short period of time to give priority to its transmission. In addition, for short videos, which have low latency and unpredictable user viewing time, the application layer of the service server should not set too large a number of video encoding frames in the first sequence, or even in the entire sequence, to avoid wasting transmission resources.

[0068] The wireless access network identifies the first time sequence of video data and processes it as follows:

[0069] 1) The service server sends a data packet carrying a neural network to the user plane function, and the wireless access network receives the data packet carrying the neural network sent by the user plane function, the data packet carries a first identifier or a second parameter, the first identifier is used to indicate whether the transmission timing of the neural network is the first timing, and the value of the second parameter is used to determine whether the transmission timing of the neural network is the first timing.

[0070] Here, the data packet of the neural network is a data packet obtained by sealing the parameters of the neural network.

[0071] 2) The wireless access network identifies the first identifier or the second parameter in the data packet.

[0072] 3) If the recognition result indicates that the transmission timing of the neural network is the first timing, the data packet is mapped to a data bearer with a first priority, and a first bandwidth resource is allocated to the data packet. The data packet is transmitted to the terminal using the first bandwidth resource. The first priority is not lower than the preset priority, and the first bandwidth resource is not less than the preset bandwidth resource.

[0073] Here, the data bearer of the first priority may be, for example, a data bearer with the highest priority among all data bearers. The first bandwidth resource may be, for example, a maximum bandwidth resource supported by the terminal.

[0074] 4) If the identification result indicates that the transmission timing of the neural network is not the first timing, the wireless access network allocates a second bandwidth resource to the neural network based on the transmission bandwidth obtained in step 501 above, and transmits the neural network to the terminal using the second bandwidth resource.

[0075] The transmission path of the data packet is service server → user plane function → wireless access network → terminal, wherein the service server is to clarify whether the transmission timing of the neural network (that is, the transmission timing of the video data) is the first timing. When the service server sends the data packet carrying the neural network to the user plane function, the first identifier or the second parameter is encapsulated in the RTP header of the data packet; the value of the first identifier is 0 for the first timing, and the value is 1 for non-first timing; the second parameter (N s-1 ) can refer to the above scheme, N s-1 When the value of is 0 or less than the preset value, it represents the first timing, otherwise it represents a non-first timing. The user plane function has a functional module for identifying the first identifier or the second parameter. After the user plane function receives the data packet sent by the service server, it identifies the first identifier or the second parameter to determine whether the transmission timing of the neural network (that is, the video data) carried in the data packet is the first timing, and encapsulates the identification result in the NG-U header of the data packet and sends it to the wireless access network. The NG-U header of the data packet can also encapsulate the first identifier or the second parameter. Alternatively, the user plane function does not need to identify the first identifier or the second parameter, but directly encapsulates the first identifier or the second parameter into the NG-U header and transparently transmits it to the wireless access network. The wireless access network has a functional module for identifying the first identifier or the second parameter. After the wireless access network receives the data packet sent by the user plane function, it identifies the first identifier or the second parameter to determine whether the transmission timing of the neural network (that is, the video data) carried in the data packet is the first timing. If it is the first timing, the data packet is mapped to the highest priority data bearer, and the maximum bandwidth resource is allocated to the data packet, so that the data packet can be transmitted to the terminal first.

[0076] like Figure 7 As shown, Figure 7 The flowchart of the wireless access network identifying the first timing sequence is shown, comprising the following steps:

[0077] 1. The service server encapsulates parameter N in the RTP header of the data packet s-1 , through N s-1 The transmission timing of the marked data packet is the first timing.

[0078] Here, the data packet refers to a data packet carrying a neural network (ie, video data). The transmission timing of the data packet can be understood as the transmission timing of the neural network (ie, video data).

[0079] 2. The business server sends the data packet marked with the first timing sequence to the UPF.

[0080] 3. UPF Identification N s-1 , when N s-1If the value of is 0, the transmission sequence of the data packet is determined to be the first sequence; the identification result is encapsulated in the NG-U header of the data packet. Alternatively, UPF sets the parameter N s-1 Encapsulated in an NG-U header.

[0081] Exemplarily, the recognition result is characterized by a first identifier, which may be, but is not limited to, named as a sequential characteristic (Sequential Characteristic), and its value may be 0 or 1, where a value of 0 represents the first time sequence, and a value of 1 represents a non-first time sequence. The position of the sequential characteristic (Sequential Characteristic) in the NG-U header may refer to the following Table 1.

[0082] Table 1

[0083]

[0084] For example, the parameter N s-1 When the value is 0, it represents the first time series, otherwise it represents a non-first time series. s-1 The position in the NG-U header can be referred to as shown in the following Table 2.

[0085] Table 2

[0086]

[0087] 4. UPF sends the data packet to RAN.

[0088] 5. The RAN SDAP layer identifies the first identifier or parameter N in the data packet s-1 , determine the transmission timing of the data packet as the first timing, map the data packet to the highest priority data bearer, and allocate the maximum bandwidth resource to the data packet.

[0089] 6. RAN sends the data packet to the terminal using the bandwidth resources allocated by it.

[0090] The technical solution of the embodiment of the present application proposes a new solution for video transmission based on neural network coding. The service server calculates the transmission bandwidth according to the coding parameters (i.e., the first parameter, the second parameter, and the third parameter) and the network transmission delay, and notifies the transmission bandwidth to the wireless access network, or the wireless access network calculates the transmission bandwidth according to the coding parameters and the network transmission delay, thereby obtaining a more accurate bandwidth, which is conducive to the wireless access network to accurately schedule resources, improve the utilization of wireless resources, and avoid congestion caused by traffic bursts. In addition, a mechanism is designed to enable the wireless access network to identify the first time sequence of video data, thereby allocating larger bandwidth resources to it, and improving the user experience of the service.

[0091] Figure 8 Schematic diagram of the structure of a video transmission device based on neural network coding provided by an embodiment of the present application, which is applied to a wireless access network, such as Figure 8 As shown, the video transmission device based on neural network coding includes:

[0092] An acquisition unit 801 is used to acquire a transmission bandwidth, where the transmission bandwidth is determined based on a first parameter, a second parameter, a third parameter, and a fourth parameter, wherein the first parameter represents the amount of data of a neural network, the second parameter represents the number of video encoding frames of a previous time-series neural network, the third parameter represents the frame rate of a previous time-series video, and the fourth parameter represents a one-way transmission delay of the network;

[0093] An allocating unit 802, configured to allocate bandwidth resources based on the transmission bandwidth;

[0094] The sending unit 803 is used to transmit the neural network using the bandwidth resources.

[0095] In some embodiments, the acquisition unit 801 is specifically used to: acquire a transmission bandwidth from a service server; the transmission bandwidth is determined by the service server based on the first parameter, the second parameter, the third parameter and the fourth parameter, wherein the first parameter, the second parameter and the third parameter are obtained at the application layer of the service server, and the fourth parameter is measured by the service server.

[0096] In some implementations, the acquisition unit 801 is further configured to acquire an updated transmission bandwidth from the service server when the transmission bandwidth is updated.

[0097] In some implementations, the acquisition unit 801 is specifically used to: determine the transmission bandwidth based on the first parameter, the second parameter, the third parameter and the fourth parameter, wherein the first parameter, the second parameter and the third parameter are sent by the service server to the wireless access network, and the fourth parameter is measured by the wireless access network.

[0098] In some implementations, the acquiring unit 801 is further configured to, when at least one of the first parameter, the second parameter, the third parameter, and the fourth parameter is updated, re-determine the transmission bandwidth based on the updated parameter.

[0099] In some implementations, the network one-way transmission delay is a one-way transmission delay between the wireless access network and the terminal.

[0100] In some implementations, the apparatus further includes: a receiving unit 804 and a processing unit 805;

[0101] The receiving unit 804 is used to receive a data packet carrying a neural network sent by a user plane function, the data packet carrying a first identifier or the second parameter, the first identifier is used to indicate whether the transmission timing of the neural network is the first timing, and the value of the second parameter is used to determine whether the transmission timing of the neural network is the first timing;

[0102] The processing unit 805 is configured to identify the first identifier or the second parameter in the data packet; if the identification result indicates that the transmission timing of the neural network is the first timing, the data packet is mapped to a data bearer with a first priority, and a first bandwidth resource is allocated to the data packet;

[0103] The sending unit 803 is further configured to transmit the data packet to the terminal by using the first bandwidth resources, the first priority is not lower than a preset priority, and the first bandwidth resources are not less than a preset bandwidth resources.

[0104] In some implementations, if the identification result indicates that the transmission timing of the neural network is not the first timing, the sending unit 803 allocates a second bandwidth resource to the neural network based on the acquired transmission bandwidth, and transmits the neural network to the terminal using the second bandwidth resource.

[0105] In some implementations, the first identifier or the second parameter is encapsulated by the user plane function in the NG-U header of the data packet.

[0106] In some implementations, the data packet is sent to the user plane function by a service server, and the service server encapsulates the first identifier or the second parameter in an RTP header of the data packet.

[0107] In some implementations, the transmission bandwidth is determined by the following formula:

[0108] B=D nn ÷[(N s-1 ÷f s-1 )+T]

[0109] Wherein, B represents the transmission bandwidth, D nn represents the first parameter, N s-1 represents the second parameter, f s-1 represents the third parameter, and T represents the fourth parameter.

[0110] Those skilled in the art should understand that Figure 8 The implementation functions of each unit in the video transmission device based on neural network coding shown can be understood by referring to the relevant description of the aforementioned method. Figure 8The functions of each unit in the video transmission device based on neural network coding shown can be implemented by a program running on a processor, or by a specific logic circuit.

[0111] Fig. 9 The structure of the video transmission device based on neural network coding provided by the embodiment of the present application is shown in FIG. Figure 2 , applied to business servers, such as Fig. 9 As shown, the video transmission device based on neural network coding includes:

[0112] A determination unit 901 is used to determine a transmission bandwidth based on a first parameter, a second parameter, a third parameter and a fourth parameter, wherein the first parameter represents the amount of data of a neural network, the second parameter represents the number of video encoding frames of a previous time-series neural network, the third parameter represents the frame rate of a previous time-series video, and the fourth parameter represents a one-way transmission delay of the network;

[0113] The sending unit 902 is used to indicate the transmission bandwidth to the wireless access network, and the transmission bandwidth is used by the wireless access network to allocate bandwidth resources for transmitting the transmission neural network.

[0114] In some implementations, the first parameter, the second parameter, and the third parameter are obtained at an application layer of the service server, and the fourth parameter is measured by the service server.

[0115] In some implementations, the sending unit 902 is further configured to indicate an updated transmission bandwidth to the updated transmission bandwidth when the transmission bandwidth is updated.

[0116] In some embodiments, the sending unit 902 is further used to send a data packet carrying a neural network to a user plane function, wherein the data packet carries a first identifier or the second parameter, wherein the first identifier is used to indicate whether the transmission timing of the neural network is the first timing, and the value of the second parameter is used to determine whether the transmission timing of the neural network is the first timing.

[0117] In some implementations, the service server encapsulates the first identifier or the second parameter in an RTP header of the data packet.

[0118] In some implementations, the determining unit 901 is specifically configured to determine the transmission bandwidth based on the following formula:

[0119] B=D nn ÷[(N s-1 ÷f s-1 )+T]

[0120] Wherein, B represents the transmission bandwidth, Dnn represents the first parameter, N s-1 represents the second parameter, f s-1 represents the third parameter, and T represents the fourth parameter.

[0121] Those skilled in the art should understand that Fig. 9 The implementation functions of each unit in the video transmission device based on neural network coding shown can be understood by referring to the relevant description of the aforementioned method. Fig. 9 The functions of each unit in the video transmission device based on neural network coding shown can be implemented by a program running on a processor, or by a specific logic circuit.

[0122] Fig.10 It is a schematic structural diagram of a communication device 1000 provided in an embodiment of the present application. Fig.10 The communication device 1000 shown includes a processor 1010, and the processor 1010 can call and run a computer program from a memory to implement the method in the embodiment of the present application.

[0123] Alternatively, if Fig.10 As shown, the communication device 1000 may further include a memory 1020. The processor 1010 may call and run a computer program from the memory 1020 to implement the method in the embodiment of the present application.

[0124] The memory 1020 may be a separate device independent of the processor 1010 , or may be integrated into the processor 1010 .

[0125] Alternatively, if Fig.10 As shown, the communication device 1000 may further include a transceiver 1030, and the processor 1010 may control the transceiver 1030 to communicate with other devices, specifically, may send information or data to other devices, or receive information or data sent by other devices.

[0126] The transceiver 1030 may include a transmitter and a receiver. The transceiver 1030 may further include an antenna, and the number of antennas may be one or more.

[0127] Optionally, the communication device 1000 may specifically be a device (such as a base station) corresponding to the wireless access network of the embodiment of the present application, and the communication device 1000 may implement the corresponding processes implemented by the wireless access network in each method of the embodiment of the present application. For the sake of brevity, they will not be repeated here.

[0128] Optionally, the communication device 1000 may specifically be a service server of an embodiment of the present application, and the communication device 1000 may implement corresponding processes implemented by the service server in various methods of the embodiment of the present application, which will not be described in detail here for the sake of brevity.

[0129] Fig.11 It is a schematic structural diagram of the chip of an embodiment of the present application. Fig.11 The chip 1100 shown includes a processor 1110, which can call and run a computer program from a memory to implement the method in the embodiment of the present application.

[0130] Alternatively, if Fig.11 As shown, the chip 1100 may further include a memory 1120. The processor 1110 may call and run a computer program from the memory 1120 to implement the method in the embodiment of the present application.

[0131] The memory 1120 may be a separate device independent of the processor 1110 , or may be integrated into the processor 1110 .

[0132] Optionally, the chip 1100 may further include an input interface 1130. The processor 1110 may control the input interface 1130 to communicate with other devices or chips, and specifically, may obtain information or data sent by other devices or chips.

[0133] Optionally, the chip 1100 may further include an output interface 1140. The processor 1110 may control the output interface 1140 to communicate with other devices or chips, and specifically, may output information or data to other devices or chips.

[0134] Optionally, the chip can be applied to a device (such as a base station) corresponding to the wireless access network in the embodiments of the present application, and the chip can implement the corresponding processes implemented by the wireless access network in each method of the embodiments of the present application. For the sake of brevity, they will not be repeated here.

[0135] Optionally, the chip can be applied to the business server in the embodiments of the present application, and the chip can implement the corresponding processes implemented by the business server in each method of the embodiments of the present application, which will not be described here for the sake of brevity.

[0136] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0137] It should be understood that the processor of the embodiment of the present application may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiment can be completed by the hardware integrated logic circuit or software instructions in the processor. The above processor can be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or the hardware and software modules in the decoding processor are combined and performed. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, and other mature storage media in the art. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0138] It can be understood that the memory in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0139] It should be understood that the above-mentioned memory is exemplary but not restrictive. For example, the memory in the embodiments of the present application may also be static random access memory (static RAM, SRAM), dynamic random access memory (dynamic RAM, DRAM), synchronous dynamic random access memory (synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (double data rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (synch link DRAM, SLDRAM) and direct memory bus random access memory (Direct Rambus RAM, DR RAM), etc. That is to say, the memory in the embodiments of the present application is intended to include but not limited to these and any other suitable types of memory.

[0140] An embodiment of the present application also provides a computer-readable storage medium for storing a computer program.

[0141] Optionally, the computer-readable storage medium can be applied to a device (such as a base station) corresponding to the wireless access network in the embodiments of the present application, and the computer program enables the computer to execute the corresponding processes implemented by the wireless access network in the various methods of the embodiments of the present application. For the sake of brevity, they will not be repeated here.

[0142] Optionally, the computer-readable storage medium can be applied to the business server in the embodiments of the present application, and the computer program enables the computer to execute the corresponding processes implemented by the business server in the various methods of the embodiments of the present application. For the sake of brevity, they are not repeated here.

[0143] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0144] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0145] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0146] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0147] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0148] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0149] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A video transmission method based on neural network coding, characterized in that: The method comprises: The wireless access network acquires a transmission bandwidth, where the transmission bandwidth is determined based on a first parameter, a second parameter, a third parameter, and a fourth parameter, wherein the first parameter represents the amount of data of the neural network, the second parameter represents the number of video encoding frames of the previous time-series neural network, the third parameter represents the frame rate of the previous time-series video, and the fourth parameter represents the network unidirectional transmission delay; The wireless access network allocates bandwidth resources based on the transmission bandwidth and uses the bandwidth resources to transmit the neural network.

2. The method according to claim 1, characterized in that The wireless access network acquires the transmission bandwidth, including: The wireless access network obtains a transmission bandwidth from a service server; the transmission bandwidth is determined by the service server based on the first parameter, the second parameter, the third parameter and the fourth parameter, wherein the first parameter, the second parameter and the third parameter are obtained at the application layer of the service server, and the fourth parameter is measured by the service server.

3. The method according to claim 2, characterized in that The method further comprises: When the transmission bandwidth is updated, the wireless access network obtains the updated transmission bandwidth from the service server.

4. The method according to claim 1, characterized in that: The wireless access network acquires the transmission bandwidth, including: The wireless access network determines the transmission bandwidth based on the first parameter, the second parameter, the third parameter and the fourth parameter, wherein the first parameter, the second parameter and the third parameter are sent to the wireless access network by a service server, and the fourth parameter is measured by the wireless access network.

5. The method according to claim 4, characterized in that The method further comprises: When at least one of the first parameter, the second parameter, the third parameter and the fourth parameter is updated, the radio access network re-determines the transmission bandwidth based on the updated parameter.

6. The method according to claim 4, characterized in that The network one-way transmission delay is the one-way transmission delay between the wireless access network and the terminal.

7. The method according to any one of claims 1 to 6, characterized in that The method further comprises: The radio access network receives a data packet carrying a neural network sent by a user plane function, the data packet carrying a first identifier or the second parameter, the first identifier is used to indicate whether a transmission timing of the neural network is a first timing, and a value of the second parameter is used to determine whether a transmission timing of the neural network is the first timing; The wireless access network identifies the first identifier or the second parameter in the data packet; If the identification result indicates that the transmission timing of the neural network is the first timing, the data packet is mapped to a data bearer with a first priority, and a first bandwidth resource is allocated to the data packet. The data packet is transmitted to the terminal using the first bandwidth resource, and the first priority is not lower than the preset priority, and the first bandwidth resource is not less than the preset bandwidth resource.

8. The method according to claim 7, characterized in that The wireless access network allocates bandwidth resources based on the transmission bandwidth, and uses the bandwidth resources to transmit the neural network, including: If the identification result indicates that the transmission timing of the neural network is not the first timing, a second bandwidth resource is allocated to the neural network based on the acquired transmission bandwidth, and the neural network is transmitted to the terminal using the second bandwidth resource.

9. The method according to claim 7, characterized in that: The first identifier or the second parameter is encapsulated by the user plane function in the NG-U header of the data packet.

10. The method according to claim 7, characterized in that The data packet is sent by the service server to the user plane function, and the service server encapsulates the first identifier or the second parameter in the RTP header of the data packet.

11. The method according to any one of claims 1 to 6, characterized in that The transmission bandwidth is determined based on the first parameter, the second parameter, the third parameter and the fourth parameter, including: The transmission bandwidth is determined by the following formula: B=D nn ÷[(N s-1 ÷f s-1 )+T] Wherein, B represents the transmission bandwidth, D nn represents the first parameter, N s-1 represents the second parameter, f s-1 represents the third parameter, and T represents the fourth parameter.

12. A video transmission method based on neural network coding, characterized in that: The method comprises: The service server determines the transmission bandwidth based on the first parameter, the second parameter, the third parameter and the fourth parameter, wherein the first parameter represents the data volume of the neural network, the second parameter represents the number of video encoding frames of the previous time sequence neural network, the third parameter represents the frame rate of the previous time sequence video, and the fourth parameter represents the network one-way transmission delay; The service server indicates the transmission bandwidth to the wireless access network, and the transmission bandwidth is used by the wireless access network to allocate bandwidth resources for transmitting the transmission neural network.

13. A video transmission device based on neural network coding, characterized in that: Applied to a wireless access network, the device comprises: An acquisition unit, used to acquire a transmission bandwidth, wherein the transmission bandwidth is determined based on a first parameter, a second parameter, a third parameter, and a fourth parameter, wherein the first parameter represents the amount of data of a neural network, the second parameter represents the number of video encoding frames of a previous time-series neural network, the third parameter represents the frame rate of a previous time-series video, and the fourth parameter represents a one-way transmission delay of the network; an allocating unit, configured to allocate bandwidth resources based on the transmission bandwidth; A sending unit is used to transmit the neural network using the bandwidth resources.

14. A video transmission device based on neural network coding, characterized in that: Applied to a business server, the device comprises: A determination unit, configured to determine a transmission bandwidth based on a first parameter, a second parameter, a third parameter, and a fourth parameter, wherein the first parameter represents the amount of data of a neural network, the second parameter represents the number of video encoding frames of a previous time-series neural network, the third parameter represents the frame rate of a previous time-series video, and the fourth parameter represents a one-way transmission delay of a network; A sending unit is used to indicate the transmission bandwidth to the wireless access network, and the transmission bandwidth is used by the wireless access network to allocate bandwidth resources for transmitting the transmission neural network.

15. A communication device, characterized in that: include: A processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory to execute the method according to any one of claims 1 to 12.

16. A chip, characterized in that: The invention comprises: a processor, configured to call and run a computer program from a memory, so that a device equipped with the chip executes a method as claimed in any one of claims 1 to 12.

17. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein the computer program causes a computer to execute the method according to any one of claims 1 to 12.

Images