Video transmission adaptive optimization method and device, equipment and storage medium

Abstract

The invention provides a video transmission adaptive optimization method and device, equipment and a storage medium, and the method comprises the steps: carrying out the adaptive optimization of the ratio of a common feature to a personalized feature corresponding to a target video frame group if video quality evaluation result data of a historical video frame group is obtained; and performing semantic coding on the target video frame group on the basis of the current ratio of the generality to the personality characteristics so as to obtain generality characteristic data and personality characteristic data of the target video frame group, packaging the generality characteristic data and the personality characteristic data, and transmitting the packaged data to a receiving end, so that the receiving end generates video quality evaluation result data. According to the method and the device, the common personality characteristic ratio of video communication can be dynamically adjusted according to the network channel quality and the video quality change, so that the stability and the quality of video transmission are effectively improved.

Description

Technical Field

[0001] The present application relates to the field of video data processing technology, and in particular to a method, device, equipment and storage medium for adaptive optimization of video transmission. Background Art

[0002] With the continuous development of the digital age, the proportion of video traffic in the network has gradually increased, and traditional communication methods can no longer meet the growing demand for video transmission. In order to solve this problem, video transmission based on semantic communication has gradually become a new trend in recent years. In semantic communication, more emphasis is placed on the semantic understanding and transmission of video content to improve transmission efficiency and user experience.

[0003] In a video transmission system based on semantic communication, the video semantic features extracted by the system can be divided into common features and individual features, where common features refer to the shared feature information between each video frame in a group of video frames, and individual features refer to the unique feature information of each video frame in a group of video frames. The ratio of common features to individual features will affect the effect of video transmission. Existing video transmission methods based on semantic communication usually extract features from video data based on a fixed ratio of common features to individual features, but this method cannot adapt to real-time changes in the network environment, and cannot balance the video transmission efficiency and the video effect after transmission, which leads to a decrease in transmission efficiency or video effect after transmission, and cannot meet user needs. Summary of the invention

[0004] In view of this, the embodiments of the present application provide a method, apparatus, device and storage medium for adaptive optimization of video transmission to eliminate or improve one or more defects existing in the prior art.

[0005] One aspect of the present application provides a method for adaptive optimization of video transmission, comprising:

[0006] If the video quality assessment result data of the historical video frame group is obtained, the commonality-individuality feature ratio corresponding to the target video frame group is adaptively optimized; the historical video frame group is the video frame group corresponding to the video data and has been transmitted to the receiving end; the target video frame group is the video frame group corresponding to the video data and currently to be transmitted;

[0007] Based on the current ratio of the common and individual features, the target video frame group is semantically encoded to obtain the common feature data and the individual feature data of the target video frame group respectively, and the common feature data and the individual feature data are packaged and transmitted to the receiving end, so that the receiving end obtains the corresponding video frame group based on the received common feature data and the individual feature data, and the receiving end uses the video frame group as a new historical video frame group for video quality assessment, and then determines whether to generate video quality assessment result data of the new historical video frame group.

[0008] In some embodiments of the present application, the video quality assessment result data is used to represent the first quality feedback information or the second quality feedback information;

[0009] The first quality feedback information is used to indicate that the video quality assessment values ​​corresponding to the plurality of consecutive video frames in the historical video frame group are all higher than a preset upper threshold;

[0010] The second quality feedback information is used to indicate that the video quality assessment values ​​corresponding to each of a plurality of consecutive video frames in the historical video frame group are all lower than a preset lower threshold; wherein the lower threshold is smaller than the upper threshold.

[0011] In some embodiments of the present application, if the video quality assessment result data of the historical video frame group is obtained, the ratio of commonality to individuality features corresponding to the target video frame group is adaptively optimized, including:

[0012] If video quality assessment result data of a first historical video frame group is received and the video quality assessment result data currently represents the first quality feedback information, then reducing the commonality-to-individuality feature ratio corresponding to the current target video frame group;

[0013] The first historical video frame group is a historical video frame group in the video data that is located before the current target video frame group, and the first historical video frame group and the target video frame group are adjacent to or spaced apart by a preset number of video frame groups.

[0014] In some embodiments of the present application, if the video quality assessment result data of the historical video frame group is obtained, the ratio of commonality to individuality features corresponding to the target video frame group is adaptively optimized, including:

[0015] If video quality assessment result data of a second historical video frame group is received and the video quality assessment result data currently represents the second quality feedback information, then increasing the commonality-individuality feature ratio corresponding to the current target video frame group;

[0016] The second historical video frame group is a historical video frame group in the video data that is located before the current target video frame group, and the second historical video frame group is adjacent to or spaced apart from the target video frame group by a preset number of video frame groups.

[0017] In some embodiments of the present application, before semantically encoding the target video frame group based on the current commonality-to-individuality feature ratio to obtain the commonality feature data and individuality feature data of the target video frame group, the method further includes:

[0018] If, within a preset time period after the common feature data and individual feature data corresponding to the third historical video frame group are encapsulated and transmitted to the receiving end, the video quality assessment result data for the third historical video frame group is not received, the common feature to individual feature ratio corresponding to a historical video frame group located before the third historical video frame group in the video data is used as the common feature to individual feature ratio corresponding to the target video frame group to be transmitted currently;

[0019] The third historical video frame group is a historical video frame group in the video data that is located before the current target video frame group, and the third historical video frame group is adjacent to or spaced apart from the target video frame group by a preset number of video frame groups.

[0020] A second aspect of the present application provides a video transmission adaptive optimization method, comprising:

[0021] Based on the common feature data and individual feature data currently received from the sending end, a video frame group belonging to the video data corresponding to the common feature data and the individual feature data is obtained, and the video frame group is used as a historical video frame group for video quality assessment to determine whether to generate video quality assessment result data of the historical video frame group;

[0022] If video quality assessment result data of the historical video frame group is generated, the video quality assessment result data of the historical video frame group is sent to the sending end, so that the sending end adaptively optimizes the commonality-to-individuality feature ratio corresponding to the current target video frame group according to the video quality assessment result data, and enables the sending end to semantically encode the target video frame group based on the current commonality-to-individuality feature ratio to respectively obtain the commonality feature data and individuality feature data of the target video frame group, so as to encapsulate the commonality feature data and individuality feature data and transmit them, and the target video frame group is the video frame group to be transmitted corresponding to the video data.

[0023] In some embodiments of the present application, the step of obtaining a video frame group belonging to video data corresponding to the common feature data and the individual feature data based on the common feature data and the individual feature data currently received from the sending end includes:

[0024] Receiving a data packet encapsulating common feature data and individual feature data from a transmitting end, and extracting the common feature data and individual feature data from the data packet;

[0025] It is determined whether the common feature data and individual feature data currently extracted belong to the same target video frame group in the video data. If so, semantic decoding is performed on the common feature data and individual feature data to obtain a corresponding video frame group.

[0026] In some embodiments of the present application, the step of using the video frame group as a historical video frame group to perform video quality assessment to determine whether to generate video quality assessment result data of the historical video frame group includes:

[0027] The video frame group obtained after semantic decoding is used as a historical video frame group, and the video quality assessment is performed on the historical video frame group to obtain the video quality assessment values ​​corresponding to each of the consecutive multiple video frames in the historical video frame group, and it is determined whether the video quality assessment values ​​corresponding to each of the consecutive multiple video frames in the historical video frame group are all higher than a preset upper limit threshold or are all lower than a preset lower limit threshold, wherein the lower limit threshold is less than the upper limit threshold;

[0028] If the video quality assessment values ​​corresponding to each of the plurality of consecutive video frames in the historical video frame group are all higher than a preset upper threshold, first quality feedback information is generated and video quality assessment result data corresponding to the historical video frame group is generated for representing the first quality feedback information;

[0029] If the video quality assessment values ​​corresponding to each of the plurality of consecutive video frames in the historical video frame group are lower than a preset lower threshold, second quality feedback information is generated and video quality assessment result data corresponding to the historical video frame group is generated for representing the second quality feedback information.

[0030] A third aspect of the present application provides a first video transmission adaptive optimization device, which is arranged in a transmitting end, and includes:

[0031] A commonality and individuality feature ratio selection module is used to adaptively optimize the commonality and individuality feature ratio corresponding to the target video frame group if the video quality evaluation result data of the historical video frame group is obtained; the historical video frame group is the video frame group corresponding to the video data that has been transmitted to the receiving end; the target video frame group is the video frame group currently to be transmitted corresponding to the video data;

[0032] A semantic coding and channel transmission module is used to semantically encode the target video frame group based on the current ratio of the common and individual features to obtain the common feature data and individual feature data of the target video frame group respectively, and to encapsulate the common feature data and the individual feature data and transmit them to the receiving end, so that the receiving end can obtain the corresponding video frame group based on the received common feature data and the individual feature data, and the receiving end can use the video frame group as a new historical video frame group for video quality assessment, and then determine whether to generate video quality assessment result data of the new historical video frame group.

[0033] A fourth aspect of the present application provides a second video transmission adaptive optimization device, which is arranged in a receiving end, and the second video transmission adaptive optimization device includes:

[0034] A video quality assessment module, for obtaining a video frame group belonging to the video data corresponding to the common feature data and the individual feature data based on the common feature data and the individual feature data currently received from the sending end, and using the video frame group as a historical video frame group for video quality assessment to determine whether to generate video quality assessment result data for the historical video frame group;

[0035] An evaluation result feedback module is used to send the video quality evaluation result data of the historical video frame group to the sending end if the video quality evaluation result data of the historical video frame group is generated, so that the sending end adaptively optimizes the commonality and individuality feature ratio corresponding to the current target video frame group according to the video quality evaluation result data, and enables the sending end to semantically encode the target video frame group based on the current commonality and individuality feature ratio to respectively obtain the commonality feature data and individuality feature data of the target video frame group, so as to encapsulate the commonality feature data and individuality feature data and transmit them, and the target video frame group is the video frame group to be transmitted corresponding to the video data.

[0036] The fifth aspect of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for adaptively optimizing video transmission described in the first aspect is implemented, and / or the method for adaptively optimizing video transmission described in the second aspect is implemented.

[0037] The sixth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for adaptively optimizing video transmission described in the first aspect and / or the method for adaptively optimizing video transmission described in the second aspect are implemented.

[0038] The seventh aspect of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the video transmission adaptive optimization method described in the first aspect, and / or implements the video transmission adaptive optimization method described in the second aspect.

[0039] The video transmission adaptive optimization method provided by the present application, if the video quality evaluation result data of the historical video frame group is obtained, then the commonality and individuality feature ratio corresponding to the target video frame group is adaptively optimized; the historical video frame group is the video frame group corresponding to the video data that has been transmitted to the receiving end; the target video frame group is the video frame group currently to be transmitted corresponding to the video data; based on the current commonality and individuality feature ratio, the target video frame group is semantically encoded to obtain the commonality feature data and individuality feature data of the target video frame group respectively, and the commonality feature data and individuality feature data are packaged and transmitted to the receiving end, so as to The receiving end is enabled to obtain the corresponding video frame group based on the received common feature data and individual feature data, and the receiving end is enabled to use the video frame group as a new historical video frame group for video quality assessment, and then determine whether to generate the video quality assessment result data of the new historical video frame group, which can effectively reduce data redundancy and reduce the amount of data to be transmitted, thereby supporting real-time video communication; it can dynamically adjust the ratio of common and individual features according to changes in network channel quality and video quality, and improve the stability and quality of video transmission; and it can more flexibly adapt to complex and changeable network environments, and effectively improve the efficiency of video transmission and user experience. Therefore, this application has important application prospects and technical value, and brings new ideas and solutions to the development of the field of video communication.

[0040] Additional advantages, purposes, and features of the present application will be partially described in the following description, and will become partially apparent to those skilled in the art after studying the following, or may be learned from the practice of the present application. The purposes and other advantages of the present application can be achieved and obtained by the structures specifically pointed out in the specification and the drawings.

[0041] Those skilled in the art will understand that the purposes and advantages that can be achieved by the present application are not limited to the above specific description, and the above and other purposes that can be achieved by the present application will be more clearly understood based on the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The drawings described herein are used to provide a further understanding of the present application, constitute a part of the present application, and do not constitute a limitation of the present application. The components in the drawings are not drawn to scale, but are only for the purpose of illustrating the principles of the present application. In order to facilitate the illustration and description of some parts of the present application, the corresponding parts in the drawings may be enlarged, that is, they may become larger relative to other components in the exemplary device actually manufactured according to the present application. In the drawings:

[0043] Figure 1 Schematic diagram of the overall process of semantic communication.

[0044] Figure 2 Schematic diagram of the execution logic of video transmission based on semantic communication.

[0045] Figure 3 The first flowchart of the video transmission adaptive optimization method executed at the sending end in one embodiment of the present application is shown.

[0046] Figure 4 The second flowchart of the video transmission adaptive optimization method executed at the sending end in one embodiment of the present application is shown.

[0047] Figure 5 The first flowchart of the video transmission adaptive optimization method executed at the receiving end in one embodiment of the present application is shown.

[0048] Figure 6 Schematic diagram of a second flow chart of a method for adaptive optimization of video transmission performed at a receiving end in one embodiment of the present application.

[0049] Figure 7 Schematic diagram of the structure of a first video transmission adaptive optimization device in one embodiment of the present application.

[0050] Figure 8 Schematic diagram of the structure of a second video transmission adaptive optimization device in one embodiment of the present application.

[0051] Fig. 9 This is a schematic diagram of the execution logic of the adaptive commonality-individuality feature ratio adjustment method based on video semantic communication in the application example of this application.

[0052] Fig.10 The following is a schematic diagram of the execution flow of the receiving end quality assessment feedback module in the application example of this application.

[0053] Fig.11 This is a schematic diagram of the execution flow of the common and individual feature comparison module at the sending end in the application example of this application. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the implementation modes and the accompanying drawings. Here, the illustrative implementation modes and descriptions of the present application are used to explain the present application, but are not intended to limit the present application.

[0055] It should also be noted here that in order to avoid obscuring the present application due to unnecessary details, only the structures and / or processing steps closely related to the scheme according to the present application are shown in the accompanying drawings, while other details that are not very relevant to the present application are omitted.

[0056] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0057] It should also be noted that, unless otherwise specified, the term “connection” herein may refer not only to a direct connection but also to an indirect connection involving an intermediate.

[0058] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0059] In traditional communication, even if a small number of bits are lost or erroneous, the integrity of the entire data packet will be affected, resulting in the inability of the receiving end to correctly parse and restore the original information. Therefore, when designing and implementing traditional communication systems, attention is paid to the stability and reliability of data transmission, and various error correction and fault tolerance technologies are used to minimize errors that may occur during transmission to ensure that information can be accurately transmitted and received.

[0060] Transmission based on semantic communication extracts the information of the video to be sent at the source, rather than the bit stream of the message. More meaningful information can be transmitted under the same bandwidth conditions, which can effectively compress data redundancy, improve the effectiveness of information transmission, reduce the pressure of network transmission, and reduce the processing delay of intelligent tasks. Compared with traditional networks, semantic communication can better meet the needs when the communication channel quality is poor and the packet loss rate is high.

[0061] The process of semantic communication is as follows Figure 1 As shown in the figure, the transmitter first extracts features from the information to be sent, and then performs semantic and channel coding on the extracted semantic information. The data after semantic and channel coding will reach the receiver in the form of a bit stream through the network channel at the physical layer. After receiving the bit stream information, the receiver will perform semantic and channel decoding on it to recover the information from the bit stream. The process of extracting semantic information at the transmitter and recovering semantic information at the receiver mainly depends on the semantic knowledge base shared between the transmitter and the receiver.

[0062] Since video traffic accounts for a large proportion of all mobile data traffic, the increase in video traffic puts higher requirements on the network environment. Therefore, semantic communication has broad development prospects when facing video transmission tasks.

[0063] Video transmission systems based on semantic communication Figure 2 As shown, it includes five steps: video data splitting, implicit conversion, source-channel joint coding, semantic feature extraction, and variable length coding. Video data splitting divides the video into groups of video frames (Group of Frames, GoF); implicit conversion aims to reduce the amount of data to be transmitted by downsampling, convolution, and residual network processing of GOF, thereby supporting real-time video communication; source-channel joint coding is divided into two steps: source coding and channel coding. The purpose of source coding is to remove redundant information inside the source and improve effectiveness; channel coding requires adding check bits to the original bit sequence to realize error detection and correction functions, so as to increase the reliability of bit sequence transmission for noisy channels; semantic feature extraction obtains the common features of GOF and personality traits In video semantic communication, data containing key video feature information is usually defined as "common features", which refers to the feature information shared between each video frame in a group of video frames, and each group of GOF has only one common feature. Data containing non-key feature information is defined as "individual features", which refers to the unique feature information of each video frame in a group of video frames. Each group of GOF has n individual features, where n is the number of video frames in GOF; the entropy model structure shared by the sender and the receiver is generated during the variable length coding process.

[0064] The receiving end combines the received common features and individual features through source-channel joint decoding, and then converts the received data into implicit expression data through deconvolution and activation function; then the receiving end converts the implicit expression data into a video image observable to the human eye through inverse implicit conversion using deconvolution and residual blocks.

[0065] In the above process, the common characteristics and personality traits The ratio of will affect the transmission quality of the video. Here, the Common Individual Feature Ratio (CIFR) is defined as:

[0066]

[0067] Different CIFRs have different effects on the video quality of semantic communication. The setting of CIFR should not be too large or too small. It should be set to ensure effective transmission of M c While ensuring Mi A reasonable amount of data.

[0068] Therefore, when video semantic communication faces a complex and changing network environment, the fixed CIFR method cannot fully adapt to the needs of network channel transmission. This is because under different network channel conditions, and The loss and interference levels of different packets vary, and a fixed CIFR may make it impossible for the receiver to improve the video quality in some cases. Therefore, it is necessary to adjust the CIFR for different network channel environments to adapt to network channel environments with different packet loss rates, improve video quality and the robustness of the video semantic communication system.

[0069] In one or more embodiments of the present application, the common personality feature ratio refers to the ratio of the dimension of the personality feature to the dimension of the common feature.

[0070] In order to improve the quality of video transmission based on semantic communication, thereby improving the stability and quality of video transmission, the embodiments of the present application respectively provide a video transmission adaptive optimization method performed at a sending end, a first video transmission adaptive optimization device for performing the video transmission adaptive optimization method, a video transmission adaptive optimization method performed at a receiving end, a second video transmission adaptive optimization device for performing the video transmission adaptive optimization method, an electronic device, a computer-readable storage medium, and a computer program product, which can dynamically adjust the commonality and individuality feature ratio of video semantic communication according to feedback information, thereby effectively improving the stability and quality of video transmission.

[0071] The details are described in detail through the following examples.

[0072] Based on this, the embodiment of the present application provides a video transmission adaptive optimization method that can be performed by a first video transmission adaptive optimization device arranged at a transmitting end, see Figure 3 The video transmission adaptive optimization method performed by the first video transmission adaptive optimization device arranged at the transmitting end specifically includes the following contents:

[0073] Step 100: If the video quality assessment result data of the historical video frame group is obtained, the ratio of common and individual features corresponding to the target video frame group is adaptively optimized; the historical video frame group is the video frame group corresponding to the video data and has been transmitted to the receiving end; the target video frame group is the video frame group currently to be transmitted corresponding to the video data.

[0074] In step 100, if the first video transmission adaptive optimization device arranged at the sending end receives the video quality assessment result data of the historical video frame group sent by the receiving end, the commonality and individuality feature ratio corresponding to the target video frame group is reduced or increased according to the video quality assessment result data, so as to realize adaptive adjustment of the commonality and individuality feature ratio corresponding to the target video frame group.

[0075] In one or more embodiments of the present application, the historical video frame group and the target video frame group both refer to one of the video frame groups corresponding to the same video data, wherein the historical video frame group is the video frame group corresponding to the video data that has been transmitted to the receiving end; the target video frame group is the video frame group corresponding to the video data that is currently to be transmitted. It is understandable that the target video frame group, the first historical video frame group, the second historical video frame group, and the first historical video frame group mentioned later all refer to video frame groups with the same data structure, and the differentiated expressions are only for the convenience of expression.

[0076] Before step 100, the transmitter may split the video data to be transmitted into various video frame groups (Group of Frames, GoF), and each of the video frame groups contains various video frames sorted in time sequence; and then perform downsampling, convolution and residual network processing on each video frame group through implicit conversion.

[0077] Step 200: Based on the current ratio of the common and individual features, the target video frame group is semantically encoded to obtain the common feature data and the individual feature data of the target video frame group respectively, and the common feature data and the individual feature data are packaged and transmitted to the receiving end, so that the receiving end obtains the corresponding video frame group based on the received common feature data and the individual feature data, and the receiving end uses the video frame group as a new historical video frame group for video quality assessment, and then determines whether to generate video quality assessment result data of the new historical video frame group.

[0078] In step 200, the first video transmission adaptive optimization device set at the transmitting end extracts semantic information from the target video frame group in a specific manner that each video frame group after implicit conversion is subjected to source-channel joint coding (can also be written as source and channel joint coding) to obtain semantic information corresponding to the target video frame group, and the source-channel joint coding is divided into two steps: source coding and channel coding, wherein the purpose of source coding is to remove redundant information inside the source and improve effectiveness; channel coding requires adding check bits to the original bit sequence to realize the functions of error detection and correction, thereby increasing the reliability of bit sequence transmission for noisy channels; and then based on the current commonality and individuality feature ratio, the semantic information corresponding to the target video frame group subjected to source-channel joint coding is subjected to semantic feature extraction to obtain commonality feature data and individuality feature data of the target video frame group respectively. Then, the commonality feature data and individuality feature data are encapsulated (channel coding) to obtain a data packet corresponding to the target video frame group, and the data packet is transmitted to the receiving end via a network channel.

[0079] After receiving the data packet, the receiving end will first extract the common feature data and individual feature data from the data packet, and then based on the consistency between the corresponding identifiers of the common feature data and individual feature data currently received, it can determine whether the common feature data and individual feature data currently extracted belong to the same target video frame group in the video data. If so, the common feature data and individual feature data are semantically decoded (that is, the inverse process of the semantic encoding performed by the sending end) to obtain the corresponding video frame group. And use the video frame group as a historical video frame group for video quality assessment to determine whether to generate video quality assessment result data for the historical video frame group; then if the receiving end generates video quality assessment result data for the historical video frame group, send the video quality assessment result data for the historical video frame group to the sending end, so that the sending end adaptively optimizes the commonality and individuality feature ratio corresponding to the current target video frame group according to the video quality assessment result data, and enables the sending end to semantically encode the target video frame group based on the current commonality and individuality feature ratio to respectively obtain the commonality feature data and individuality feature data of the target video frame group, so as to encapsulate the commonality feature data and individuality feature data and transmit them, and the target video frame group is the video frame group to be transmitted corresponding to the video data.

[0080] In one or more embodiments of the present application, the sending end and the receiving end refer to the two ends of video transmission, and the sending end and the receiving end can both be electronic devices, and the electronic device can be a server or a client device. It can be understood that the same device can also be a sending end and a receiving end at the same time. For example, a client device can semantically encode and transmit video data A based on the video transmission adaptive optimization method executed by the first video transmission adaptive optimization device, and can also receive and semantically decode video data B based on the video transmission adaptive optimization method executed by the second video transmission adaptive optimization device mentioned in the following embodiments.

[0081] From the above description, it can be seen that the video transmission adaptive optimization method provided by the embodiment of the present application and executed by the first video transmission adaptive optimization device set at the transmitting end can effectively reduce data redundancy and reduce the amount of data to be transmitted, thereby supporting real-time video communication; it can dynamically adjust the ratio of commonality to individual characteristics according to changes in network channel quality and video quality, and improve the stability and quality of video transmission; and it can more flexibly adapt to complex and changeable network environments, effectively improving the efficiency of video transmission and user experience. Therefore, this application has important application prospects and technical value, and brings new ideas and solutions to the development of the field of video communication.

[0082] In order to further avoid the transmission loss of invalid information, simplify the adaptive optimization process at the receiving end, and further improve the video transmission efficiency, in a video transmission adaptive optimization method performed by a first video transmission adaptive optimization device provided at a transmitting end provided in an embodiment of the present application, the video quality evaluation result data is used to represent the first quality feedback information or the second quality feedback information;

[0083] The first quality feedback information is used to indicate that the video quality assessment values ​​corresponding to the plurality of consecutive video frames in the historical video frame group are all higher than a preset upper threshold;

[0084] The second quality feedback information is used to indicate that the video quality assessment values ​​corresponding to each of a plurality of consecutive video frames in the historical video frame group are all lower than a preset lower threshold; wherein the lower threshold is smaller than the upper threshold.

[0085] That is to say, the video quality assessment result data is only used to indicate that the video quality assessment values ​​corresponding to multiple consecutive video frames in the historical video frame group are all higher than the preset upper threshold or the video quality assessment values ​​corresponding to multiple consecutive video frames in the historical video frame group are all lower than the preset lower threshold, but does not indicate the information that the video quality assessment value is between the lower threshold and the upper threshold. This setting is because the video quality assessment value between the lower threshold and the upper threshold indicates that the video quality of the corresponding historical video frame group is in a state that can balance the transmission efficiency and video time quality. Therefore, there is no need to optimize the commonality and individuality feature ratio of the target video frame group after the historical video frame group in this state, and thus there is no need to generate video quality assessment result data containing information corresponding to this state, and there is no need to send data to the sending end when the commonality and individuality feature ratio is not optimized. This can avoid the transmission loss of invalid information and simplify the adaptive optimization process of the receiving end to further improve the video transmission efficiency.

[0086] In order to further improve the effectiveness, flexibility and intelligence of the ratio of commonality to individuality features corresponding to the adaptive optimization target video frame group, in the video transmission adaptive optimization method performed by the first video transmission adaptive optimization device provided at the transmitting end provided in the embodiment of the present application, see Figure 4 Step 100 of the video transmission adaptive optimization method executed by the first video transmission adaptive optimization device disposed at the transmitting end specifically includes the following contents:

[0087] Step 110: if the video quality assessment result data of the first historical video frame group is received and the video quality assessment result data currently represents the first quality feedback information, then reducing the commonality-individuality feature ratio corresponding to the current target video frame group;

[0088] The first historical video frame group is a historical video frame group in the video data that is located before the current target video frame group, and the first historical video frame group and the target video frame group are adjacent to or spaced apart by a preset number of video frame groups.

[0089] In order to further improve the effectiveness, flexibility and intelligence of the ratio of commonality to individuality features corresponding to the adaptive optimization target video frame group, in a video transmission adaptive optimization method provided in an embodiment of the present application, see Figure 4 , step 100 in the video transmission adaptive optimization method further specifically includes the following contents:

[0090] Step 120: if the video quality assessment result data of the second historical video frame group is received and the video quality assessment result data currently represents the second quality feedback information, then the commonality-individuality feature ratio corresponding to the current target video frame group is increased;

[0091] The second historical video frame group is a historical video frame group in the video data that is located before the current target video frame group, and the second historical video frame group is adjacent to or spaced apart from the target video frame group by a preset number of video frame groups.

[0092] In order to further avoid the transmission loss of invalid information, simplify the adaptive optimization process at the receiving end, and further improve the video transmission efficiency, in the video transmission adaptive optimization method performed by the first video transmission adaptive optimization device provided at the transmitting end provided in the embodiment of the present application, see Figure 4 The video transmission adaptive optimization method executed by the first video transmission adaptive optimization device disposed at the transmitting end further specifically includes the following contents before step 200:

[0093] Step 010: If the video quality assessment result data for the third historical video frame group is not received within a preset time period after the common feature data and individual feature data corresponding to the third historical video frame group are encapsulated and transmitted to the receiving end, the common feature to individual feature ratio corresponding to a historical video frame group located before the third historical video frame group in the video data is used as the common feature to individual feature ratio corresponding to the target video frame group to be transmitted currently;

[0094] The third historical video frame group is a historical video frame group in the video data that is located before the current target video frame group, and the third historical video frame group is adjacent to or spaced apart from the target video frame group by a preset number of video frame groups.

[0095] It can be understood that in one or more embodiments of the present application, the first historical video frame group, the second historical video frame group and the third historical video frame group can adopt the same or different historical video frame groups, which can be specifically set according to the actual application scenario.

[0096] For example, in executing steps 110 and 120, in order to avoid the receiving end's evaluation of video quality affecting the sending end's efficiency in video transmission, the first historical video frame group and the second historical video frame group can be selected as historical video frame groups that are separated from the target video frame group by a preset number of video frame groups. For example, a historical video frame group that is separated by three video frame groups before the current target video frame group is selected as the first historical video frame group and the second historical video frame, rather than adjacent video frame groups. In this way, the transmission time separated by three video frame groups can be used to enable the receiving end to evaluate the video quality and send the video quality evaluation result data to the sending end, thereby effectively avoiding the receiving end's evaluation of video quality affecting the sending end's efficiency in video transmission.

[0097] For another example, in executing step 010, in order to prevent the transmitter from waiting for a long time for the video quality assessment result data and affecting the transmission efficiency of the target video frame group, a historical video frame group three video frame groups before the current target video frame group may be selected as the third historical video frame group.

[0098] At the same time, since the video quality assessment result data only indicates information that the video quality is too good or too poor, in order to further avoid time-consuming waiting, if the video quality assessment result data for the third historical video frame group is not received within a preset time period after the common feature data and individual feature data corresponding to the third historical video frame group are encapsulated and transmitted to the receiving end, the commonality-to-individuality feature ratio corresponding to a historical video frame group in the video data that is located before the third historical video frame group is used as the commonality-to-individuality feature ratio corresponding to the target video frame group to be transmitted.

[0099] Among them, the preset time period can be set according to the actual situation. For example, based on the common feature data and individual feature data extracted from the data packet received by the receiving end, it is judged whether the common feature data and individual feature data currently extracted belong to the same target video frame group in the video data. If so, the common feature data and individual feature data are semantically decoded to obtain the corresponding video frame group, and the video frame group is used as a historical video frame group for video quality assessment to determine whether to generate video quality assessment result data of the historical video frame group; and then if the receiving end generates video quality assessment result data of the historical video frame group, the total time required to send the video quality assessment result data to the sending end is set.

[0100] Based on this, the embodiment of the present application provides a video transmission adaptive optimization method that can be implemented by a second video transmission adaptive optimization device arranged at a receiving end, see Figure 5 The video transmission adaptive optimization method provided by the second video transmission adaptive optimization device at the receiving end specifically includes the following contents:

[0101] Step 300: Based on the common feature data and individual feature data currently received from the sending end, obtain the video frame group belonging to the video data corresponding to the common feature data and the individual feature data, and use the video frame group as a historical video frame group to perform video quality assessment to determine whether to generate video quality assessment result data for the historical video frame group.

[0102] Step 400: If the video quality assessment result data of the historical video frame group is generated, the video quality assessment result data of the historical video frame group is sent to the sending end, so that the sending end adaptively optimizes the commonality-to-individuality feature ratio corresponding to the current target video frame group according to the video quality assessment result data, and enables the sending end to semantically encode the target video frame group based on the current commonality-to-individuality feature ratio to respectively obtain the commonality feature data and individuality feature data of the target video frame group, so as to encapsulate the commonality feature data and individuality feature data and transmit them, and the target video frame group is the video frame group to be transmitted corresponding to the video data.

[0103] The technical implementation details involved in the embodiment of the video transmission adaptive optimization method performed by the second video transmission adaptive optimization device set at the receiving end provided in the present application can be specifically implemented by adopting the technical implementation details recorded in the video transmission adaptive optimization method performed by the first video transmission adaptive optimization device set at the sending end in the above-mentioned embodiment. Its functions will not be repeated here, and reference can be made to the detailed description of the embodiment of the video transmission adaptive optimization method performed by the first video transmission adaptive optimization device set at the sending end.

[0104] From the above description, it can be seen that the video transmission adaptive optimization method provided by the embodiment of the present application and executed by the second video transmission adaptive optimization device set at the receiving end can effectively reduce data redundancy and reduce the amount of data to be transmitted, thereby supporting real-time video communication; it can dynamically adjust the ratio of commonality to individual characteristics to improve the stability and quality of video transmission; and it can more flexibly adapt to complex and changeable network environments, effectively improving the efficiency of video transmission and user experience. Therefore, this application has important application prospects and technical value, and brings new ideas and solutions to the development of the field of video communication.

[0105] In order to further improve the effectiveness and reliability of obtaining the video frame group corresponding to the common feature data and the individual feature data belonging to the video data, in the video transmission adaptive optimization method provided by the second video transmission adaptive optimization device provided at the receiving end in the embodiment of the present application, see Figure 6 The step 300 of the video transmission adaptive optimization method of the second video transmission adaptive optimization device provided at the receiving end specifically includes the following contents:

[0106] Step 310: Receive a data packet encapsulating common feature data and individual feature data from a transmitting end, and extract the common feature data and individual feature data from the data packet.

[0107] Step 320: Determine whether the common feature data and individual feature data currently extracted belong to the same target video frame group in the video data. If so, perform semantic decoding on the common feature data and individual feature data to obtain a corresponding video frame group.

[0108] In order to further improve the effectiveness and reliability of using the video frame group as a historical video frame group for video quality assessment, in the video transmission adaptive optimization method provided by the second video transmission adaptive optimization device provided at the receiving end in the embodiment of the present application, see Figure 6 Step 300 of the video transmission adaptive optimization method provided by the second video transmission adaptive optimization device at the receiving end further specifically includes the following contents:

[0109] Step 330: The video frame group obtained after semantic decoding is used as a historical video frame group, and video quality assessment is performed on the historical video frame group to obtain video quality assessment values ​​corresponding to each of the multiple consecutive video frames in the historical video frame group, and determine whether the video quality assessment values ​​corresponding to each of the multiple consecutive video frames in the historical video frame group are all higher than a preset upper threshold or are all lower than a preset lower threshold, wherein the lower threshold is less than the upper threshold.

[0110] Step 340: If the video quality assessment values ​​corresponding to each of the plurality of consecutive video frames in the historical video frame group are higher than a preset upper threshold, first quality feedback information is generated and video quality assessment result data corresponding to the historical video frame group is generated for representing the first quality feedback information.

[0111] Step 350: If the video quality assessment values ​​corresponding to the plurality of consecutive video frames in the historical video frame group are all lower than the preset lower threshold, second quality feedback information is generated and video quality assessment result data corresponding to the historical video frame group is generated for representing the second quality feedback information.

[0112] From the software level, the present application also provides a method for executing Figure 3 or Figure 4 The first video transmission adaptive optimization device in all or part of the video transmission adaptive optimization method shown in the figure is arranged in the transmitting end, see Figure 7 The first video transmission adaptive optimization device specifically includes the following contents:

[0113] The commonality and individuality feature ratio selection module 10 is used to adaptively optimize the commonality and individuality feature ratio corresponding to the target video frame group if the video quality evaluation result data of the historical video frame group is obtained; the historical video frame group is the video frame group corresponding to the video data that has been transmitted to the receiving end; the target video frame group is the video frame group corresponding to the video data that is currently to be transmitted;

[0114] The semantic coding and channel transmission module 20 is used to semantically encode the target video frame group based on the current ratio of the common and individual features to obtain the common feature data and individual feature data of the target video frame group respectively, and to encapsulate the common feature data and the individual feature data and transmit them to the receiving end, so that the receiving end can obtain the corresponding video frame group based on the received common feature data and the individual feature data, and the receiving end can use the video frame group as a new historical video frame group for video quality assessment, and then determine whether to generate video quality assessment result data of the new historical video frame group.

[0115] The embodiment of the first video transmission adaptive optimization device provided in the present application can be specifically used to perform the following steps: Figure 3 or Figure 4 The processing flow of the embodiment of the video transmission adaptive optimization method in the above embodiment is shown in FIG. Figure 3 or Figure 4 A detailed description of the above-mentioned video transmission adaptive optimization method embodiment is shown.

[0116] The first video transmission adaptive optimization device performs the following steps: Figure 3 or Figure 4 The portion of the video transmission adaptive optimization shown can be completed in the client device. The specific selection can be based on the processing capability of the client device and the limitations of the user's usage scenario. This application does not limit this. If all operations are completed in the client device, the client device may also include a processor for Figure 3 or Figure 4 The specific process of adaptive optimization of video transmission is shown.

[0117] The client device may have a communication module (i.e., a communication unit) that can communicate with a remote server to achieve data transmission with the server. The server may include a server on the task scheduling center side, and other implementation scenarios may also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, or a server cluster consisting of multiple servers, or a server structure of a distributed device.

[0118] The server and the client device may communicate with each other using any suitable network protocol, including network protocols that have not yet been developed on the date of filing this application. The network protocols may include, for example, TCP / IP, UDP / IP, HTTP, HTTPS, etc. Of course, the network protocols may also include, for example, RPC (Remote Procedure Call Protocol) and REST (Representational State Transfer) protocols used on top of the above protocols.

[0119] From the above description, it can be seen that the first video transmission adaptive optimization device provided by the embodiment of the present application can effectively reduce data redundancy and reduce the amount of data to be transmitted, thereby supporting real-time video communication; it can dynamically adjust the ratio of commonality to individual characteristics to improve the stability and quality of video transmission; and it can more flexibly adapt to complex and changeable network environments, effectively improving the efficiency of video transmission and user experience. Therefore, this application has important application prospects and technical value, and brings new ideas and solutions to the development of the field of video communication.

[0120] From the software level, the present application also provides a method for executing Figure 5 or Figure 6 The second video transmission adaptive optimization device in all or part of the video transmission adaptive optimization method shown in the figure, the first video transmission adaptive optimization device is set in the receiving end, see Figure 8 The second video transmission adaptive optimization device specifically includes the following contents:

[0121] The video quality assessment module 30 is used to obtain a video frame group belonging to the video data corresponding to the common feature data and the individual feature data based on the common feature data and the individual feature data currently received from the sending end, and use the video frame group as a historical video frame group to perform video quality assessment to determine whether to generate video quality assessment result data of the historical video frame group;

[0122] The evaluation result feedback module 40 is used to send the video quality evaluation result data of the historical video frame group to the sending end if the video quality evaluation result data of the historical video frame group is generated, so that the sending end adaptively optimizes the commonality and individuality feature ratio corresponding to the current target video frame group according to the video quality evaluation result data, and enables the sending end to semantically encode the target video frame group based on the current commonality and individuality feature ratio to respectively obtain the commonality feature data and individuality feature data of the target video frame group, so as to encapsulate the commonality feature data and individuality feature data and transmit them, and the target video frame group is the video frame group to be transmitted corresponding to the video data.

[0123] The embodiment of the second video transmission adaptive optimization device provided in this application can be specifically used to perform the following steps: Figure 5 or Figure 6 The processing flow of the embodiment of the video transmission adaptive optimization method in the above embodiment is shown in FIG. Figure 5 or Figure 6 A detailed description of the above-mentioned video transmission adaptive optimization method embodiment is shown.

[0124] From the above description, it can be seen that the second video transmission adaptive optimization device provided by the embodiment of the present application can effectively reduce data redundancy and reduce the amount of data to be transmitted, thereby supporting real-time video communication; it can dynamically adjust the ratio of commonality to individual characteristics to improve the stability and quality of video transmission; and it can more flexibly adapt to complex and changeable network environments, effectively improving the efficiency of video transmission and user experience. Therefore, this application has important application prospects and technical value, and brings new ideas and solutions to the development of the field of video communication.

[0125] To further illustrate the above embodiments, the present application also provides an adaptive common individual feature ratio adjustment method based on video semantic communication, which is interactively executed by a transmitter and a receiver. In actual video transmission scenarios, if video semantic communication adopts an unchanging CIFR in the face of a complex and changeable network channel environment, it is not conducive to maintaining the high quality of the video. Therefore, the CIFR extracted by video semantics can be adjusted to adapt to the changeable network channel environment. The application example of the present application designs and implements an adaptive common individual feature ratio adjustment method based on video semantic communication based on the above ideas. As video traffic increases, traditional communications cannot meet the demand, and video transmission based on semantic communication is gradually emerging. Semantic communication focuses on the semantic understanding and transmission of video content to improve transmission efficiency and user experience. The application example of the present application proposes an adaptive common individual feature ratio adjustment method, which is dynamically adjusted according to the network channel quality and video quality information to improve the stability and quality of video transmission. See. Fig. 9 , the specific steps are as follows:

[0126] Step 1. The sender uses the current CIFR to semantically encode the video to be transmitted and generate common features of the video. and personality traits

[0127] Step 2. The sender After encapsulation, it is transmitted to the receiving end;

[0128] Step 3. The receiving end analyzes the common features and personality traits Perform quality assessment. If the video quality changes, the receiving end will generate feedback information about the adjustment of common and individual features and send it to the sending end. At the same time, the receiving end performs semantic decoding to restore the semantic features into video.

[0129] Step 4. The sender adjusts the CIFR accordingly based on the received feedback information;

[0130] Step 5. The above process is iterated periodically until the video transmission is completed.

[0131] The specific implementation of this application mainly relies on two modules: a receiving end video quality assessment feedback module and a sending end commonality and individuality feature ratio selection module. A specific description is given below.

[0132] 1. Receiver video quality assessment feedback module

[0133] The main function of the receiving end video quality assessment feedback module is to evaluate the quality of the network channel parameters and the transmitted video, and then generate different feedback information and send it to the common individual feature ratio selection module.

[0134] This module evaluates the quality of the network channel parameters and the transmitted video. If the evaluation value is continuously lower than the set threshold V l , then generate quality feedback information M l If the evaluation value is continuously higher than the set threshold V h , then generate quality feedback information M h , and feed it back to the sender to reduce CIFR to save network bandwidth. l With V h If the quality feedback information is between , no quality feedback information is generated and the current CIFR is maintained. The specific algorithm of the receiving end quality assessment feedback module is shown in Table 1. The process is as follows Fig.10 shown.

[0135] Table 1 Specific algorithm description of the receiving end video quality assessment feedback module

[0136]

[0137] 2. Common and individual characteristics ratio selection module of the sending end

[0138] The sender first parses the received quality feedback information, and then passes the parsed information to the common individual feature ratio selection module. The main function of the common individual feature ratio selection module is to actively adjust the CIFR of the semantic code according to the different feedback information sent by the receiver to select the CIFR that best suits the current network channel environment to maintain high video quality.

[0139] When the parsed feedback information is M lWhen the feedback information is M, it indicates that the quality of the current video semantic communication is poor, so it is necessary to increase CIFR to improve the video quality; h When , it indicates that the quality of the current video semantic communication is high, and the packet loss rate of the network channel has little effect on the video quality, which will not affect the user's intuitive experience. In this case, the commonality and individuality feature ratio selection module at the sender can reduce CIFR to reduce the consumption of network bandwidth and improve transmission efficiency and video quality.

[0140] Subsequently, according to the CIFR provided by the commonality and individuality feature ratio selection module at the sending end, the video is semantically encoded to generate the commonality and individuality features of the video. After encapsulating the header information, the video data is transmitted. The specific algorithm of the commonality and individuality feature ratio selection module is shown in Table 2, and the process is as follows: Fig.11 shown.

[0141] Table 2 Specific algorithm description of the commonality and individuality feature ratio selection module at the sender

[0142]

[0143] The adaptive commonality and individuality feature ratio adjustment method proposed in the application example of this application brings significant advantages to the video transmission system based on semantic communication. First, by extracting the commonality and individuality features of the video, data redundancy can be more effectively reduced, and the amount of data to be transmitted can be reduced, thereby supporting real-time video communication. Secondly, by adopting an adaptive commonality and individuality feature ratio adjustment strategy, the commonality and individuality feature ratio can be dynamically adjusted according to the real-time network channel quality and video quality information, thereby improving the stability and quality of video transmission. Compared with the traditional fixed feature ratio method, this application can adapt to complex and changeable network environments more flexibly, and effectively improve the efficiency of video transmission and user experience. Therefore, this application has important application prospects and technical value, and brings new ideas and solutions to the development of the field of video communication.

[0144] The embodiment of the present application also provides an electronic device, which may include a processor, a memory, a receiver and a transmitter, wherein the processor is used to execute the following Figure 3 or Figure 4 The video transmission adaptive optimization method shown and / or Figure 5 or Figure 6 In the video transmission adaptive optimization method shown, the processor and the memory can be connected via a bus or other methods, taking the bus connection as an example. The receiver can be connected to the processor and the memory via a wired or wireless method.

[0145] The processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0146] As a non-transitory computer-readable storage medium, the memory can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as the embodiment of the present application. Figure 3 or Figure 4 The video transmission adaptive optimization method shown and / or Figure 5 or Figure 6 The processor executes various functional applications and data processing of the processor by running the non-transient software programs, instructions and modules stored in the memory, that is, implementing the above method embodiments as shown in the program instructions / modules corresponding to the video transmission adaptive optimization method. Figure 3 or Figure 4 The video transmission adaptive optimization method shown and / or Figure 5 or Figure 6 The video transmission adaptive optimization method shown.

[0147] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0148] The one or more modules are stored in the memory, and when executed by the processor, perform the following steps in the embodiment: Figure 3 or Figure 4 The video transmission adaptive optimization method shown and / or Figure 5 or Figure 6 The video transmission adaptive optimization method shown.

[0149] In some embodiments of the present application, the user equipment may include a processor, a memory, and a transceiver unit, which may include a receiver and a transmitter. The processor, memory, receiver, and transmitter may be connected through a bus system. The memory is used to store computer instructions, and the processor is used to execute the computer instructions stored in the memory to control the transceiver unit to send and receive signals.

[0150] As an implementation method, the functions of the receiver and the transmitter in the present application can be considered to be implemented through a transceiver circuit or a dedicated chip for transceiver, and the processor can be considered to be implemented through a dedicated processing chip, a processing circuit or a general chip.

[0151] As another implementation method, it is possible to use a general-purpose computer to implement the server provided in the embodiment of the present application, that is, to store the program code for implementing the functions of the processor, receiver, and transmitter in a memory, and the general-purpose processor implements the functions of the processor, receiver, and transmitter by executing the code in the memory.

[0152] The present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program can be used to implement the following Figure 3 or Figure 4 The video transmission adaptive optimization method shown and / or Figure 5 or Figure 6 The steps of the video transmission adaptive optimization method shown. The computer-readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0153] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the following Figure 3 or Figure 4 The video transmission adaptive optimization method shown and / or Figure 5 or Figure 6 The video transmission adaptive optimization method shown.

[0154] It should be understood by those skilled in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is specifically performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier.

[0155] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0156] In the present application, features described and / or illustrated for one embodiment may be used in the same manner or in a similar manner in one or more other embodiments, and / or combined with features of other embodiments or replace features of other embodiments.

[0157] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the embodiments of the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A video transmission adaptive optimization method, characterized in that: include: If the video quality assessment result data of the historical video frame group is obtained, the ratio of commonality to individuality features corresponding to the target video frame group is adaptively optimized; The historical video frame group is a video frame group corresponding to the video data that has been transmitted to the receiving end; the target video frame group is a video frame group corresponding to the video data that is currently to be transmitted; Based on the current ratio of the common and individual features, the target video frame group is semantically encoded to obtain the common feature data and the individual feature data of the target video frame group respectively, and the common feature data and the individual feature data are packaged and transmitted to the receiving end, so that the receiving end obtains the corresponding video frame group based on the received common feature data and the individual feature data, and the receiving end uses the video frame group as a new historical video frame group for video quality assessment, and then determines whether to generate video quality assessment result data of the new historical video frame group.

2. The video transmission adaptive optimization method according to claim 1, characterized in that: The video quality assessment result data is used to represent the first quality feedback information or the second quality feedback information; The first quality feedback information is used to indicate that the video quality assessment values ​​corresponding to the plurality of consecutive video frames in the historical video frame group are all higher than a preset upper threshold; The second quality feedback information is used to indicate that the video quality assessment values ​​corresponding to each of a plurality of consecutive video frames in the historical video frame group are all lower than a preset lower threshold; wherein the lower threshold is smaller than the upper threshold.

3. The video transmission adaptive optimization method according to claim 2, characterized in that: If the video quality assessment result data of the historical video frame group is obtained, the ratio of commonality to individuality features corresponding to the target video frame group is adaptively optimized, including: If video quality assessment result data of a first historical video frame group is received and the video quality assessment result data currently represents the first quality feedback information, then reducing the commonality-to-individuality feature ratio corresponding to the current target video frame group; The first historical video frame group is a historical video frame group in the video data that is located before the current target video frame group, and the first historical video frame group and the target video frame group are adjacent to or spaced apart by a preset number of video frame groups.

4. The video transmission adaptive optimization method according to claim 2, characterized in that: If the video quality assessment result data of the historical video frame group is obtained, the ratio of commonality to individuality features corresponding to the target video frame group is adaptively optimized, including: If video quality assessment result data of a second historical video frame group is received and the video quality assessment result data currently represents the second quality feedback information, then increasing the commonality-individuality feature ratio corresponding to the current target video frame group; The second historical video frame group is a historical video frame group in the video data that is located before the current target video frame group, and the second historical video frame group is adjacent to or spaced apart from the target video frame group by a preset number of video frame groups.

5. The video transmission adaptive optimization method according to claim 1, characterized in that: Before semantically encoding the target video frame group based on the current commonality-to-individuality feature ratio to obtain the commonality feature data and individuality feature data of the target video frame group, the method further includes: If, within a preset time period after the common feature data and individual feature data corresponding to the third historical video frame group are encapsulated and transmitted to the receiving end, the video quality assessment result data for the third historical video frame group is not received, the common feature to individual feature ratio corresponding to a historical video frame group located before the third historical video frame group in the video data is used as the common feature to individual feature ratio corresponding to the target video frame group to be transmitted currently; The third historical video frame group is a historical video frame group in the video data that is located before the current target video frame group, and the third historical video frame group is adjacent to or spaced apart from the target video frame group by a preset number of video frame groups.

6. A video transmission adaptive optimization method, characterized in that: include: Based on the common feature data and individual feature data currently received from the sending end, a video frame group belonging to the video data corresponding to the common feature data and the individual feature data is obtained, and the video frame group is used as a historical video frame group for video quality assessment to determine whether to generate video quality assessment result data of the historical video frame group; If video quality assessment result data of the historical video frame group is generated, the video quality assessment result data of the historical video frame group is sent to the sending end, so that the sending end adaptively optimizes the commonality-to-individuality feature ratio corresponding to the current target video frame group according to the video quality assessment result data, and enables the sending end to semantically encode the target video frame group based on the current commonality-to-individuality feature ratio to respectively obtain the commonality feature data and individuality feature data of the target video frame group, so as to encapsulate the commonality feature data and individuality feature data and transmit them, and the target video frame group is the video frame group to be transmitted corresponding to the video data.

7. The video transmission adaptive optimization method according to claim 6, characterized in that: The step of obtaining a video frame group belonging to the video data corresponding to the common feature data and the individual feature data based on the common feature data and the individual feature data currently received from the sending end includes: Receiving a data packet encapsulating common feature data and individual feature data from a transmitting end, and extracting the common feature data and individual feature data from the data packet; It is determined whether the common feature data and individual feature data currently extracted belong to the same target video frame group in the video data. If so, semantic decoding is performed on the common feature data and individual feature data to obtain a corresponding video frame group.

8. The video transmission adaptive optimization method according to claim 6, characterized in that: The step of using the video frame group as a historical video frame group to perform video quality assessment to determine whether to generate video quality assessment result data of the historical video frame group includes: The video frame group obtained after semantic decoding is used as a historical video frame group, and the video quality assessment is performed on the historical video frame group to obtain the video quality assessment values ​​corresponding to each of the consecutive multiple video frames in the historical video frame group, and it is determined whether the video quality assessment values ​​corresponding to each of the consecutive multiple video frames in the historical video frame group are all higher than a preset upper limit threshold or are all lower than a preset lower limit threshold, wherein the lower limit threshold is less than the upper limit threshold; If the video quality assessment values ​​corresponding to each of the plurality of consecutive video frames in the historical video frame group are all higher than a preset upper threshold, first quality feedback information is generated and video quality assessment result data corresponding to the historical video frame group is generated for representing the first quality feedback information; If the video quality assessment values ​​corresponding to each of the plurality of consecutive video frames in the historical video frame group are lower than a preset lower threshold, second quality feedback information is generated and video quality assessment result data corresponding to the historical video frame group is generated for representing the second quality feedback information.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements the video transmission adaptive optimization method as described in any one of claims 1 to 5, and / or implements the video transmission adaptive optimization method as described in any one of claims 6 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the video transmission adaptive optimization method as described in any one of claims 1 to 5, and / or implements the video transmission adaptive optimization method as described in any one of claims 6 to 8.

Images