Adaptive Network Video Transmission Method, Device, Equipment and Medium

Abstract

The present invention belongs to the technical field of video data processing, and solves the technical problem in the prior art that the video picture quality is poor due to network differences in different stages of the video transmission process, resulting in a poor user experience. The present invention provides an adaptive network video transmission method, device, equipment and medium. The method includes: obtaining network information at the initial stage of establishing a video data transmission channel; generating a first encoding method corresponding to the network information, and then encoding the initial video data; after the initial video data is successfully decoded, when the decoding is successful, output a first encoding instruction; then encoding subsequent video data using a second encoding method; wherein, the key frame interval time of the video data encoded by the first encoding method is less than the key frame interval time of the video data encoded by the second encoding method. Using this method can ensure video fluency in different stages of the video transmission process and improve the user experience effect.

Description

[0001] This application is a divisional application of the invention patent application with the application number 202110650703.9, titled "Video Compression and Transmission Method, Device, Equipment and Medium Based on Dynamically Encoded Frames", which was filed on June 11, 2021. Technical Field

[0002] The present invention relates to the technical field of video data processing, and particularly to an adaptive network video transmission method, device, equipment and medium. Background Art

[0003] With the development of video technology, it is already possible to view the real-time images collected by a camera terminal in real time through a mobile device, so as to master the activities of the target objects in the monitoring area of the camera; this technology is of great significance in the care of the elderly and infants, and can improve the care efficiency of infants and the elderly. The specific process of viewing the real-time images of the camera terminal on the mobile device is that the camera terminal caches the collected real-time images in the form of image frames to form a video stream; a complete video stream includes multiple I-frames and multiple P-frames, where the I-frame is a key frame and the P-frame is an ordinary frame.

[0004] In the prior art, after establishing video communication between the mobile device and the camera device, video data transmission begins. The quality of the video is closely related to the network bandwidth. If the network is good, a higher bitrate can be selected for video data transmission, and at this time the video image quality is high; if the network is not good, only a suitable bitrate can be selected for video data transmission, and at this time the video image quality is low. Then, as people's requirements for video image quality are getting higher and higher, when the image quality is not good, it will seriously affect the user experience. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide an adaptive network video transmission method, device, equipment and medium, which are used to solve the technical problem that the poor video image quality caused by network differences in different stages of the video transmission process in the prior art leads to a poor user experience.

[0006] The technical solution adopted by the present invention is as follows:

[0007] The present invention provides an adaptive network video transmission method, and the method includes:

[0008] Obtain the network information at the initial stage of establishing a video data transmission channel for video data transmission;

[0009] According to the network information, generate a first coding method corresponding to the video coding adapted to the network information, and encode the initial video data according to the first coding method;

[0010] Decode the transmitted video data encoded by the first encoding method. When the decoding is successful, output the first encoding instruction;

[0011] According to the first encoding instruction, encode and transmit the subsequent video data using the second encoding method;

[0012] Among them, the key frame interval time of the video data encoded by the first encoding method is less than the key frame interval time of the video data encoded by the second encoding method.

[0013] Preferably, during the subsequent video data transmission process, after encoding and transmitting the subsequent video data using the second encoding method according to the first encoding instruction, it further includes:

[0014] Supervise the transmitted video data encoded by the second encoding method. If there is data abnormality, output the second encoding instruction;

[0015] According to the second encoding instruction, encode and transmit the subsequent video data using the first encoding method corresponding to the initial video data;

[0016] Repeat decoding the transmitted video data encoded by the first encoding method. When the decoding is successful, output the first encoding instruction to encoding and transmitting the video data using the first encoding method corresponding to the initial video data according to the second encoding instruction until the video ends.

[0017] Preferably, the supervising the transmitted video data encoded by the second encoding method and outputting the second encoding instruction if there is data abnormality includes:

[0018] Real-time obtain the reception time of each frame image of the video data;

[0019] Calculate the reception speed of the current image frame according to the reception time of each frame image, so as to predict the reception prediction speed of receiving the next image frame;

[0020] According to the reception prediction speed of the next image frame, from the formula Calculate the first delay duration of receiving each frame image;

[0021] Determine whether to adjust the video encoding method of the subsequent video according to the first delay duration;

[0022] Among them, S t+1 is the reception prediction speed of the (n + 1)-th frame image, T' is the first delay time, β is the buffer allocation coefficient, 0 < β < 1, M s is the remaining buffer capacity of the video data sender, M R is the remaining buffer capacity of the video data receiver.

[0023] Preferably, the determining whether to adjust the video encoding method of subsequent videos according to the first delay duration includes:

[0024] Obtain the delay thresholds of each frame image preset;

[0025] Compare the first delay duration of each frame image with the corresponding preset delay threshold. If the first delay duration is greater than the delay threshold, output a second encoding instruction.

[0026] Preferably, the delay threshold is a dynamic value, and according to the formula delay threshold = τ * T, where τ is the delay jitter coefficient of data transmission and T is the total time required to receive n frame images.

[0027] Preferably, the determining whether to adjust the video encoding method of subsequent videos according to the first delay duration includes:

[0028] Obtain each of the first delay durations corresponding to each frame image in the video data;

[0029] Establish a data transceiver delay curve according to each of the first delay durations;

[0030] According to the data transceiver delay curve, statistically analyze the slope change direction of adjacent sampling points on the data transceiver curve, and output the positive statistical times corresponding to the continuous increase in delay;

[0031] Compare the positive statistical times with the preset threshold times of delay increase. If the positive statistical times are greater than or equal to the threshold times, adjust the video encoding method of subsequent videos and output a second encoding instruction.

[0032] Preferably, the calculating the receiving speed of the current image frame according to the receiving time of each frame image, so as to predict the receiving prediction speed of the next image frame includes:

[0033] According to the receiving time of each frame image, from the formula Calculate the receiving speed of the current image frame;

[0034] According to the receiving speed of the current image frame, from the formula Calculate the receiving prediction speed of the next image frame;

[0035] where n is the number of image frames, S t is the receiving speed of the nth frame image, T is the total time required to receive n frame images, B i is the data size of the ith frame image, S t+1 is the receiving prediction speed of the (n + 1)th frame image, S t-1 is the receiving speed of the (n - 1)th frame image, The smoothing jitter coefficient for video data transmission

[0036] The present invention also provides a video compression and transmission device based on dynamic coding frames, including:

[0037] The first coding instruction module: used to decode the video data encoded by the first coding method for transmission, and when the decoding is successful, output the first coding instruction;

[0038] The second data coding module: used to encode and transmit the video data according to the first coding instruction by using the second coding method;

[0039] The second coding instruction module: used to supervise the video data encoded by the second coding method for transmission, and if there is data abnormality, output the second coding instruction;

[0040] The first data coding module: used to encode and transmit the video data according to the second coding instruction by using the first coding method;

[0041] The data loop coding module: used to repeatedly decode the video data encoded by the first coding method for transmission, and when the decoding is successful, output the first coding instruction to encode and transmit the video data by using the first coding method according to the second coding instruction until the video ends;

[0042] Wherein, the key frame interval time of the video data encoded by the first coding method is less than the key frame interval time of the video data encoded by the second coding method.

[0043] The present invention also provides an electronic device, including: at least one processor, at least one memory, and computer program instructions stored in the memory, and when the computer program instructions are executed by the processor, the method described in any one of the above is implemented.

[0044] The present invention also provides a medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the method described in any one of the above is implemented.

[0045] In summary, the beneficial effects of the present invention are as follows:

[0046] An adaptive network video transmission method, device, equipment and medium provided by the present invention, in the initial stage of video data transmission, set the first coding method for video data coding according to the initial network state. When the initial video decoding is successful, generate the first coding instruction, and then encode the video data in the coding method of normal video data, so as to avoid poor video quality caused by transmission differences in different stages of video transmission and improve the user experience effect. Description of the Drawings

[0047] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the accompanying drawings required for the embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings, and all of them are within the protection scope of the present invention.

[0048] Figure 1 It is a schematic flowchart of the video compression and transmission method based on dynamic coding frames in Embodiment 1 of the present invention;

[0049] Figure 2 It is a schematic flowchart of obtaining the second coding instruction in Embodiment 1 of the present invention;

[0050] Figure 3 It is a schematic flowchart of obtaining the conditions for the second coding instruction in Embodiment 1 of the present invention;

[0051] Figure 4 It is a schematic structural diagram of the time curve of video data transceiver delay in Embodiment 1 of the present invention;

[0052] Figure 5 It is a schematic flowchart of obtaining the reliable code rate for video data transmission in Embodiment 1 of the present invention;

[0053] Figure 6 It is a schematic flowchart of screening the reliable transmission code rate from the test code rate in Embodiment 1 of the present invention;

[0054] Figure 7 It is a schematic flowchart of screening the reliable transmission code rate according to the traffic type in Embodiment 1 of the present invention;

[0055] Figure 8 It is a schematic structural diagram of the video compression and transmission device based on dynamic coding frames in Embodiment 2 of the present invention;

[0056] Figure 9 It is a schematic structural diagram of the electronic device in Embodiment 3 of the present invention. Detailed implementation manners

[0057] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. It should be noted that in this document, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. In the description of the present invention, it should be understood that the terms "center", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus cannot be construed as a limitation of the present invention. Moreover, the terms "include", "comprise", or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device including a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article, or device. Without further limitation, elements defined by the statement "including..." do not exclude the existence of additional identical elements in the process, method, article, or device including the said elements. If there is no conflict, the various features in the present invention and its embodiments can be combined with each other, and all are within the protection scope of the present invention.

[0058] Embodiment 1

[0059] Please refer to Figure 1 , Figure 1 which is a schematic flowchart of a video compression and transmission method based on dynamic coding frames in Embodiment 1 of the present invention. The method includes:

[0060] S1: Decode the video data transmitted and encoded by the first encoding method. When the decoding is successful, output the first encoding instruction;

[0061] Specifically, after the video data transmission is established between the user's App side and the camera terminal, determine a reliable video data transmission bitrate according to the current network conditions. After determining the reliable transmission bitrate, first encode the video data generated by the camera terminal using the first encoding method and then transmit it. Decode the video data encoded by the first encoding method on the user's App side. When the decoding is completed, that is, when the image is normally displayed, generate a first encoding instruction and feedback it to the camera terminal. This first encoding instruction is used to guide the video terminal to start encoding video data using the second encoding method. After receiving the first encoding instruction, it indicates that the established video connection can perform stable video data transmission. At this time, after the camera terminal completes the encoding of the video data packet corresponding to the current key frame, start encoding subsequent real-time video data using the second encoding method from the next key frame.

[0062] S2: According to the first encoding instruction, encode the video data using the second encoding method and then transmit it;

[0063] Specifically, after receiving the first encoding instruction, the camera terminal adjusts the encoding method of the video data and starts encoding subsequent real-time video data using the second encoding method.

[0064] S3: Supervise the transmitted video data encoded using the second encoding method. If there is data abnormality, output a second encoding instruction;

[0065] Specifically, during the transmission of the video data encoded using the second encoding method, supervise the transmitted video data to prevent video data abnormality. If video data abnormality is found, generate a second encoding instruction. The second encoding instruction is used to guide the video terminal to start encoding video data using the first encoding method.

[0066] In one embodiment, the parameters for supervising the transmitted video data encoded using the second encoding method include at least one of the following: video data decoding result, video data integrity, and delay information of video data sending and receiving.

[0067] In one embodiment, please refer to Figure 2 , in the supervision of the delay information of video data sending and receiving, S3 includes:

[0068] S31: Obtain the reception time of each frame image of the video data in real time;

[0069] Specifically, denote each frame image of the video data as F1, F2,... F n ; denote the size of each frame image as B1, B2,... B n ; the reception time of each frame image received by the App side is T1, T2,... T n; where n is the number of image frames included in the statistical window corresponding to each image frame. For example, if the statistical window contains 20 image frames and the number of image frames participating in the statistics is 20 frames, when starting the statistics from the 5th frame, the image frames within the statistical window are from the 5th frame to the 25th frame. As the video data is continuously transmitted, the specific image frames within the statistical window change, but the total number of image frames remains unchanged. For example, after the 26th frame image is received, the image frames within the statistical window at this time are from the 6th frame image to the 26th frame image, F n is the nth frame image, B n is the data size of the nth frame image, T n is the reception time of the nth frame image, where n is a positive integer and n is an integer multiple of the video frame rate.

[0070] S32: According to the reception time of each frame image, use the formula to calculate the reception speed of the current image frame;

[0071] Specifically, according to the reception time of the statistically n frame images, use the formula to calculate the reception speed of the nth frame image, where S t is the reception speed of the nth frame image, T is the total time required to receive n frame images, that is, T = T n - T1.

[0072] S33: According to the reception speed of the current image frame, use the formula to calculate the reception prediction speed of the next image frame;

[0073] Specifically, according to the reception speeds of the current image frame and the previous image frame, use the formula to predict the reception prediction speed of the next image frame, where S t+1 is the reception prediction speed of the (n + 1)th frame image, S t-1 is the reception speed of the (n - 1)th frame image, is the smoothing jitter coefficient of video data transmission,

[0074] S34: According to the reception prediction speed of the next image frame, use the formula to calculate the first delay duration for the reception of each frame image;

[0075] Specifically, according to the reception prediction speed of the next image frame, use the formula to calculate the first delay duration for the reception of each frame image, where T' is the first delay time, β is the buffer allocation coefficient, M s is the remaining buffer capacity of the video data sender, M Ris the remaining buffer capacity of the video data receiving end. The remaining buffer capacity of the sending end is directly obtained through the tag information carried by the received image frames. The remaining buffer capacity of the receiving end is directly obtained at the receiving end.

[0076] S35: Compare the first delay duration with a preset delay threshold. If the first delay duration is greater than the delay threshold, output a second encoding instruction.

[0077] Specifically, based on the remaining buffer capacities of the receiving end and the sending end, the predicted receiving speed of the next image frame can be obtained according to the receiving speed of the current image frame, and it can be estimated that the buffer capacity will be exhausted within a certain period of time in this state; therefore, on this basis, a threshold for the exhaustion time corresponding to the predicted speed is set, denoted as the delay threshold; when the first delay duration corresponding to the predicted receiving speed of a certain image frame to be received is greater than the delay threshold, output a second encoding instruction, and according to the first encoding instruction, the device terminal that generates the video data starts to encode the video data in the second encoding method.

[0078] In an embodiment, the delay threshold = τ * T, where τ is the delay jitter coefficient of data transmission and T is the total time required to receive n frames of images.

[0079] Specifically, by statistically analyzing the differences in the receiving times of each frame of images, T is a dynamic value, so that the generation of the second encoding instruction is dynamically changed, optimizing the generation of the second encoding instruction and improving the video quality.

[0080] In an embodiment, please refer to Figure 3 , the S35 includes:

[0081] S351: Obtain each of the first delay durations corresponding to each frame of images in the video data;

[0082] Specifically, it is calculated by the formula to obtain the first delay duration of each frame of image received.

[0083] S352: Establish a data transceiver delay curve according to each of the first delay durations;

[0084] Specifically, please refer to Figure 4 , Figure 4The data transmission and reception delay curve is a curve formed by using the first delay duration corresponding to each image frame as each sampling point. Sampling points a, b, c, d, e, f, g, h, and j respectively correspond to the delay sampling points corresponding to the 1st to 9th frames. Denote the slope corresponding to point a and point b as k1, the slope corresponding to point b and point c as k2, the slope corresponding to point c and point d as k3, the slope corresponding to point d and point e as k4, the slope corresponding to point e and point f as k5, the slope corresponding to point f and point g as k6, the slope corresponding to point g and point h as k7, and the slope corresponding to point h and point j as k8; from Figure 4 it can be seen that the slopes corresponding to k1, k2, k3, and k4 are positive, the slopes corresponding to k6, k7, and k8 are negative, and the slope of k5 is 0.

[0085] S353: According to the data transmission and reception delay curve, statistically analyze the slope change direction of adjacent sampling points on the data transmission and reception curve, and output the positive statistical count corresponding to the continuously increasing delay;

[0086] Specifically, calculate the slopes of the sampling points corresponding to all adjacent image frames, and successively statistically analyze the positive and negative of each slope, so as to obtain the delay change situation of each image frame transmission; the statistical method is: start counting from the first positive slope, add 1 to the count for one positive slope, and when encountering a negative slope, this statistical analysis ends, and start the next statistical analysis at the next positive slope; because it is the count of the number of positive slopes, it can quickly obtain that during this statistical process, the delay is continuously increasing; therefore, it is recorded as positive statistical analysis; please refer to Figure 4 and start counting from k1. When counting to k6, k6 is a negative slope, this positive statistical analysis ends, the positive statistical count for this time is 5, and start the next count.

[0087] S354: Compare the positive statistical count with a preset threshold count for the increasing delay. If the positive statistical count is greater than or equal to the threshold count, output a second coding instruction.

[0088] Specifically, during the statistical process, compare the positive statistical count with the threshold count in real time. When the positive statistical count is equal to the threshold count, generate a second coding instruction, and encode the subsequent video data using the first coding method. Please refer to Figure 4 If the threshold count is less than or equal to 5 times, and the positive statistical count corresponding to k1 is 6, and the statistical count is greater than the threshold count, generate a second coding instruction, so as to change the coding method of the video data to the first coding method; reduce the number of ordinary frames between adjacent key frames, thereby reducing the details of the video picture, so that the network can ensure the normal progress of the video.

[0089] S4: According to the second coding instruction, encode and transmit the video data using the first coding method;

[0090] Specifically, when abnormal situations such as video data delay, frame freezing, and data loss are detected, the camera terminal encodes the newly generated video data using the first encoding method for transmission through the second encoding instruction.

[0091] S5: Repeat S1 to S4 until the video ends.

[0092] Specifically, continuously switch the encoding method of the video data between the first encoding method and the second encoding method until the video ends, so as to realize the dynamic encoding of the video data when the video is played.

[0093] Among them, the key frame interval time of the video data encoded by the first encoding method is less than the key frame interval time of the video data encoded by the second encoding method.

[0094] Specifically, the basic data of the video data is mainly reflected in the key frames, and the details of the video data are reflected by the ordinary frames. When the video data transmission is abnormal, the first encoding method is selected to encode the video data; when the video data transmission is normal, the second encoding method is used to encode the video data. The video frame rates of the first encoding method and the second encoding method are the same. The number of ordinary frames corresponding to the key frame of the second encoding method is more than that of the first encoding method. The video data encoded by the first encoding method can ensure the normal progress of the video, and the video data encoded by the second encoding method can improve the picture clarity. The video data encoded by the first encoding method can ensure the normal progress of the video; for example, if the video frame rate is 20 frames per second, the first encoding method includes 1 key frame and 19 ordinary frames after the key frame per second, and the second encoding method includes 1 key frame and 79 ordinary frames after the key frame every 4 seconds.

[0095] It should be noted that: The key frame is also called the I frame, and the ordinary frames include P frames and / or B frames.

[0096] In one embodiment, the video frame rate of the first encoding method is the same as the video frame rate of the second encoding method.

[0097] In one embodiment, please refer to Figure 5 , before the S1, it further includes:

[0098] S01: Send a video request instruction.

[0099] S02: Establish a video connection according to the video request instruction.

[0100] S03: After the video connection is successful, perform a bit rate test for video output transmission and output a reliable transmission bit rate.

[0101] Specifically, when the user needs to view the live video at the camera end from a non-key frame, a video request is sent. After establishing a video connection between the non-key frame and the camera terminal, the current network latency is tested by sending test data, and the video bitrate is adjusted according to the latency situation to obtain the reliable transmission bitrate for the current video data transmission.

[0102] In one embodiment, please refer to Figure 6 , where S03 includes:

[0103] S031: Obtain multiple test bitrates for bitrate testing;

[0104] Specifically, before the camera leaves the factory, multiple bitrates for video data transmission are preset in the processor of the camera so that after the user establishes a video connection with the camera, the transmission bitrate can be tested to determine the reliable transmission bitrate for this video.

[0105] S032: Transmit video data in sequence at each of the test bitrates to obtain test video data corresponding to each of the test bitrates;

[0106] S033: Analyze the upload speed and download speed of each of the test data to obtain a second delay duration for data transmission and reception corresponding to each of the test bitrates;

[0107] Specifically, transmit test video data at each test bitrate, obtain the upload speed and download speed of the video data corresponding to each test bitrate, so as to determine the delay duration corresponding to each test bitrate, denoted as the second delay duration.

[0108] S034: Compare each of the second delay durations and output the test bitrate corresponding to the data transmission and reception delay duration that meets the requirements as the reliable transmission bitrate.

[0109] Specifically, compare the second delay durations corresponding to each test bitrate, and use the test bitrate corresponding to the delay duration that meets the requirements as the reliable transmission bitrate. Further, if there are multiple reliable transmission bitrates, they can be selected according to the network type selected by the user at this time, or according to the user's historical network; for example, when the user uses mobile data for video, a low bitrate that meets the requirements can be selected as the reliable bitrate to avoid using too much mobile traffic due to high-definition picture quality, resulting in insufficient mobile traffic for the user later and affecting the user experience.

[0110] In one embodiment, if there are multiple test bitrates that meet the requirements, the reliable transmission bitrate is selected according to the traffic type of the user's Internet access at this time.

[0111] Specifically, the traffic types include at least one of the following: mobile traffic, traffic provided by a hotspot, and Wi-Fi traffic. When the user accesses the Internet using non-Wi-Fi traffic, a low code rate that meets the requirements is selected as the reliable transmission code rate. When the user accesses the Internet using Wi-Fi traffic, a high code rate that meets the requirements is selected as the reliable transmission code rate.

[0112] In an embodiment, if the traffic type for the user to access the Internet at this time is mobile traffic, S034 includes:

[0113] S0341: Obtain the total amount of mobile traffic available to the user and the proportion of the remaining traffic to the total mobile traffic corresponding to the available traffic;

[0114] S0342: Determine the reliable transmission code rate from multiple test code rates that meet the requirements according to the total mobile traffic and the available traffic proportion.

[0115] Specifically, when the user accesses the Internet using mobile traffic, analyze the total amount of remaining traffic in the user's mobile package at this time and the proportion of the remaining traffic. If the total traffic volume and / or the available traffic proportion is insufficient, a low code rate is selected for video data transmission. Further, during or before the start of video transmission, if the traffic exceeds the total available traffic volume, a prompt is given to avoid waste of traffic caused by the user's incorrect operation and improve the user experience.

[0116] S04: Transmit the video data in the first encoding method at the reliable transmission code rate.

[0117] Specifically, after determining the reliable transmission code rate of the video data, in the initial stage, the video data is encoded in the first encoding method, and the encoded video data is transmitted at the reliable transmission code rate.

[0118] Using the video compression and transmission method based on dynamic encoding frames in this embodiment, during the video data transmission process, in case of abnormal situations such as unstable network, the first encoding method is selected to encode and transmit the video data to ensure normal video playback. When the network is stable, the second encoding method is selected to encode and transmit the video data, thereby improving video clarity; according to the actual network fluctuations that cause changes in video quality, the dynamic encoding method is used to select a suitable video encoding method for encoding the video data, which can ensure the smoothness of the user's video and improve the user experience.

[0119] Embodiment 2

[0120] Please refer to Figure 7 , Figure 7FIG. 0 is a schematic structural diagram of the video compression and transmission device based on dynamic coding frames in Embodiment 2 of the present invention. Embodiment 2 correspondingly provides a video compression and transmission device based on dynamic coding frames based on the video compression and transmission method in Embodiment 1. The device includes:

[0121] The first coding instruction module: It is used to decode the transmitted video data encoded by the first coding method. When the decoding is successful, it outputs the first coding instruction;

[0122] The second data coding module: It is used to encode the video data by using the second coding method according to the first coding instruction and then transmit it;

[0123] The second coding instruction module: It is used to supervise the transmitted video data encoded by the second coding method. If there is data abnormality, it outputs the second coding instruction;

[0124] The first data coding module: It is used to encode the video data by using the first coding method according to the second coding instruction and then transmit it;

[0125] The data loop coding module: It is used to repeatedly decode the transmitted video data encoded by the first coding method. When the decoding is successful, it outputs the first coding instruction to encode the video data by using the first coding method according to the second coding instruction and then transmit it until the video ends;

[0126] Among them, the key frame interval time of the video data encoded by the first coding method is less than the key frame interval time of the video data encoded by the second coding method.

[0127] By using the video compression and transmission device based on dynamic coding frames in this embodiment, during the video data transmission process, in case of abnormal situations such as unstable network, the first coding method is selected to encode and transmit the video data to ensure the normal progress of the video. When the network is stable, the second coding method is selected to encode and transmit the video data, thereby improving the video clarity; according to the actual network fluctuation causing the change of video quality, the dynamic coding method is used to select the appropriate video coding method to encode the video data, which can ensure the video fluency of the user and improve the user experience effect.

[0128] In one embodiment, the parameters for supervising the transmitted video data encoded by the second coding method at least include one of the following: the video data decoding result, the video data integrity, and the delay information of video data reception and transmission.

[0129] In one embodiment, the second coding instruction module includes:

[0130] The duration receiving and transmitting unit: It obtains the reception time of each frame image of the video data in real time;

[0131] Receiving speed unit: According to the receiving time of each frame of image, calculate the receiving speed of the current image frame by the formula ;

[0132] Receiving predicted speed unit: According to the receiving speed of the current image frame, calculate the receiving predicted speed of the next image frame by the formula ;

[0133] First delay unit: According to the receiving predicted speed of the next image frame, calculate the first delay duration for receiving each frame of image by the formula ;

[0134] Second encoding instruction unit: Compare the first delay duration with a preset delay threshold. If the first delay duration is greater than the delay threshold, output a second encoding instruction;

[0135] where n is the number of image frames, S t is the receiving speed of the nth frame of image, T is the total time required to receive n frames of images, B i is the data size of the ith frame of image, S t+1 is the receiving predicted speed of the (n + 1)th frame of image, S t-1 is the receiving speed of the (n - 1)th frame of image, is the smoothing jitter coefficient of video data transmission, T' is the first delay time, β is the buffer allocation coefficient, 0 < β < 1, M s is the remaining buffer capacity of the video data sender, M R is the remaining buffer capacity of the video data receiver.

[0136] In one embodiment, the second instruction unit includes:

[0137] Delay acquisition unit: Acquire each of the first delay durations corresponding to each frame of image in the video data;

[0138] Delay curve unit: Establish a data transceiver delay curve according to each of the first delay durations;

[0139] Delay statistics unit: According to the data transceiver delay curve, statistically analyze the slope change direction of adjacent sampling points on the data transceiver curve, and output the positive statistical times corresponding to the continuous increase of the delay;

[0140] Delay processing unit: Compare the positive statistical times with a preset threshold number of times for delay increase. If the positive statistical times are greater than or equal to the threshold number of times, output a second encoding instruction.

[0141] In one embodiment, the first encoding instruction module further includes:

[0142] Video request unit: issue a video request instruction;

[0143] Video connection unit: establish a video connection according to the video request instruction;

[0144] Bit rate test unit: after the video connection is successful, perform a bit rate test on the video output transmission and output the reliable transmission bit rate;

[0145] Initial video decoding unit: transmit video data at the reliable transmission bit rate according to the first coding method.

[0146] In one embodiment, the bit rate test unit includes:

[0147] Test bit rate unit: obtain multiple test bit rates for bit rate testing;

[0148] Test data processing unit: sequentially transmit video data at each of the test bit rates to obtain test video data corresponding to each of the test bit rates;

[0149] Network speed analysis unit: analyze the upload speed and download speed of each of the test data to obtain a second delay duration of data transmission and reception corresponding to each of the test bit rates;

[0150] Reliable bit rate unit: compare each of the second delay durations and output the test bit rate corresponding to the data transmission and reception delay duration that meets the requirements as the reliable transmission bit rate.

[0151] In one embodiment, if the traffic type of the user's Internet access at this time is mobile traffic, the reliable bit rate unit includes:

[0152] Traffic data unit: obtain the total amount of mobile traffic available to the user and the available traffic ratio of the remaining traffic to the total mobile traffic;

[0153] Traffic data processing unit: determine the reliable transmission bit rate from multiple test bit rates that meet the requirements according to the total mobile traffic and the available traffic ratio.

[0154] With the video compression and transmission device based on dynamic coding frames of this embodiment, during the video data transmission process, in case of abnormal situations such as unstable network, the first coding method is selected to encode and transmit the video data to ensure the normal progress of the video. When the network is stable, the second coding method is selected to encode and transmit the video data, thereby improving the video clarity; according to the actual network fluctuations causing changes in video quality, the dynamic coding method is used to select a suitable video coding method for encoding the video data, which can ensure the smoothness of the user's video and improve the user experience effect.

[0155] Embodiment 3

[0156] The present invention provides an electronic device and a medium, as Figure 8 shown, including at least one processor, at least one memory, and computer program instructions stored in the memory.

[0157] Specifically, the above-mentioned processor may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. The electronic device includes at least one of the following: a camera, a mobile device with a camera, and a wearable device with a camera.

[0158] The memory may include a mass storage for data or instructions. By way of example and not limitation, the memory may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. In a suitable case, the memory may include a removable or non-removable (or fixed) medium. In a suitable case, the memory may be internal or external to the data processing device. In a specific embodiment, the memory is a non-volatile solid-state memory. In a specific embodiment, the memory includes a read-only memory (ROM). In a suitable case, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or a flash memory, or a combination of two or more of these.

[0159] The processor reads and executes the computer program instructions stored in the memory to implement any one of the video compression and transmission methods based on dynamic coding frames in the first mode of the above embodiments.

[0160] In an example, the electronic device may further include a communication interface and a bus. Among them, the processor, the memory, and the communication interface are connected through the bus and complete communication with each other.

[0161] The communication interface is mainly used to implement communication between the modules, devices, units, and / or devices in the embodiments of the present invention.

[0162] A bus includes hardware, software, or both, and couples components of an electronic device to each other. By way of example and not limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable bus or a combination of two or more of these. Where appropriate, the bus may include one or more buses. Although embodiments of the present invention describe and illustrate specific buses, the present invention contemplates any suitable bus or interconnect.

[0163] In summary, embodiments of the present invention provide a video compression and transmission method, apparatus, device, and medium based on dynamically encoded frames. During the video data transmission process, when abnormal situations such as unstable network occur, a first encoding method is selected to encode and transmit the video data to ensure the normal progress of the video. When the network becomes stable, a second encoding method is selected to encode and transmit the video data, thereby improving the video clarity. According to the actual network fluctuations that cause changes in video quality, a suitable video encoding method is selected for the video data using the dynamic encoding method, which can ensure the smoothness of the user's video and improve the user experience effect.

[0164] It should be clear that the present invention is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and illustrated as examples. However, the method process of the present invention is not limited to the specific steps described and illustrated. Those skilled in the art can make various changes, modifications, and additions, or change the order between steps after understanding the spirit of the present invention.

[0165] The functional blocks shown in the above-described structural block diagrams can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a functional card, and so on. When implemented in software, the elements of the present invention are programs or code segments for performing the required tasks. The program or code segment can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via a data signal carried in a carrier wave. A "machine-readable medium" can include any medium that can store or transmit information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, and so on. The code segment can be downloaded via a computer network such as the Internet, an intranet, and so on.

[0166] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An adaptive network video transmission method, characterized in that The method includes: Obtaining network information at the initial stage of establishing a video data transmission channel for video data transmission; Generating a first encoding method corresponding to video encoding adapted to the network information according to the network information, and encoding initial video data according to the first encoding method; Decoding the transmitted video data encoded by the first encoding method, and when the decoding is successful, outputting a first encoding instruction; Encoding and transmitting subsequent video data using a second encoding method according to the first encoding instruction; Wherein, the key frame interval time of the video data encoded by the first encoding method is less than the key frame interval time of the video data encoded by the second encoding method; After encoding and transmitting subsequent video data using the second encoding method according to the first encoding instruction, it further includes: Supervising the transmitted video data encoded by the second encoding method, and if there is data abnormality, outputting a second encoding instruction; Encoding and transmitting subsequent video data in the first encoding method corresponding to the initial video data according to the second encoding instruction; Repeating decoding the transmitted video data encoded by the first encoding method, and when the decoding is successful, outputting the first encoding instruction to encoding and transmitting the video data in the first encoding method corresponding to the initial video data according to the second encoding instruction until the video ends; The supervising the transmitted video data encoded by the second encoding method, and if there is data abnormality, outputting a second encoding instruction includes: Obtaining the reception time of each frame image of the video data in real time; Calculating the reception speed of the current image frame according to the reception time of each frame image, so as to predict the reception prediction speed of receiving the next image frame; According to the predicted speed of the next image frame, the first delay duration for receiving each frame of image is calculated by the formula ​ Determining whether to adjust the video encoding method of subsequent videos according to the first delay duration; Among them, S t+1 is the received prediction speed of the (n + 1)-th frame image, T' is the first delay time, β is the buffer allocation coefficient, 0 < β < 1, M s is the remaining buffer capacity of the video data sender, M R is the remaining buffer capacity of the video data receiver.

2. The adaptive network video transmission method according to claim 1, wherein, The determining whether to adjust the video encoding method of subsequent videos according to the first delay duration includes: Obtaining the preset delay threshold for each frame image; Comparing the first delay duration of each frame image with the preset corresponding delay threshold, and if the first delay duration is greater than the delay threshold, outputting a second encoding instruction.

3. The adaptive network video transmission method according to claim 2, wherein The delay threshold is a dynamic value, and according to the formula delay threshold = τ * T, τ is the delay jitter coefficient of data transmission, and T is the total time required to receive n frames of images.

4. The adaptive network video transmission method according to claim 1, characterized in that, The determining whether to adjust the video encoding method of subsequent videos according to the first delay duration includes: Obtaining each of the first delay durations corresponding to each frame image in the video data; Establishing a data transceiver delay curve according to each of the first delay durations; According to the data transceiver delay curve, statistically analyzing the slope change direction of adjacent sampling points on the data transceiver curve, and outputting the positive statistical times corresponding to the continuously increasing delay; Comparing the positive statistical times with the preset threshold times of delay increase, and if the positive statistical times are greater than or equal to the threshold times, adjusting the video encoding method of subsequent videos and outputting a second encoding instruction.

5. The adaptive network video transmission method according to any one of claims 1 to 4, characterized in that The calculating the reception speed of the current image frame according to the reception time of each frame image, so as to predict the reception prediction speed of receiving the next image frame includes: According to the reception time of each frame image, the reception speed of the current image frame is calculated by the formula ​ According to the reception speed of the current image frame, the reception prediction speed of the next image frame is calculated by the formula ; Among them, n is the number of image frames, S t is the reception speed of the nth frame image, T is the total time required to receive n frame images, B i is the data size of the ith frame image, S t+1 is the reception prediction speed of the (n + 1)th frame image, S t-1 is the reception speed of the (n - 1)th frame image, ∂ is the smoothing jitter coefficient of video data transmission, 0 < ∂ < 1.

6. An adaptive network video transmission device, characterized in that Including: Network information acquisition module: Acquire network information at the initial stage of establishing a video data transmission channel for video data transmission; Initial video data encoding module: Generate a first encoding method corresponding to video encoding adapted to the network information according to the network information, and encode the initial video data according to the first encoding method; Initial video data decoding module: Decode the transmitted video data encoded by the first encoding method, and output a first encoding instruction when the decoding is successful; Conventional video data encoding module: Encode and transmit subsequent video data using a second encoding method according to the first encoding instruction; Wherein, the key frame interval time of the video data encoded by the first encoding method is less than the key frame interval time of the video data encoded by the second encoding method; After encoding and transmitting subsequent video data using the second encoding method according to the first encoding instruction, it further includes: Supervise the transmitted video data encoded by the second encoding method. If data anomalies exist, output a second encoding instruction; Encode and transmit subsequent video data in the first encoding method corresponding to the initial video data according to the second encoding instruction; Repeat decoding the transmitted video data encoded by the first encoding method. When the decoding is successful, output the first encoding instruction to encoding and transmitting the video data in the first encoding method corresponding to the initial video data according to the second encoding instruction until the video ends; The supervising the transmitted video data encoded by the second encoding method and outputting a second encoding instruction if data anomalies exist includes: Obtain the reception time of each frame image of the video data in real time; Calculate the reception speed of the current image frame according to the reception time of each frame image, so as to predict the reception prediction speed of receiving the next image frame; According to the predicted speed of the next image frame, the first delay duration received for each frame of image is calculated by the formula ​ Determine whether to adjust the video encoding method of subsequent videos according to the first delay duration; Wherein, St+1 is the reception prediction speed of the (n + 1)-th frame image, T' is the first delay time, β is the buffer allocation coefficient, 0 < β < 1, Ms is the remaining buffer capacity of the video data sender, and MR is the remaining buffer capacity of the video data receiver.

7. An electronic device, characterized in that, It includes: At least one processor, at least one memory, and computer program instructions stored in the memory, which implement the method according to any one of claims 1-5 when the computer program instructions are executed by the processor.

8. A medium on which computer program instructions are stored, characterized in that, When the computer program instructions are executed by the processor, the method according to any one of claims 1-5 is implemented.

Images