Video transmission delay prediction method, electronic equipment and computer readable storage medium

Abstract

The invention discloses a video transmission time delay prediction method, electronic equipment and a computer readable storage medium, the method is applied to a sending end, the sending end obtains a wireless signal state in real time, and the method comprises the following steps: in response to establishment of connection with a receiving end, obtaining a video packet corresponding to a video frame, and sending a target video packet of a current period to the receiving end; in response to feedback information received by a receiving end for the target video packet in the current period, determining a network transmission parameter of the current period based on the target video packet and the feedback information, and obtaining an estimated time delay of a subsequent specified period based on the current wireless signal state and the network transmission parameter of the current period; in response to the situation that feedback information of the receiving end for the target video packet is not received in the current period, obtaining an estimated time delay of a subsequent specified period based on the current wireless signal state; wherein the pre-estimated time delay is used for adjusting a sending strategy of the video packet in a subsequent specified period. Through the method, the timeliness and the accuracy of time delay prediction can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of data transmission, and in particular to a method for predicting video transmission delay, an electronic device, and a computer-readable storage medium. Background Art

[0002] As wireless networks are increasingly used in data transmission, in the process of video transmission, the sender usually connects to the cloud platform through a wireless network, and the receiver accesses the cloud platform through a wireless network or a mobile network to view the video. After the sender accesses the wireless network, it is affected by many factors such as installation location, wireless interference, network delay, and wireless resource competition. The existing video transmission delay prediction method is difficult to predict the round-trip time (RTT) in a timely manner, and the accuracy of the delay is not high. In view of this, how to improve the timeliness and accuracy of delay prediction has become an urgent problem to be solved. Summary of the invention

[0003] The main technical problem solved by the present application is to provide a video transmission delay prediction method, an electronic device and a computer-readable storage medium, which can improve the timeliness and accuracy of delay prediction.

[0004] To solve the above-mentioned technical problems, the first aspect of the present application provides a video transmission delay prediction method, which is applied to a transmitting end, and the transmitting end obtains a wireless signal status in real time, including: in response to establishing a connection with a receiving end, obtaining a video packet corresponding to a video frame, and sending a target video packet of the current period to the receiving end; in response to receiving feedback information from the receiving end for the target video packet within the current period, determining the network transmission parameters of the current period based on the target video packet and the feedback information, and obtaining an estimated delay for a subsequent specified period based on the current wireless signal status and the network transmission parameters of the current period; in response to not receiving feedback information from the receiving end for the target video packet within the current period, obtaining an estimated delay for a subsequent specified period based on the current wireless signal status; wherein the estimated delay is used to adjust the sending strategy of the video packet in the subsequent specified period.

[0005] To solve the above technical problem, the second aspect of the present application provides an electronic device, which includes: a memory and a processor coupled to each other, wherein the memory stores program data, and the processor calls the program data to execute the method described in the first aspect.

[0006] In order to solve the above technical problem, the third aspect of the present application provides a computer-readable storage medium on which program data is stored. When the program data is executed by a processor, the method described in the first aspect is implemented.

[0007] In the above scheme, after the transmitting end establishes a connection with the receiving end, the video packet corresponding to the video frame required by the receiving end is obtained, and the target video packet to be sent in the current cycle is sent to the receiving end. When the feedback information of the receiving end for the target video packet is received in the current cycle, the network transmission parameters measured in the current cycle are determined based on the target video packet and the feedback information, and the wireless signal state obtained in real time by the transmitting end is obtained. Based on the current wireless signal state and the network transmission parameters measured in the current cycle, the round-trip delay of the subsequent specified cycle is predicted, and the estimated delay of the subsequent specified cycle is obtained. In this way, when the feedback information of the receiving end is obtained, the delay of the subsequent specified cycle is predicted through multi-dimensional data, thereby improving the accuracy of the delay prediction. When no feedback information for a target video packet is received from the receiving end in the current cycle, the wireless signal state obtained in real time by the transmitting end is obtained, and based on the current wireless signal state, the round-trip delay of a subsequent specified cycle is predicted to obtain the estimated delay of the subsequent specified cycle, thereby reducing the dependence on feedback information when no feedback information is obtained from the receiving end, and timely using the wireless state information to predict the delay of the subsequent specified cycle to improve the timeliness of the delay prediction, wherein the estimated delay is used to adjust the sending strategy of the video packet in the subsequent specified cycle, thereby timely adjusting and confirming the sending strategy according to the estimated delay, and improving the quality of the video received by the receiving end. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them: Figure 1 It is a flowchart of an implementation method of a video transmission delay prediction method of the present application; Figure 2 It is a flowchart of another implementation method of the video transmission delay prediction method of the present application; Figure 3 This is a schematic diagram of an application scenario of an implementation method of a video transmission delay prediction method of the present application; Figure 4 It is a structural schematic diagram of an embodiment of the electronic device of the present application; Figure 5 It is a structural schematic diagram of an implementation method of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0009] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments, and different implementation methods can be adaptively combined. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0010] The terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship. In addition, "many" in this article means two or more than two.

[0011] The video transmission delay prediction method provided in the present application is applied to the sending end, and the sending end obtains the wireless signal status in real time to predict the round-trip delay RTT of the video transmission. The corresponding execution subject is the processing unit of the sending end.

[0012] See also Figure 1 , Figure 1 1 is a flow chart of an implementation method of a video transmission delay prediction method of the present application, the method comprising: S101: In response to establishing a connection with a receiving end, a video packet corresponding to a video frame is obtained, and a target video packet of a current period is sent to the receiving end.

[0013] Specifically, after the sending end establishes a connection with the receiving end, the video packet corresponding to the video frame required by the receiving end is obtained, and the target video packet required to be sent in the current cycle is sent to the receiving end.

[0014] It should be noted that the need to establish a connection is usually initiated by the receiving end. After receiving the need to establish a connection, the handshake phase begins, and the sending end attempts to establish a connection with the receiving end. For example, when the sending end is a network camera and the receiving end is a mobile terminal, when the mobile terminal attempts to receive the video stream, the network camera attempts to establish a connection with the mobile terminal.

[0015] Optionally, the sending end and the receiving end communicate through a reliable User Datagram Protocol (UDP), so as to establish a reliable connection based on the reliable UDP protocol and to be able to obtain feedback information sent by the receiving end to the sending end in a timely manner.

[0016] In some implementation scenarios, after a connection is established between a sending end and a receiving end, the sending end obtains video frames required by the receiving end, divides the video frames into multiple video packets based on the data volume of the video frames, and determines the video packets that need to be sent to the receiving end in the current cycle as target video packets, wherein the number of target video packets is at least one, and the target video packets that need to be sent in the current cycle are sent to the receiving end.

[0017] In some implementation scenarios, after a connection is established between a sending end and a receiving end, the video frame required by the receiving end is obtained, and based on the image size of the video frame, the video frame is divided into multiple video packets of the same size and the amount of data corresponding to the video packet is obtained, and the video packet that needs to be sent to the receiving end in the current cycle is determined as the target video packet, wherein the number of the target video packet is at least one, and the target video packet that needs to be sent in the current cycle is sent to the receiving end.

[0018] S102: In response to feedback information received from the receiving end for a target video packet in the current period, determine the network transmission parameters of the current period based on the target video packet and the feedback information, and obtain an estimated delay for a subsequent specified period based on the current wireless signal state and the network transmission parameters of the current period.

[0019] Specifically, when feedback information about the target video packet is received from the receiving end in the current cycle, the network transmission parameters measured in the current cycle are determined based on the target video packet and the feedback information, and the wireless signal status obtained in real time by the sending end is obtained. Based on the current wireless signal status and the network transmission parameters measured in the current cycle, the round-trip delay of the subsequent specified cycle is predicted to obtain the estimated delay of the subsequent specified cycle.

[0020] It is understandable that, when feedback information is obtained from the receiving end, the delay of the subsequent specified period is predicted through multi-dimensional data to improve the accuracy of the delay prediction.

[0021] Optionally, the feedback information may be an acknowledgment character (ACK) or other reply message based on the communication protocol setting, and this application does not impose any specific restrictions on this.

[0022] In some implementation scenarios, when feedback information for a target video packet is received from the receiving end in the current cycle, the network transmission parameters corresponding to the process from the target video packet being sent to the receiving end in the current cycle are determined based on the target video packet and the feedback information, the wireless signal state obtained in real time by the sending end is obtained, and the current wireless signal state and the network transmission parameters measured in the current cycle are input into a pre-trained dual-input prediction model to obtain the estimated delay of the subsequent specified cycle output by the dual-input prediction model. The dual-input prediction model is obtained by supervised training using training data with training labels.

[0023] In some implementation scenarios, when feedback information for a target video packet is received from the receiving end in the current cycle, the network transmission parameters corresponding to the process from the target video packet being sent to the receiving end in the current cycle are determined based on the target video packet and the feedback information, the wireless signal status obtained in real time by the sending end is obtained, a prompt text matching the network transmission parameters and the wireless signal status is constructed, and the prompt text, the current wireless signal status, and the network transmission parameters measured in the current cycle are input into the fine-tuned large language model to obtain the estimated delay of the subsequent specified cycle output by the large language model. The large language model is fine-tuned using the wireless signal status, network transmission parameters, and actual delay of the historical cycle.

[0024] In a specific implementation scenario, the actual delay of the current period is determined based on the timestamps of the target video packet and the feedback information, and the actual delay is used as the network transmission parameter of the current period.

[0025] In a specific implementation scenario, the actual delay of the current cycle is determined based on the timestamp of the target video packet and the feedback information, and the data transmission rate of the current cycle is determined based on the data volume corresponding to the target video packet. The actual delay and data transmission rate are used as network transmission parameters of the current cycle.

[0026] It should be noted that the subsequent designated period may be the period next to the current period, or may be the Nth period after the current period, where N is greater than 1.

[0027] Optionally, if feedback information about the target video packet is received from the receiving end within the current period, the number of intervals between the subsequent specified period and the current period is determined based on the current wireless signal state and the network transmission parameters measured in the current period, wherein the number of intervals is greater than 1, so that high-precision estimated delay can be obtained by delay prediction based on multi-dimensional parameters, thereby increasing the number of intervals and reducing resource consumption caused by delay prediction.

[0028] S103: In response to not receiving feedback information from the receiving end for the target video packet in the current period, obtaining an estimated delay in a subsequent specified period based on the current wireless signal state; wherein the estimated delay is used to adjust a sending strategy for the video packet in the subsequent specified period.

[0029] Specifically, when no feedback information for the target video packet is received from the receiving end in the current cycle, the real-time wireless signal state of the transmitting end is obtained, and the round-trip delay of the subsequent specified cycle is predicted based on the current wireless signal state to obtain the estimated delay of the subsequent specified cycle.

[0030] It can be understood that in the absence of feedback information from the receiving end, the reliance on feedback information is reduced, and the wireless status information is used in a timely manner to predict the delay of the subsequent specified period, thereby improving the timeliness of the delay prediction. The estimated delay is used to adjust the sending strategy of the video packet in the subsequent specified period, so as to adjust and confirm the sending strategy in time according to the estimated delay, thereby improving the quality of the video received by the receiving end.

[0031] Optionally, if feedback information of the receiving end for the target video packet is received in the current cycle, the subsequent designated cycle is the next cycle of the current cycle, thereby reducing the number of intervals and improving the accuracy of delay prediction when delay prediction is performed only based on the wireless signal state.

[0032] In some implementation scenarios, when no feedback information is received from the receiving end for the target video packet in the current cycle, the wireless signal state obtained by the transmitting end in real time is obtained, and the current wireless signal state is input into a pre-trained single-input prediction model to obtain the estimated delay of the subsequent specified cycle output by the single-input prediction model. The single-input prediction model is obtained by supervised training using training data with training labels.

[0033] In some implementation scenarios, when no feedback information is received from the receiving end for the target video packet in the current cycle, the wireless signal state obtained by the transmitting end in real time is obtained, and the current wireless signal state is input into the fine-tuned large language model to obtain the estimated delay of the subsequent specified cycle output by the large language model. The large language model is fine-tuned using the wireless signal state and actual delay of the historical cycle.

[0034] Optionally, when feedback information is obtained in the current cycle, the estimated delay of the subsequent specified cycle is obtained using a dual-input prediction model; wherein the dual-input prediction model is trained using multiple training samples, and the training samples of the dual-input prediction model include the wireless signal status and network transmission parameters of the historical cycle as training data, and the actual delay of the subsequent specified cycle of the historical cycle as training labels; when no feedback information is obtained in the current cycle, the estimated delay of the subsequent specified cycle is obtained using a single-input prediction model; wherein the single-input prediction model is trained using multiple training samples, and the training samples of the single-input prediction model include the wireless signal status of the historical cycle as training data, and the actual delay of the subsequent specified cycle of the historical cycle as training labels.

[0035] It is understandable that the training data is usually obtained offline, and the actual data obtained in the historical period is used as the training data, and the actual delay in the subsequent specified period of the historical period is used as the training label. For different situations where the network transmission parameters can be obtained based on feedback information, and where the network transmission parameters cannot be obtained, prediction models matching different situations are trained respectively to improve the efficiency and accuracy of delay prediction.

[0036] Optionally, the estimated delay of a subsequent specified period is obtained using a large language model, which may include but is not limited to deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), long short-term memory networks (LSTM), and generative pre-trained Transformer models, etc. No specific restrictions are imposed on the specific construction and deployment of the large language model.

[0037] In the above scheme, after the transmitting end establishes a connection with the receiving end, the video packet corresponding to the video frame required by the receiving end is obtained, and the target video packet to be sent in the current cycle is sent to the receiving end. When the feedback information of the receiving end for the target video packet is received in the current cycle, the network transmission parameters measured in the current cycle are determined based on the target video packet and the feedback information, and the wireless signal state obtained in real time by the transmitting end is obtained. Based on the current wireless signal state and the network transmission parameters measured in the current cycle, the round-trip delay of the subsequent specified cycle is predicted, and the estimated delay of the subsequent specified cycle is obtained. In this way, when the feedback information of the receiving end is obtained, the delay of the subsequent specified cycle is predicted through multi-dimensional data, thereby improving the accuracy of the delay prediction. When no feedback information for a target video packet is received from the receiving end in the current cycle, the wireless signal state obtained in real time by the transmitting end is obtained, and based on the current wireless signal state, the round-trip delay of a subsequent specified cycle is predicted to obtain the estimated delay of the subsequent specified cycle, thereby reducing the dependence on feedback information when no feedback information is obtained from the receiving end, and timely using the wireless state information to predict the delay of the subsequent specified cycle to improve the timeliness of the delay prediction, wherein the estimated delay is used to adjust the sending strategy of the video packet in the subsequent specified cycle, thereby timely adjusting and confirming the sending strategy according to the estimated delay, and improving the quality of the video received by the receiving end.

[0038] See also Figure 2 , Figure 2 : is a flow chart of another embodiment of the method for predicting video transmission delay of the present application, the method comprising: S201: Obtain an initial estimated delay based on the current wireless signal state.

[0039] Specifically, after the receiving end initiates a connection establishment requirement, the sending end prepares to establish a connection with the receiving end. Since no data is sent for this connection, the current network transmission parameters cannot be obtained. Thus, the wireless signal state obtained by the sending end in real time is acquired, and based on the current wireless signal state, the delay in the handshake phase is predicted to obtain an initial estimated delay, so as to use the initial estimated delay for transmission control in the handshake phase and improve the control accuracy.

[0040] Optionally, in the case where the network transmission parameters are not obtained, the initial estimated delay is obtained using a single-input prediction model. Among them, the single-input prediction model is trained using multiple training samples. The training samples of the single-input prediction model include the wireless signal state of the historical period as training data, and the actual delay of the subsequent specified period of the historical period as training labels.

[0041] S202: Try to establish a connection with the receiving end in the handshake phase using the initial estimated delay; among them, when the handshake phase times out, use the initial estimated delay for timeout retransmission.

[0042] Specifically, try to establish a connection with the receiving end in the handshake phase using the initial estimated delay, and determine whether the handshake phase times out. If it times out, use the initial estimated delay for timeout retransmission to establish a connection with the receiving end as soon as possible.

[0043] S203: In response to establishing a connection with the receiving end, obtain the video packet corresponding to the video frame, and send the target video packet of the current period to the receiving end.

[0044] Specifically, after the sending end and the receiving end establish a connection, obtain the video packet corresponding to the video frame required by the receiving end, and send the target video packet that needs to be sent in the current period to the receiving end.

[0045] Please refer to Figure 3 , Figure 3 which is a schematic diagram of the application scenario of an embodiment of the video transmission delay prediction method of the present application. After the sending end and the receiving end establish a connection, the video frame is segmented based on the data volume of the video frame to obtain the video packet corresponding to the video frame. Among them, the key frame and the non-key frame correspond to different data volumes, and different numbers of video packets can be obtained.

[0046] In some implementation scenarios, in response to establishing a connection with a receiving end, obtaining a video packet corresponding to a video frame, and sending a target video packet of the current period to the receiving end, includes: in response to establishing a connection with a receiving end, obtaining a video frame received by the receiving end, and dividing the video frame into multiple video packets based on the data amount of the video frame; obtaining a target video packet that matches the sending strategy of the current period, and sending the target video packet of the current period to the receiving end.

[0047] Specifically, when the sending end establishes a connection with the receiving end, the video frame received by the receiving end is obtained, and based on the data amount of the video frame, the video frame is divided into multiple video packets so that the video frames with different data amounts can be divided as evenly as possible, making the video packet transmission more stable and facilitating the adjustment of the transmission strategy according to the estimated delay.

[0048] Further, the sending strategy of the current cycle is determined, the video packets required to be sent by the sending strategy are obtained as target video packets, and the target video packets of the current cycle are sent to the receiving end, thereby ensuring that the video packets are sent to the receiving end according to the sending strategy of each cycle.

[0049] S204: In response to feedback information received from the receiving end for the target video packet in the current period, determine the network transmission parameters of the current period based on the target video packet and the feedback information, and obtain the estimated delay of the subsequent specified period based on the current wireless signal state and the network transmission parameters of the current period.

[0050] Specifically, when feedback information about the target video packet is received from the receiving end in the current cycle, the network transmission parameters measured in the current cycle are determined based on the target video packet and the feedback information, and the wireless signal status obtained in real time by the sending end is obtained. Based on the current wireless signal status and the network transmission parameters measured in the current cycle, the round-trip delay of the subsequent specified cycle is predicted to obtain the estimated delay of the subsequent specified cycle.

[0051] In some implementation scenarios, based on the target video packet and feedback information, the network transmission parameters of the current period are determined, and based on the current wireless signal state and the network transmission parameters of the current period, the estimated delay of the subsequent specified period is obtained, including: based on the target video packet and feedback information, the actual delay of the current period is determined, and based on the data volume corresponding to the target video packet, the data transmission rate of the current period is determined; wherein the network transmission parameters include the actual delay and the data transmission rate; based on the current wireless signal state, the actual delay of the current period and the data transmission rate, the estimated delay of the subsequent specified period is obtained.

[0052] For details, please refer again to Figure 3The receiving end successfully receives the target video packet in the current cycle and sends feedback information to the sending end. When the receiving end receives the feedback information for the target video packet in the current cycle, it is determined that the receiving end has successfully received the target video packet. Based on the timestamps corresponding to the target video packet and the feedback information, the actual delay of the current cycle is determined. Based on the data volume corresponding to the target video packet, the data sending rate of the current cycle is determined. The network transmission parameters include the actual delay and the data sending rate. Thus, the network status of the current cycle is evaluated through the actual delay and the data sending rate obtained by actual measurement, and a detailed and accurate network status is obtained.

[0053] Furthermore, based on the current wireless signal state, the actual delay of the current cycle and the parameters of multiple dimensions consisting of the data transmission rate, the delay of the subsequent specified cycle is predicted to obtain the estimated delay of the subsequent specified cycle.

[0054] It should be noted that based on the current wireless signal state, the actual delay of the current cycle and the data transmission rate, the estimated delay of the subsequent specified cycle is obtained, including: based on the current wireless signal state, determining the current signal state characteristics, based on the actual delay of the current cycle and the data transmission rate, determining the network transmission characteristics of the current cycle; fusing the signal state characteristics and the network transmission characteristics to obtain the target fusion characteristics, and determining the estimated delay of the subsequent specified cycle based on the target fusion characteristics.

[0055] Specifically, the current wireless signal state is normalized and the current signal state characteristics are extracted. The actual delay and data transmission rate of the current period are normalized and the network transmission characteristics of the current period are extracted. The signal state characteristics that can provide feedback on signal quality and the network transmission characteristics of network transmission quality are fused to obtain high-precision target fusion characteristics after multi-dimensional fusion. The delay of the subsequent specified period is predicted based on the target fusion characteristics, the estimated delay of the subsequent specified period is determined, and the accuracy of delay prediction is improved.

[0056] S205: In response to not receiving feedback information from the receiving end for the target video packet in the current period, obtaining an estimated delay in a subsequent specified period based on the current wireless signal state; wherein the estimated delay is used to adjust a sending strategy for the video packet in the subsequent specified period.

[0057] Specifically, when no feedback information for the target video packet is received from the receiving end in the current cycle, the real-time wireless signal state of the transmitting end is obtained, and the round-trip delay of the subsequent specified cycle is predicted based on the current wireless signal state to obtain the estimated delay of the subsequent specified cycle.

[0058] In some implementation scenarios, the wireless signal state corresponds to a signal change threshold; in response to not receiving feedback information from the receiving end for the target video packet in the current cycle, obtaining an estimated delay for a subsequent specified cycle based on the current wireless signal state, including: in response to not receiving feedback information from the receiving end for the target video packet in the current cycle, and the change in the wireless signal state before the end of the current cycle exceeds the signal change threshold, obtaining an estimated delay for a subsequent specified cycle based on the current wireless signal state; in response to not receiving feedback information from the receiving end for the target video packet until the end of the current cycle, obtaining an estimated delay for a subsequent specified cycle based on the current wireless signal state.

[0059] For details, please refer again to Figure 3 The video packet sends data to the receiving end through the wireless network transmission control module and the wireless driver module. The wireless driver module obtains the wireless signal status in real time and determines whether the wireless signal status exceeds the signal change threshold. The wireless network transmission control module is used to predict the delay of the subsequent specified period and adjust the sending strategy of the video packet in the subsequent specified period.

[0060] It can be understood that when no feedback information is received from the receiving end for the target video packet within the current cycle, it is determined whether the current wireless signal state change exceeds the signal change threshold. If the change in the wireless signal state exceeds the signal change threshold before the end of the current cycle, the delay of the subsequent specified cycle is immediately predicted based on the current wireless signal state. Therefore, when the wireless signal state fluctuates greatly, the delay of the subsequent specified cycle is predicted in time to obtain the estimated delay of the subsequent specified cycle, thereby improving the timeliness of the delay prediction.

[0061] Furthermore, if the change in the wireless signal state does not exceed the signal change threshold before the end of the current cycle, continue to monitor whether feedback information is obtained from the receiving end and monitor changes in the wireless signal state until the end of the current cycle. If no feedback information is received from the receiving end for the target video packet, the delay of the subsequent specified cycle is predicted based on the current wireless signal state in a timely manner to obtain the estimated delay of the subsequent specified cycle.

[0062] It should be noted that the wireless signal status includes wireless signal strength, channel utilization, wireless packet loss rate, wireless negotiation rate and wireless receiving signal-to-noise ratio as signal status parameters. Various signal status parameters can affect the amount of data transmitted, and the signal change threshold corresponds to the data amount change threshold.

[0063] Specifically, the wireless signal status includes wireless signal strength, channel utilization, wireless packet loss rate, wireless negotiation rate and wireless receiving signal-to-noise ratio as signal status parameters, so that the wireless signal status is comprehensively fed back through multiple signal status parameters to improve the accuracy of the wireless signal status.

[0064] It should be noted that multiple signal state parameters can affect the amount of data transmitted, and the signal change threshold corresponds to the data amount change threshold, so as to quantify the impact of different signal state parameters on data transmission. When the impact of changes in multiple signal state parameters on the data amount exceeds the data amount change threshold, the delay of the subsequent specified period is predicted in time, thereby improving the accuracy of the time node for predicting the delay of the subsequent specified period.

[0065] In this embodiment, after the receiving end initiates the need to establish a connection, the sending end prepares to establish a connection with the receiving end. Since the connection does not send data, the current network transmission parameters cannot be obtained, thereby obtaining the wireless signal state obtained by the sending end in real time, and predicting the delay in the handshake phase based on the current wireless signal state to obtain the initial estimated delay, so as to use the initial estimated delay to control the transmission in the handshake phase to improve the control accuracy, and use the initial estimated delay to try to establish a connection with the receiving end in the handshake phase, and determine whether the handshake phase has timed out. If it has timed out, the initial estimated delay is used to perform timeout retransmission, so as to establish a connection with the receiving end as soon as possible. When the sending end establishes a connection with the receiving end, the video frame received by the receiving end is obtained, and based on the data volume of the video frame, the video frame is divided into multiple video packets, so as to divide the video frames with different data volumes as evenly as possible, so that the video packet transmission is more stable, and it is convenient to adjust the transmission strategy according to the estimated delay. When receiving feedback information from the receiving end for the target video packet in the current cycle, the delay of the subsequent specified cycle is predicted based on the current wireless signal state, the actual delay of the current cycle, and the parameters of multiple dimensions composed of the data transmission rate, and the estimated delay of the subsequent specified cycle is obtained to improve the accuracy of the delay prediction. When no feedback information from the receiving end for the target video packet is received in the current cycle, it is determined whether the current wireless signal state change exceeds the signal change threshold. If the change of the wireless signal state exceeds the signal change threshold before the end of the current cycle, the delay of the subsequent specified cycle is immediately predicted based on the current wireless signal state. Therefore, when the wireless signal state fluctuates greatly, the delay of the subsequent specified cycle is predicted in time, and the estimated delay of the subsequent specified cycle is obtained, thereby improving the timeliness of the delay prediction.

[0066] See also Figure 4 , Figure 4 It is a structural diagram of an embodiment of an electronic device of the present application, wherein the electronic device 30 includes a memory 301 and a processor 302 coupled to each other, wherein the memory 301 stores program data (not shown), and the processor 302 calls the program data to implement the method in any of the above embodiments. For descriptions of related contents, please refer to the detailed description of the above method embodiments, which will not be repeated here.

[0067] See also Figure 5 , Figure 5It is a structural diagram of an embodiment of a computer-readable storage medium of the present application. The computer-readable storage medium 40 stores program data 400. When the program data 400 is executed by a processor, the method in any of the above embodiments is implemented. For descriptions of related contents, please refer to the detailed description of the above method embodiments, which will not be repeated here.

[0068] It should be noted that the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present implementation scheme.

[0069] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0070] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0071] The above description is only an implementation method of the present application, and does not limit the protection scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the protection scope of the present application.

Claims

1. A method for predicting video transmission delay, characterized in that: Applied to a transmitting end, the transmitting end obtains the wireless signal status in real time, and the method includes: In response to establishing a connection with a receiving end, obtaining a video packet corresponding to a video frame, and sending a target video packet of a current period to the receiving end; In response to receiving feedback information from the receiving end regarding the target video packet in the current period, determining a network transmission parameter of the current period based on the target video packet and the feedback information, and obtaining an estimated delay of a subsequent specified period based on a current wireless signal state and the network transmission parameter of the current period; In response to not receiving feedback information from the receiving end for the target video packet in the current period, an estimated delay for a subsequent specified period is obtained based on a current wireless signal state; wherein the estimated delay is used to adjust a sending strategy for the video packet in the subsequent specified period.

2. The video transmission delay prediction method according to claim 1, characterized in that: The step of establishing a connection between the transmitting end and the receiving end comprises: Based on the current wireless signal status, an initial estimated delay is obtained; Using the initial estimated time delay to try to establish a connection with the receiving end in the handshake phase; wherein, when the handshake phase times out, using the initial estimated time delay to perform timeout retransmission.

3. The method for predicting video transmission delay according to claim 1, characterized in that: The step of obtaining a video packet corresponding to a video frame in response to establishing a connection with a receiving end, and sending a target video packet of a current period to the receiving end includes: In response to establishing a connection with the receiving end, obtaining a video frame received by the receiving end, and dividing the video frame into a plurality of video packets based on a data amount of the video frame; A target video packet matching the sending strategy of the current period is acquired, and the target video packet of the current period is sent to the receiving end.

4. The video transmission delay prediction method according to claim 1, characterized in that: The determining, based on the target video packet and the feedback information, the network transmission parameters of the current cycle, and obtaining the estimated delay of a subsequent specified cycle based on the current wireless signal state and the network transmission parameters of the current cycle, includes: Based on the target video packet and the feedback information, determine the actual delay of the current cycle, and based on the data volume corresponding to the target video packet, determine the data transmission rate of the current cycle; wherein the network transmission parameters include the actual delay and the data transmission rate; Based on the current wireless signal state, the actual delay of the current period and the data transmission rate, an estimated delay of a subsequent specified period is obtained.

5. The method for predicting video transmission delay according to claim 4, characterized in that: The obtaining, based on the current wireless signal state, the actual delay of the current cycle and the data transmission rate, an estimated delay of a subsequent specified cycle includes: Determine the current signal state characteristics based on the current wireless signal state, and determine the network transmission characteristics of the current period based on the actual delay and data transmission rate of the current period; The signal state feature and the network transmission feature are fused to obtain a target fusion feature, and an estimated delay of a subsequent specified period is determined based on the target fusion feature.

6. The method for predicting video transmission delay according to claim 1, characterized in that: The wireless signal state corresponds to a signal change threshold; The step of obtaining an estimated delay of a subsequent designated period based on a current wireless signal state in response to not receiving feedback information from the receiving end for the target video packet within the current period includes: In response to not receiving feedback information from the receiving end for the target video packet within the current period, and the change of the wireless signal state exceeds the signal change threshold before the end of the current period, obtaining an estimated delay of a subsequent specified period based on the current wireless signal state; In response to not receiving feedback information from the receiving end for the target video packet until the end of the current period, an estimated delay of a subsequent designated period is obtained based on a current wireless signal state.

7. The method for predicting video transmission delay according to claim 6, characterized in that: The wireless signal status includes wireless signal strength, channel utilization, wireless packet loss rate, wireless negotiation rate and wireless receiving signal-to-noise ratio as signal status parameters. Multiple signal status parameters can affect the amount of data transmitted, and the signal change threshold corresponds to the data amount change threshold.

8. The method for predicting video transmission delay according to any one of claims 1 to 7, characterized in that: When the feedback information is obtained in the current cycle, the estimated delay of the subsequent specified cycle is obtained by using a dual-input prediction model; wherein the dual-input prediction model is trained using a plurality of training samples, and the training samples of the dual-input prediction model include wireless signal states and network transmission parameters of historical cycles as training data, and actual delays of subsequent specified cycles of the historical cycles as training labels; When the feedback information is not obtained in the current cycle, the estimated delay of the subsequent specified cycle is obtained using a single-input prediction model; wherein the single-input prediction model is trained using multiple training samples, and the training samples of the single-input prediction model include the wireless signal status of the historical cycle as training data, and the actual delay of the subsequent specified cycle of the historical cycle as a training label.

9. An electronic device, characterized in that: include: A memory and a processor coupled to each other, wherein the memory stores program data, and the processor calls the program data to execute the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having program data stored thereon, characterized in that: When the program data is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images