A frame rate control method, device, screen mirroring device, and program product

Abstract

Embodiments of the present application provide a frame rate control method, apparatus, screen mirroring device, and program product, which relate to the technical field of data processing. Among them, a frame rate control method includes: during the screen mirroring process, detecting the bottleneck bandwidth of the wireless network as the current bottleneck bandwidth, predicting the data volume of the target frame to obtain a predicted data volume, and based on the predicted data volume, predicting the number of image frames that can be transmitted per unit time if the bitstream is transmitted under the current bottleneck bandwidth to obtain a predicted frame rate. Based on the change value of the predicted frame rate relative to the reference frame rate, adjusting the reference frame rate to determine a screen capture frame rate adapted to the current bottleneck bandwidth, and setting the determined screen capture frame rate as the screen capture frame rate for capturing the display data of the source end, so as to perform encoding processing and transmission on the display data captured at the determined screen capture frame rate. It can be seen that this solution can achieve both the screen mirroring effect and the utilization rate of the network bandwidth.

Description

Technical Field

[0001] The present application relates to the technical field of data processing, and in particular, to a frame rate control method, apparatus, screen mirroring device, and program product. Background Art

[0002] In a screen mirroring scenario, a screen mirroring device usually captures the display data of a source end at a fixed frame rate, and encodes and transmits the captured display data to a screen mirroring receiving end according to a bit rate corresponding to the fixed frame rate. Among them, the bit rate and the frame rate usually have a positive correlation.

[0003] However, during the process of bit stream transmission, it may be affected by the bottleneck bandwidth of the wireless network, thereby affecting the subsequent screen mirroring effect based on the encoded bit stream; where the bottleneck bandwidth is the bandwidth of the device / link that limits the data transmission rate in the network, the bottleneck bandwidth is also the maximum transmission rate allowed by the network bandwidth, and the bottleneck bandwidth also changes in real time.

[0004] Specifically, when the bit rate is much greater than the bottleneck bandwidth, network congestion will occur, and screen tearing and stuttering may occur during screen mirroring, that is, the screen mirroring effect is poor; and when the bit rate is much less than the bottleneck bandwidth, the maximum transmission rate characterized by the bottleneck bandwidth cannot be achieved, resulting in low utilization rate of the network bandwidth.

[0005] Currently, there is an urgent need for a frame rate control method to achieve both the screen mirroring effect and the utilization rate of the network bandwidth. Summary of the Invention

[0006] The purpose of the embodiments of the present application is to provide a frame rate control method, apparatus, screen mirroring device, and program product to achieve both the screen mirroring effect and the utilization rate of the network bandwidth. The specific technical solutions are as follows:

[0007] In a first aspect, the embodiments of the present application provide a frame rate control method, which is applied to a screen mirroring device, and the screen mirroring device communicates with a screen mirroring receiving end through a wireless network; the method includes:

[0008] During screen mirroring, detect the bottleneck bandwidth of the wireless network as the current bottleneck bandwidth;

[0009] Predict the data amount of a target frame to obtain a predicted data amount; where the target frame is a first type of frame to be encoded during the encoding process of the captured display data of the source end, and the first type of frame is a frame that needs to refer to other frames during encoding;

[0010] Based on the predicted data amount, predict the number of image frames that can be transmitted per unit time if bit stream transmission is performed under the current bottleneck bandwidth to obtain a predicted frame rate;

[0011] Based on the change value of the predicted frame rate relative to the reference frame rate, adjust the reference frame rate to determine a screen capture frame rate adapted to the current bottleneck bandwidth; wherein, the reference frame rate is the frame rate set as the adjustment basis according to the network state of the wireless network.

[0012] Set the determined screen capture frame rate as the screen capture frame rate for capturing the display data of the source end, so that the display data captured according to the determined screen capture frame rate is encoded and transmitted to the screen mirroring receiving end, and the bit rate of the bitstream used in the encoding process has a positive correlation with the determined screen capture frame rate.

[0013] In a second aspect, an embodiment of the present application provides a frame rate control device, which is applied to a screen mirroring device, and the screen mirroring device communicates with a screen mirroring receiving end through a wireless network; the device includes:

[0014] A detection module, configured to detect the bottleneck bandwidth of the wireless network during screen mirroring as the current bottleneck bandwidth.

[0015] A first prediction module, configured to predict the data volume of a target frame to obtain a predicted data volume; wherein, the target frame is a first type of frame to be encoded during the encoding process of the captured display data of the source end, and the first type of frame is a frame that needs to refer to other frames during encoding.

[0016] A second prediction module, configured to predict the number of image frames that can be transmitted per unit time if bitstream transmission is performed under the current bottleneck bandwidth based on the predicted data volume to obtain a predicted frame rate.

[0017] An adjustment processing module, configured to adjust the reference frame rate based on the change value of the predicted frame rate relative to the reference frame rate to determine a screen capture frame rate adapted to the current bottleneck bandwidth; wherein, the reference frame rate is the frame rate set as the adjustment basis according to the network state of the wireless network.

[0018] A setting module, configured to set the determined screen capture frame rate as the screen capture frame rate for capturing the display data of the source end, so that the display data captured according to the determined screen capture frame rate is encoded and transmitted to the screen mirroring receiving end, and the bit rate of the bitstream used in the encoding process has a positive correlation with the determined screen capture frame rate.

[0019] In a third aspect, an embodiment of the present application provides a screen mirroring device, including:

[0020] A memory, configured to store a computer program.

[0021] A processor, configured to implement any of the above frame rate control methods when executing the program stored in the memory.

[0022] In a fourth aspect, an embodiment of the present application provides a computer program product, including a computer program which, when executed by a processor, implements any of the above frame rate control methods.

[0023] Advantageous effects of the embodiments of the present application:

[0024] The frame rate control method provided by the embodiments of the present application can detect the bottleneck bandwidth of the wireless network during the screen mirroring process, use it as the current bottleneck bandwidth, predict the data volume of the target frame, obtain the predicted data volume, and based on the predicted data volume, predict the number of image frames that can be transmitted per unit time if the bitstream is transmitted under the current bottleneck bandwidth, obtain the predicted frame rate. Subsequently, based on the change value of the predicted frame rate relative to the reference frame rate, the reference frame rate can be adjusted to determine the screen capture frame rate adapted to the current bottleneck bandwidth, and the determined screen capture frame rate is set as the screen capture frame rate for capturing the display data of the source end. Among them, the reference frame rate is the frame rate set as the adjustment basis according to the network state of the wireless network; it can be seen that the predicted frame rate (the predicted frame rate can to a certain extent reflect the bit rate required in the current network environment, and the required bit rate can be considered as the actually encoded data volume) and the network state of the wireless network are referred to, so that the reference frame rate can be dynamically adjusted and can respond quickly to the changing network state; and, since the network state of the wireless network fluctuates, the bottleneck bandwidth can also change, and the determined screen capture frame rate adapted to the current bottleneck bandwidth can adapt to the changing bottleneck bandwidth. In this way, when encoding the display data captured according to the determined screen capture frame rate, the bitstream bit rate used for the encoding process has a positive correlation with the determined screen capture frame rate, and there will be no situation where the bitstream bit rate is much greater than the bottleneck bandwidth, thus affecting the screen mirroring effect, nor will there be a situation where the bitstream bit rate is much less than the bottleneck bandwidth, resulting in a low utilization rate of the network bandwidth. Therefore, both the screen mirroring effect and the utilization rate of the network bandwidth are taken into account.

[0025] Of course, implementing any product or method of the present application does not necessarily require achieving all the above advantages simultaneously. Description of the Drawings

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application, and those of ordinary skill in the art can also obtain other embodiments according to these drawings.

[0027] Figure 1 It is a schematic flowchart of a frame rate control method provided by an embodiment of the present application;

[0028] Figure 2 A schematic diagram of the effect regarding a predetermined time window provided by an embodiment of the present application;

[0029] Figure 3 A schematic flowchart of obtaining an empirical frame rate corresponding to any bottleneck bandwidth provided by an embodiment of the present application;

[0030] Figure 4 A schematic diagram of the principle of a frame rate control method provided by an embodiment of the present application;

[0031] Figure 5 A schematic structural diagram of a frame rate control device provided by an embodiment of the present application;

[0032] Figure 6 A schematic structural diagram of a screen mirroring device provided by an embodiment of the present application. Detailed implementation manners

[0033] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art based on the present application belong to the scope of protection of the present application.

[0034] First, some professional terms in the embodiments of the present application are briefly introduced:

[0035] Bitrate detection: It can also be called bandwidth detection, which detects the bottleneck bandwidth in the network.

[0036] Screen capture: The process of capturing display data from the source end (device).

[0037] Screen mirroring system: A system composed of a screen mirroring device, a wireless network, and a screen mirroring receiver for displaying the captured display data on the screen mirroring receiver.

[0038] GCC: Google Congest Control, a commonly used congestion control algorithm.

[0039] Secondly, in order to better understand the present application, before introducing the method provided by the embodiments of the present application, the frame rate control methods in the related technologies are first introduced:

[0040] In the related technologies, the frame rate is usually dynamically adjusted by using the network maximum bandwidth (bottleneck bandwidth) and the actual coding situation. However, since the network is changing in real time and the network maximum bandwidth can also be changing in real time, the frame rate control methods in the related technologies do not consider the change situation of the network. Therefore, the utilization rate of the network bandwidth may be low, and the screen mirroring effect cannot be satisfied.

[0041] Based on the problems existing in the related art, the present application provides a frame rate control method, device, screen mirroring device and program product to achieve both the screen mirroring effect and the utilization rate of network bandwidth.

[0042] Again, a frame rate control method provided by an embodiment of the present application will be introduced below.

[0043] Among them, a frame rate control method provided by an embodiment of the present application can be applied to a screen mirroring device, which can also be called a screen capture encoding end. The screen mirroring device is specifically used to capture display data from a source end, encode the display data, and then transmit the encoded bitstream to a screen mirroring receiving end. Exemplarily, in one implementation, the screen mirroring device can be a hardware device, for example, a screen mirroring device; in another implementation, the screen mirroring device can also be screen mirroring software, for example: a screen mirroring client. The present application embodiment does not make a specific limitation on the specific form of the screen mirroring device.

[0044] In addition, the screen mirroring device can communicate with the screen mirroring receiving end through a wireless network. The screen mirroring receiving end can also be considered as the receiving end that receives the display data captured by the screen mirroring device. Subsequently, the screen mirroring receiving end can also display the display data; specifically, the screen mirroring receiving end can be a personal computer (PC) device, or a mobile device, for example: a mobile phone, a tablet computer, etc., or can also be screen mirroring receiving software installed in the device. The present application embodiment does not make a specific limitation on the specific form of the screen mirroring receiving end. In addition, the source end can also communicate with the screen mirroring device through a wireless network. The source end can be considered as the video source end that displays the display data. Specifically, the source end can also be a personal computer (PC) device or a mobile device. The present application embodiment does not make a specific limitation on this.

[0045] It should be noted that the terms "device" and "end" in the embodiments of the present application can be a hardware device, a software program, or a necessary carrier for the software program to run. With the evolution of technology, the steps to be executed or the functions to be performed by these "devices" and "ends" (for example, the steps described after "a certain device is used for") may use hardware, software, or a combination of hardware and software. As long as it is the hardware, software, or the combination of hardware and software that can execute the steps required by the "device" and "end", it can be considered as the "device" and "end" in the embodiments of the present application.

[0046] Next, a simple introduction to the application scenario of the frame rate control method provided by the embodiment of the present application will be given:

[0047] The embodiments of the present application can be used in various screen mirroring scenarios, for example: video conferencing scenarios, online teaching scenarios, etc. The present application embodiment does not make a specific limitation on this.

[0048] Among them, a frame rate control method is applied to a screen mirroring device, and the screen mirroring device communicates with a screen mirroring receiving end through a wireless network; the method includes:

[0049] During the screen mirroring process, detect the bottleneck bandwidth of the wireless network as the current bottleneck bandwidth;

[0050] Predict the data volume of a target frame to obtain a predicted data volume; wherein, the target frame is a first type of frame to be encoded during the encoding process of the captured display data of the source end, and the first type of frame is a frame that needs to refer to other frames during encoding;

[0051] Based on the predicted data volume, predict the number of image frames that can be transmitted per unit time if the bitstream is transmitted under the current bottleneck bandwidth to obtain a predicted frame rate;

[0052] Based on the change value of the predicted frame rate relative to the reference frame rate, adjust the reference frame rate to determine a screen capture frame rate adapted to the current bottleneck bandwidth; wherein, the reference frame rate is a frame rate set as an adjustment basis according to the network state of the wireless network;

[0053] Set the determined screen capture frame rate as the screen capture frame rate for capturing the display data of the source end, so that the display data captured at the determined screen capture frame rate is encoded and transmitted to the screen mirroring receiving end, and the bitstream bit rate used in the encoding process has a positive correlation with the determined screen capture frame rate.

[0054] The frame rate control method provided by the embodiments of the present application can detect the bottleneck bandwidth of the wireless network during the screen mirroring process as the current bottleneck bandwidth, predict the data volume of the target frame to obtain the predicted data volume, and based on the predicted data volume, predict the number of image frames that can be transmitted per unit time if the bitstream is transmitted under the current bottleneck bandwidth to obtain the predicted frame rate. Subsequently, based on the change value of the predicted frame rate relative to the reference frame rate, the reference frame rate can be adjusted to determine the screen capture frame rate adapted to the current bottleneck bandwidth, and the determined screen capture frame rate is set as the screen capture frame rate for capturing the display data of the source end. Among them, the reference frame rate is the frame rate set as the adjustment basis according to the network state of the wireless network. It can be seen that the predicted frame rate (the predicted frame rate can reflect to a certain extent the bit rate required in the current network environment, and the required bit rate can be considered as the actual encoded data volume) and the network state of the wireless network are referred to, so that the reference frame rate can be dynamically adjusted and can respond quickly to the changing network state. In addition, since the network state of the wireless network fluctuates, the bottleneck bandwidth can also change, and the determined screen capture frame rate adapted to the current bottleneck bandwidth can adapt to the changing bottleneck bandwidth. In this way, when encoding the display data captured according to the determined screen capture frame rate, the bitstream bit rate used in the encoding process has a positive correlation with the determined screen capture frame rate, and the situation where the bitstream bit rate is much greater than the bottleneck bandwidth, thus affecting the screen mirroring effect, will not occur, nor will the situation where the bitstream bit rate is much less than the bottleneck bandwidth, resulting in low utilization rate of the network bandwidth. Therefore, both the screen mirroring effect and the utilization rate of the network bandwidth are taken into account.

[0055] The following introduces a frame rate control method provided by the embodiments of the present application with reference to the accompanying drawings.

[0056] As Figure 1 shown, a frame rate control method provided by the embodiments of the present application is applied to a screen mirroring device, and the screen mirroring device communicates with a screen mirroring receiving end through a wireless network. The method includes:

[0057] S101, during the screen mirroring process, detect the bottleneck bandwidth of the wireless network as the current bottleneck bandwidth;

[0058] It can be understood that the embodiments of the present application are only directed to the screen mirroring scenario. Then, during the screen mirroring process, the bottleneck bandwidth of the wireless network can be detected as the current bottleneck bandwidth. Among them, the bottleneck bandwidth of the wireless network can also be considered as the maximum transmission rate allowed by the network bandwidth of the wireless network. It should be emphasized that during the process of detecting the bottleneck bandwidth of the wireless network, it can be implemented by means of congestion control algorithms such as GCC. Of course, the present application does not make specific limitations in this regard, and any congestion control algorithm can be used as the method for detecting the bottleneck bandwidth of the wireless network in the present application. In addition, the bottleneck bandwidth can also be referred to as the bottleneck link bandwidth, and the unit of the bottleneck bandwidth is bps, that is, bits per second, specifically it can be kbps, kilobits per second, Mbps, megabits per second; and, the unit of the bottleneck bandwidth is the same as the unit of the bit rate.

[0059] And, when the screen mirroring device is a hardware device (such as a screen mirroring device), the bottleneck bandwidth of the wireless network can be detected as the current bottleneck bandwidth based on the bandwidth detection software in the screen mirroring device. This bandwidth detection software can be software implemented using congestion control algorithms such as GCC. When the screen mirroring device is screen mirroring software, the screen mirroring software can be software with a bandwidth detection function, so that the bandwidth detection function can be enabled during the screen mirroring process to directly detect the bottleneck bandwidth of the wireless network as the current bottleneck bandwidth; or, when the screen mirroring device is screen mirroring software, the screen mirroring device can call the bandwidth detection module in the source end to detect the bottleneck bandwidth of the wireless network as the current bottleneck bandwidth. Of course, the screen mirroring device can also send a detection instruction to the screen mirroring receiving end to instruct the screen mirroring receiving end to detect the bottleneck bandwidth of the wireless network, which is also reasonable. It can be understood that for different forms of the screen mirroring device, the specific implementation methods for detecting the bottleneck bandwidth of the wireless network may be different, and the embodiments of the present application do not make specific limitations in this regard.

[0060] Exemplarily, in one implementation, the detecting the bottleneck bandwidth of the wireless network as the current bottleneck bandwidth includes step A1:

[0061] Step A1, in response to satisfying the frame rate control condition, detect the bottleneck bandwidth of the wireless network as the current bottleneck bandwidth;

[0062] Wherein, the frame rate control condition includes: reaching the target period, and / or, detecting that the network state of the wireless network meets a preset unstable condition.

[0063] It can be understood that, by way of example, when entering each target period, it can be considered that the frame rate control condition is satisfied. At this time, frame rate control will be performed periodically, that is, the bottleneck bandwidth of the wireless network will be detected periodically, and the currently detected bottleneck bandwidth will be used as the current bottleneck bandwidth, so as to perform subsequent processing procedures based on the current bottleneck bandwidth to achieve frame rate control; by way of example, when it is detected that the network state of the wireless network meets the preset unstable condition, it can also be considered that the frame rate control condition is satisfied; by way of example, when entering a target period and detecting that the network state of the wireless network meets the preset unstable condition at the same time, it can still be considered that the frame rate control condition is satisfied. The embodiments of the present application do not make specific limitations on the specific implementation manner of satisfying the frame rate control condition.

[0064] By way of example, the duration of the target period can usually be set to 1 s. Of course, it can also be set to other durations, such as 1.5 s, 2 s, etc. The embodiments of the present application do not make specific limitations on this. In specific applications, different period durations can be selected according to the actual network state. In addition, the network state of the wireless network meeting the preset unstable condition can be considered as a strong fluctuation in the network state of the wireless network, triggering a network state fluctuation event (this network state fluctuation event can be set according to the specific application scenario. For example, when the change amount of the network bandwidth detected relative to the previous detection exceeds a predetermined threshold, it is determined that the network state fluctuation event is triggered), so as to satisfy the frame rate control condition. Of course, the above is only an exemplary introduction, and the embodiments of the present application do not make specific limitations on this.

[0065] It can be seen that when the embodiments of the present application respond to the satisfaction of the frame rate control condition, they can detect the bottleneck bandwidth of the wireless network as the current bottleneck bandwidth, which facilitates frame rate control in subsequent steps and provides an implementation basis for balancing the screen mirroring effect and the utilization rate of the network bandwidth; in addition, when the frame rate control condition includes entering each target period, the frequency of frame rate control can be increased, improving the accuracy of the determined capture frame rate adapted to the current bottleneck bandwidth. When the frame rate control condition includes detecting that the network state of the wireless network meets the preset unstable condition, the frequency of frame rate control can be reduced, thereby reducing the consumption of computing power.

[0066] S102. Predict the data volume of the target frame to obtain a predicted data volume;

[0067] Wherein, the target frame is a first type of frame to be encoded during the encoding of the captured display data at the source end, and the first type of frame is a frame that needs to refer to other frames during encoding;

[0068] It can be understood that the target frame is a first - type frame to be encoded in the captured display data of the source end. The first - type frame is a frame that needs to refer to other frames during encoding. For example, a frame encoded as a P - frame. Of course, there are also frames that do not need to refer to other frames during encoding, such as frames encoded as I - frames. Specific details will be introduced in subsequent embodiments and will not be elaborated here. Also, during the process of predicting the data volume, the data volume encoded from other first - type frames can also be referred to, so as to achieve referring to the actual encoded data volume during the frame rate control process. Additionally, due to the high correlation between the front and back of the screen - mirroring image, the data volume of the first - type frames (frames that need to refer to other frames during encoding) can be predicted here. When encoding any first - type frame, the encoded data of the previous first - type frame of this first - type frame can be referred to. Therefore, predicting the data volume of this first - type frame can reflect the correlation between the front and back of the screen - mirroring image. In other words, the data volume of frames that need to refer to other frames during encoding is relatively fixed (when performing JPEG compression encoding on the full - frame image information, it can be a preset value or an empirical value, etc.). Instead, the first - type frame is related to the reference frame and is related to the motion scene of the image. It is relatively not fixed. Therefore, as long as the timing of frame rate control is reached, the data volume of the first - type frame can be predicted to obtain the predicted data volume. Taking the first - type frame as a P - frame and the frame that needs to refer to other frames during encoding as an I - frame as an example, since the data volume of the I - frame is relatively fixed (for example, when performing JPEG compression encoding on the full - frame image information, it can be a preset value or an empirical value, etc.), the P - frame is instead related to the reference frame and is related to the motion scene of the image. It is relatively not fixed. Therefore, as long as the timing of frame rate control is reached, the data volume of the P - frame can be predicted.

[0069] Also, when the screen - mirroring device is a hardware device (screen - mirroring adapter), based on the data - volume prediction module in the screen - mirroring device, the data volume of the target frame can be predicted to obtain the predicted data volume. When the screen - mirroring device is screen - mirroring software, the screen - mirroring software can be software with a data - volume prediction function and can directly predict the data volume of the target frame to obtain the predicted data volume. The embodiments of this application do not make specific limitations on this.

[0070] For the sake of clear layout, the specific implementation of predicting the data volume of the target frame to obtain the predicted data volume will be introduced in other embodiments and will not be elaborated here.

[0071] S103, based on the predicted data volume, predict the number of image frames that can be transmitted per unit time if the bitstream is transmitted under the current bottleneck bandwidth, to obtain the predicted frame rate;

[0072] It can be understood that, based on the predicted data volume obtained in step S102, the number of image frames that can be transmitted per unit time can be predicted with the current bottleneck bandwidth as the transmission rate of the bitstream, which is used as the predicted frame rate. Among them, the number of image frames that can be transmitted per unit time can include the number of frames of the first type of frame, and can also include the number of frames that do not need to refer to other frames during encoding. And, the unit time can be 1s, and the present application does not make specific limitations in this regard. For the sake of clear layout, the specific implementation of predicting the number of image frames that can be transmitted per unit time and obtaining the predicted frame rate based on the predicted data volume when the bitstream is transmitted under the current bottleneck bandwidth will be introduced in other embodiments and will not be elaborated here.

[0073] S104. Based on the change value of the predicted frame rate relative to the reference frame rate, perform adjustment processing on the reference frame rate to determine the screen capture frame rate adapted to the current bottleneck bandwidth.

[0074] Among them, the reference frame rate is the frame rate set as the adjustment basis according to the network state of the wireless network.

[0075] It can be understood that the reference frame rates set under different network states of the wireless network are also different. For example, the reference frame rate set when the network state of the wireless network is stable is different from the reference frame rate set when the network state of the wireless network fluctuates violently. Then, based on the change value of the predicted frame rate relative to the reference frame rate, the reference frame rate can be directly adjusted. The determined screen capture frame rate adapted to the current bottleneck bandwidth can be the same as the predicted frame rate. Or, the change value of the predicted frame rate relative to the reference frame rate can be adjusted first, and then based on the adjusted change value, the reference frame rate can be adjusted to determine the screen capture frame rate adapted to the current bottleneck bandwidth. Adjusting the change value of the predicted frame rate relative to the reference frame rate can ensure the smoothness of the frame rate change and prevent the situation of drastic frame rate changes. The embodiments of the present application do not make specific limitations in this regard.

[0076] Exemplarily, if the change value of the predicted frame rate relative to the reference frame rate is 6fps and the reference frame rate is 20fps, then the screen capture frame rate adapted to the current bottleneck bandwidth can be determined to be 26fps.

[0077] In addition, there is a positive correlation between the frame rate and the bit rate. The capture frame rate adapted to the current bottleneck bandwidth also corresponds to a bitstream bit rate, and the bitstream bit rate corresponding to the capture frame rate adapted to the current bottleneck bandwidth matches the current bottleneck bandwidth. Moreover, the bitstream bit rate corresponding to the capture frame rate adapted to the current bottleneck bandwidth can also be considered as the bitstream bit rate used for encoding the display data obtained by capturing the screen at the capture frame rate adapted to the current bottleneck bandwidth. For the sake of clear layout, the setting method for the reference frame rate will be introduced in other embodiments and will not be elaborated here too much.

[0078] S105. Set the determined capture frame rate as the capture frame rate for capturing the display data at the source end, so that the display data captured at the determined capture frame rate is encoded and transmitted to the screen mirroring receiving end, and the bitstream bit rate used for the encoding process has a positive correlation with the determined capture frame rate.

[0079] It can be understood that the display data at the source end can be captured according to the captured screen frame rate adapted to the currently determined bottleneck bandwidth. When encoding the captured display data, since the bit rate of the encoded stream has a positive correlation with the captured screen frame rate (for example: the larger the bit rate of the encoded stream, the larger the captured screen frame rate), the captured display data can be encoded according to the bit rate corresponding to the captured screen frame rate adapted to the currently determined bottleneck bandwidth, and the encoded stream is transmitted to the screen mirroring receiving end. It should be emphasized that the determined captured screen frame rate is the captured screen frame rate adapted to the currently determined bottleneck bandwidth, so the bit rate corresponding to the captured screen frame rate adapted to the currently determined bottleneck bandwidth also matches the currently determined bottleneck bandwidth. The frame rate control method provided in the embodiments of the present application can detect the bottleneck bandwidth of the wireless network as the currently determined bottleneck bandwidth during the screen mirroring process, predict the data volume of the target frame, obtain the predicted data volume, and based on the predicted data volume, predict the number of image frames that can be transmitted per unit time if the encoded stream is transmitted under the currently determined bottleneck bandwidth, obtain the predicted frame rate. Subsequently, based on the change value of the predicted frame rate relative to the reference frame rate, the reference frame rate can be adjusted to determine the captured screen frame rate adapted to the currently determined bottleneck bandwidth, and the determined captured screen frame rate is set as the captured screen frame rate for capturing the display data at the source end, where the reference frame rate is the frame rate set as the adjustment basis according to the network state of the wireless network; it can be seen that the predicted frame rate (the predicted frame rate can to a certain extent reflect the required bit rate in the current network environment, and the required bit rate can be considered as the actual encoded data volume) and the network state of the wireless network are referred to, so that the reference frame rate can be dynamically adjusted and can respond quickly to the changing network state; and, since the network state of the wireless network fluctuates, the bottleneck bandwidth can also change, and the determined captured screen frame rate adapted to the currently determined bottleneck bandwidth can adapt to the changing bottleneck bandwidth. In this way, when encoding the captured display data according to the determined captured screen frame rate, the bit rate of the encoded stream used for encoding has a positive correlation with the determined captured screen frame rate, and the situation where the bit rate of the encoded stream is much larger than the bottleneck bandwidth, thus affecting the screen mirroring effect, will not occur, nor will the situation where the bit rate of the encoded stream is much smaller than the bottleneck bandwidth, thus resulting in a low utilization rate of the network bandwidth, and thus the screen mirroring effect and the utilization rate of the network bandwidth are taken into account.

[0080] Optionally, in another embodiment, the setting method of the reference frame rate includes Method B1 or Method B2:

[0081] Method B1: When the network state meets the network stability condition, a specified frame rate is used as the reference frame rate;

[0082] Method B2: When the network state does not meet the network stability condition, the empirical frame rate corresponding to the currently determined bottleneck bandwidth is used as the reference frame rate;

[0083] The specified frame rate is a screen capture frame rate determined based on the previously detected bottleneck bandwidth and adapted to the bottleneck bandwidth. The empirical frame rate corresponding to any bottleneck bandwidth is the frame rate at which the video stream transmitted with the bottleneck bandwidth as the bit rate can clearly display the image.

[0084] It can be understood that Method B1 is for the case where the network state meets the network stability condition. It can be determined that the network state is relatively stable without significant fluctuations. At this time, the specified frame rate can be determined as the reference frame rate. The specified frame rate is a screen capture frame rate determined based on the previously detected bottleneck bandwidth and adapted to the bottleneck bandwidth. Then, when the network state meets the network stability condition, that is, the difference between the current network state and the network state at the previous detection is small, the screen capture frame rate determined based on the previously detected bottleneck bandwidth and adapted to the bottleneck bandwidth can be directly used as the frame rate for adjustment.

[0085] It can be understood that Method B2 is for the case where the network state does not meet the network stability condition. It can be determined that the network state is relatively unstable and may have significant fluctuations. At this time, the empirical frame rate corresponding to the current bottleneck bandwidth can be determined as the reference frame rate. The empirical frame rate corresponding to the current bottleneck bandwidth is the frame rate at which the video stream transmitted with the current bottleneck bandwidth as the bit rate can clearly display the image. Then, when the network state does not meet the network stability condition, that is, the difference between the current network state and the network state at the previous detection is large, there will be an error if the screen capture frame rate determined based on the previously detected bottleneck bandwidth and adapted to the bottleneck bandwidth is used as the frame rate for adjustment. Therefore, the frame rate at which the video stream transmitted with the current bottleneck bandwidth as the bit rate can clearly display the image can be used as the frame rate for adjustment.

[0086] It should be emphasized that in one implementation, when initially setting the reference frame rate, it can be considered that the network state does not meet the network stability condition, and the empirical frame rate corresponding to the current bottleneck bandwidth can be directly used as the reference frame rate. In another implementation, when initially setting the reference frame rate, it can also be considered that the network state meets the network stability condition, and the previously determined screen capture frame rate recorded in the screen mirroring device can be used as the reference frame rate. The embodiments of the present application do not make specific limitations on this.

[0087] For the sake of clear layout, the method for obtaining the empirical frame rate corresponding to the bottleneck bandwidth will be introduced in other embodiments and will not be elaborated here.

[0088] It can be seen that in the embodiment of the present application, when the network state meets the network stability condition, the specified frame rate is used as the reference frame rate, and when the network state does not meet the network stability condition, the empirical frame rate corresponding to the current bottleneck bandwidth is used as the reference frame rate, so that accurate reference frame rates can be provided for the above two situations, laying a foundation for subsequent frame rate control.

[0089] Optionally, in another embodiment, the method for determining the network state of the wireless network includes Method C1:

[0090] Method C1: Based on the difference relationship between the empirical frame rate corresponding to the current bottleneck bandwidth and the empirical frame rate corresponding to the bottleneck bandwidth detected last time, determine the network state of the wireless network according to a predetermined determination method;

[0091] Among them, the predetermined determination method includes:

[0092] If the empirical frame rate corresponding to the current bottleneck bandwidth is the same as the empirical frame rate corresponding to the bottleneck bandwidth detected last time, it is determined that the network state of the wireless network meets the network stability condition;

[0093] If the empirical frame rate corresponding to the current bottleneck bandwidth is different from the empirical frame rate corresponding to the bottleneck bandwidth detected last time, it is determined that the network state of the wireless network does not meet the network stability condition.

[0094] It should be emphasized that each bottleneck bandwidth is located in a bandwidth interval, and each bandwidth interval corresponds to an empirical frame rate. Then, the current bottleneck bandwidth and the bottleneck bandwidth detected last time may not be exactly the same. However, if the current bottleneck bandwidth and the bottleneck bandwidth detected last time are both located in the same bandwidth interval, the empirical frame rate corresponding to the current bottleneck bandwidth is also the same as the empirical frame rate corresponding to the bottleneck bandwidth detected last time. The specific content will be introduced in other embodiments and will not be elaborated here.

[0095] It can be understood that when determining the network state of the wireless network, the network state of the wireless network can be determined according to the predetermined determination method. When the empirical frame rate corresponding to the current bottleneck bandwidth is the same as the empirical frame rate corresponding to the bottleneck bandwidth detected last time, it indicates that the current bottleneck bandwidth and the bottleneck bandwidth detected last time are located in the same bandwidth interval, so it can be determined that the network state of the wireless network meets the network stability condition; on the contrary, when the empirical frame rate corresponding to the current bottleneck bandwidth is different from the empirical frame rate corresponding to the bottleneck bandwidth detected last time, it indicates that the current bottleneck bandwidth and the bottleneck bandwidth detected last time are not located in the same bandwidth interval, so it can be determined that the network state of the wireless network does not meet the network stability condition.

[0096] It can be seen that based on the difference relationship between the empirical frame rate corresponding to the current bottleneck bandwidth and the empirical frame rate corresponding to the bottleneck bandwidth detected last time, the network state of the wireless network can be determined. During the frame rate control process, the network state of the wireless network is referred to, and the changing network state can be adapted to.

[0097] In another implementation, adjusting the reference frame rate based on the change value of the predicted frame rate relative to the reference frame rate to determine the screen capture frame rate adapted to the current bottleneck bandwidth includes step D1 or step D2:

[0098] Step D1, if the reference frame rate is the specified frame rate, then according to the first formula, based on the change value of the predicted frame rate relative to the specified frame rate, adjust the specified frame rate to determine the screen capture frame rate adapted to the current bottleneck bandwidth;

[0099] Step D2, if the reference frame rate is the empirical frame rate corresponding to the current bottleneck bandwidth, then according to the second formula, based on the change value of the predicted frame rate relative to the empirical frame rate corresponding to the current bottleneck bandwidth, adjust the empirical frame rate corresponding to the current bottleneck bandwidth to determine the screen capture frame rate adapted to the current bottleneck bandwidth;

[0100] Among them, the first formula includes:

[0101] ;

[0102] The second formula includes:

[0103] ;

[0104] is the screen capture frame rate adapted to the current bottleneck bandwidth, is a parameter used to make the frame rate change smoothly, is the predicted frame rate, is the empirical frame rate corresponding to the bottleneck bandwidth detected last time, is the empirical frame rate corresponding to the current bottleneck bandwidth, is the specified frame rate.

[0105] It can be understood that step D1 is the case where the reference frame rate is the specified frame rate. At this time, according to the first formula, based on the change value of the predicted frame rate relative to the specified frame rate, the specified frame rate can be adjusted to determine the screen capture frame rate adapted to the current bottleneck bandwidth; among them, in the first formula can represent that the empirical frame rate corresponding to the current bottleneck bandwidth is the same as the empirical frame rate corresponding to the bottleneck bandwidth detected last time, that is, the network state of the wireless network meets the network stability condition, can be considered as the change value of the predicted frame rate relative to the specified frame rate, If it is a parameter for making the frame rate change smoothly, then it can be updated based on the specified frame rate to obtain a screen capture frame rate adapted to the current bottleneck bandwidth; The range of can be (0, 1), and the empirical value is 0.2. The embodiments of the present application do not make specific limitations on this. Exemplarily, the specified frame rate is 50fps, and the predicted frame rate is 45fps. If is 0.2, the screen capture frame rate adapted to the current bottleneck bandwidth can be calculated as 49fps.

[0106] It can be understood that step D2 is the case where the reference frame rate is the empirical frame rate corresponding to the current bottleneck bandwidth. At this time, according to the second formula, based on the change value of the predicted frame rate relative to the empirical frame rate corresponding to the current bottleneck bandwidth, the empirical frame rate corresponding to the current bottleneck bandwidth is adjusted to determine the screen capture frame rate adapted to the current bottleneck bandwidth; among them, in the second formula, can represent that the empirical frame rate corresponding to the current bottleneck bandwidth is different from the empirical frame rate corresponding to the bottleneck bandwidth detected last time, that is, the network state of the wireless network does not meet the network stability condition. Then, it can be updated based on the empirical frame rate corresponding to the current bottleneck bandwidth to obtain the screen capture frame rate adapted to the current bottleneck bandwidth; can be considered as the change value of the predicted frame rate relative to the empirical frame rate corresponding to the current bottleneck bandwidth. Exemplarily, the empirical frame rate corresponding to the current bottleneck bandwidth is 30fps, and the predicted frame rate is 45fps. If is 0.2, the screen capture frame rate adapted to the current bottleneck bandwidth can be calculated as 33fps.

[0107] Exemplarily, in one implementation, the first formula and the second formula can also be combined into a third formula.

[0108] Among them, the third formula is:

[0109] ;

[0110] It can be understood that the elements in this third formula are the same as those in the first formula and the second formula, and will not be elaborated here.

[0111] It can be seen that for different reference frame rates, the screen capture frame rate adapted to the current bottleneck bandwidth can be determined according to different formulas. When the reference frame rate is the specified frame rate, the specified frame rate is adjusted according to the first formula to determine the screen capture frame rate adapted to the current bottleneck bandwidth; when the reference frame rate is the empirical frame rate corresponding to the current bottleneck bandwidth, the empirical frame rate corresponding to the current bottleneck bandwidth is adjusted according to the second formula to determine the screen capture frame rate adapted to the current bottleneck bandwidth, so as to improve the accuracy of the determined screen capture frame rate.

[0112] Optionally, in another embodiment, predicting the data volume of the target frame to obtain a predicted data volume includes steps E1 - E2:

[0113] Step E1, calculate the sum of the data volumes of each first - type frame encoded within a predetermined time window before the target frame to obtain a target sum value;

[0114] It can be understood that since there is a correlation between each frame of the display data for screen mirroring before and after, the sum of the data volumes of each first - type frame encoded within a predetermined time window can be used to predict the data volume of the target frame; among them, the predetermined time window can be 3s, 5s, etc., and the embodiments of the present application do not make specific limitations on this. Also, the process of calculating the sum of the data volumes of each first - type frame encoded within a predetermined time window to obtain a target sum value can also be referred to as the moving average method. Exemplarily, when the predetermined time window is 3s and there are 5 P - frames within the predetermined time window with data volumes of 10kb, 20kb, 40kb, 5kb, and 15kb respectively, the target sum value can be calculated as 90kb.

[0115] To better understand the content regarding the predetermined time window, taking the first - type frame encoded as a P - frame as an example, it will be introduced in combination with the accompanying drawings:

[0116] As Figure 2 shown, the predetermined time window is located before the target frame, and the sum of the data volumes of each P - frame within the predetermined time window can be calculated as the target sum value.

[0117] Step E2, calculate the ratio of the target sum value to the number of frames of each first - type frame encoded within the predetermined time window to obtain the predicted data volume.

[0118] It can be understood that by determining the number of frames of each first - type frame encoded within the predetermined time window and then calculating the ratio of the target sum value to it, the predicted data volume can be obtained. Exemplarily, when the target sum value is 90kb and there are 5 P - frames within the predetermined time window, the predicted data volume can be calculated as 18kb.

[0119] Exemplarily, in one implementation, the process of calculating the predicted data volume can refer to the fourth formula;

[0120] Among them, the fourth formula is:

[0121] ;

[0122] Among them, is the predicted data volume, is the average value of the data volume of the P frame in the predetermined time window w, w represents the time represented by the predetermined time window, and can also be called a setting parameter. Of course, the above is only an exemplary introduction, and other methods of predicting the data volume are also applicable to the embodiment of the present application, and the embodiment of the present application does not make specific limitations on this.

[0123] It can be seen that by calculating the sum of the data amounts of each first-category frame that has been encoded within the predetermined time window before the target frame, the target sum value is obtained, and the ratio of the target sum value to the number of frames of each first-category frame that has been encoded within the predetermined time window is calculated. The predicted data amount can be obtained. Compared with other methods, the predicted data amount calculated by this method has lower requirements for computing power.

[0124] Optionally, in another embodiment, the step of predicting the number of image frames that can be transmitted per unit time if bitstream transmission is performed under the current bottleneck bandwidth based on the predicted data volume to obtain a predicted frame rate includes step F1:

[0125] Step F1, predicting the total number of first-category frames and second-category frames transmitted per unit time under the current bottleneck bandwidth when the code stream is transmitted in the target transmission state, and obtaining a predicted frame rate;

[0126] The target transmission state is a code stream transmission state in which the predicted data amount is used as the data amount of the first type of frames, the specified data amount is used as the data amount of the second type of frames, and the predetermined empirical value is used as the number of the second type of frames; wherein the second type of frames are frames that do not require reference to other frames during encoding.

[0127] Among them, the specified data amount is the data amount of the encoded second-category frame closest to the target frame, and the second-category frame is a frame that does not need to refer to other frames when encoding. For example, the second-category frame can be a frame encoded as an I frame; exemplarily, when the frame after the second-category frame is encoded is an I frame, the data amount of the I frame closest to the frame is the specified data amount.

[0128] It can be understood that when predicting the bitstream transmission using the predicted data amount as the data amount of the first type of frames, the specified data amount as the data amount of the second type of frames, and the predetermined empirical value as the number of the second type of frames, the total number of frames of the first type of frames and the second type of frames transmitted per unit time under the current bottleneck bandwidth can be used as the predicted frame rate.

[0129] Exemplarily, in one implementation, the first type of frame is a frame encoded as a P frame, and the second type of frame is a frame encoded as an I frame. The process of calculating the predicted frame rate can refer to the fifth formula;

[0130] Among them, the fifth formula is:

[0131] ;

[0132] Among them, is the predicted frame rate, x is a predetermined empirical value, which can also be considered as the number of I-frames transmitted per unit time, and the default value is 1. is the value of the bottleneck bandwidth at the current time. is the specified data volume (the data volume of the I-frame closest to the target frame). is the predicted data volume. is the number of P-frames transmitted per unit time.

[0133] In addition, the fifth formula can be considered as follows: under the current bandwidth detected, if the number of I-frames and the data volume of a single I-frame are determined (the predetermined empirical value x and the data volume of the already encoded I-frame closest to the target frame ), the bandwidth occupied by the I-frames is determined. After removing the bandwidth occupied by the I-frames, the remaining is the bandwidth occupied by the P-frames. Dividing it by the predicted data volume of a single P-frame can calculate how many P-frames can be transmitted, that is, the predicted frame rate.

[0134] It can be seen that when transmitting the bitstream in the target transmission state, predicting the total number of frames of the first type of frames and the second type of frames transmitted per unit time under the current bottleneck bandwidth to obtain the predicted frame rate provides a basis for subsequent frame rate control.

[0135] Optionally, in another embodiment, the method for obtaining the empirical frame rate corresponding to any bottleneck bandwidth includes, as Figure 3 shown:

[0136] S301, determine the bandwidth interval in which the bottleneck bandwidth is located from the preset bandwidth intervals as the target bandwidth interval;

[0137] It can be understood that the bandwidth intervals can be set in advance. For each bottleneck bandwidth, the bandwidth interval in which the bottleneck bandwidth is located can be determined as the target bandwidth. Exemplarily, the bandwidth intervals can be divided into: the first bandwidth interval is (0, 512 kbps], the second bandwidth interval is (512 kbps, 1 Mbps], the third bandwidth interval is (1 Mbps, 2 Mbps], the fourth bandwidth interval is (2 Mbps, 3 Mbps], the fifth bandwidth interval is (3 Mbps, 4 Mbps], and the sixth bandwidth interval is greater than 4 Mbps. If the bottleneck bandwidth is 600 kbps, it can be considered that the bottleneck bandwidth is located in the second bandwidth interval, and the second bandwidth interval can be used as the target bandwidth interval.

[0138] Of course, the above division of the bandwidth intervals is only for exemplary introduction, and the embodiments of the present application do not make specific limitations on this.

[0139] S302. Determine the empirical frame rate corresponding to the target bandwidth range from the empirical frame rate correspondence table as the empirical frame rate corresponding to the bottleneck bandwidth.

[0140] The empirical frame rate correspondence table is used to record each bandwidth range and the empirical frame rate corresponding to each bandwidth range. The empirical frame rate corresponding to each bandwidth range is the frame rate at which the video stream transmitted at the bandwidth of this bandwidth range can clearly display the picture.

[0141] It can be understood that the empirical frame rate correspondence table is used to record each bandwidth range and the empirical frame rate corresponding to each bandwidth range. Based on the empirical frame rate corresponding to this bandwidth range, the display data is captured, and the video stream transmitted at the bandwidth of this bandwidth range can clearly display the picture at the receiving end of the screen mirroring. Then, based on the empirical frame rate correspondence table, the empirical frame rate corresponding to the target bandwidth range can be determined as the empirical frame rate corresponding to the bottleneck bandwidth.

[0142] Exemplarily, in one implementation, the empirical frame rate correspondence table is shown in Table 1:

[0143]

[0144] Table 1

[0145] Among them, the empirical frame rate corresponding to the bandwidth range (0, 512 kbps] is 3 fps, the empirical frame rate corresponding to the bandwidth range (512 kbps, 1 Mbps] is 5 fps, the empirical frame rate corresponding to the bandwidth range (1 Mbps, 2 Mbps] is 13 fps, the empirical frame rate corresponding to the bandwidth range (2 Mbps, 3 Mbps] is 18 fps, the empirical frame rate corresponding to the bandwidth range (3 Mbps, 4 Mbps] is 25 fps, and the empirical frame rate corresponding to the bandwidth range greater than 4 Mbps is 30 fps. Of course, the above-introduced empirical frame rate correspondence table is only for exemplary introduction and has no specific limitation on the embodiments of the present application.

[0146] Exemplarily, in another implementation, the empirical frame rate correspondence table can be expressed as:

[0147] ;

[0148] Among them, , is the empirical frame rate corresponding to the current bottleneck bandwidth, is the current bottleneck bandwidth, (0, is the bandwidth range with the smallest represented data volume among each bandwidth range, (( , is the bandwidth range with the largest represented data volume among each bandwidth range.

[0149] Exemplarily, if the current bottleneck bandwidth is 200 kbps, then the corresponding empirical frame rate can be determined as 3 fps using the empirical frame rate correspondence table.

[0150] It can be seen that from the preset bandwidth intervals, the bandwidth interval where the bottleneck bandwidth is located is determined as the target bandwidth interval, and from the empirical frame rate correspondence table, the empirical frame rate corresponding to the target bandwidth interval is determined as the empirical frame rate corresponding to the bottleneck bandwidth, so that the empirical frame rate corresponding to any bottleneck bandwidth can be quickly determined, enabling a quick response to the changing network state.

[0151] In another implementation, the method for determining the empirical frame rate corresponding to each bandwidth interval in the empirical frame rate correspondence table includes steps G1 - G2:

[0152] Step G1, for each test scenario, according to a predetermined test method, determine the screen capture frame rate of the test source end for the display data matching the test scenario in this bandwidth interval.

[0153] Step G2, calculate the average value of the screen capture frame rates of each test scenario corresponding to this bandwidth interval to obtain the empirical frame rate corresponding to this bandwidth interval.

[0154] Among them, the predetermined test method includes:

[0155] Taking the current test frame rate as the screen capture frame rate of the display data matching the test scenario at the test source end, capturing the screen of the test source end, encoding the captured display data according to the fixed bit rate corresponding to this bandwidth interval and transmitting it to the test receiving end, determining the picture clarity of the display data at the test receiving end, and when the picture clarity meets the picture clarity condition, obtaining the screen capture frame rate of the test scenario corresponding to this bandwidth interval; otherwise, adjusting the current test frame rate to obtain a new test frame rate, and returning to the step of taking the current test frame rate as the screen capture frame rate of the display data matching the test scenario at the test source end and capturing the screen of the test source end; among them, the value of the fixed bit rate corresponding to this bandwidth interval is the data volume represented by the maximum bandwidth in this bandwidth interval.

[0156] It can be understood that for step G1, the test scenarios may include scenarios of still images, scenarios of displaying tables, scenarios of displaying presentation documents, etc., with different styles, and the embodiments of the present application do not make specific limitations thereto; moreover, both the test source end and the test receiving end can be regarded as servers for performing screen mirroring tests for each test scenario. Additionally, each test scenario can be divided according to a predetermined division dimension, and the predetermined division dimension can be the type of business data as the division dimension, or the degree of picture movement as the division dimension, and the embodiments of the present application do not make specific limitations thereto; for example, when the type of business data is used as the division dimension, when the type of business data is PPT, the scenario of displaying the PPT can be regarded as a test scenario, and the screen capture frame rate in the scenario of displaying the PPT within the bandwidth range (3 Mbps, 4 Mbps] can be determined according to a predetermined test method; when the type of business data is document data, the scenario of displaying the document data can be regarded as another test scenario, and the screen capture frame rate in the scenario of displaying the document data within the bandwidth range (3 Mbps, 4 Mbps] can be determined according to a predetermined test method; when the degree of picture movement is used as the division dimension, the scenario of displaying a still picture with a small degree of picture movement can be used as a test scenario, and the screen capture frame rate in the scenario of displaying the still picture within the bandwidth range (3 Mbps, 4 Mbps] can be determined according to a predetermined test method, and the scenario of displaying a video with a large degree of picture movement can be used as another test scenario, and the screen capture frame rate in the scenario of displaying the video within the bandwidth range (3 Mbps, 4 Mbps] can be determined according to a predetermined test method.

[0157] And for each test scenario, the capture frame rate corresponding to the test scenario for this bandwidth range can be determined according to a predetermined test method. Specifically, using the current test frame rate as the capture frame rate of the display data matching the test scenario at the test source end, the test source end is captured. Here, the test frame rate can also be regarded as the initial frame rate. Since the test frame rate can be adjusted later, there are no special requirements for setting the test frame rate. And the display data obtained by capturing the screen is encoded according to the fixed bit rate corresponding to the bandwidth range and transmitted to the test receiver. The value of the fixed bit rate corresponding to the bandwidth range can be the data volume represented by the maximum bandwidth in this bandwidth range. Of course, it can also be the data volume represented by the median bandwidth in this bandwidth range. This application embodiment does not make specific limitations on this. Moreover, after receiving the encoded bitstream, the test receiver can display the display data represented by the bitstream. At this time, the picture clarity of the display data at the test receiver can be determined. When the picture clarity meets the picture clarity condition, the capture frame rate corresponding to the test scenario for this bandwidth range is determined. Whether it meets the picture clarity condition can be judged based on a picture clarity algorithm or subjectively by the user. Then, when the picture clarity is determined to be clear through the picture clarity algorithm, it can be considered that the picture clarity condition is met. Or when the user subjectively determines that the picture is clear, it can also be considered that the picture clarity condition is met. This application embodiment does not make specific limitations on the manner of meeting the picture clarity condition. Exemplarily, user a sees that the puppy image displayed at the test receiver is clear enough. Then, it can be considered that the picture clarity condition is met, and thus the capture frame rate corresponding to the puppy image test scenario for this bandwidth range can be obtained. Exemplarily, in the test scenario of displaying a PPT, the bandwidth range is (3 Mbps, 4 Mbps]. Using the current test frame rate of 15 fps as the capture frame rate of the PPT display data matching the PPT display scenario at the test source end, the test source end is captured. The captured PPT display data is encoded at 4 Mbps and transmitted to the test receiver. At this time, the current test receiver does not meet the picture clarity condition. The current test frame rate is adjusted, and the step of capturing the test source end by using the current test frame rate as the capture frame rate of the PPT display data matching the PPT display scenario at the test source end is returned until the picture clarity condition is met. The current test frame rate is used as the capture frame rate in the bandwidth range (3 Mbps, 4 Mbps] in the scenario of displaying the PPT.It should be emphasized that when the screen clarity does not meet the screen clarity condition, the current test frame rate can be adjusted to obtain a new test frame rate, and the current test frame rate is returned as the capture frame rate of the display data matching the test scenario at the test source end, and the steps of capturing the screen at the test source end are performed to achieve a loop until the screen clarity meets the screen clarity condition, and the capture frame rate in the test scenario corresponding to the bandwidth interval is determined. Exemplarily, user b sees that the table image displayed on the test receiver is not clear enough, adjusts the test frame rate from 20fps to 17fps, and re-executes capturing the table image displayed on the test source end at 17fps, encoding the captured table image and transmitting it to the test receiver. User b checks again whether the displayed table image on the test receiver is clear enough. If user b sees that the displayed table image is clear enough, it is determined that 17fps can be used as the capture frame rate of the table image.

[0158] It can be understood that for step G2, after obtaining the capture frame rates in each test scenario corresponding to the bandwidth interval, the average value of the capture frame rates in each test scenario corresponding to the bandwidth interval can be calculated, so that the empirical frame rate corresponding to the bandwidth interval can be obtained. Exemplarily, the capture frame rate in test scenario 1 corresponding to the bandwidth interval (0, 512kbps] is 5fps, the capture frame rate in test scenario 2 corresponding to it is 3fps, and the capture frame rate in test scenario 3 corresponding to it is 1fps. It can be calculated that the empirical frame rate corresponding to the bandwidth interval (0, 512kbps] is 3fps. In addition, in each test scenario, the capture frame rate in the test scenario with a large screen motion scenario will be larger. Similarly, the capture frame rate in the test scenario with a small screen motion scenario will be smaller. However, by calculating the average value of the capture frame rates in each test scenario corresponding to the bandwidth interval, a more accurate empirical frame rate corresponding to the bandwidth interval can be obtained.

[0159] It can be seen that through the determination method introduced above, the empirical frame rate corresponding to each bandwidth interval can be determined. In this process, the capture frame rates in each test scenario are referred to, and the average value is calculated to determine the empirical frame rate corresponding to the bandwidth interval, so that an accurate empirical frame rate corresponding to each bandwidth interval can be obtained, providing a basis for subsequent frame rate control. And, the value of the fixed bit rate corresponding to the bandwidth interval is the data volume represented by the maximum bandwidth in the bandwidth interval. At this fixed bit rate, a capture frame rate can be determined to ensure the screen clarity. Then, capturing the screen at other bandwidths within the bandwidth interval according to this capture frame rate will surely be able to ensure the screen clarity, thereby improving the accuracy of the test.

[0160] Optionally, in another embodiment, the embodiments of the present application further provide a schematic diagram of the principle of frame rate control, as Figure 4 shown:

[0161] During the screen mirroring process, detect the bottleneck bandwidth of the wireless network as the current bottleneck bandwidth. This process can also be called bitrate detection. Predict the data volume of the target frame to obtain the predicted data volume. This process can also be called single-frame data volume prediction. At the same time, based on the current bottleneck bandwidth, an empirical frame rate can be selected. Based on the predicted data volume, predict the number of image frames that can be transmitted per unit time under the current bottleneck bandwidth to obtain the predicted frame rate. This process can also be called frame rate prediction. In addition, before frame rate decision-making, a specified frame rate can be obtained. The specified frame rate is the screen capture frame rate determined based on the previously detected bottleneck bandwidth and adapted to the bottleneck bandwidth. Based on the network state of the wireless network, it can be determined whether to select the empirical frame rate corresponding to the current bottleneck bandwidth or the specified frame rate as the reference bandwidth. This process can also be called frame rate decision-making. Based on the change value of the predicted frame rate relative to the reference frame rate, adjust the reference frame rate to determine the screen capture frame rate adapted to the current bottleneck bandwidth. This process can also be called determining the target frame rate, where the target frame rate is the screen capture frame rate adapted to the current bottleneck bandwidth introduced in the above embodiments. Subsequently, the determined screen capture frame rate can be set as the screen capture frame rate for capturing the display data at the source end, so that the captured display data is encoded and transmitted to the screen mirroring receiver end according to the determined screen capture frame rate. This process can also be called frame rate update.

[0162] The frame rate control method provided by the embodiments of the present application can detect the bottleneck bandwidth of the wireless network during the screen mirroring process, use it as the current bottleneck bandwidth, predict the data volume of the target frame, obtain the predicted data volume, and based on the predicted data volume, predict the number of image frames that can be transmitted per unit time if the bitstream is transmitted under the current bottleneck bandwidth, to obtain the predicted frame rate. Subsequently, based on the change value of the predicted frame rate relative to the reference frame rate, the reference frame rate can be adjusted to determine the screen capture frame rate adapted to the current bottleneck bandwidth, and the determined screen capture frame rate is set as the screen capture frame rate for capturing the display data of the source end. Among them, the reference frame rate is the frame rate set as the adjustment basis according to the network state of the wireless network. It can be seen that by referring to the predicted frame rate (the predicted frame rate can to a certain extent reflect the bit rate required in the current network environment, and this required bit rate can be considered as the actual encoded data volume) and the network state of the wireless network, the reference frame rate can be dynamically adjusted, and a quick response can be made to the changing network state. In addition, since the network state of the wireless network fluctuates, the bottleneck bandwidth can also change, and the determined screen capture frame rate adapted to the current bottleneck bandwidth can adapt to the changing bottleneck bandwidth. In this way, when encoding the display data captured at the determined screen capture frame rate, the bitstream bit rate used for the encoding process has a positive correlation with the determined screen capture frame rate, and there will be no situation where the bitstream bit rate is much greater than the bottleneck bandwidth, thus affecting the screen mirroring effect, nor will there be a situation where the bitstream bit rate is much less than the bottleneck bandwidth, resulting in a low utilization rate of the network bandwidth. Therefore, both the screen mirroring effect and the utilization rate of the network bandwidth are taken into account.

[0163] Based on the above method embodiments, as Figure 5 shown, the embodiments of the present application provide a frame rate control device, which is applied to a screen mirroring device. The screen mirroring device communicates with a screen mirroring receiving end through a wireless network. The device includes:

[0164] A detection module 510, configured to detect the bottleneck bandwidth of the wireless network during the screen mirroring process and use it as the current bottleneck bandwidth;

[0165] A first prediction module 520, configured to predict the data volume of a target frame to obtain a predicted data volume. Wherein, the target frame is a first type of frame to be encoded during the encoding process of the captured display data of the source end, and the first type of frame is a frame that needs to refer to other frames during encoding;

[0166] A second prediction module 530, configured to predict the number of image frames that can be transmitted per unit time if the bitstream is transmitted under the current bottleneck bandwidth based on the predicted data volume, to obtain a predicted frame rate;

[0167] An adjustment processing module 540 is configured to adjust the reference frame rate based on the change value of the predicted frame rate relative to the reference frame rate, so as to determine a screen capture frame rate adapted to the current bottleneck bandwidth; wherein, the reference frame rate is a frame rate set as an adjustment basis according to the network state of the wireless network.

[0168] A setting module 550 is configured to set the determined screen capture frame rate as the screen capture frame rate for capturing the display data of the source end, so that the display data captured at the determined screen capture frame rate is encoded and transmitted to the screen projection receiving end, and the bit rate of the bitstream used in the encoding process has a positive correlation with the determined screen capture frame rate.

[0169] Optionally, the setting method of the reference frame rate includes:

[0170] When the network state meets the network stability condition, a specified frame rate is used as the reference frame rate;

[0171] When the network state does not meet the network stability condition, an empirical frame rate corresponding to the current bottleneck bandwidth is used as the reference frame rate;

[0172] The specified frame rate is a screen capture frame rate determined based on the previously detected bottleneck bandwidth and adapted to the bottleneck bandwidth, and the empirical frame rate corresponding to any bottleneck bandwidth is: the frame rate at which the bitstream transmitted at the bottleneck bandwidth can clearly display the image.

[0173] Optionally, the determining method of the network state of the wireless network includes:

[0174] Based on the difference relationship between the empirical frame rate corresponding to the current bottleneck bandwidth and the empirical frame rate corresponding to the previously detected bottleneck bandwidth, the network state of the wireless network is determined according to a predetermined determination method;

[0175] Wherein, the predetermined determination method includes:

[0176] If the empirical frame rate corresponding to the current bottleneck bandwidth is the same as the empirical frame rate corresponding to the previously detected bottleneck bandwidth, it is determined that the network state of the wireless network meets the network stability condition;

[0177] If the empirical frame rate corresponding to the current bottleneck bandwidth is different from the empirical frame rate corresponding to the previously detected bottleneck bandwidth, it is determined that the network state of the wireless network does not meet the network stability condition.

[0178] Optionally, the adjusting the reference frame rate based on the change value of the predicted frame rate relative to the reference frame rate to determine a screen capture frame rate adapted to the current bottleneck bandwidth includes:

[0179] If the reference frame rate is the specified frame rate, then according to the first formula, based on the change value of the predicted frame rate relative to the specified frame rate, the specified frame rate is adjusted to determine the screen capture frame rate adapted to the current bottleneck bandwidth;

[0180] If the reference frame rate is the empirical frame rate corresponding to the current bottleneck bandwidth, then according to the second formula, based on the change value of the predicted frame rate relative to the empirical frame rate corresponding to the current bottleneck bandwidth, the empirical frame rate corresponding to the current bottleneck bandwidth is adjusted to determine the screen capture frame rate adapted to the current bottleneck bandwidth;

[0181] Among them, the first formula includes:

[0182] ;

[0183] The second formula includes:

[0184] ;

[0185] is the screen capture frame rate adapted to the current bottleneck bandwidth, is a parameter for making the frame rate change smoothly, is the predicted frame rate, is the empirical frame rate corresponding to the bottleneck bandwidth detected last time, is the empirical frame rate corresponding to the current bottleneck bandwidth, is the specified frame rate.

[0186] Optionally, the first prediction module is specifically configured to:

[0187] Calculate the sum of the data amounts of each first type of frame encoded within a predetermined time window before the target frame to obtain a target sum value;

[0188] Calculate the ratio of the target sum value to the number of frames of each first type of frame encoded within the predetermined time window to obtain a predicted data amount.

[0189] Optionally, the second prediction module is specifically configured to:

[0190] Predict the total number of frames of the first type of frame and the second type of frame transmitted per unit time at the current bottleneck bandwidth when transmitting the bitstream in the target transmission state to obtain a predicted frame rate;

[0191] The target transmission state is a bitstream transmission state with the predicted data amount as the data amount of the first type of frame, the specified data amount as the data amount of the second type of frame, and a predetermined empirical value as the number of the second type of frame; among them, the second type of frame is a frame that does not need to refer to other frames during encoding.

[0192] Optionally, the method for obtaining the empirical frame rate corresponding to any bottleneck bandwidth includes:

[0193] Determine the bandwidth interval in which the bottleneck bandwidth is located from each preset bandwidth interval as the target bandwidth interval;

[0194] Determine the empirical frame rate corresponding to the target bandwidth interval from the empirical frame rate correspondence table as the empirical frame rate corresponding to the bottleneck bandwidth; wherein, the empirical frame rate correspondence table is used to record each bandwidth interval and the empirical frame rate corresponding to each bandwidth interval, and the empirical frame rate corresponding to each bandwidth interval is: the frame rate at which the video stream transmitted at the bandwidth of this bandwidth interval as the transmission rate can clearly display the picture.

[0195] Optionally, the method for determining the empirical frame rate corresponding to each bandwidth interval in the empirical frame rate correspondence table includes:

[0196] For each test scenario, determine the screen capture frame rate in this test scenario corresponding to this bandwidth interval according to a predetermined test method;

[0197] Calculate the average value of the screen capture frame rates in each test scenario corresponding to this bandwidth interval to obtain the empirical frame rate corresponding to this bandwidth interval;

[0198] Wherein, the predetermined test method includes:

[0199] Use the current test frame rate as the screen capture frame rate of the display data matching this test scenario at the test source end, capture the screen of the test source end, encode the captured display data at the fixed bit rate corresponding to this bandwidth interval and transmit it to the test receiver, determine the picture clarity of the display data at the test receiver, and when the picture clarity meets the picture clarity condition, obtain the screen capture frame rate in this test scenario corresponding to this bandwidth interval; otherwise, adjust the current test frame rate to obtain a new test frame rate, and return to the step of using the current test frame rate as the screen capture frame rate of the display data matching this test scenario at the test source end and capturing the screen of the test source end; wherein, the value of the fixed bit rate corresponding to this bandwidth interval is the data volume represented by the maximum bandwidth in this bandwidth interval.

[0200] Optionally, the detection module is specifically configured to:

[0201] In response to meeting the frame rate control condition, detect the bottleneck bandwidth of the wireless network as the current bottleneck bandwidth;

[0202] Wherein, the frame rate control condition includes: reaching the target period, and / or, detecting that the network state of the wireless network meets the preset unstable condition.

[0203] In the technical solution of this application, operations such as the acquisition, storage, use, processing, transmission, provision, and disclosure of the user's personal information are all carried out under the condition of obtaining the user's authorization.

[0204] The embodiment of this application also provides a screen mirroring device, such as Figure 6 shown, including:

[0205] A memory 601 for storing computer programs;

[0206] A processor 602, when executing the program stored on the memory 601, implements any of the above frame rate control methods.

[0207] And the above screen mirroring device may further include a communication bus and / or a communication interface. The processor 602, the communication interface, and the memory 601 complete mutual communication through the communication bus.

[0208] The communication bus mentioned in the above screen mirroring device may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity, only a thick line is used in the figure to represent it, but it does not mean that there is only one bus or one type of bus.

[0209] The communication interface is used for communication between the above screen mirroring device and other devices.

[0210] The memory may include a Random Access Memory (RAM), or may also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located far from the aforementioned processor.

[0211] The above-mentioned processor may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0212] In another embodiment provided by the present application, a computer-readable storage medium is further provided. A computer program is stored in the computer-readable storage medium. When the computer program is executed by a processor, the above-mentioned any frame rate control method is implemented.

[0213] In another embodiment provided by the present application, a computer program product including instructions is further provided. When it runs on a computer, the computer is caused to execute any frame rate control method in the above embodiments.

[0214] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that the computer can access, or a data storage device such as a server or data center that includes one or more integrated available media. The available medium may be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a solid-state disk (SSD), etc.

[0215] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variation thereof is intended to cover a non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the element.

[0216] Each embodiment in this specification is described in a related manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for the related parts, reference can be made to the partial description of the method embodiment.

[0217] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are all included in the protection scope of the present application.

Claims

1. A frame rate control method, characterized in that: Applied to a screen projection device, the screen projection device communicates with a screen projection receiving end via a wireless network; the method comprises: During the screen projection process, the bottleneck bandwidth of the wireless network is detected as the current bottleneck bandwidth; Predicting the data amount of the target frame to obtain the predicted data amount; wherein the target frame is a first-class frame to be encoded in the encoding process of the captured display data of the source end, and the first-class frame is a frame that needs to refer to other frames during encoding; Based on the predicted data volume, predict the number of image frames that can be transmitted per unit time if bitstream transmission is performed under the current bottleneck bandwidth, and obtain a predicted frame rate; Based on the change value of the predicted frame rate relative to the reference frame rate, the reference frame rate is adjusted to determine a screen capture frame rate that matches the current bottleneck bandwidth; wherein the reference frame rate is a frame rate set as an adjustment basis according to the network status of the wireless network; The determined screen capture frame rate is set as the screen capture frame rate for capturing the display data of the source end, so that the display data captured according to the determined screen capture frame rate is encoded and transmitted to the screen projection receiving end, and the bit stream rate used for the encoding process has a positive correlation with the determined screen capture frame rate.

2. The method according to claim 1, characterized in that The reference frame rate is set in the following manner: When the network state satisfies a network stability condition, using the specified frame rate as a reference frame rate; When the network state does not meet the network stability condition, the empirical frame rate corresponding to the current bottleneck bandwidth is used as the reference frame rate; The specified frame rate is a screen capture frame rate determined based on the bottleneck bandwidth detected last time and adapted to the bottleneck bandwidth. The empirical frame rate corresponding to any bottleneck bandwidth is: a frame rate that enables a code stream transmitted using the bottleneck bandwidth as the code rate to clearly display the image.

3. The method according to claim 2, characterized in that The method for determining the network status of the wireless network includes: Determining the network state of the wireless network according to a predetermined determination method based on a difference relationship between an empirical frame rate corresponding to the current bottleneck bandwidth and an empirical frame rate corresponding to the bottleneck bandwidth detected last time; The predetermined determination method includes: If the empirical frame rate corresponding to the current bottleneck bandwidth is the same as the empirical frame rate corresponding to the bottleneck bandwidth detected last time, it is determined that the network state of the wireless network meets the network stability condition; If the empirical frame rate corresponding to the current bottleneck bandwidth is different from the empirical frame rate corresponding to the bottleneck bandwidth detected last time, it is determined that the network state of the wireless network does not meet the network stability condition.

4. The method according to claim 2, characterized in that: The adjusting process of the reference frame rate based on the change value of the predicted frame rate relative to the reference frame rate to determine a screen capture frame rate that matches the current bottleneck bandwidth includes: If the reference frame rate is a specified frame rate, adjusting the specified frame rate based on a change value of the predicted frame rate relative to the specified frame rate according to the first formula to determine a screen capture frame rate that matches the current bottleneck bandwidth; If the reference frame rate is the empirical frame rate corresponding to the current bottleneck bandwidth, then according to the second formula, based on the change value of the predicted frame rate relative to the empirical frame rate corresponding to the current bottleneck bandwidth, the empirical frame rate corresponding to the current bottleneck bandwidth is adjusted to determine a screen capture frame rate that matches the current bottleneck bandwidth; Wherein, the first formula includes: ； The second formula includes: ； is the screen capture frame rate adapted to the current bottleneck bandwidth, is a parameter used to make the frame rate change smooth. is the predicted frame rate, is the empirical frame rate corresponding to the bottleneck bandwidth detected last time, is the empirical frame rate corresponding to the current bottleneck bandwidth, Specifies the frame rate for the video.

5. The method according to any one of claims 1 to 4, characterized in that: The step of predicting the data amount of the target frame to obtain the predicted data amount includes: Calculating the sum of the data amounts of the first-category frames that have been encoded within a predetermined time window before the target frame to obtain a target sum value; The ratio of the target sum value to the number of frames of each first category that have been encoded within a predetermined time window is calculated to obtain the predicted data volume.

6. The method according to any one of claims 1 to 4, characterized in that: The step of predicting the number of image frames that can be transmitted per unit time based on the predicted data volume if the code stream is transmitted under the current bottleneck bandwidth to obtain the predicted frame rate includes: Predicting the total number of first-category frames and second-category frames transmitted per unit time under the current bottleneck bandwidth when the code stream is transmitted in the target transmission state, and obtaining the predicted frame rate; The target transmission state is: a code stream transmission state in which the predicted data amount is used as the data amount of the first type of frames, the specified data amount is used as the data amount of the second type of frames, and a predetermined empirical value is used as the number of the second type of frames; wherein the second type of frames are frames that do not require reference to other frames during encoding.

7. The method according to any one of claims 1 to 4, characterized in that: The methods for obtaining the empirical frame rate corresponding to any bottleneck bandwidth include: From various preset bandwidth intervals, determine the bandwidth interval where the bottleneck bandwidth is located as the target bandwidth interval; From the empirical frame rate correspondence table, determine the empirical frame rate corresponding to the target bandwidth interval as the empirical frame rate corresponding to the bottleneck bandwidth; wherein the empirical frame rate correspondence table is used to record each bandwidth interval and the empirical frame rate corresponding to each bandwidth interval, and the empirical frame rate corresponding to each bandwidth interval is: the frame rate at which the code stream transmitted according to the bandwidth of the bandwidth interval as the transmission rate can clearly display the picture.

8. The method according to claim 7, characterized in that The method for determining the empirical frame rate corresponding to each bandwidth interval in the empirical frame rate corresponding table includes: For each test scenario, determine the screen capture frame rate of the test scenario corresponding to the bandwidth interval according to a predetermined test method; Calculate the average of the screen capture frame rates in each test scenario corresponding to the bandwidth interval to obtain the empirical frame rate corresponding to the bandwidth interval; Wherein, the predetermined test method includes: The test source end is captured using the current test frame rate as the screen capture frame rate of the display data matching the test scenario at the test source end, the display data obtained by the screen capture is encoded according to the fixed bit rate corresponding to the bandwidth interval and transmitted to the test receiving end, the picture clarity of the display data of the test receiving end is determined, and when the picture clarity meets the picture clarity condition, the screen capture frame rate of the test scenario corresponding to the bandwidth interval is obtained; otherwise, the current test frame rate is adjusted to obtain a new test frame rate, and the step of capturing the screen of the test source end using the current test frame rate as the screen capture frame rate of the display data matching the test scenario at the test source end is returned; wherein the value of the fixed bit rate corresponding to the bandwidth interval is the amount of data represented by the maximum bandwidth in the bandwidth interval.

9. The method according to any one of claims 1 to 4, characterized in that: The detecting the bottleneck bandwidth of the wireless network as the current bottleneck bandwidth includes: In response to satisfying the frame rate control condition, detecting a bottleneck bandwidth of the wireless network as a current bottleneck bandwidth; The frame rate control conditions include: reaching a target period, and / or detecting that a network state of the wireless network meets a preset unstable condition.

10. A frame rate control device, characterized in that: Applied to a screen projection device, the screen projection device communicates with a screen projection receiving end via a wireless network; the device comprises: A detection module, used to detect the bottleneck bandwidth of the wireless network as the current bottleneck bandwidth during the screen projection process; A first prediction module is used to predict the data amount of a target frame to obtain a predicted data amount; wherein the target frame is a first-type frame to be encoded during the encoding process of the captured display data of the source end, and the first-type frame is a frame that needs to refer to other frames during encoding; A second prediction module is used to predict, based on the predicted data volume, the number of image frames that can be transmitted per unit time if bitstream transmission is performed under the current bottleneck bandwidth, and obtain a predicted frame rate; An adjustment processing module, configured to adjust the reference frame rate based on a change value of the predicted frame rate relative to the reference frame rate, so as to determine a screen capture frame rate that is compatible with the current bottleneck bandwidth; wherein the reference frame rate is a frame rate set as an adjustment basis according to a network status of the wireless network; A setting module is used to set the determined screen capture frame rate as the screen capture frame rate for capturing the display data of the source end, so that the display data captured according to the determined screen capture frame rate is encoded and transmitted to the screen projection receiving end, and the bit stream rate used for the encoding process has a positive correlation with the determined screen capture frame rate.

11. A screen projection device, characterized in that: include: Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-9 when executing a program stored in a memory.

12. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 9.

Images