Frame rate control method and device, screen projection equipment and program product

Abstract

The embodiment of the invention provides a frame rate control method and device, screen projection equipment and a program product, and relates to the technical field of data processing.The frame rate control method comprises the steps that in the screen projection process, the bottleneck bandwidth of a wireless network is detected and serves as the current bottleneck bandwidth, the data size of a target frame is predicted, and the predicted data size is obtained; and predicting the number of image frames which can be transmitted in unit time if code stream transmission is carried out under the current bottleneck bandwidth based on the predicted data volume to obtain a predicted frame rate, and carrying out adjustment processing on the reference frame rate based on a change value of the predicted frame rate relative to the reference frame rate so as to determine a screen capture frame rate matched with the current bottleneck bandwidth, and setting the determined screen capture frame rate as the screen capture frame rate for capturing the display data of the source end, so that the display data captured according to the determined screen capture frame rate is coded and transmitted. Therefore, according to the scheme, the screen projection effect and the utilization rate for the network bandwidth can be considered.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a frame rate control method, device, screen projection equipment and program product. Background Art

[0002] In the screen projection scenario, the screen projection device usually uses a fixed frame rate to capture the display data of the source end, and encodes the captured display data according to the bit stream rate corresponding to the fixed frame rate and transmits it to the screen projection receiving end, where the bit stream rate and frame rate usually have a positive correlation.

[0003] However, during the transmission of the bit stream, it may be affected by the bottleneck bandwidth of the wireless network, thereby affecting the subsequent screen projection effect based on the encoded bit stream; the bottleneck bandwidth is the bandwidth of the device / link that limits the data transmission rate in the network, and the bottleneck bandwidth is also the maximum transmission rate allowed by the network bandwidth, and the bottleneck bandwidth also changes in real time.

[0004] Specifically, when the bit rate is much larger than the bottleneck bandwidth, network congestion will occur, and screen distortion and freezes may occur during screen projection, that is, the screen projection effect is poor; and when the bit rate is much smaller than the bottleneck bandwidth, the maximum transmission rate represented by the bottleneck bandwidth cannot be reached, resulting in low utilization of the network bandwidth.

[0005] Currently, there is an urgent need for a frame rate control method to achieve both screen projection effect and network bandwidth utilization. Summary of the invention

[0006] The purpose of the embodiments of the present application is to provide a frame rate control method, device, screen projection device and program product to achieve both screen projection effect and network bandwidth utilization. The specific technical solution is as follows:

[0007] In a first aspect, an embodiment of the present application provides a frame rate control method, which is applied to a screen projection device, wherein the screen projection device communicates with a screen projection receiving end via a wireless network; the method comprises:

[0008] During the screen projection process, the bottleneck bandwidth of the wireless network is detected as the current bottleneck bandwidth;

[0009] Predicting the data amount of the target frame to obtain the predicted data amount; wherein the target frame is a first-class frame to be encoded in the encoding process of the captured display data of the source end, and the first-class frame is a frame that needs to refer to other frames during encoding;

[0010] Based on the predicted data volume, predict the number of image frames that can be transmitted per unit time if bitstream transmission is performed under the current bottleneck bandwidth, and obtain a predicted frame rate;

[0011] Based on the change value of the predicted frame rate relative to the reference frame rate, the reference frame rate is adjusted to determine a screen capture frame rate that matches the current bottleneck bandwidth; wherein the reference frame rate is a frame rate set as an adjustment basis according to the network status of the wireless network;

[0012] The determined screen capture frame rate is set as the screen capture frame rate for capturing the display data of the source end, so that the display data captured according to the determined screen capture frame rate is encoded and transmitted to the screen projection receiving end, and the bit stream rate used for the encoding process has a positive correlation with the determined screen capture frame rate.

[0013] In a second aspect, an embodiment of the present application provides a frame rate control device, which is applied to a screen projection device, wherein the screen projection device communicates with a screen projection receiving end via a wireless network; the device includes:

[0014] A detection module, used to detect the bottleneck bandwidth of the wireless network as the current bottleneck bandwidth during the screen projection process;

[0015] A first prediction module is used to predict the data amount of a target frame to obtain a predicted data amount; wherein the target frame is a first-type frame to be encoded during the encoding process of the captured display data of the source end, and the first-type frame is a frame that needs to refer to other frames during encoding;

[0016] A second prediction module is used to predict, based on the predicted data volume, the number of image frames that can be transmitted per unit time if bitstream transmission is performed under the current bottleneck bandwidth, and obtain a predicted frame rate;

[0017] An adjustment processing module, configured to adjust the reference frame rate based on a change value of the predicted frame rate relative to the reference frame rate, so as to determine a screen capture frame rate that is compatible with the current bottleneck bandwidth; wherein the reference frame rate is a frame rate set as an adjustment basis according to a network status of the wireless network;

[0018] A setting module is used to set the determined screen capture frame rate as the screen capture frame rate for capturing the display data of the source end, so that the display data captured according to the determined screen capture frame rate is encoded and transmitted to the screen projection receiving end, and the bit stream rate used for the encoding process has a positive correlation with the determined screen capture frame rate.

[0019] In a third aspect, an embodiment of the present application provides a screen projection device, including:

[0020] Memory, used to store computer programs;

[0021] The processor is used to implement any of the above frame rate control methods when executing the program stored in the memory.

[0022] In a fourth aspect, an embodiment of the present application provides a computer program product, including a computer program, which implements any of the above-mentioned frame rate control methods when executed by a processor.

[0023] Beneficial effects of the embodiments of the present application:

[0024] The frame rate control method provided in the embodiment of the present application can detect the bottleneck bandwidth of the wireless network during the screen projection process, use it as the current bottleneck bandwidth, predict the data volume of the target frame, and obtain the predicted data volume. Based on the predicted data volume, it is predicted that if the code stream is transmitted under the current bottleneck bandwidth, the number of image frames that can be transmitted per unit time is obtained to obtain the predicted frame rate. Subsequently, based on the change value of the predicted frame rate relative to the reference frame rate, the reference frame rate can be adjusted to determine the screen capture frame rate that is compatible with the current bottleneck bandwidth, and the determined screen capture frame rate is set as the screen capture frame rate for capturing the display data of the source end, wherein the reference frame rate is a frame rate set as the adjustment basis according to the network status of the wireless network; it can be seen that the predicted frame rate is referred to (the predicted frame rate can reflect the code rate required in the current network environment to a certain extent). , the required bit rate can be considered as the actual encoded data volume) and the network status of the wireless network, so that the reference frame rate can be dynamically adjusted, and a quick response can be made to the changing network status; and, since the network status of the wireless network fluctuates, the bottleneck bandwidth may also be changing, and the determined screen capture frame rate that is adapted to the current bottleneck bandwidth can adapt to the changing bottleneck bandwidth. In this way, when encoding the display data captured according to the determined screen capture frame rate, the bit stream rate used for the encoding process has a positive correlation with the determined screen capture frame rate, and the bit stream rate will not be too much greater than the bottleneck bandwidth, thereby affecting the screen projection effect, nor will the bit stream rate be too much less than the bottleneck bandwidth, resulting in a low utilization rate of the network bandwidth, thereby taking into account both the screen projection effect and the utilization rate of the network bandwidth.

[0025] Of course, implementing any product or method of the present application does not necessarily require achieving all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0027] Figure 1 A flowchart of a frame rate control method provided in an embodiment of the present application;

[0028] Figure 2 A schematic diagram of the effect of a predetermined time window provided in an embodiment of the present application;

[0029] Figure 3 A schematic diagram of a process for obtaining an empirical frame rate corresponding to any bottleneck bandwidth provided in an embodiment of the present application;

[0030] Figure 4 A schematic diagram of the principle of a frame rate control method provided in an embodiment of the present application;

[0031] Figure 5 A structural schematic diagram of a frame rate control device provided in an embodiment of the present application;

[0032] Figure 6 A schematic diagram of the structure of a screen projection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field based on the present application belong to the scope of protection of the present application.

[0034] First, some professional terms in the embodiments of the present application are briefly introduced:

[0035] Bit rate detection: also known as bandwidth detection, which detects the bottleneck bandwidth in the network.

[0036] Screen capture: The process of capturing display data from the source (device).

[0037] Screen projection system: A system consisting of a screen projection device, a wireless network and a screen projection receiver, used to display the captured display data on the screen projection receiver.

[0038] GCC: Google Congest Control, a commonly used congestion control algorithm.

[0039] Secondly, in order to better understand the present application, before introducing the method provided in the embodiment of the present application, the frame rate control method in the related art is first introduced:

[0040] In the related art, the frame rate is usually adjusted dynamically by using the maximum network bandwidth (bottleneck bandwidth) and the actual encoding situation. However, since the network changes in real time, the maximum network bandwidth may also change in real time. The frame rate control method in the related art does not take the changes in the network into consideration. Therefore, the utilization rate of the network bandwidth may be low and cannot meet the screen projection effect.

[0041] Based on the problems existing in the related technology, the present application provides a frame rate control method, device, screen projection equipment and program product to achieve both the screen projection effect and the utilization of network bandwidth.

[0042] Next, a frame rate control method provided in an embodiment of the present application is introduced below.

[0043] Among them, a frame rate control method provided in an embodiment of the present application can be applied to a screen projection device, which can also be called a screen capture encoding end. The screen projection device is specifically used to capture display data from a source end, encode the display data, and then transmit the encoded code stream to a screen projection receiving end. Exemplarily, in one implementation, the screen projection device can be a hardware device, such as a screen projector; in another implementation, the screen projection device can also be screen projection software, such as a screen projection client. The embodiment of the present application does not specifically limit the specific form of the screen projection device.

[0044] Furthermore, the screen projection device can communicate with the screen projection receiving end through a wireless network. The screen projection receiving end can also be considered as a receiving end that receives the display data captured by the screen projection device. The subsequent screen projection receiving end can also display the display data; the specific screen projection receiving end can be a personal computer PC device, or a mobile device, such as a mobile phone, tablet computer, etc., or a screen projection receiving software set in the device. The embodiment of the present application does not specifically limit the specific form of the screen projection receiving end. In addition, the source end can also communicate with the screen projection device through a wireless network. The source end can be considered as a video source end that displays display data. Specifically, the source end can also be a personal computer PC device, or a mobile device. The embodiment of the present application does not specifically limit this.

[0045] It should be noted that the term "device" or "end" in the embodiments of the present application may refer to a hardware device, a software program, or a software program and the necessary carrier for the operation of the software program. With the evolution of technology, the steps required to be performed or the functions undertaken by these "devices" or "ends" (for example, the steps described after a certain device is "used" for) may be performed using hardware, software, or a combination of hardware and software. As long as the hardware, software, or a combination of hardware and software can perform the steps required to be performed by the "device" or "end", they can be considered as the "device" or "end" in the embodiments of the present application.

[0046] The following is a brief introduction to the application scenarios of the frame rate control method provided in the embodiment of the present application:

[0047] The embodiments of the present application can be used in various screen projection scenarios, such as video conferencing scenarios, online teaching scenarios, etc., and the embodiments of the present application do not make specific limitations on this.

[0048] Among them, a frame rate control method is applied to a screen projection device, wherein the screen projection device communicates with a screen projection receiving end via a wireless network; the method comprises:

[0049] During the screen projection process, the bottleneck bandwidth of the wireless network is detected as the current bottleneck bandwidth;

[0050] Predicting the data amount of the target frame to obtain the predicted data amount; wherein the target frame is a first-class frame to be encoded in the encoding process of the captured display data of the source end, and the first-class frame is a frame that needs to refer to other frames during encoding;

[0051] Based on the predicted data volume, predict the number of image frames that can be transmitted per unit time if bitstream transmission is performed under the current bottleneck bandwidth, and obtain a predicted frame rate;

[0052] Based on the change value of the predicted frame rate relative to the reference frame rate, the reference frame rate is adjusted to determine a screen capture frame rate that matches the current bottleneck bandwidth; wherein the reference frame rate is a frame rate set as an adjustment basis according to the network status of the wireless network;

[0053] The determined screen capture frame rate is set as the screen capture frame rate for capturing the display data of the source end, so that the display data captured according to the determined screen capture frame rate is encoded and transmitted to the screen projection receiving end, and the bit stream rate used for the encoding process has a positive correlation with the determined screen capture frame rate.

[0054] The frame rate control method provided in the embodiment of the present application can detect the bottleneck bandwidth of the wireless network during the screen projection process, use it as the current bottleneck bandwidth, predict the data volume of the target frame, and obtain the predicted data volume. Based on the predicted data volume, it is predicted that if the code stream is transmitted under the current bottleneck bandwidth, the number of image frames that can be transmitted per unit time is obtained to obtain the predicted frame rate. Subsequently, based on the change value of the predicted frame rate relative to the reference frame rate, the reference frame rate can be adjusted to determine the screen capture frame rate that is compatible with the current bottleneck bandwidth, and the determined screen capture frame rate is set as the screen capture frame rate for capturing the display data of the source end, wherein the reference frame rate is a frame rate set as the adjustment basis according to the network status of the wireless network; it can be seen that the predicted frame rate is referred to (the predicted frame rate can reflect the code rate required in the current network environment to a certain extent). , the required bit rate can be considered as the actual encoded data volume) and the network status of the wireless network, so that the reference frame rate can be dynamically adjusted, and a quick response can be made to the changing network status; and, since the network status of the wireless network fluctuates, the bottleneck bandwidth may also be changing, and the determined screen capture frame rate that is adapted to the current bottleneck bandwidth can adapt to the changing bottleneck bandwidth. In this way, when encoding the display data captured according to the determined screen capture frame rate, the bit stream rate used for the encoding process has a positive correlation with the determined screen capture frame rate, and the bit stream rate will not be too much greater than the bottleneck bandwidth, thereby affecting the screen projection effect, nor will the bit stream rate be too much less than the bottleneck bandwidth, resulting in a low utilization rate of the network bandwidth, thereby taking into account both the screen projection effect and the utilization rate of the network bandwidth.

[0055] A frame rate control method provided in an embodiment of the present application is described below in conjunction with the accompanying drawings.

[0056] like Figure 1 As shown, a frame rate control method provided in an embodiment of the present application is applied to a screen projection device, wherein the screen projection device communicates with a screen projection receiving end via a wireless network; the method comprises:

[0057] S101, during screen projection, detecting the bottleneck bandwidth of the wireless network as the current bottleneck bandwidth;

[0058] It can be understood that the embodiments of the present application are only for the scenario of screen projection. Then, during the screen projection process, the bottleneck bandwidth of the wireless network can be detected as the current bottleneck bandwidth, wherein the bottleneck bandwidth of the wireless network can also be considered as the maximum transmission rate allowed by the network bandwidth of the wireless network. It should be emphasized that in the process of detecting the bottleneck bandwidth of the wireless network, it can be implemented with the help of congestion control algorithms such as GCC. Of course, the present application does not make specific limitations on this, and any congestion control algorithm can be used as a way to detect the bottleneck bandwidth of the wireless network in the present application. In addition, the bottleneck bandwidth can also be called the bottleneck link bandwidth, and the unit of the bottleneck bandwidth is bps, that is, bits per second, which can be kbps, kilobits per second, Mbps, megabits per second; and the unit of the bottleneck bandwidth is consistent with the unit of the bit rate.

[0059] And, when the screen projection device is a hardware device (such as a screen projector), the bottleneck bandwidth of the wireless network can be detected based on the bandwidth detection software in the screen projection device as the current bottleneck bandwidth. The bandwidth detection software can be software implemented using congestion control algorithms such as GCC. When the screen projection device is a screen projection software, the screen projection software can be software with a bandwidth detection function, so that the bandwidth detection function can be turned on during the screen projection process to directly detect the bottleneck bandwidth of the wireless network as the current bottleneck bandwidth; or, when the screen projection device is a screen projection software, the screen projection device can call the bandwidth detection module in the source end to perform the bottleneck bandwidth of the wireless network as the current bottleneck bandwidth. Of course, the screen projection device can also send a detection instruction to the screen projection receiving end to instruct the screen projection receiving end to detect the bottleneck bandwidth of the wireless network, which is also reasonable. It can be understood that the specific implementation method of detecting the bottleneck bandwidth of the wireless network may be different for different forms of the screen projection device, and the embodiments of the present application do not specifically limit this.

[0060] Exemplarily, in one implementation, the detecting the bottleneck bandwidth of the wireless network as the current bottleneck bandwidth includes step A1:

[0061] Step A1, in response to satisfying the frame rate control condition, detecting the bottleneck bandwidth of the wireless network as the current bottleneck bandwidth;

[0062] The frame rate control conditions include: reaching a target period, and / or detecting that a network state of the wireless network meets a preset unstable condition.

[0063] It is understandable that, exemplarily, when entering each target cycle, it can be considered that the frame rate control condition is met. At this time, the frame rate control will be performed periodically, that is, the bottleneck bandwidth of the wireless network is periodically detected, and the bottleneck bandwidth currently detected is used as the current bottleneck bandwidth, so that the subsequent processing flow is performed based on the current bottleneck bandwidth to achieve frame rate control; exemplarily, when it is detected that the network state of the wireless network meets the preset unstable condition, it can also be considered that the frame rate control condition is met; exemplarily, when entering a target cycle at the same time, and when it is detected that the network state of the wireless network meets the preset unstable condition, it can also be considered that the frame rate control condition is met. The embodiment of the present application does not specifically limit the specific implementation method for meeting the frame rate control condition.

[0064] Exemplarily, the duration of the target cycle can usually be set to 1s. Of course, it can also be set to other durations, such as 1.5s, 2s, etc. The embodiments of the present application do not make specific limitations on this. In specific applications, different cycle durations can be selected according to the actual network status. In addition, the network status of the wireless network meets the preset unstable conditions, which can be considered as a strong fluctuation in the network status of the wireless network, triggering a network status fluctuation event (the network status fluctuation event can be set according to the specific application scenario, for example, the change in the network bandwidth obtained relative to the previous detection is detected, exceeding a predetermined threshold, and it is determined to trigger a network status fluctuation event), thereby satisfying the frame rate control conditions. Of course, the above is only an exemplary introduction, and the embodiments of the present application do not make specific limitations on this.

[0065] It can be seen that the embodiment of the present application can detect the bottleneck bandwidth of the wireless network as the current bottleneck bandwidth in response to meeting the frame rate control conditions, thereby facilitating frame rate control in subsequent steps, and providing a basis for taking into account both the screen projection effect and the utilization of the network bandwidth; and, when the frame rate control conditions include entering each target cycle, the frequency of frame rate control can be increased, thereby improving the accuracy of the determined screen capture frame rate that is compatible with the current bottleneck bandwidth; when the frame rate control conditions include detecting that the network state of the wireless network meets the preset unstable conditions, the frequency of frame rate control can be reduced, thereby reducing the consumption of computing power.

[0066] S102, predicting the data volume of the target frame to obtain a predicted data volume;

[0067] The target frame is a first-category frame to be encoded during the encoding process of the captured display data of the source end, and the first-category frame is a frame that needs to refer to other frames during encoding;

[0068] It can be understood that the target frame is a first-class frame to be encoded in the display data captured from the source end. The first-class frame is a frame that needs to refer to other frames when encoding, for example: a frame encoded as a P frame. Of course, there are also frames that do not need to refer to other frames when encoding, for example: a frame encoded as an I frame. The details will be introduced in the subsequent embodiments and will not be described in detail here. In addition, in the process of predicting the amount of data, the amount of data encoded by other first-class frames can also be referenced, so as to achieve reference to the actual amount of data encoded during the frame rate control process. In addition, since there is a great correlation between the front and back of the projected screen, the data amount of the first-class frame (the frame that needs to refer to other frames when encoding) can be predicted here. When encoding any first-class frame, the encoded data of the previous first-class frame of the first-class frame can be referenced. Therefore, predicting the data amount of the first-class frame can reflect the correlation between the front and back of the projected screen. In other words, the amount of data that needs to be referenced to other frames during encoding is relatively fixed (for example, the full-frame image information is compressed and encoded using JPEG, which can be preset or based on empirical values, etc.), while the first type of frame is related to the reference frame and the motion scene of the picture, etc., and its relative time is not fixed. Therefore, as long as the timing of frame rate control is reached, the amount of data of the first type of frame can be predicted to obtain the predicted amount of data. For example, if the first type of frame is a P frame and the frame that needs to be referenced to other frames during encoding is an I frame, since the amount of data of an I frame is relatively fixed (for example, the full-frame image information is compressed and encoded using JPEG, which can be preset or based on empirical values, etc.), the P frame is related to the reference frame and the motion scene of the picture, etc., and its relative time is not fixed. Therefore, as long as the timing of frame rate control is reached, the amount of data of the P frame can be predicted.

[0069] Furthermore, when the screen projection device is a hardware device (screen projector), the data amount of the target frame can be predicted based on the data amount prediction module in the screen projection device to obtain the predicted data amount; when the screen projection device is screen projection software, the screen projection software can be software with a data amount prediction function, which can directly predict the data amount of the target frame to obtain the predicted data amount. The embodiments of the present application do not make specific limitations on this.

[0070] For the sake of clarity of the layout, the data volume of the target frame is predicted, and the specific implementation of obtaining the predicted data volume will be introduced in other embodiments and will not be elaborated on here.

[0071] S103, based on the predicted data volume, predicting the number of image frames that can be transmitted per unit time if bitstream transmission is performed under the current bottleneck bandwidth, and obtaining a predicted frame rate;

[0072] It can be understood that based on the predicted data volume predicted in step S102, the number of image frames that can be transmitted per unit time with the current bottleneck bandwidth as the transmission rate of the code stream can be predicted as the predicted frame rate; wherein the number of image frames that can be transmitted per unit time can include the number of frames of the first category, and can also include the number of frames that do not require reference to other frames during encoding; and the unit time can be 1s, which is not specifically limited in this application. For the sake of clarity of the layout, based on the predicted data volume, the number of image frames that can be transmitted per unit time if the code stream is transmitted under the current bottleneck bandwidth is predicted, and the specific implementation of the predicted frame rate is obtained, which will be introduced in other embodiments and will not be elaborated here.

[0073] S104, adjusting the reference frame rate based on the change value of the predicted frame rate relative to the reference frame rate to determine a screen capture frame rate that is compatible with the current bottleneck bandwidth;

[0074] Wherein, the reference frame rate is a frame rate set as an adjustment basis according to the network status of the wireless network;

[0075] It is understandable that the reference frame rate set under different network states of the wireless network is also different. For example, the reference frame rate set when the network state of the wireless network is stable is different from the reference frame rate set when the network state of the wireless network fluctuates violently. Therefore, the reference frame rate can be directly adjusted based on the change value of the predicted frame rate relative to the reference frame rate, and the determined screen capture frame rate that is adapted to the current bottleneck bandwidth can be consistent with the predicted frame rate. Alternatively, the change value of the predicted frame rate relative to the reference frame rate can be adjusted first, and then the reference frame rate is adjusted based on the adjusted change value to determine the screen capture frame rate that is adapted to the current bottleneck bandwidth. By adjusting the change value of the predicted frame rate relative to the reference frame rate, the stability of the frame rate change can be ensured, and the frame rate will not change violently. The embodiments of the present application do not make specific limitations on this.

[0076] Exemplarily, the change value of the predicted frame rate relative to the reference frame rate is 6fps, and the reference frame rate is 20fps. Then, it can be determined that the screen capture frame rate that is compatible with the current bottleneck bandwidth is 26fps.

[0077] In addition, there is a positive correlation between the frame rate and the bit rate. The screen capture frame rate that matches the current bottleneck bandwidth also corresponds to the bit stream rate. The bit stream rate corresponding to the screen capture frame rate that matches the current bottleneck bandwidth matches the current bottleneck bandwidth; and the bit stream rate corresponding to the screen capture frame rate that matches the current bottleneck bandwidth can also be considered as: the bit stream rate used when encoding the display data obtained by capturing the screen at the screen capture frame rate that matches the current bottleneck bandwidth. For the sake of clarity, the setting method for the reference frame rate will be introduced in other embodiments, and will not be elaborated on here.

[0078] S105, setting the determined screen capture frame rate as the screen capture frame rate for capturing the display data of the source end, so that the display data captured according to the determined screen capture frame rate is encoded and transmitted to the screen projection receiving end, and the bit stream rate used for the encoding process has a positive correlation with the determined screen capture frame rate.

[0079] It is understandable that the display data of the source end can be captured according to the determined screen capture frame rate that is compatible with the current bottleneck bandwidth. When the captured display data is encoded, since the bitstream rate of the encoding process is positively correlated with the screen capture frame rate (for example, the larger the bitstream rate of the encoding process, the larger the screen capture frame rate), the captured display data can be encoded according to the bitstream rate corresponding to the screen capture frame rate that is compatible with the current bottleneck bandwidth, and the encoded bitstream is transmitted to the screen projection receiving end. It should be emphasized that the determined screen capture frame rate is the screen capture frame rate that is compatible with the current bottleneck bandwidth, so the bitstream rate corresponding to the screen capture frame rate that is compatible with the current bottleneck bandwidth also matches the current bottleneck bandwidth. The frame rate control method provided in the embodiment of the present application can detect the bottleneck bandwidth of the wireless network during the screen projection process, use it as the current bottleneck bandwidth, predict the data volume of the target frame, and obtain the predicted data volume. Based on the predicted data volume, it is predicted that if the code stream is transmitted under the current bottleneck bandwidth, the number of image frames that can be transmitted per unit time is obtained to obtain the predicted frame rate. Subsequently, based on the change value of the predicted frame rate relative to the reference frame rate, the reference frame rate can be adjusted to determine the screen capture frame rate that is compatible with the current bottleneck bandwidth, and the determined screen capture frame rate is set as the screen capture frame rate for capturing the display data of the source end, wherein the reference frame rate is a frame rate set as the adjustment basis according to the network status of the wireless network; it can be seen that the predicted frame rate is referred to (the predicted frame rate can reflect the code rate required in the current network environment to a certain extent). , the required bit rate can be considered as the actual encoded data volume) and the network status of the wireless network, so that the reference frame rate can be dynamically adjusted, and a quick response can be made to the changing network status; and, since the network status of the wireless network fluctuates, the bottleneck bandwidth may also be changing, and the determined screen capture frame rate that is adapted to the current bottleneck bandwidth can adapt to the changing bottleneck bandwidth. In this way, when encoding the display data captured according to the determined screen capture frame rate, the bit stream rate used for the encoding process has a positive correlation with the determined screen capture frame rate, and the bit stream rate will not be too much greater than the bottleneck bandwidth, thereby affecting the screen projection effect, nor will the bit stream rate be too much less than the bottleneck bandwidth, resulting in a low utilization rate of the network bandwidth, thereby taking into account both the screen projection effect and the utilization rate of the network bandwidth.

[0080] Optionally, in another embodiment, the reference frame rate is set in a manner including manner B1 or manner B2:

[0081] Mode B1, when the network state satisfies the network stability condition, the specified frame rate is used as the reference frame rate;

[0082] Mode B2, when the network state does not meet the network stability condition, the empirical frame rate corresponding to the current bottleneck bandwidth is used as the reference frame rate;

[0083] The specified frame rate is a screen capture frame rate determined based on the bottleneck bandwidth detected last time and adapted to the bottleneck bandwidth. The empirical frame rate corresponding to any bottleneck bandwidth is: a frame rate that enables a code stream transmitted using the bottleneck bandwidth as the code rate to clearly display the image.

[0084] It can be understood that method B1 is for the case where the network status meets the network stability condition. It can be determined that the network status is relatively stable without drastic fluctuations. At this time, the specified frame rate can be determined as the reference frame rate. The specified frame rate is the screen capture frame rate determined based on the last detected bottleneck bandwidth and adapted to the bottleneck bandwidth. Then, when the network status meets the network stability condition, that is, the difference between the current network status and the network status at the last detection is small, the screen capture frame rate determined by the last detected bottleneck bandwidth and adapted to the bottleneck bandwidth can be directly used as the frame rate for adjustment.

[0085] It can be understood that method B2 is for the case where the network status does not meet the network stability conditions. It can be determined that the network status is relatively unstable and may fluctuate violently. At this time, the empirical frame rate corresponding to the current bottleneck bandwidth can be determined as the reference frame rate. The empirical frame rate corresponding to the current bottleneck bandwidth is the frame rate at which the code stream transmitted by the current bottleneck bandwidth as the code rate can clearly display the image. Then, when the network status does not meet the network stability conditions, that is, the difference between the current network status and the network status at the last detection is large, the frame rate based on the adjustment of the screen capture frame rate determined by the last detected bottleneck bandwidth and adapted to the bottleneck bandwidth will have errors, so the frame rate based on the adjustment of the code stream transmitted by the current bottleneck bandwidth as the code rate can be used.

[0086] It should be emphasized that in one implementation, when the reference frame rate is set for the first time, it can be considered that the network status does not meet the network stability condition, and the empirical frame rate corresponding to the current bottleneck bandwidth can be directly used as the reference frame rate; in another implementation, when the reference frame rate is set for the first time, it can also be considered that the network status meets the network stability condition, and the last determined screen capture frame rate recorded in the projection device can be used as the reference frame rate. The embodiments of the present application do not make specific limitations on this.

[0087] For the sake of clarity, the method for obtaining the empirical frame rate corresponding to the bottleneck bandwidth will be introduced in other embodiments and will not be elaborated on here.

[0088] It can be seen that in the embodiment of the present application, when the network status meets the network stability condition, the specified frame rate is used as the reference frame rate. When the network status does not meet the network stability condition, the empirical frame rate corresponding to the current bottleneck bandwidth is used as the reference frame rate. This can provide accurate reference frame rates for the above two situations and provide a basis for subsequent frame rate control.

[0089] Optionally, in another embodiment, the method for determining the network status of the wireless network includes method C1:

[0090] Mode C1, determining the network state of the wireless network according to a predetermined determination method based on a difference relationship between an experience frame rate corresponding to the current bottleneck bandwidth and an experience frame rate corresponding to the bottleneck bandwidth detected last time;

[0091] The predetermined determination method includes:

[0092] If the empirical frame rate corresponding to the current bottleneck bandwidth is the same as the empirical frame rate corresponding to the bottleneck bandwidth detected last time, it is determined that the network state of the wireless network meets the network stability condition;

[0093] If the empirical frame rate corresponding to the current bottleneck bandwidth is different from the empirical frame rate corresponding to the bottleneck bandwidth detected last time, it is determined that the network state of the wireless network does not meet the network stability condition.

[0094] It should be emphasized that each bottleneck bandwidth is located in a bandwidth interval, and each bandwidth interval corresponds to an empirical frame rate. Therefore, the current bottleneck bandwidth may not be exactly the same as the bottleneck bandwidth detected last time, but if the current bottleneck bandwidth and the bottleneck bandwidth detected last time are located in the same bandwidth interval, the empirical frame rate corresponding to the current bottleneck bandwidth is also the same as the empirical frame rate corresponding to the bottleneck bandwidth detected last time. The specific content will be introduced in other embodiments and will not be elaborated here.

[0095] It can be understood that when determining the network status of the wireless network, the network status of the wireless network can be determined according to a predetermined determination method. When the empirical frame rate corresponding to the current bottleneck bandwidth is the same as the empirical frame rate corresponding to the bottleneck bandwidth detected last time, it indicates that the current bottleneck bandwidth and the bottleneck bandwidth detected last time are located in the same bandwidth interval, so it can be determined that the network status of the wireless network meets the network stability condition; conversely, when the empirical frame rate corresponding to the current bottleneck bandwidth is different from the empirical frame rate corresponding to the bottleneck bandwidth detected last time, it indicates that the current bottleneck bandwidth and the bottleneck bandwidth detected last time are not located in the same bandwidth interval, so it can be determined that the network status of the wireless network does not meet the network stability condition.

[0096] It can be seen that based on the difference between the empirical frame rate corresponding to the current bottleneck bandwidth and the empirical frame rate corresponding to the bottleneck bandwidth detected last time, the network status of the wireless network can be determined. During the frame rate control process, the network status of the wireless network is referenced to adapt to the changing network status.

[0097] In another implementation, the adjusting process of the reference frame rate based on the change value of the predicted frame rate relative to the reference frame rate to determine the screen capture frame rate adapted to the current bottleneck bandwidth includes step D1 or step D2:

[0098] Step D1, if the reference frame rate is a specified frame rate, adjusting the specified frame rate based on a change value of the predicted frame rate relative to the specified frame rate according to a first formula, so as to determine a screen capture frame rate that is compatible with the current bottleneck bandwidth;

[0099] Step D2, if the reference frame rate is the empirical frame rate corresponding to the current bottleneck bandwidth, then according to the second formula, based on the change value of the predicted frame rate relative to the empirical frame rate corresponding to the current bottleneck bandwidth, the empirical frame rate corresponding to the current bottleneck bandwidth is adjusted to determine a screen capture frame rate that is compatible with the current bottleneck bandwidth;

[0100] Wherein, the first formula includes:

[0101] ;

[0102] The second formula includes:

[0103] ;

[0104] is the screen capture frame rate that matches the current bottleneck bandwidth, is a parameter used to make the frame rate change smooth. To predict the frame rate, is the empirical frame rate corresponding to the bottleneck bandwidth detected last time, is the empirical frame rate corresponding to the current bottleneck bandwidth, To specify the frame rate.

[0105] It can be understood that step D1 is a case where the reference frame rate is the specified frame rate. At this time, the specified frame rate can be adjusted according to the first formula based on the change value of the predicted frame rate relative to the specified frame rate to determine the screen capture frame rate that is compatible with the current bottleneck bandwidth; wherein, It can be indicated that the empirical frame rate corresponding to the current bottleneck bandwidth is the same as the empirical frame rate corresponding to the bottleneck bandwidth detected last time, that is, the network state of the wireless network meets the network stability condition. It can be considered as the change in the predicted frame rate relative to the specified frame rate. As a parameter for making the frame rate change smooth, it can be updated based on the specified frame rate to obtain a screen capture frame rate that is compatible with the current bottleneck bandwidth; The range of can be (0, 1), and the empirical value is 0.2, which is not specifically limited in the present embodiment. For example, the specified frame rate is 50fps and the predicted frame rate is 45fps. is 0.2, we can calculate that the screen capture frame rate that is compatible with the current bottleneck bandwidth is 49fps.

[0106] It can be understood that step D2 is a case where the reference frame rate is the empirical frame rate corresponding to the current bottleneck bandwidth. At this time, the empirical frame rate corresponding to the current bottleneck bandwidth can be adjusted according to the second formula based on the change value of the predicted frame rate relative to the empirical frame rate corresponding to the current bottleneck bandwidth to determine the screen capture frame rate that is compatible with the current bottleneck bandwidth; wherein, in the second formula, It can be indicated that the empirical frame rate corresponding to the current bottleneck bandwidth is different from the empirical frame rate corresponding to the bottleneck bandwidth detected last time, that is, the network state of the wireless network does not meet the network stability condition. Then, it can be updated based on the empirical frame rate corresponding to the current bottleneck bandwidth to obtain a screen capture frame rate that is compatible with the current bottleneck bandwidth; It can be considered as the change in the predicted frame rate relative to the empirical frame rate corresponding to the current bottleneck bandwidth. For example, the empirical frame rate corresponding to the current bottleneck bandwidth is 30fps, and the predicted frame rate is 45fps. is 0.2, we can calculate that the screen capture frame rate that is compatible with the current bottleneck bandwidth is 33fps.

[0107] Exemplarily, in one implementation, the first formula and the second formula may also be combined into a third formula.

[0108] Among them, the third formula is:

[0109] ;

[0110] It can be understood that the various elements in the third formula are consistent with those in the first formula and the second formula, and will not be elaborated herein.

[0111] It can be seen that according to different reference frame rates, different formulas can be used to determine the screen capture frame rate that is compatible with the current bottleneck bandwidth. When the reference frame rate is a specified frame rate, the specified frame rate is adjusted according to the first formula to determine the screen capture frame rate that is compatible with the current bottleneck bandwidth; when the reference frame rate is the empirical frame rate corresponding to the current bottleneck bandwidth, the empirical frame rate corresponding to the current bottleneck bandwidth is adjusted according to the second formula to determine the screen capture frame rate that is compatible with the current bottleneck bandwidth, thereby improving the accuracy of the determined screen capture frame rate.

[0112] Optionally, in another embodiment, predicting the data amount of the target frame to obtain the predicted data amount includes steps E1-E2:

[0113] Step E1, calculating the sum of the data amounts of the first-category frames encoded within a predetermined time window before the target frame to obtain a target sum value;

[0114] It can be understood that since there is a correlation between each frame of the display data of the projection, the sum of the data amounts of each first-class frame encoded within the predetermined time window can be used to predict the data amount of the target frame; wherein, the predetermined time window can be 3s, or 5s, etc., and the embodiment of the present application does not specifically limit this. And, the process of calculating the sum of the data amounts of each first-class frame encoded within the predetermined time window to obtain the target sum value can also be called a sliding average method. Exemplarily, the predetermined time window is 3s, and there are 5 P frames in the predetermined time window, with data amounts of 10kb, 20kb, 40kb, 5kb and 15kb respectively, and the target sum value can be calculated to be 90kb.

[0115] In order to better understand the content of the predetermined time window, the first type of frame is encoded as a P frame as an example, and is introduced in conjunction with the accompanying drawings:

[0116] like Figure 2 As shown, the predetermined time window is located before the target frame, and the sum of the data amounts of each P frame within the predetermined time window can be calculated as the target sum value.

[0117] Step E2, calculating the ratio of the target sum value to the number of frames of each first category that have been encoded within a predetermined time window, and obtaining the predicted data amount.

[0118] It can be understood that the predicted data volume can be obtained by determining the number of frames of each first type that have been encoded within the predetermined time window and then calculating the ratio of the target sum value to the target sum value. For example, the target sum value is 90kb, and there are 5 P frames within the predetermined time window, and the predicted data volume can be calculated to be 18kb.

[0119] Exemplarily, in one implementation, the process of calculating the predicted data volume can refer to the fourth formula;

[0120] Among them, the fourth formula is:

[0121] ;

[0122] in, To predict the amount of data, is the average value of the data volume of the P frame in the predetermined time window w, w represents the time represented by the predetermined time window, and can also be called a setting parameter. Of course, the above is only an exemplary introduction, and other methods of predicting the data volume are also applicable to the embodiment of the present application, and the embodiment of the present application does not make specific limitations on this.

[0123] It can be seen that by calculating the sum of the data amounts of each first-category frame that has been encoded within the predetermined time window before the target frame, the target sum value is obtained, and the ratio of the target sum value to the number of frames of each first-category frame that has been encoded within the predetermined time window is calculated. The predicted data amount can be obtained. Compared with other methods, the predicted data amount calculated by this method has lower requirements for computing power.

[0124] Optionally, in another embodiment, the step of predicting the number of image frames that can be transmitted per unit time if bitstream transmission is performed under the current bottleneck bandwidth based on the predicted data volume to obtain a predicted frame rate includes step F1:

[0125] Step F1, predicting the total number of first-category frames and second-category frames transmitted per unit time under the current bottleneck bandwidth when the code stream is transmitted in the target transmission state, and obtaining a predicted frame rate;

[0126] The target transmission state is a code stream transmission state in which the predicted data amount is used as the data amount of the first type of frames, the specified data amount is used as the data amount of the second type of frames, and the predetermined empirical value is used as the number of the second type of frames; wherein the second type of frames are frames that do not require reference to other frames during encoding.

[0127] Among them, the specified data amount is the data amount of the encoded second-category frame closest to the target frame, and the second-category frame is a frame that does not need to refer to other frames when encoding. For example, the second-category frame can be a frame encoded as an I frame; exemplarily, when the frame after the second-category frame is encoded is an I frame, the data amount of the I frame closest to the frame is the specified data amount.

[0128] It can be understood that when predicting the bitstream transmission using the predicted data amount as the data amount of the first type of frames, the specified data amount as the data amount of the second type of frames, and the predetermined empirical value as the number of the second type of frames, the total number of frames of the first type of frames and the second type of frames transmitted per unit time under the current bottleneck bandwidth can be used as the predicted frame rate.

[0129] Exemplarily, in one implementation, the first type of frame is a frame encoded as a P frame, and the second type of frame is a frame encoded as an I frame. The process of calculating the predicted frame rate can refer to the fifth formula;

[0130] Among them, the fifth formula is:

[0131] ;

[0132] in, To predict the frame rate, x is a predetermined empirical value, which can also be considered as the number of I frames transmitted per unit time. The default value is 1. is the bandwidth value of the current bottleneck bandwidth, is the specified data amount (the amount of data of the I frame closest to the target frame), To predict the amount of data, It is the number of P frames transmitted per unit time.

[0133] Furthermore, the fifth formula can be considered as that under the current bandwidth detected, if the number of I frames and the single-frame data volume of I frames are determined (the predetermined empirical value x and the data volume of an already encoded I frame closest to the target frame ), the bandwidth occupied by I frames is determined. After deducting the bandwidth occupied by I frames, the remaining bandwidth is the bandwidth occupied by P frames. By dividing it by the predicted data amount of a single P frame, we can calculate how many frames a P frame can transmit, that is, the predicted frame rate.

[0134] It can be seen that when the code stream is transmitted in the target transmission state, the total number of frames of the first type and the second type transmitted per unit time under the current bottleneck bandwidth is predicted to obtain the predicted frame rate, which provides a basis for subsequent frame rate control.

[0135] Optionally, in another embodiment, the method for obtaining the empirical frame rate corresponding to any bottleneck bandwidth includes: Figure 3 As shown:

[0136] S301, determining a bandwidth interval where the bottleneck bandwidth is located from various preset bandwidth intervals as a target bandwidth interval;

[0137] It is understandable that the bandwidth interval can be set in advance, and for each bottleneck bandwidth, the bandwidth interval where the bottleneck bandwidth is located can be determined as the target bandwidth. Exemplarily, the bandwidth interval can be divided into (0, 512kbps] as the first bandwidth interval, (512kbps, 1Mbps] as the second bandwidth interval, (1Mbps, 2Mbps] as the third bandwidth interval, (2Mbps, 3Mbps] as the fourth bandwidth interval, (3Mbps, 4Mbps] as the fifth bandwidth interval, and greater than 4Mbps as the sixth bandwidth interval. The bottleneck bandwidth is 600kbps, and it can be considered that the bottleneck bandwidth is located in the second bandwidth interval, and the second bandwidth interval can be used as the target bandwidth interval.

[0138] Of course, the above division of bandwidth intervals is only for illustrative purposes, and the embodiments of the present application do not make any specific limitations on this.

[0139] S302, determining an empirical frame rate corresponding to the target bandwidth interval from an empirical frame rate corresponding table as the empirical frame rate corresponding to the bottleneck bandwidth;

[0140] Among them, the empirical frame rate correspondence table is used to record each bandwidth interval and the empirical frame rate corresponding to each bandwidth interval. The empirical frame rate corresponding to each bandwidth interval is: the frame rate that enables the code stream transmitted according to the bandwidth of the bandwidth interval as the transmission rate to clearly display the picture.

[0141] It can be understood that the empirical frame rate correspondence table is used to record each bandwidth interval and the empirical frame rate corresponding to each bandwidth interval, and to capture and display data based on the empirical frame rate corresponding to the bandwidth interval, and to transmit the code stream according to the bandwidth of the bandwidth interval as the transmission rate, so that the picture can be clearly displayed at the projection receiving end; then, based on the empirical frame rate correspondence table, the empirical frame rate corresponding to the target bandwidth interval can be determined as the empirical frame rate corresponding to the bottleneck bandwidth.

[0142] Exemplarily, in one implementation, the empirical frame rate correspondence table is shown in Table 1:

[0143]

[0144] Table 1

[0145] Among them, the experience frame rate corresponding to the bandwidth interval (0, 512kbps] is 3fps, the experience frame rate corresponding to the bandwidth interval (512kbps, 1Mbps] is 5fps, the experience frame rate corresponding to the bandwidth interval (1Mbps, 2Mbps] is 13fps, the experience frame rate corresponding to the bandwidth interval (2Mbps, 3Mbps] is 18fps, the experience frame rate corresponding to the bandwidth interval (3Mbps, 4Mbps] is 25fps, and the experience frame rate corresponding to the bandwidth interval greater than 4Mbps is 30fps. Of course, the above-mentioned experience frame rate correspondence table is only used as an exemplary introduction and is not specifically limited to the embodiments of the present application.

[0146] Exemplarily, in another implementation, the empirical frame rate correspondence table may be expressed as:

[0147] ;

[0148] in, , is the empirical frame rate corresponding to the current bottleneck bandwidth, is the current bottleneck bandwidth, (0, ] is the bandwidth interval with the smallest amount of representation data in each bandwidth interval, ( , ] is the bandwidth interval with the largest amount of representation data among all bandwidth intervals.

[0149] For example, the current bottleneck bandwidth is 200 kbps, and the corresponding empirical frame rate can be determined to be 3 fps using the empirical frame rate correspondence table.

[0150] It can be seen that from the preset bandwidth intervals, the bandwidth interval where the bottleneck bandwidth is located is determined as the target bandwidth interval, and the empirical frame rate corresponding to the target bandwidth interval is determined from the empirical frame rate correspondence table as the empirical frame rate corresponding to the bottleneck bandwidth. In this way, the empirical frame rate corresponding to any bottleneck bandwidth can be quickly determined, and a quick response can be made to the changing network status.

[0151] In another implementation, the method for determining the empirical frame rate corresponding to each bandwidth interval in the empirical frame rate correspondence table includes steps G1-G2:

[0152] Step G1, for each test scenario, according to a predetermined test method, determining the screen capture frame rate of the test scenario corresponding to the bandwidth interval;

[0153] Step G2, calculating the average value of the screen capture frame rate in each test scenario corresponding to the bandwidth interval to obtain the empirical frame rate corresponding to the bandwidth interval;

[0154] Wherein, the predetermined test method includes:

[0155] The test source end is captured using the current test frame rate as the screen capture frame rate of the display data matching the test scenario at the test source end, the display data obtained by the screen capture is encoded according to the fixed bit rate corresponding to the bandwidth interval and transmitted to the test receiving end, the picture clarity of the display data of the test receiving end is determined, and when the picture clarity meets the picture clarity condition, the screen capture frame rate of the test scenario corresponding to the bandwidth interval is obtained; otherwise, the current test frame rate is adjusted to obtain a new test frame rate, and the step of capturing the screen of the test source end using the current test frame rate as the screen capture frame rate of the display data matching the test scenario at the test source end is returned; wherein the value of the fixed bit rate corresponding to the bandwidth interval is the amount of data represented by the maximum bandwidth in the bandwidth interval.

[0156] It can be understood that, for step G1, the test scenario may include a still picture scenario, a table display scenario, a presentation document display scenario, and other different styles of scenarios, and the present application embodiment does not make specific limitations on this; and, both the test source end and the test receiving end can be considered as a server end for performing screen projection tests on each test scenario. And, each test scenario can be divided according to a predetermined division dimension, and the predetermined division dimension can be the type of business data as a division dimension, or the degree of picture motion as a division dimension, and the present application embodiment does not make specific limitations on this; illustratively, when the type of business data is used as the division dimension, when the type of business data is PPT, it can be considered that the scenario for displaying the PPT is a test scenario, and the bandwidth interval (3Mbps, 4Mbps] can be determined according to a predetermined test method. The screen capture frame rate in the scenario where the PPT is displayed; when the type of business data is document data, it can be considered that the scenario for displaying the document data is another test scenario, The screen capture frame rate of a bandwidth interval (3Mbps, 4Mbps) in the scenario where the document data is displayed can be determined according to a predetermined test method; when the degree of picture motion is used as a division dimension, a scenario where a still picture with a small degree of picture motion is displayed can be used as a test scenario, and the screen capture frame rate of a bandwidth interval (3Mbps, 4Mbps) in the scenario where the still picture is displayed can be determined according to a predetermined test method; a scenario where a video with a large degree of picture motion is displayed can be used as another test scenario, and the screen capture frame rate of a bandwidth interval (3Mbps, 4Mbps) in the scenario where the video is displayed can be determined according to a predetermined test method.

[0157] Furthermore, for each test scenario, the screen capture frame rate of the test scenario corresponding to the bandwidth interval can be determined according to a predetermined test method. Specifically, the current test frame rate is used as the screen capture frame rate of the display data matching the test scenario at the test source end, and the test source end is captured, wherein the test frame rate can also be considered as the initial frame rate. Since the test frame rate can be adjusted later, there are no special requirements for setting the test frame rate; and, the display data obtained by the screen capture is encoded according to the fixed bit rate corresponding to the bandwidth interval and transmitted to the test receiving end, wherein the value of the fixed bit rate corresponding to the bandwidth interval can be the amount of data represented by the maximum bandwidth in the bandwidth interval, and of course, it can also be the amount of data represented by the median bandwidth of the bandwidth interval. The amount of data is not specifically limited in the embodiments of the present application; and after receiving the encoded bitstream, the test receiving end can display the display data represented by the bitstream. At this time, the picture clarity of the display data of the test receiving end can be determined. When the picture clarity meets the picture clarity condition, the screen capture frame rate of the test scene corresponding to the bandwidth interval is determined. Whether the picture clarity condition is met can be judged based on the picture clarity algorithm, or it can be judged subjectively by the user. Then, when the picture clarity is determined by the picture clarity algorithm, it can be considered that the picture clarity condition is met, or when the picture clarity is determined by the user subjectively, it can also be considered that the picture clarity condition is met. The embodiments of the present application do not specifically limit the method of meeting the picture clarity condition. For example, user a sees that the puppy image displayed by the test receiving end is clear enough, then it can be considered that the picture clarity condition is met, so that the screen capture frame rate of the puppy image test scene corresponding to the bandwidth interval can be obtained. Exemplarily, when the test scenario is a scenario of presenting PPT, the bandwidth interval is (3Mbps, 4Mbps], and the current test frame rate of 15fps is used as the screen capture frame rate of the PPT display data of the test source end that matches the scenario of presenting PPT. The test source end is captured, and the PPT display data obtained by the screen capture is encoded at 4Mbps and transmitted to the test receiving end. At this time, the current test receiving end does not meet the picture clarity condition, and the current test frame rate is adjusted. The test source end is returned to the step of capturing the screen according to the current test frame rate as the screen capture frame rate of the PPT display data of the test source end that matches the scenario of presenting PPT, until the picture clarity condition is met, and the current test frame rate is used as the screen capture frame rate in the bandwidth interval (3Mbps, 4Mbps] in the scenario of presenting PPT.It should be emphasized that when the picture clarity does not meet the picture clarity condition, the current test frame rate can be adjusted to obtain a new test frame rate, and the current test frame rate is returned as the screen capture frame rate of the display data matching the test scene at the test source end, and the test source end is screen captured, thereby realizing a cycle until the picture clarity meets the picture clarity condition, and the screen capture frame rate under the test scene corresponding to the bandwidth interval is determined. Exemplarily, user B sees that the table image displayed on the test receiving end is not clear enough, adjusts the test frame rate from 20fps to 17fps, and re-executes the screen capture of the table image displayed on the test source end at 17fps, and encodes the table image obtained by the screen capture and transmits it to the test receiving end. User B re-checks whether the displayed table image is clear enough at the test receiving end. If user B sees that the displayed table image is clear enough, it is determined that 17fps can be used as the screen capture frame rate of the table image.

[0158] It can be understood that, for step G2, after obtaining the screen capture frame rate in each test scenario corresponding to the bandwidth interval, the average value of the screen capture frame rate in each test scenario corresponding to the bandwidth interval can be calculated, so as to obtain the empirical frame rate corresponding to the bandwidth interval. Exemplarily, the screen capture frame rate in test scenario 1 corresponding to the bandwidth interval (0, 512kbps] is 5fps, the screen capture frame rate in test scenario 2 is 3fps, and the screen capture frame rate in test scenario 3 is 1fps. It can be calculated that the empirical frame rate corresponding to the bandwidth interval (0, 512kbps] is 3fps. In addition, in each test scenario, the screen capture frame rate in the test scenario with large picture motion scene will also be larger. Similarly, the screen capture frame rate in the test scenario with small picture motion scene will also be smaller. However, by calculating the average value of the screen capture frame rate in each test scenario corresponding to the bandwidth interval, a more accurate empirical frame rate corresponding to the bandwidth interval can be obtained.

[0159] It can be seen that through the determination method introduced above, the empirical frame rate corresponding to each bandwidth interval can be determined. In this process, the screen capture frame rate in each test scenario is referred to, and the empirical frame rate corresponding to the bandwidth interval is determined by calculating its average value, so that the accurate empirical frame rate corresponding to each bandwidth interval can be obtained, which provides a basis for subsequent frame rate control. In addition, the value of the fixed bit rate corresponding to the bandwidth interval is the amount of data represented by the maximum bandwidth in the bandwidth interval. Under this fixed bit rate, a screen capture frame rate can be determined to ensure a clear picture. Then, according to the screen capture frame rate, the screen can be captured at other bandwidths within the bandwidth interval, which will inevitably ensure a clear picture, thereby improving the accuracy of the test.

[0160] Optionally, in another embodiment, the present application embodiment further provides a schematic diagram of the principle of frame rate control, such as Figure 4 As shown:

[0161] During the screen projection process, the bottleneck bandwidth of the wireless network is detected as the current bottleneck bandwidth. This process can also be called bit rate detection. The data volume of the target frame is predicted to obtain the predicted data volume. This process can also be called single-frame data volume prediction. At the same time, based on the current bottleneck bandwidth, the empirical frame rate can be selected. Based on the predicted data volume, the number of image frames that can be transmitted per unit time under the current bottleneck bandwidth is predicted to obtain the predicted frame rate. This process can also be called frame rate prediction. In addition, before the frame rate decision is made, the specified frame rate can be obtained. The specified frame rate is the screen capture frame rate determined based on the bottleneck bandwidth detected last time and adapted to the bottleneck bandwidth. Based on the network status of the wireless network, it can be determined The process of selecting the empirical frame rate or the specified frame rate corresponding to the current bottleneck bandwidth as the reference bandwidth can also be called frame rate decision; adjusting the reference frame rate based on the change value of the predicted frame rate relative to the reference frame rate to determine the screen capture frame rate that matches the current bottleneck bandwidth can also be called determining the target frame rate, wherein the target frame rate is the screen capture frame rate that matches the current bottleneck bandwidth as described in the above embodiment; subsequently, the determined screen capture frame rate can be set as the screen capture frame rate for capturing the display data of the source end, so that the display data captured according to the determined screen capture frame rate is encoded and transmitted to the screen projection receiving end, and this process can also be called frame rate update.

[0162] The frame rate control method provided in the embodiment of the present application can detect the bottleneck bandwidth of the wireless network during the screen projection process, use it as the current bottleneck bandwidth, predict the data volume of the target frame, and obtain the predicted data volume. Based on the predicted data volume, it is predicted that if the code stream is transmitted under the current bottleneck bandwidth, the number of image frames that can be transmitted per unit time is obtained to obtain the predicted frame rate. Subsequently, based on the change value of the predicted frame rate relative to the reference frame rate, the reference frame rate can be adjusted to determine the screen capture frame rate that is compatible with the current bottleneck bandwidth, and the determined screen capture frame rate is set as the screen capture frame rate for capturing the display data of the source end, wherein the reference frame rate is a frame rate set as the adjustment basis according to the network status of the wireless network; it can be seen that the predicted frame rate is referred to (the predicted frame rate can reflect the code rate required in the current network environment to a certain extent). , the required bit rate can be considered as the actual encoded data volume) of the wireless network status, so that the reference frame rate can be dynamically adjusted, and a quick response can be made to the changing network status; and, since the network status of the wireless network fluctuates, the bottleneck bandwidth may also be changing, and the determined screen capture frame rate that is adapted to the current bottleneck bandwidth can adapt to the changing bottleneck bandwidth. In this way, when encoding the display data captured according to the determined screen capture frame rate, the bit stream rate used for the encoding process has a positive correlation with the determined screen capture frame rate, and the bit stream rate will not be too much greater than the bottleneck bandwidth, thereby affecting the screen projection effect, nor will the bit stream rate be too much less than the bottleneck bandwidth, resulting in a low utilization rate of the network bandwidth, thereby taking into account both the screen projection effect and the utilization rate of the network bandwidth.

[0163] Based on the above method embodiment, Figure 5 As shown, an embodiment of the present application provides a frame rate control device, which is applied to a screen projection device, wherein the screen projection device communicates with a screen projection receiving end via a wireless network; the device includes:

[0164] The detection module 510 is used to detect the bottleneck bandwidth of the wireless network as the current bottleneck bandwidth during the screen projection process;

[0165] The first prediction module 520 is used to predict the data amount of the target frame to obtain the predicted data amount; wherein the target frame is a first-type frame to be encoded in the encoding process of the captured display data of the source end, and the first-type frame is a frame that needs to refer to other frames during encoding;

[0166] A second prediction module 530 is used to predict, based on the predicted data volume, the number of image frames that can be transmitted per unit time if bitstream transmission is performed under the current bottleneck bandwidth, and obtain a predicted frame rate;

[0167] An adjustment processing module 540 is used to adjust the reference frame rate based on the change value of the predicted frame rate relative to the reference frame rate to determine a screen capture frame rate that is compatible with the current bottleneck bandwidth; wherein the reference frame rate is a frame rate set as an adjustment basis according to the network status of the wireless network;

[0168] The setting module 550 is used to set the determined screen capture frame rate as the screen capture frame rate for capturing the display data of the source end, so that the display data captured according to the determined screen capture frame rate is encoded and transmitted to the screen projection receiving end, and the bit stream rate used for the encoding process has a positive correlation with the determined screen capture frame rate.

[0169] Optionally, the reference frame rate is set in a manner including:

[0170] When the network state satisfies a network stability condition, using the specified frame rate as a reference frame rate;

[0171] When the network state does not meet the network stability condition, the empirical frame rate corresponding to the current bottleneck bandwidth is used as the reference frame rate;

[0172] The specified frame rate is a screen capture frame rate determined based on the bottleneck bandwidth detected last time and adapted to the bottleneck bandwidth. The empirical frame rate corresponding to any bottleneck bandwidth is: a frame rate that enables a code stream transmitted using the bottleneck bandwidth as the code rate to clearly display the image.

[0173] Optionally, the method for determining the network status of the wireless network includes:

[0174] Determining the network state of the wireless network according to a predetermined determination method based on a difference relationship between an empirical frame rate corresponding to the current bottleneck bandwidth and an empirical frame rate corresponding to the bottleneck bandwidth detected last time;

[0175] The predetermined determination method includes:

[0176] If the empirical frame rate corresponding to the current bottleneck bandwidth is the same as the empirical frame rate corresponding to the bottleneck bandwidth detected last time, it is determined that the network state of the wireless network meets the network stability condition;

[0177] If the empirical frame rate corresponding to the current bottleneck bandwidth is different from the empirical frame rate corresponding to the bottleneck bandwidth detected last time, it is determined that the network state of the wireless network does not meet the network stability condition.

[0178] Optionally, adjusting the reference frame rate based on the change value of the predicted frame rate relative to the reference frame rate to determine a screen capture frame rate that matches the current bottleneck bandwidth includes:

[0179] If the reference frame rate is a specified frame rate, adjusting the specified frame rate based on a change value of the predicted frame rate relative to the specified frame rate according to the first formula to determine a screen capture frame rate that matches the current bottleneck bandwidth;

[0180] If the reference frame rate is the empirical frame rate corresponding to the current bottleneck bandwidth, then according to the second formula, based on the change value of the predicted frame rate relative to the empirical frame rate corresponding to the current bottleneck bandwidth, the empirical frame rate corresponding to the current bottleneck bandwidth is adjusted to determine a screen capture frame rate that matches the current bottleneck bandwidth;

[0181] Wherein, the first formula includes:

[0182] ;

[0183] The second formula includes:

[0184] ;

[0185] is the screen capture frame rate adapted to the current bottleneck bandwidth, is a parameter used to make the frame rate change smooth. is the predicted frame rate, is the empirical frame rate corresponding to the bottleneck bandwidth detected last time, is the empirical frame rate corresponding to the current bottleneck bandwidth, Specifies the frame rate for the video.

[0186] Optionally, the first prediction module is specifically used to:

[0187] Calculating the sum of the data amounts of the first-category frames that have been encoded within a predetermined time window before the target frame to obtain a target sum value;

[0188] The ratio of the target sum value to the number of frames of each first category that have been encoded within a predetermined time window is calculated to obtain the predicted data volume.

[0189] Optionally, the second prediction module is specifically used to:

[0190] Predicting the total number of first-category frames and second-category frames transmitted per unit time under the current bottleneck bandwidth when the code stream is transmitted in the target transmission state, and obtaining the predicted frame rate;

[0191] The target transmission state is a code stream transmission state in which the predicted data amount is used as the data amount of the first type of frames, the specified data amount is used as the data amount of the second type of frames, and the predetermined empirical value is used as the number of the second type of frames; wherein the second type of frames are frames that do not require reference to other frames during encoding.

[0192] Optionally, a method for obtaining an empirical frame rate corresponding to any bottleneck bandwidth includes:

[0193] From various preset bandwidth intervals, determine the bandwidth interval where the bottleneck bandwidth is located as the target bandwidth interval;

[0194] From the empirical frame rate correspondence table, determine the empirical frame rate corresponding to the target bandwidth interval as the empirical frame rate corresponding to the bottleneck bandwidth; wherein the empirical frame rate correspondence table is used to record each bandwidth interval and the empirical frame rate corresponding to each bandwidth interval, and the empirical frame rate corresponding to each bandwidth interval is: the frame rate at which the code stream transmitted according to the bandwidth of the bandwidth interval as the transmission rate can clearly display the picture.

[0195] Optionally, a method for determining the empirical frame rate corresponding to each bandwidth interval in the empirical frame rate correspondence table includes:

[0196] For each test scenario, determine the screen capture frame rate of the test scenario corresponding to the bandwidth interval according to a predetermined test method;

[0197] Calculate the average of the screen capture frame rates in each test scenario corresponding to the bandwidth interval to obtain the empirical frame rate corresponding to the bandwidth interval;

[0198] Wherein, the predetermined test method includes:

[0199] The test source end is captured using the current test frame rate as the screen capture frame rate of the display data matching the test scenario at the test source end, the display data obtained by the screen capture is encoded according to the fixed bit rate corresponding to the bandwidth interval and transmitted to the test receiving end, the picture clarity of the display data of the test receiving end is determined, and when the picture clarity meets the picture clarity condition, the screen capture frame rate of the test scenario corresponding to the bandwidth interval is obtained; otherwise, the current test frame rate is adjusted to obtain a new test frame rate, and the step of capturing the screen of the test source end using the current test frame rate as the screen capture frame rate of the display data matching the test scenario at the test source end is returned; wherein the value of the fixed bit rate corresponding to the bandwidth interval is the amount of data represented by the maximum bandwidth in the bandwidth interval.

[0200] Optionally, the detection module is specifically used to:

[0201] In response to satisfying the frame rate control condition, detecting a bottleneck bandwidth of the wireless network as a current bottleneck bandwidth;

[0202] The frame rate control conditions include: reaching a target period, and / or detecting that a network state of the wireless network meets a preset unstable condition.

[0203] In the technical solution of this application, the operations involved in obtaining, storing, using, processing, transmitting, providing and disclosing user personal information are all carried out with the user's authorization.

[0204] The present application also provides a screen projection device, such as Figure 6 As shown, including:

[0205] Memory 601, used for storing computer programs;

[0206] The processor 602 is used to implement any of the above frame rate control methods when executing the program stored in the memory 601.

[0207] In addition, the above-mentioned screen projection device may also include a communication bus and / or a communication interface, and the processor 602, the communication interface, and the memory 601 communicate with each other through the communication bus.

[0208] The communication bus mentioned in the above projection device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0209] The communication interface is used for communication between the above-mentioned screen projection device and other devices.

[0210] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0211] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0212] In another embodiment provided in the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, any of the above-mentioned frame rate control methods is implemented.

[0213] In another embodiment provided in the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any frame rate control method in the above embodiments.

[0214] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a solid-state hard disk (SSD), etc.

[0215] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0216] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0217] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A frame rate control method, characterized in that: Applied to a screen projection device, the screen projection device communicates with a screen projection receiving end via a wireless network; the method comprises: During the screen projection process, the bottleneck bandwidth of the wireless network is detected as the current bottleneck bandwidth; Predicting the data amount of the target frame to obtain the predicted data amount; wherein the target frame is a first-class frame to be encoded in the encoding process of the captured display data of the source end, and the first-class frame is a frame that needs to refer to other frames during encoding; Based on the predicted data volume, predict the number of image frames that can be transmitted per unit time if bitstream transmission is performed under the current bottleneck bandwidth, and obtain a predicted frame rate; Based on the change value of the predicted frame rate relative to the reference frame rate, the reference frame rate is adjusted to determine a screen capture frame rate that matches the current bottleneck bandwidth; wherein the reference frame rate is a frame rate set as an adjustment basis according to the network status of the wireless network; The determined screen capture frame rate is set as the screen capture frame rate for capturing the display data of the source end, so that the display data captured according to the determined screen capture frame rate is encoded and transmitted to the screen projection receiving end, and the bit stream rate used for the encoding process has a positive correlation with the determined screen capture frame rate.

2. The method according to claim 1, characterized in that: The reference frame rate is set in the following manner: When the network state satisfies a network stability condition, using the specified frame rate as a reference frame rate; When the network state does not meet the network stability condition, the empirical frame rate corresponding to the current bottleneck bandwidth is used as the reference frame rate; The specified frame rate is a screen capture frame rate determined based on the bottleneck bandwidth detected last time and adapted to the bottleneck bandwidth. The empirical frame rate corresponding to any bottleneck bandwidth is: a frame rate that enables a code stream transmitted using the bottleneck bandwidth as the code rate to clearly display the image.

3. The method according to claim 2, characterized in that The method for determining the network status of the wireless network includes: Determining the network state of the wireless network according to a predetermined determination method based on a difference relationship between an empirical frame rate corresponding to the current bottleneck bandwidth and an empirical frame rate corresponding to the bottleneck bandwidth detected last time; The predetermined determination method includes: If the empirical frame rate corresponding to the current bottleneck bandwidth is the same as the empirical frame rate corresponding to the bottleneck bandwidth detected last time, it is determined that the network state of the wireless network meets the network stability condition; If the empirical frame rate corresponding to the current bottleneck bandwidth is different from the empirical frame rate corresponding to the bottleneck bandwidth detected last time, it is determined that the network state of the wireless network does not meet the network stability condition.

4. The method according to claim 2, characterized in that: The adjusting process of the reference frame rate based on the change value of the predicted frame rate relative to the reference frame rate to determine a screen capture frame rate that matches the current bottleneck bandwidth includes: If the reference frame rate is a specified frame rate, adjusting the specified frame rate based on a change value of the predicted frame rate relative to the specified frame rate according to the first formula to determine a screen capture frame rate that matches the current bottleneck bandwidth; If the reference frame rate is the empirical frame rate corresponding to the current bottleneck bandwidth, then according to the second formula, based on the change value of the predicted frame rate relative to the empirical frame rate corresponding to the current bottleneck bandwidth, the empirical frame rate corresponding to the current bottleneck bandwidth is adjusted to determine a screen capture frame rate that matches the current bottleneck bandwidth; Wherein, the first formula includes: ； The second formula includes: ； is the screen capture frame rate adapted to the current bottleneck bandwidth, is a parameter used to make the frame rate change smooth. is the predicted frame rate, is the empirical frame rate corresponding to the bottleneck bandwidth detected last time, is the empirical frame rate corresponding to the current bottleneck bandwidth, Specifies the frame rate for the video.

5. The method according to any one of claims 1 to 4, characterized in that: The step of predicting the data amount of the target frame to obtain the predicted data amount includes: Calculating the sum of the data amounts of the first-category frames that have been encoded within a predetermined time window before the target frame to obtain a target sum value; The ratio of the target sum value to the number of frames of each first category that have been encoded within a predetermined time window is calculated to obtain the predicted data volume.

6. The method according to any one of claims 1 to 4, characterized in that: The step of predicting the number of image frames that can be transmitted per unit time based on the predicted data volume if the code stream is transmitted under the current bottleneck bandwidth to obtain the predicted frame rate includes: Predicting the total number of first-category frames and second-category frames transmitted per unit time under the current bottleneck bandwidth when the code stream is transmitted in the target transmission state, and obtaining the predicted frame rate; The target transmission state is: a code stream transmission state in which the predicted data amount is used as the data amount of the first type of frames, the specified data amount is used as the data amount of the second type of frames, and a predetermined empirical value is used as the number of the second type of frames; wherein the second type of frames are frames that do not require reference to other frames during encoding.

7. The method according to any one of claims 1 to 4, characterized in that: The methods for obtaining the empirical frame rate corresponding to any bottleneck bandwidth include: From various preset bandwidth intervals, determine the bandwidth interval where the bottleneck bandwidth is located as the target bandwidth interval; From the empirical frame rate correspondence table, determine the empirical frame rate corresponding to the target bandwidth interval as the empirical frame rate corresponding to the bottleneck bandwidth; wherein the empirical frame rate correspondence table is used to record each bandwidth interval and the empirical frame rate corresponding to each bandwidth interval, and the empirical frame rate corresponding to each bandwidth interval is: the frame rate at which the code stream transmitted according to the bandwidth of the bandwidth interval as the transmission rate can clearly display the picture.

8. The method according to claim 7, characterized in that The method for determining the empirical frame rate corresponding to each bandwidth interval in the empirical frame rate corresponding table includes: For each test scenario, determine the screen capture frame rate of the test scenario corresponding to the bandwidth interval according to a predetermined test method; Calculate the average of the screen capture frame rates in each test scenario corresponding to the bandwidth interval to obtain the empirical frame rate corresponding to the bandwidth interval; Wherein, the predetermined test method includes: The test source end is captured using the current test frame rate as the screen capture frame rate of the display data matching the test scenario at the test source end, the display data obtained by the screen capture is encoded according to the fixed bit rate corresponding to the bandwidth interval and transmitted to the test receiving end, the picture clarity of the display data of the test receiving end is determined, and when the picture clarity meets the picture clarity condition, the screen capture frame rate of the test scenario corresponding to the bandwidth interval is obtained; otherwise, the current test frame rate is adjusted to obtain a new test frame rate, and the step of capturing the screen of the test source end using the current test frame rate as the screen capture frame rate of the display data matching the test scenario at the test source end is returned; wherein the value of the fixed bit rate corresponding to the bandwidth interval is the amount of data represented by the maximum bandwidth in the bandwidth interval.

9. The method according to any one of claims 1 to 4, characterized in that: The detecting the bottleneck bandwidth of the wireless network as the current bottleneck bandwidth includes: In response to satisfying the frame rate control condition, detecting a bottleneck bandwidth of the wireless network as a current bottleneck bandwidth; The frame rate control conditions include: reaching a target period, and / or detecting that a network state of the wireless network meets a preset unstable condition.

10. A frame rate control device, characterized in that: Applied to a screen projection device, the screen projection device communicates with a screen projection receiving end via a wireless network; the device comprises: A detection module, used to detect the bottleneck bandwidth of the wireless network as the current bottleneck bandwidth during the screen projection process; A first prediction module is used to predict the data amount of a target frame to obtain a predicted data amount; wherein the target frame is a first-type frame to be encoded during the encoding process of the captured display data of the source end, and the first-type frame is a frame that needs to refer to other frames during encoding; A second prediction module is used to predict, based on the predicted data volume, the number of image frames that can be transmitted per unit time if bitstream transmission is performed under the current bottleneck bandwidth, and obtain a predicted frame rate; An adjustment processing module, configured to adjust the reference frame rate based on a change value of the predicted frame rate relative to the reference frame rate, so as to determine a screen capture frame rate that is compatible with the current bottleneck bandwidth; wherein the reference frame rate is a frame rate set as an adjustment basis according to a network status of the wireless network; A setting module is used to set the determined screen capture frame rate as the screen capture frame rate for capturing the display data of the source end, so that the display data captured according to the determined screen capture frame rate is encoded and transmitted to the screen projection receiving end, and the bit stream rate used for the encoding process has a positive correlation with the determined screen capture frame rate.

11. A screen projection device, characterized in that: include: Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-9 when executing a program stored in a memory.

12. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 9.

Images