Screen projection optimization method and device, equipment and medium

Abstract

The invention is suitable for the technical field of video communication, and particularly relates to a projection screen optimization method and device, equipment and a medium. A receiving end calculates a current smoothing parameter according to a video frame sent by a sending end, the sending end obtains the current smoothing parameter fed back by the receiving end and a current code rate of the video frame sent by the sending end, and when it is detected that the current smoothing parameter is larger than a first parameter threshold value or smaller than a second parameter threshold value, the current code rate is adjusted. And controlling the transmitting end to transmit the video frame according to the adjusted code rate, and returning to execute the operation of acquiring the current smoothing parameter fed back by the receiving end and the current code rate of the video frame transmitted by the transmitting end until the current smoothing parameter is detected to be between the first parameter threshold and the second parameter threshold. The function of adjusting the video transmission code rate according to the smooth characteristic of the video frame is realized, so that reasonable transmission adjustment is performed on the video frame with large fluctuation, the time delay can be reduced, and smooth playing of the video can be ensured.

Description

Technical Field

[0001] The present application is applicable to the field of video communication technology, and in particular, relates to a screen projection optimization method, device, equipment and medium. Background Art

[0002] Real-time screen projection is widely used in various scenarios such as gaming, entertainment, office meetings, etc. It generally refers to displaying the screen display of one device on another device in real time within a local area network. Among them, low-latency screen projection is an important indicator.

[0003] At present, common screen projection protocols include micracast, airplay, RTSP, or custom protocols modified based on some network transmission algorithms. These screen projection protocols have achieved the effect of reducing latency to varying degrees. However, I frames in video transmission are much larger than P frames. When sending data, smooth sending is required to better reduce network congestion. However, smooth sending of video data with large fluctuations will cause unevenness at the receiving end, ultimately leading to unsmooth playback at the receiving end.

[0004] Currently, relatively good low-latency video data transmission and display are achieved through congestion control algorithms, smooth transmission, jiter buffer and other technologies. Among them, a jiter buffer is introduced to solve the smoothness problem when displaying data, but this will introduce new delays. In particular, when the network fluctuations are large, the delay introduced by the jiter buffer is relatively large.

[0005] Therefore, how to adjust video transmission to reduce the smooth transmission and cause the receiving end to play unsmoothly to optimize screen projection has become an urgent problem to be solved. Summary of the invention

[0006] In view of this, the embodiments of the present application provide a screen projection optimization method, device, equipment and medium to solve the problem of how to adjust video transmission to reduce the problem of optimizing screen projection by reducing the unsmooth playback at the receiving end caused by smooth transmission.

[0007] In a first aspect, an embodiment of the present application provides a screen projection optimization method, which is applied to a transmitting end in a screen projection system, wherein the screen projection system also includes a receiving end, including: Acquire a current smoothing parameter fed back by the receiving end and a current bit rate of the video frame sent by the sending end, wherein the current smoothing parameter is calculated by the receiving end according to the video frame sent by the sending end; When it is detected that the current smoothing parameter is greater than a first parameter threshold or the current smoothing parameter is less than a second parameter threshold, adjusting the current bit rate to obtain an adjusted bit rate; According to the adjusted bit rate, the sending end is controlled to send video frames, and the process of obtaining the current smoothing parameters fed back by the receiving end and the current bit rate of the video frames sent by the sending end is returned to execute until the screen projection is completed.

[0008] In a second aspect, an embodiment of the present application provides a screen projection optimization method, which is applied to a receiving end in a screen projection system, wherein the screen projection system also includes a sending end, including: Receiving the video frame sent by the transmitting end, and calculating the current smoothing parameter according to the received video frame; Feeding back the current smoothing parameter to the sending end, returning to execute the receiving of the video frame sent by the sending end, and calculating the current smoothing parameter according to the received video frame; wherein, The sending end is used to implement the screen projection optimization method as described in the first aspect above.

[0009] In a third aspect, an embodiment of the present application provides a screen projection optimization device, which is applied to a transmitting end in a screen projection system, and the screen projection system also includes a receiving end, including: A sending end acquisition module, used to acquire a current smoothing parameter fed back by the receiving end and a current bit rate of the video frame sent by the sending end, wherein the current smoothing parameter is calculated by the receiving end according to the video frame sent by the sending end; A transmitting end optimization module, configured to adjust the current bit rate to obtain an adjusted bit rate when it is detected that the current smoothing parameter is greater than a first parameter threshold or the current smoothing parameter is less than a second parameter threshold; The transmitting end control module is used to control the transmitting end to send video frames according to the adjusted bit rate, and return to execute the current smoothing parameters fed back by the receiving end and the current bit rate of the video frames sent by the transmitting end until the screen projection is completed.

[0010] In a fourth aspect, an embodiment of the present application provides a screen projection optimization device, which is applied to a receiving end in a screen projection system, and the screen projection system also includes a transmitting end, including: A receiving end calculation module, used for receiving the video frame sent by the sending end, and calculating the current smoothing parameter according to the received video frame; The receiving end feedback module is used to feed back the current smoothing parameter to the sending end, return to execute the receiving of the video frame sent by the sending end, and calculate the current smoothing parameter according to the received video frame; wherein, The sending end is used to implement the screen projection optimization method as described in the first aspect above.

[0011] In the fifth aspect, an embodiment of the present application provides a computer device, which includes a processor, a memory, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, it implements the screen projection optimization method described in the first aspect or the screen projection optimization method described in the second aspect.

[0012] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the screen projection optimization method described in the first aspect or the screen projection optimization method described in the second aspect is implemented.

[0013] Compared with the prior art, the embodiments of the present application have the following beneficial effects: in the present application, the receiving end calculates the current smoothing parameter according to the video frame sent by the sending end, the sending end obtains the current smoothing parameter fed back by the receiving end and the current bit rate of the video frame sent by the sending end, and when it is detected that the current smoothing parameter is greater than the first parameter threshold or the current smoothing parameter is less than the second parameter threshold, the current bit rate is adjusted to obtain the adjusted bit rate, and according to the adjusted bit rate, the sending end is controlled to send the video frame, and the execution returns to obtain the current smoothing parameter fed back by the receiving end and the current bit rate of the video frame sent by the sending end until it is detected that the current smoothing parameter is between the first parameter threshold and the second parameter threshold, thereby realizing the function of adjusting the video transmission bit rate according to the smoothing characteristics of the video frame, thereby making reasonable transmission adjustments to the video frames with large fluctuations, which can both reduce the delay and ensure the smooth playback of the video. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0015] Figure 1 This is a schematic diagram of an application environment of a screen projection optimization method provided in Embodiment 1 of the present application; Figure 2 It is a flowchart of a screen projection optimization method provided in Embodiment 2 of the present application; Figure 3 It is a flowchart of a screen projection optimization method provided in Embodiment 3 of the present application; Figure 4 It is a structural schematic diagram of a screen projection optimization device provided in Embodiment 4 of the present application; Figure 5 It is a structural schematic diagram of a screen projection optimization device provided in Embodiment 5 of the present application; Figure 6 It is a structural diagram of a computer device provided in Example 6 of the present application. DETAILED DESCRIPTION

[0016] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0017] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0018] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0019] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0020] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0021] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0022] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0023] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0024] It should be understood that the size of the serial numbers of the steps in the following embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0025] In order to illustrate the technical solution of the present application, a specific embodiment is provided below for illustration.

[0026] A screen projection optimization method provided in the first embodiment of the present application can be applied in the following cases: Figure 1 In the application environment, the transmitting end and the receiving end are connected to communicate to form a screen projection system, the transmitting end smoothly sends the screen projection data to the receiving end, the receiving end feeds back the corresponding state and performs bandwidth estimation together with the transmitting end, and the receiving end continuously receives jitter information and feeds it back to the transmitting end. The transmitting end and the receiving end may include but are not limited to PDAs, desktop computers, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, cloud computing devices, personal digital assistants (PDAs), and other computer devices, and the computer devices have corresponding displays.

[0027] See also Figure 2 , is a flow chart of a screen projection optimization method provided in Embodiment 2 of the present application, wherein the screen projection optimization method is applied to Figure 1 The sending end in . Figure 2 As shown, the screen projection optimization method may include the following steps: Step S201, obtaining the current smoothing parameter fed back by the receiving end and the current bit rate of the video frame sent by the sending end.

[0028] The receiving end feeds back a smoothing parameter to the sending end, and the current smoothing parameter is calculated by the receiving end according to the video frame sent by the sending end. The specific processing process of the receiving end may refer to the content of the following embodiment 3.

[0029] At the current moment, the video frame sent by the transmitter can collect the sending bit rate of sending the video frame, that is, the current bit rate.

[0030] Step S202: When it is detected that the current smoothing parameter is greater than a first parameter threshold or the current smoothing parameter is less than a second parameter threshold, the current bit rate is adjusted to obtain an adjusted bit rate.

[0031] The current smoothing parameter is compared with the first parameter threshold and the second parameter threshold respectively, and the first parameter threshold is greater than or equal to the second parameter, that is, the current smoothing parameter cannot simultaneously satisfy the requirement of being greater than the first parameter threshold and less than the second parameter threshold. When the encoded video data (for example, H264, H265) is transmitted using the smooth transmission method, there is a characteristic that the I frame is much larger than the P frame between video data frames.

[0032] The bit rate is adjusted only when the current smoothing parameter is greater than the first parameter threshold or the current smoothing parameter is less than the second parameter threshold, that is, the current bit rate is adjusted to obtain the adjusted bit rate (that is, the adjusted bit rate).

[0033] Step S203, according to the adjusted bit rate, control the sending end to send video frames, return to execute the acquisition of the current smoothing parameters fed back by the receiving end and the current bit rate of the video frames sent by the sending end, until the screen projection is completed.

[0034] The video frames are sent by adjusting the bit rate, and the bit rate is a parameter of the sending end. After adjustment, the rate at which the sending end sends the video frames to the receiving end can be adjusted.

[0035] In the screen projection system, as long as the sending end and the receiving end are in the screen projection state, they need to loop through the above steps S201 to S203. If it is detected that the current smoothing parameter is between the first parameter threshold and the second parameter threshold, there is no need to adjust the corresponding bit rate.

[0036] In the present application, the feedback frequency of the receiving end feeding back the smoothing parameter to the sending end is 1 GOP, and accordingly, the bit rate adjustment process is adjusted once every 1 GOP, and GOP is a video group.

[0037] Optionally, the screen projection optimization method further includes: Obtaining a first estimated bit rate based on a bandwidth estimation algorithm; The adjusting the current bit rate to obtain the adjusted bit rate includes: If it is detected that the current smoothing parameter is greater than the first parameter threshold, lowering the current bit rate to obtain a first bit rate; A minimum value between the first code rate and the first estimated code rate is determined as an adjusted code rate.

[0038] In order to reduce the current bit rate to obtain the first bit rate, a first weight may be set, the first weight is a positive number less than 1, and the first weight is multiplied by the current bit rate to obtain the first bit rate. In one embodiment, the first weight is 0.9.

[0039] The sender also estimates the bandwidth, where the bandwidth estimation algorithm (Bottleneck Bandwidth and Round-trip propagation time, BBR) can be used to estimate the estimated bit rate, compare the estimated bit rate with the adjusted bit rate, and take the minimum value as the adjusted bit rate to avoid the adjusted bit rate exceeding the bandwidth limit. Taking into account both bandwidth utilization and low latency, the bit rate is adjusted to a reasonable value based on the actual receiving jitter of the receiving end, which has better bandwidth utilization than simply setting the bit rate to the bandwidth estimate multiplied by the fixed system.

[0040] Optionally, the screen projection optimization method further includes: Obtaining a second estimated bit rate based on a bandwidth estimation algorithm; The adjusting the current bit rate to obtain the adjusted bit rate includes: If it is detected that the current smoothing parameter is less than the second parameter threshold, the current bit rate is increased to obtain a second bit rate; A minimum value between the second code rate and the second estimated code rate is determined as the adjusted code rate.

[0041] In order to reduce the current bit rate to obtain the first bit rate, a second weight may be set, the second weight is a positive number greater than 1, and the second weight is multiplied by the current bit rate to obtain the second bit rate. In one embodiment, the second weight is 1.1.

[0042] The sender also estimates the bandwidth, where the bandwidth estimation algorithm (Bottleneck Bandwidth and Round-trip propagation time, BBR) can be used to estimate the estimated bit rate, compare the estimated bit rate with the adjusted bit rate, and take the minimum value as the adjusted bit rate to avoid the adjusted bit rate exceeding the bandwidth limit.

[0043] In the embodiment of the present application, the receiving end calculates the current smoothing parameter based on the video frame sent by the sending end, and the sending end obtains the current smoothing parameter fed back by the receiving end and the current bit rate of the video frame sent by the sending end. When it is detected that the current smoothing parameter is greater than the first parameter threshold or the current smoothing parameter is less than the second parameter threshold, the current bit rate is adjusted to obtain the adjusted bit rate. According to the adjusted bit rate, the sending end is controlled to send the video frame, and the execution returns to obtain the current smoothing parameter fed back by the receiving end and the current bit rate of the video frame sent by the sending end until it is detected that the current smoothing parameter is between the first parameter threshold and the second parameter threshold, thereby realizing the function of adjusting the video transmission bit rate according to the smoothing characteristics of the video frame, thereby making reasonable transmission adjustments to the video frames with large fluctuations, which can both reduce the delay and ensure the smooth playback of the video.

[0044] See also Figure 3 , is a flow chart of a screen projection optimization method provided in Embodiment 3 of the present application, wherein the screen projection optimization method is applied to Figure 1 The receiving end in Figure 3 As shown, the screen projection optimization method may include the following steps: Step S301: receiving a video frame sent by the transmitting end, and calculating a current smoothing parameter according to the received video frame.

[0045] As long as the receiving end receives the video frame, it will calculate a smoothing parameter according to the video frame as the current smoothing parameter.

[0046] Step S302, feeding back the current smoothing parameter to the sending end, returning to execute the receiving of the video frame sent by the sending end, and calculating the current smoothing parameter according to the received video frame.

[0047] Among them, the sending end is used to implement the screen projection optimization method as in the above-mentioned embodiment 2.

[0048] Optionally, the calculating a current smoothing parameter according to the received video frame includes: Calculating the variance of the video frame according to the received video frame; According to the variance, a current smoothing parameter is calculated.

[0049] The variance of the video frame can be calculated using a variance formula, and the smoothing parameter is determined according to the variance.

[0050] Optionally, the calculating a current smoothing parameter according to the variance includes: Get the historical smoothing parameters and weight parameters of the previous calculation moment; Multiplying the historical smoothing parameter by the weight parameter to obtain a first parameter item; Multiply the difference between 1 and the weight parameter by the variance to obtain a second parameter term; Add the first parameter term and the second parameter term to obtain the current smoothing parameter.

[0051] Among them, the calculation formula of the smoothing parameter is as follows: S = S(t - 1) * f + D * (1 - f) In the formula, 0 < f < 1, S is the current smoothing parameter, S(t - 1) is the smoothing parameter obtained in the previous calculation (i.e., the historical smoothing parameter), and D is the variance.

[0052] Therefore, the first video frame sent by the sending end is sent according to a preset bit rate or an estimated bit rate, and the receiving end feeds back the current smoothing parameter when receiving the second video frame.

[0053] In the embodiment of the present application, the receiving end calculates the current smoothing parameter according to the video frame sent by the sending end. The sending end obtains the current smoothing parameter fed back by the receiving end and the current bit rate of the video frame sent by the sending end. When it is detected that the current smoothing parameter is greater than the first parameter threshold or the current smoothing parameter is less than the second parameter threshold, adjust the current bit rate to obtain an adjusted bit rate. According to the adjusted bit rate, control the sending end to send video frames, and return to execute obtaining the current smoothing parameter fed back by the receiving end and the current bit rate of the video frame sent by the sending end until it is detected that the current smoothing parameter is between the first parameter threshold and the second parameter threshold, realizing the function of adjusting the video transmission bit rate according to the smoothing feature of the video frame, so as to perform reasonable transmission adjustment on video frames with large fluctuations, which can not only reduce latency but also ensure smooth playback of the video.

[0054] Corresponding to the screen mirroring optimization method in the above embodiment, Figure 4 shows the structural block diagram of the screen mirroring optimization device provided in the fifth embodiment of the present application. The above screen mirroring optimization device is applied to Figure 1 the sending end in. For the sake of simplicity of description, only the parts related to the embodiment of the present application are shown.

[0055] See Figure 4 This screen mirroring optimization device includes: A sending end acquisition module 41, configured to acquire the current smoothing parameter fed back by the receiving end and the current bit rate of the video frame sent by the sending end, where the current smoothing parameter is calculated by the receiving end according to the video frame sent by the sending end; A sending end optimization module 42, configured to adjust the current bit rate to obtain an adjusted bit rate when it is detected that the current smoothing parameter is greater than the first parameter threshold or the current smoothing parameter is less than the second parameter threshold; The sending end control module 43 is used to control the sending end to send video frames according to the adjusted bit rate, and return to execute the current smoothing parameters fed back by the receiving end and the current bit rate of the video frames sent by the sending end until the screen projection is completed.

[0056] Optionally, the screen projection optimization device further includes: A first bit rate acquisition module, used to obtain a first estimated bit rate based on a bandwidth estimation algorithm; The sending end optimization module 42 includes: A bit rate lowering unit, configured to lower the current bit rate to obtain a first bit rate if it is detected that the current smoothing parameter is greater than the first parameter threshold; The first code rate determining unit is configured to determine a minimum value between the first code rate and the first estimated code rate as an adjusted code rate.

[0057] Optionally, the screen projection optimization device further includes: A second bit rate acquisition module, used to obtain a second estimated bit rate based on a bandwidth estimation algorithm; The sending end optimization module 42 includes: A bit rate increasing unit, configured to increase the current bit rate to obtain a second bit rate if it is detected that the current smoothing parameter is less than the second parameter threshold; The second code rate determining unit is configured to determine a minimum value between the second code rate and the second estimated code rate as an adjusted code rate.

[0058] It should be noted that the information interaction, execution process and other contents between the above-mentioned modules are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0059] Corresponding to the screen projection optimization method in the above embodiment, Figure 5 The structure block diagram of the screen projection optimization device provided in the fifth embodiment of the present application is shown. The screen projection optimization device is applied to Figure 1 For ease of description, only the parts related to the embodiment of the present application are shown.

[0060] See also Figure 5 , the screen projection optimization device comprises: A receiving end calculation module 51 is used to receive the video frame sent by the sending end, and calculate the current smoothing parameter according to the received video frame; The receiving end feedback module 52 is used to feed back the current smoothing parameter to the sending end, return to execute the receiving of the video frame sent by the sending end, and calculate the current smoothing parameter based on the received video frame; wherein, the sending end is used to implement the screen projection optimization method as in Example 2.

[0061] Optionally, the receiving end computing module 51 includes: A variance calculation unit, configured to calculate the variance of the video frame according to the received video frame; The smoothing parameter calculation unit is used to calculate the current smoothing parameter according to the variance.

[0062] Optionally, the smoothing parameter calculation unit includes: The parameter acquisition subunit is used to obtain the historical smoothing parameters and weight parameters of the previous calculation moment; A first product unit, used for multiplying the historical smoothing parameter by the weight parameter to obtain a first parameter item; A second product unit, used for multiplying the difference between 1 and the weight parameter and the variance to obtain a second parameter item; The smoothing parameter calculation subunit is used to add the first parameter item and the second parameter item to obtain a current smoothing parameter.

[0063] It should be noted that the information interaction, execution process and other contents between the above-mentioned modules are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0064] Figure 6 This is a schematic diagram of the structure of a computer device provided in Example 6 of the present application. Figure 6 As shown, the computer device of this embodiment includes: at least one processor ( Figure 6 Only one is shown), a memory, and a computer program stored in the memory and executable on at least one processor, and when the processor executes the computer program, the steps in any of the above-mentioned screen projection optimization method embodiments are implemented.

[0065] The computer device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that Figure 6 This is merely an example of a computer device and does not constitute a limitation on the computer device. The computer device may include more or fewer components than those shown in the figure, or a combination of certain components, or different components. For example, it may also include a network interface, a display screen, and an input device.

[0066] The processor may be a CPU, or other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0067] The memory includes a readable storage medium, an internal memory, etc., wherein the internal memory may be the memory of a computer device, and the internal memory provides an environment for the operation of the operating system and computer-readable instructions in the readable storage medium. The readable storage medium may be a hard disk of a computer device, and in other embodiments, it may also be an external storage device of the computer device, for example, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device. Further, the memory may also include both an internal storage unit of the computer device and an external storage device. The memory is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of a computer program, etc. The memory may also be used to temporarily store data that has been output or is to be output.

[0068] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned device can refer to the corresponding process in the above-mentioned method embodiment, which will not be repeated here. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned method embodiment can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device capable of carrying computer program code, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0069] The present application implements all or part of the processes in the above-mentioned embodiment method, and may also be completed through a computer program product. When the computer program product runs on a computer device, the computer device can implement the steps in the above-mentioned method embodiment when executing the computer program product.

[0070] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0071] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0072] In the embodiments provided in the present application, it should be understood that the disclosed devices / computer equipment and methods can be implemented in other ways. For example, the device / computer equipment embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0073] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0074] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A screen projection optimization method, characterized in that: The screen projection optimization method is applied to a transmitting end in a screen projection system, and the screen projection system also includes a receiving end, including: Acquire a current smoothing parameter fed back by the receiving end and a current bit rate of the video frame sent by the sending end, wherein the current smoothing parameter is calculated by the receiving end according to the video frame sent by the sending end; When it is detected that the current smoothing parameter is greater than a first parameter threshold or the current smoothing parameter is less than a second parameter threshold, adjusting the current bit rate to obtain an adjusted bit rate; According to the adjusted bit rate, the sending end is controlled to send video frames, and the process of obtaining the current smoothing parameters fed back by the receiving end and the current bit rate of the video frames sent by the sending end is returned to execute until the screen projection is completed.

2. The screen projection optimization method according to claim 1, characterized in that: The screen projection optimization method further includes: Obtaining a first estimated bit rate based on a bandwidth estimation algorithm; The adjusting the current bit rate to obtain the adjusted bit rate includes: If it is detected that the current smoothing parameter is greater than the first parameter threshold, lowering the current bit rate to obtain a first bit rate; A minimum value between the first code rate and the first estimated code rate is determined as an adjusted code rate.

3. The screen projection optimization method according to claim 2, characterized in that: The screen projection optimization method further includes: Obtaining a second estimated bit rate based on a bandwidth estimation algorithm; The adjusting the current bit rate to obtain the adjusted bit rate includes: If it is detected that the current smoothing parameter is less than the second parameter threshold, the current bit rate is increased to obtain a second bit rate; A minimum value between the second code rate and the second estimated code rate is determined as the adjusted code rate.

4. A screen projection optimization method, characterized in that: The screen projection optimization method is applied to a receiving end in a screen projection system, and the screen projection system also includes a sending end, including: Receiving the video frame sent by the transmitting end, and calculating the current smoothing parameter according to the received video frame; Feeding back the current smoothing parameter to the sending end, returning to execute the receiving of the video frame sent by the sending end, and calculating the current smoothing parameter according to the received video frame; wherein, The sending end is used to implement the screen projection optimization method as described in any one of claims 1 to 3.

5. The screen projection optimization method according to claim 4, characterized in that: The step of calculating a current smoothing parameter according to the received video frame comprises: Calculating the variance of the video frame according to the received video frame; According to the variance, a current smoothing parameter is calculated.

6. The screen projection optimization method according to claim 5, characterized in that: The step of calculating the current smoothing parameter according to the variance includes: Get the historical smoothing parameters and weight parameters of the previous calculation moment; Multiplying the historical smoothing parameter by the weight parameter to obtain a first parameter item; Multiply the difference between 1 and the weight parameter by the variance to obtain a second parameter item; The first parameter item is added to the second parameter item to obtain a current smoothing parameter.

7. A screen projection optimization device, characterized in that: The screen projection optimization device is applied to a transmitting end in a screen projection system, and the screen projection system also includes a receiving end, including: A sending end acquisition module, used to acquire a current smoothing parameter fed back by the receiving end and a current bit rate of the video frame sent by the sending end, wherein the current smoothing parameter is calculated by the receiving end according to the video frame sent by the sending end; A transmitting end optimization module, configured to adjust the current bit rate to obtain an adjusted bit rate when it is detected that the current smoothing parameter is greater than a first parameter threshold or the current smoothing parameter is less than a second parameter threshold; The transmitting end control module is used to control the transmitting end to send video frames according to the adjusted bit rate, and return to execute the current smoothing parameters fed back by the receiving end and the current bit rate of the video frames sent by the transmitting end until the screen projection is completed.

8. A screen projection optimization device, characterized in that: The screen projection optimization device is applied to a receiving end in a screen projection system, and the screen projection system also includes a sending end, including: A receiving end calculation module, used for receiving the video frame sent by the sending end, and calculating the current smoothing parameter according to the received video frame; The receiving end feedback module is used to feed back the current smoothing parameter to the sending end, return to execute the receiving of the video frame sent by the sending end, and calculate the current smoothing parameter according to the received video frame; wherein, The sending end is used to implement the screen projection optimization method as described in any one of claims 1 to 3.

9. A computer device, characterized in that: The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the screen projection optimization method as described in any one of claims 1 to 3 or the screen projection optimization method as described in any one of claims 4 to 6.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements the screen projection optimization method as described in any one of claims 1 to 3 or the screen projection optimization method as described in any one of claims 4 to 6.

Images