Screen projection method, electronic equipment, storage medium and program product

Abstract

The embodiment of the invention provides a screen projection method, electronic equipment, a storage medium and a program product, relates to the technical field of screen projection, can dynamically adjust the code rate of a screen projection video on the premise of not influencing the subjective experience of a user as much as possible, and improves the user experience while reducing the power consumption. The screen projection method comprises the steps that a first application type weight is obtained, and the first application type weight is an application type weight corresponding to a current foreground application; whether code rate adjustment is carried out or not is determined according to code rate adjustment parameters, the code rate adjustment parameters comprise the first application type weight, if yes, coding and wireless screen projection are carried out on the content to be subjected to screen projection on the basis of a target code rate, the target code rate is lower than the original code rate of the content to be subjected to screen projection, and if not, coding and wireless screen projection are carried out on the content to be subjected to screen projection on the basis of the original code rate.

Description

Technical Field

[0001] This application relates to the field of screen mirroring technology, and in particular, to a screen mirroring method, an electronic device, a storage medium, and a program product. Background Art

[0002] With the increase in the interaction functions between electronic devices, more and more users use the screen mirroring function. For example, when a user plays a video on a mobile phone, through the function of mirror screen mirroring, the content on the mobile phone screen can be transmitted to a large-screen TV for playing via a wireless connection. However, for a mobile phone, screen mirroring requires encoding and transmitting a large amount of video stream data, resulting in high power consumption and prone to overheating during the screen mirroring process. To solve the overheating problem caused by screen mirroring, the prior art monitors the temperature of the mobile phone. If the temperature of the mobile phone is too high during the screen mirroring process, the bitrate of the mobile phone screen mirroring is reduced to reduce power consumption. However, this will reduce the clarity or smoothness of the screen mirroring, thereby reducing the user experience. Summary of the Invention

[0003] In view of this, this application provides a screen mirroring method, an electronic device, a storage medium, and a program product, which can dynamically adjust the bitrate of the screen mirroring video on the premise of minimizing the impact on the user's subjective experience, so as to improve the user experience.

[0004] In a first aspect, a screen mirroring method is provided, including: obtaining a first application type weight, where the first application type weight is the application type weight corresponding to the current foreground application; determining whether to adjust the bitrate according to the bitrate adjustment parameter, where the bitrate adjustment parameter includes the first application type weight. If so, encoding and wirelessly mirroring the content to be mirrored based on the target bitrate, where the target bitrate is lower than the original bitrate of the content to be mirrored. If not, encoding and wirelessly mirroring the content to be mirrored based on the original bitrate.

[0005] The current foreground application corresponds to the scenario of the content to be screen-cast. If it is in a relatively stable scenario, that is, the continuity of the image is relatively good and the probability of application switching is relatively low, then the application type weight corresponding to this application is relatively high; if it is in a scenario with a relatively high complexity, then the application type weight corresponding to this application is relatively high, because the higher the bit rate, the more complex the scenario is. The higher the bit rate, the smaller the impact on the user when reducing the bit rate, that is, the greater the power consumption benefit of bit rate adjustment. Therefore, the application type weight corresponding to this application is relatively high. According to the relatively high application type weight, it is easier to determine to perform bit rate adjustment. In this scenario, encoding and wireless screen-casting are performed based on the reduced bit rate. On the one hand, it can reduce power consumption, and on the other hand, it has a relatively small impact on the user's perception. For some application types, there may be scenarios with instability or relatively low complexity. In these scenarios, reducing the bit rate has a relatively large impact on the user's perception. Therefore, it is easier to determine not to perform bit rate adjustment, that is, encoding and wireless screen-casting the content to be screen-cast based on the original bit rate. In addition, the screen-casting method in the embodiments of the present application does not need to wait until the temperature rises to the threshold before adjusting the bit rate, but dynamically adjusts the bit rate based on the foreground application type, which can improve the temperature rise of the screen-casting source device due to screen-casting, extend the time of temperature rise, and reduce the probability of device abnormality due to heating.

[0006] In a possible implementation manner, the application type weight corresponding to a video or game application > the application type weight corresponding to other applications.

[0007] In a possible implementation manner, determining whether to perform bit rate adjustment according to the bit rate adjustment parameter includes: if the first application type weight is greater than the preset weight, determining to perform bit rate adjustment; if the first application type weight is not greater than the preset weight, determining not to perform bit rate adjustment.

[0008] In a possible implementation manner, the bit rate adjustment parameter further includes the frequency at which the current foreground application switches from the background to the foreground within a preset period. The frequency at which the current foreground application switches from the background to the foreground within a preset period is continuous, and this switching frequency can reflect the subsequent scenario. Therefore, dynamically adjusting the screen-casting bit rate in combination with the switching frequency can further improve the user experience.

[0009] In a possible implementation manner, determining whether to perform bit rate adjustment according to the bit rate adjustment parameter includes: determining a bit rate adjustment index according to the bit rate adjustment parameter; if the necessity of bit rate adjustment characterized by the bit rate adjustment index meets the adjustment condition, determining to perform bit rate adjustment; if the necessity of bit rate adjustment characterized by the bit rate adjustment index does not meet the adjustment condition, determining not to perform bit rate adjustment. The necessity of bit rate adjustment characterized by the bit rate adjustment index is positively correlated with the first application type weight, and the necessity of bit rate adjustment characterized by the bit rate adjustment index is negatively correlated with the frequency.

[0010] In a possible implementation manner, the bitrate adjustment parameter further includes a second application type weight, where the second application type weight is the application type weight corresponding to a target application that is switched to the foreground application within a preset period, and the target application is an application other than the current foreground application. The target application can also reflect the scenario of the screen mirroring to a certain extent. Therefore, dynamically adjusting the screen mirroring bitrate in combination with the second application type weight can further improve the user experience.

[0011] In a possible implementation manner, determining whether to perform bitrate adjustment according to the bitrate adjustment parameter includes: determining a bitrate adjustment index according to the bitrate adjustment parameter. If the necessity of bitrate adjustment characterized by the bitrate adjustment index meets the adjustment condition, it is determined to perform bitrate adjustment. If the necessity of bitrate adjustment characterized by the bitrate adjustment index does not meet the adjustment condition, it is determined not to perform bitrate adjustment. The necessity of bitrate adjustment characterized by the bitrate adjustment index is positively correlated with the first application type weight, the necessity of bitrate adjustment characterized by the bitrate adjustment index is positively correlated with the second application type weight, and the necessity of bitrate adjustment characterized by the bitrate adjustment index is negatively correlated with the above frequency.

[0012] In a possible implementation manner, determining whether to perform bitrate adjustment according to the bitrate adjustment parameter includes: determining a bitrate adjustment index k according to the following formula: k = W app1 2 / (a × f / W app2 ), if k is greater than the threshold, it is determined to perform bitrate adjustment. If k is not greater than the threshold, it is determined not to perform bitrate adjustment. W app1 is the first application type weight, f is the frequency at which the current foreground application switches from the background to the foreground within a preset period, W app2 is the second application type weight, and a is a constant. Determining the bitrate adjustment index based on this formula can more accurately achieve dynamic adjustment of encoding.

[0013] In a possible implementation manner, encoding and wirelessly mirroring the content to be mirrored based on the target bitrate includes: determining the target resolution of the content to be mirrored based on the target bitrate, where the target resolution of the content to be mirrored is lower than the original resolution of the content to be mirrored; encoding and wirelessly mirroring the content to be mirrored based on the target resolution. It is possible to minimize the adverse impact on the user experience while reducing the bitrate.

[0014] In a second aspect, a screen mirroring method is provided, including: determining whether to perform bitrate adjustment according to the frequency at which the current foreground application switches from the background to the foreground within a preset period. If so, encoding and wirelessly mirroring the content to be mirrored based on the target bitrate, where the target bitrate is lower than the original bitrate of the content to be mirrored. If not, encoding and wirelessly mirroring the content to be mirrored based on the original bitrate.

[0015] Based on the user's historical operation behaviors, it is possible to predict the probability that the user will switch the foreground application in the next period of time. If the probability that the user will switch the foreground application in the next step is high, it is more inclined not to perform bitrate adjustment, because after switching to the foreground application, if the bitrate reduction of the switched foreground application has a great impact on the user's subjective feeling, it is likely to reduce the user experience; if the probability that the user will switch the foreground application in the next step is low, it is more inclined to perform bitrate adjustment, because in a stable scenario, the impact of bitrate reduction on the user's perception is small, and bitrate reduction is not likely to reduce the user experience. Implementing dynamic bitrate adjustment according to the switching frequency of the foreground application can improve the user experience while reducing power consumption. In addition, the screen mirroring method in the embodiments of the present application does not need to wait until the temperature rises to the threshold before adjusting the bitrate, but realizes dynamic adjustment of the bitrate based on the switching frequency of the foreground application, which can improve the temperature rise of the screen mirroring source device due to screen mirroring, extend the time of temperature rise, and reduce the probability of abnormal conditions of the device due to heat generation.

[0016] In a possible implementation manner, determining whether to perform bitrate adjustment according to the frequency at which the current foreground application switches from the background to the foreground within a preset period includes: determining whether to perform bitrate adjustment according to the frequency at which the current foreground application switches from the background to the foreground within a preset period and the second application type weight, where the second application type weight is the application type weight corresponding to the target application that is switched to the foreground application within the preset period, and the target application is an application other than the current foreground application. Determining whether to perform bitrate adjustment by combining the switching frequency of the foreground application and the application type of the switched target application can further improve the user experience.

[0017] In a possible implementation manner, the application type weight corresponding to a video or game application > the application type weight corresponding to other applications.

[0018] In a possible implementation manner, determining whether to perform bitrate adjustment according to the frequency at which the current foreground application switches from the background to the foreground within a preset period and the second application type weight includes: determining a bitrate adjustment index according to the frequency at which the current foreground application switches from the background to the foreground within a preset period and the second application type weight. If the bitrate adjustment necessity characterized by the bitrate adjustment index meets the adjustment condition, it is determined to perform bitrate adjustment; if the bitrate adjustment necessity characterized by the bitrate adjustment index does not meet the adjustment condition, it is determined not to perform bitrate adjustment. The bitrate adjustment necessity characterized by the bitrate adjustment index is positively correlated with the second application type weight, and the bitrate adjustment necessity characterized by the bitrate adjustment index is negatively correlated with the above frequency. That the bitrate adjustment necessity characterized by the bitrate adjustment index meets the adjustment condition may mean that the bitrate adjustment index belongs to the first range, and that the bitrate adjustment necessity characterized by the bitrate adjustment index does not meet the adjustment condition may mean that the bitrate adjustment index belongs to the second range.

[0019] In a possible implementation, encoding and wirelessly projecting the content to be projected based on the target bit rate includes: determining a target resolution of the content to be projected based on the target bit rate, the target resolution of the content to be projected being lower than the original resolution of the content to be projected; encoding and wirelessly projecting the content to be projected based on the target resolution. This can reduce the bit rate while minimizing the adverse impact on user experience.

[0020] In a third aspect, an embodiment of the present application provides a screen projection method, including: obtaining projected content; determining a predicted bit rate of the content to be projected; determining a target bit rate of the content to be projected based on the predicted bit rate of the content to be projected and a perceived coding distortion curve, wherein the target bit rate of the content to be projected is less than the predicted bit rate; encoding and wirelessly projecting the content to be projected based on the target bit rate.

[0021] Determine the predicted bit rate of the content to be projected, and determine the target bit rate after the bit rate is reduced based on the predicted bit rate and the perceived coding distortion curve. Encode the content to be projected and wirelessly project based on the target bit rate. Since the reduced target bit rate is determined based on the predicted bit rate and the perceived coding distortion curve, the reduced target bit rate has a lower impact on user perception, and at the same time, power consumption is reduced. In other words, the embodiment of the present application can dynamically adjust the bit rate of the projected video without affecting the user's subjective experience as much as possible, thereby improving the user experience while reducing power consumption. In addition, the projection method in the embodiment of the present application does not need to wait until the temperature rises to a threshold before adjusting the bit rate, but dynamically adjusts the bit rate based on the projected content, which can improve the temperature rise of the projection source device caused by projection, prolong the temperature rise time, and reduce the probability of abnormalities in the device due to heat.

[0022] In a possible implementation, determining the predicted bit rate of the content to be projected includes determining the predicted bit rate of the content to be projected according to the picture complexity of the projected content.

[0023] In a possible implementation, the ordinate of the perceptual coding distortion curve is the image distortion rate perceived by the human eye, the abscissa of the perceptual coding distortion curve is the bit rate, the perceptual coding distortion curve has multiple steps, the width of the steps in the multiple steps is positively correlated with the bit rate, and the height of the steps in the multiple steps is negatively correlated with the bit rate; determining the target bit rate of the content to be projected based on the predicted bit rate of the content to be projected and the perceptual coding distortion curve includes: determining the step corresponding to the predicted bit rate of the content to be projected in the perceptual coding distortion curve; determining the bit rate on the left side of the predicted bit rate in the corresponding step as the target bit rate of the content to be projected. The width of the steps in the multiple steps is positively correlated with the bit rate, that is, under the condition of the same image quality, the higher the complexity of the image in the time domain and spatial domain, the stronger the visual masking effect. Therefore, if the predicted bit rate is large, a relatively large reduction in the bit rate can be based on the predicted bit rate as the target bit rate, and it can still ensure that the human eye cannot perceive the distortion caused by the reduction in the bit rate. The height of the steps in the multiple steps is negatively correlated with the bit rate, that is, under the condition of the same image quality, the higher the complexity of the image in the time domain and spatial domain, the smaller the impact on the human eye perception caused by reducing the bit rate. Dynamically adjusting the bit rate based on the above perceptual coding distortion curve can further improve the user experience.

[0024] In a possible implementation, determining the predicted bit rate of the content to be projected according to the complexity of the screen content of the projected content includes: determining the target screen complexity of the content to be projected according to the complexity of the screen content of the projected content; determining the predicted bit rate of the content to be projected according to the target screen complexity of the content to be projected. On the premise of the same image quality, there is a positive correlation between the complexity of the image and the bit rate required for the image. After obtaining the complexity of the screen content of the content to be projected, according to the correlation between the complexity of the screen content and the bit rate, the bit rate of the content to be projected can be determined.

[0025] In a possible implementation, determining the target screen complexity of the content to be projected according to the complexity of the screen content of the projected content includes: calculating the target screen complexity of the content to be projected according to the following formula: c N+1 is the target screen complexity of the content to be projected, N is the number of frames of the projected content, i is the serial number of the frame in the multiple frames of the projected content, the value of i is positively correlated with the time sequence of the multiple frames of the projected content, W i is the complexity weight coefficient of the i-th frame in the multiple frames of the projected content, C i is the screen complexity of the i-th frame in the multiple frames of the projected content; W i is positively correlated with i.

[0026] In a possible implementation, L i is the coding length corresponding to the i-th frame of the projected content, QP iis the encoding parameter corresponding to the content already cast on the i-th frame.

[0027] In a possible implementation manner, determining the predicted bitrate of the content to be cast according to the target picture complexity of the content to be cast includes: calculating the encoding length corresponding to the content to be cast according to the following formula; L N+1 is the encoding length corresponding to the content to be cast, and QP N+1 is the encoding parameter corresponding to the content to be cast, qp i is the weight coefficient of the encoding parameter of the i-th frame among multiple frames of content already cast, qp i is positively correlated with i, and C N+1 is the target picture complexity of the content to be cast; calculate the predicted bitrate of the content to be cast according to the following formula: Predicted bitrate = L N+1 × frame rate.

[0028] In a possible implementation manner, the above method further includes: obtaining a first application type weight, where the first application type weight is the application type weight corresponding to the current foreground application; determining whether to perform bitrate adjustment according to the bitrate adjustment parameter, where the bitrate adjustment parameter includes the first application type weight. If so, perform the process of encoding and wirelessly casting the content to be cast based on the target bitrate. If not, encode and wirelessly cast the content to be cast based on the original bitrate. According to the higher application type weight, it is easier to determine to perform bitrate adjustment. In this scenario, encoding and wirelessly casting based on the dynamically reduced bitrate can, on the one hand, reduce power consumption, and on the other hand, have a relatively small impact on the user's perception. For some application types, there may be scenarios with instability or low complexity. In these scenarios, reducing the bitrate has a greater impact on the user's perception. Therefore, it is easier to determine not to perform bitrate adjustment, that is, to encode and wirelessly cast the content to be cast based on the original bitrate.

[0029] In a possible implementation manner, the application type weight corresponding to a video or game application > the application type weight corresponding to other applications.

[0030] In a possible implementation, determining whether to perform bitrate adjustment according to the bitrate adjustment parameter includes: determining a bitrate adjustment index according to the bitrate adjustment parameter. If the necessity of bitrate adjustment characterized by the bitrate adjustment index meets the adjustment condition, it is determined to perform bitrate adjustment. If the necessity of bitrate adjustment characterized by the bitrate adjustment index does not meet the adjustment condition, it is determined not to perform bitrate adjustment. The necessity of bitrate adjustment characterized by the bitrate adjustment index is positively correlated with the first application type weight. The first application type weight is positively correlated with the power consumption benefit of reducing the bitrate in the scenario. By dynamically adjusting the screen projection bitrate based on the application type, the user experience can be improved. That the necessity of bitrate adjustment characterized by the bitrate adjustment index meets the adjustment condition may mean that the bitrate adjustment index belongs to the first range, and that the necessity of bitrate adjustment characterized by the bitrate adjustment index does not meet the adjustment condition may mean that the bitrate adjustment index belongs to the second range.

[0031] In a possible implementation, the bitrate adjustment parameter further includes the frequency at which the current foreground application switches from the background to the foreground within a preset period. Determining whether to perform bitrate adjustment in combination with the switching frequency of the foreground application can further improve the user experience.

[0032] In a possible implementation, the bitrate adjustment parameter further includes a second application type weight, which is the application type weight corresponding to the target application switched to the foreground application within a preset period, and the target application is an application other than the current foreground application. Determining whether to perform bitrate adjustment in combination with the switching frequency of the foreground application and the application type of the switched-to target application can further improve the user experience.

[0033] In a possible implementation, determining whether to perform bitrate adjustment according to the bitrate adjustment parameter includes: determining a bitrate adjustment index according to the bitrate adjustment parameter. If the necessity of bitrate adjustment characterized by the bitrate adjustment index meets the adjustment condition, it is determined to perform bitrate adjustment. If the necessity of bitrate adjustment characterized by the bitrate adjustment index does not meet the adjustment condition, it is determined not to perform bitrate adjustment. The necessity of bitrate adjustment characterized by the bitrate adjustment index is positively correlated with the first application type weight, the necessity of bitrate adjustment characterized by the bitrate adjustment index is positively correlated with the second application type weight, and the necessity of bitrate adjustment characterized by the bitrate adjustment index is negatively correlated with the switching frequency.

[0034] In a possible implementation, determining whether to perform bitrate adjustment according to the bitrate adjustment parameter includes: determining the bitrate adjustment index k according to the following formula: k = W app1 2 / (a × f / W app2 ), if k is greater than the threshold, it is determined to perform bitrate adjustment, if k is not greater than the threshold, it is determined not to perform bitrate adjustment, W app1 is the first application type weight, f is the frequency at which the current foreground application switches from the background to the foreground within a preset period, Wapp2 is the weight of the second application type, and a is a constant. Determining the bitrate adjustment index based on this formula can more accurately achieve dynamic adjustment of the bitrate.

[0035] In a possible implementation, encoding the content to be screen-cast and wireless screen-casting based on the target bitrate includes: determining the target resolution of the content to be screen-cast based on the target bitrate of the content to be screen-cast, where the target resolution of the content to be screen-cast is lower than the original resolution of the content to be screen-cast; encoding and wireless screen-casting the content to be screen-cast based on the target resolution. Keeping the quality of a single pixel point unchanged, determining the target resolution by reducing the resolution, so as to achieve bitrate adjustment. This way of reducing the bitrate results in a relatively small reduction in the picture quality displayed on the screen-casting target device.

[0036] In a fourth aspect, an electronic device is provided, including: a processor and a memory. The memory is used to store at least one instruction. When the instruction is loaded and executed by the processor, the electronic device executes the above-mentioned method.

[0037] In a fifth aspect, a computer-readable storage medium is provided, including a program or instruction. When the program or instruction runs on a computer, the above-mentioned method is executed.

[0038] In a sixth aspect, a computer program product is provided. The computer program product includes executable instructions. When the executable instructions are executed on a computer, the computer executes the above-mentioned method. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] To more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0040] Figure 1 Schematic diagrams of three screen-casting scenarios involved in the embodiments of the present application;

[0041] Figure 2 Schematic diagram of the structure of an electronic device involved in the embodiments of the present application;

[0042] Figure 3 A software structure block diagram of the electronic device in the embodiments of the present application;

[0043] Figure 4 A flowchart of a screen-casting method in the embodiments of the present application;

[0044] Figure 5 A schematic diagram of a perceived coding distortion curve in the embodiments of the present application;

[0045] Figure 6 It is a schematic flowchart of another screen mirroring method in the embodiments of the present application;

[0046] Figure 7 It is a schematic diagram showing the changes in resolution at different stages under three different comparative examples;

[0047] Figure 8 It is a schematic diagram of a resolution encoding relationship in the embodiments of the present application;

[0048] Figure 9 It is a schematic diagram of the temperature rise curve for comparison between the embodiments of the present application and other methods;

[0049] Figure 10 It is a schematic flowchart of another screen mirroring method in the embodiments of the present application;

[0050] Figure 11 It is a schematic flowchart of another screen mirroring method in the embodiments of the present application. Specific embodiments

[0051] To better understand the technical solution of the present application, the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0052] It should be clear that the described embodiments are only a part of the embodiments of the present application, rather than all of them. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the scope of protection of the present application.

[0053] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms "a", "the" and "said" used in the embodiments of the present application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise.

[0054] It should be understood that the term " / and" used herein is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " herein generally represents an "or" relationship between the associated objects before and after.

[0055] First, the application scenarios involved in the embodiments of the present application are described. The embodiments of the present application involve wireless screen mirroring scenarios, such as Figure 1 shown, for example, screen mirroring from a mobile phone to a computer, from a mobile phone to a TV, or from a mobile phone to a tablet, etc. The screen mirroring here can be mirror screen mirroring. The wireless screen mirroring scenario involves two types of electronic devices: a screen mirroring source device and a screen mirroring target device.

[0056] Figure 2 The structural schematic diagram of the electronic device 100 is shown.

[0057] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a wireless communication module 160, a display screen 194, an antenna 1, etc.

[0058] It can be understood that the structure schematically shown in the embodiments of the present invention does not constitute a specific limitation on the electronic device 100. In some other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0059] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors. The processor 110 may be used for functions such as audio and video stream encoding and decoding.

[0060] The controller may generate operation control signals according to the instruction operation code and timing signals to complete the control of fetching and executing instructions.

[0061] A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory may save the instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can be directly called from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0062] The wireless communication function of the electronic device 100 may be implemented through the antenna 1, the wireless communication module 160, the modem processor, the baseband processor, etc.

[0063] The wireless communication module 160 may provide solutions for wireless communications, such as wireless local area networks (WLANs) (e.g., wireless fidelity (Wi-Fi) networks), Bluetooth (BT), etc., applied to the electronic device 100. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 1, performs frequency modulation and filtering processing on the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 may also receive the signals to be sent from the processor 110, perform frequency modulation and amplification on them, and convert them into electromagnetic waves through the antenna 1 for radiation. The wireless communication module 160 includes, for example, a Bluetooth chip.

[0064] The electronic device 100 realizes the display function through the GPU, the display screen 194, the application processor, etc. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to execute mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change the display information.

[0065] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel may adopt a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include one or N display screens 194, where N is a positive integer greater than 1.

[0066] In some embodiments, the electronic device 100 may dispose the touch sensor on the display screen 194. The touch sensor and the display screen 194 form a touch screen, also known as a "touch control screen". The touch sensor is used to detect touch operations acting thereon or nearby. The touch sensor may transmit the detected touch operations to the application processor to determine the type of touch event. Visual output related to the touch operation may be provided through the display screen 194. In other embodiments, the touch sensor may also be disposed on the surface of the electronic device 100, at a different position from that of the display screen 194.

[0067] The internal memory 121 may be used to store computer-executable program code, and the executable program code includes instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.). The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.). In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121, and / or the instructions stored in the memory provided in the processor.

[0068] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservices architecture, or a cloud architecture. In the embodiments of the present invention, taking the Android system with a layered architecture as an example, the software structure of the electronic device 100 is exemplarily described.

[0069] Figure 3 It is a software structure block diagram of the electronic device 100 according to the embodiments of the present application.

[0070] The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, namely the application layer, the application framework layer, the Android runtime and the system libraries, and the kernel layer.

[0071] The application layer may include a series of application packages.

[0072] As Figure 3 shown, the application packages may include applications such as a camera, a gallery, a calendar, a call, a map, a navigation, a WLAN, a Bluetooth, music, a video, a short message, etc.

[0073] The application framework layer provides application programming interfaces (APIs) and programming frameworks for the applications in the application layer. The application framework layer includes some predefined functions.

[0074] Such as Figure 3 shown, the application framework layer may include a window manager, a content provider, a view system, a resource manager, etc.

[0075] The window manager is used to manage window programs. The content provider is used to store and obtain data, and make this data accessible to applications. The data may include videos, images, audio, dialed and answered calls, browsing history and bookmarks, phone books, etc.

[0076] The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build applications. The display interface can be composed of one or more views.

[0077] The resource manager provides various resources for applications, such as localized strings, icons, pictures, layout files, video files, and so on.

[0078] Android Runtime includes a core library and a virtual machine. Android runtime is responsible for the scheduling and management of the Android system.

[0079] The core library contains two parts: one part is the functional functions that need to be called by the Java language, and the other part is the core library of Android.

[0080] The application layer and the application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0081] The system library can include multiple functional modules. For example: surface manager, Media Libraries, 3D graphics processing library (e.g., OpenGL ES), 2D graphics engine (e.g., SGL), etc.

[0082] The surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications.

[0083] The media library supports the playback and recording of multiple common audio and video formats, as well as static image files, etc. The media library can support multiple audio and video coding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0084] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing, etc.

[0085] The 2D graphics engine is a drawing engine for 2D drawing.

[0086] The kernel layer is the layer between hardware and software. The kernel layer at least includes a display driver, a Bluetooth driver, an audio driver, and a sensor driver.

[0087] The electronic device involved in the embodiments of this application may be any product such as a smart TV, mobile phone, tablet computer, personal computer (PC), personal digital assistant (PDA), smart watch, wearable electronic device, augmented reality (AR) device, virtual reality (VR) device, vehicle-mounted device, drone device, smart car, smart speaker, robot, smart glasses, etc.

[0088] The above-mentioned electronic device may be a screen mirroring source device or a screen mirroring target device in a wireless screen mirroring scenario.

[0089] The embodiments of this application provide a screen mirroring method, which is applied to a screen mirroring source device, such as Figure 4 As shown, the screen mirroring method includes:

[0090] Step 101: Perform wireless screen mirroring;

[0091] Before performing wireless screen mirroring, a wireless connection is established between the screen mirroring source device and the screen mirroring target device. During the process of wireless screen mirroring, the screen mirroring source device captures its own display screen, which is the screen mirroring content, encodes the screen mirroring content, and then sends the encoded screen mirroring content to the screen mirroring target device through the wireless connection. After receiving the screen mirroring content, the screen mirroring target device decodes and plays the screen mirroring content, that is, wireless screen mirroring is achieved. At the beginning stage of screen mirroring, the screen mirroring source device can encode and wirelessly screen mirror the screen mirroring content based on the original bit rate.

[0092] Step 102: Obtain the screen mirroring content that has been mirrored;

[0093] During the screen mirroring process, the screen mirroring source device can obtain the screen mirroring content that has been mirrored in the recent period of time, for example, obtain the screen mirroring content that has been mirrored in the recent 1 minute.

[0094] Step 103: Determine the predicted bitrate of the content to be projected based on the complexity of the projected content's frame

[0095] The projected content is frame-by-frame images. Based on any frame image, its corresponding complexity can be determined. In a video, the frame images are continuous, and there is a strong correlation between adjacent or temporally close frame images. The image content is usually similar, so the complexity between images is also relatively close and usually does not change suddenly. Therefore, the bitrate of the content to be projected can be predicted based on the complexity of the most recently projected content's image, that is, determine the predicted bitrate of the content to be projected

[0096] It can be understood that determining the predicted bitrate of the content to be projected based on the complexity of the projected content's frame is just one implementation method. In other possible implementation methods, the predicted bitrate of the content to be projected can also be determined by other means. For example, the predicted bitrate of the content to be projected can be determined by the actual bitrate of the projected content. For example, the average value of the actual bitrates of the projected content over a period of time is statistically calculated and used as the predicted bitrate of the content to be projected

[0097] Step 104: Determine the target bitrate of the content to be projected based on the predicted bitrate of the content to be projected and the perceptual coding distortion curve. The target bitrate of the content to be projected is less than the predicted bitrate

[0098] Among them, the perceptual coding distortion curve is a curve or mapping relationship preset according to the relationship between the image distortion rate perceived by the human eye and the image bitrate. For example, as Figure 5 shown, which shows the relationship between the statistical coding distortion curve and the perceptual coding distortion curve. The statistical coding distortion curve refers to the corresponding relationship curve between the theoretical image distortion rate and the bitrate. It can be seen that from the perspective of the computer, the correlation between the image distortion rate and the bitrate is manifested as a continuous curve; the perceptual coding distortion curve refers to the corresponding relationship curve between the image distortion rate perceived by the human eye and the bitrate. For the perception of the human eye, affected by the visual masking effect, the correlation between the image distortion rate perceived by the human eye and the bitrate is manifested as a broken line. That is to say, the distortion rate perceived by the human eye will only be perceived by the human eye after the distortion of the bitrate accumulates to a certain threshold. That is to say, the perceptual coding distortion curve has multiple steps. The change in the image bitrate corresponding to each step will not correspond to the change in the image distortion rate. Only the change in the bitrate between different steps will cause a sudden change in the image distortion rate. In step 103, determine the position of the predicted bitrate of the content to be projected in the perceptual coding distortion curve. Assume that the predicted bitrate of the content to be projected is Figure 5A in it is located at the bottommost step of the perceptual coding distortion curve. Since all the bitrates on this step correspond to the same image distortion rate perceptible to the human eye, the bitrate to the left of A on this step can be set as the target bitrate of the content to be screen-cast. For example, set A' as the target bitrate of the content to be screen-cast. The smaller the target bitrate A' is, the better the power consumption reduction effect. However, the target bitrate A' cannot be equal to or less than the bitrate s1 corresponding to the rising edge on the left side of this step. Because on the same platform, the image distortion rate perceptible to the human eye remains unchanged, that is, users will not perceive the distortion caused by the reduction of the bitrate. Therefore, s1 < A' < A. The closer A' is to the bitrate s1 corresponding to the rising edge on the left side of this step, the better the bitrate reduction effect. Assume that the predicted bitrate of the content to be screen-cast is Figure 5 For B in it, the bitrate corresponding to the rising edge on the left side of the step where it is located is s2, then the bitrate B' between s2 and B can be set as the target bitrate of the content to be screen-cast.

[0099] Step 105: Encode and wirelessly screen-cast the content to be screen-cast based on the target bitrate.

[0100] Among them, the target bitrate is less than the predicted bitrate. That is to say, the bitrate of the content to be screen-cast is reduced. In this way, during the wireless screen-casting process, the image bitrate transmitted wirelessly is lower, that is, the amount of data transmitted is lower, thus reducing the power consumption. At the same time, since the reduction of the target bitrate is determined based on the perceptual coding distortion curve, the impact of the bitrate reduction on the human eye perception is smaller.

[0101] After step 105, step 102 can be executed again. That is to say, during the wireless screen-casting process, steps 102 to 105 are executed periodically. Take a specific scenario as an example for illustration. For example, during a wireless screen-casting process, the first to Nth frame images are encoded and wirelessly screen-cast at the default bitrate. Then, in step 103, based on the complexity of the first to Nth frame images that have been screen-cast, the predicted bitrate of the (N + 1)th frame image to be screen-cast is determined. In step 104, the target bitrate of the (N + 1)th frame image is determined according to the predicted bitrate of the (N + 1)th frame image. In step 105, the (N + 1)th frame image is encoded and wirelessly screen-cast according to the target bitrate; then based on the complexity of the second to (N + 1)th frame images that have been screen-cast, the predicted bitrate of the (N + 2)th frame image to be screen-cast is determined, and the target bitrate of the (N + 2)th frame image is determined, and the (N + 2)th frame image is encoded and wirelessly screen-cast according to the target bitrate; and so on. Then based on the complexity of the third to (N + 3)th frame images that have been screen-cast, the predicted bitrate of the (N + 3)th frame image to be screen-cast is determined, and the target bitrate of the (N + 3)th frame image is determined, and the (N + 3)th frame image is encoded and wirelessly screen-cast according to the target bitrate. In this way, during the wireless screen-casting process, the bitrate can be dynamically adjusted based on the screen-cast content.

[0102] In the screen mirroring method according to the embodiments of the present application, a predicted bitrate of the content to be screen mirrored is obtained based on the content that has been screen mirrored, and a target bitrate after bitrate reduction is determined according to the predicted bitrate and the perceptual coding distortion curve. The content to be screen mirrored is encoded and wirelessly screen mirrored based on the target bitrate. Since the target bitrate after reduction is determined according to the predicted bitrate and the perceptual coding distortion curve, the impact of the reduced target bitrate on user perception is low, and power consumption is reduced at the same time. That is to say, the embodiments of the present application can dynamically adjust the bitrate of the screen mirrored video on the premise of minimizing the impact on the user's subjective experience, improving the user experience while reducing power consumption. In addition, the screen mirroring method in the embodiments of the present application does not need to wait until the temperature rises to the threshold value before adjusting the bitrate, but dynamically adjusts the bitrate based on the content that has been screen mirrored, which can improve the temperature rise of the screen mirroring source device due to screen mirroring, extend the time of temperature rise, and reduce the probability of device anomalies caused by overheating.

[0103] In a possible implementation, as Figure 5 shown, the vertical coordinate of the perceptual coding distortion curve is the image distortion rate perceived by the human eye, the horizontal coordinate of the perceptual coding distortion curve is the bitrate, and the perceptual coding distortion curve has multiple steps. The above step 104 of determining the target bitrate of the content to be screen mirrored according to the predicted bitrate of the content to be screen mirrored and the perceptual coding distortion curve includes: determining the step corresponding to the predicted bitrate of the content to be screen mirrored in the perceptual coding distortion curve; determining the bitrate on the left side of the predicted bitrate in the corresponding step as the target bitrate of the content to be screen mirrored, and the bitrate on the left side of the predicted bitrate in the corresponding step does not include the rising edge on the left side of the step. Among them, the width of the step in the multiple steps is positively correlated with the bitrate. That is, under the same image quality, the higher the complexity of the image in the time domain and spatial domain, the stronger the visual masking effect. Therefore, if the predicted bitrate is large, a larger reduction in the bitrate can be based on the predicted bitrate as the target bitrate, and it can still ensure that the human eye cannot perceive the distortion caused by the bitrate reduction. The height of the step in the multiple steps is negatively correlated with the bitrate. That is, under the same image quality, the higher the complexity of the image in the time domain and spatial domain, the smaller the impact on human eye perception caused by reducing the bitrate. For example, if the screen mirrored content is a video and the complexity of video and film content is relatively high, in this scenario, even if the bitrate is reduced more, it is not easy for users to perceive the picture distortion; if the screen mirrored content is the chat interface of an instant messaging application and the complexity of the chat interface is relatively low, in this scenario, the picture distortion caused by the reduction of the bitrate is easily perceived by users. Therefore, dynamically adjusting the bitrate based on the above perceptual coding distortion curve can further improve the user experience.

[0104] The following uses a specific example to illustrate the method of determining the target bitrate of the content to be projected onto the screen as the bitrate on the left side of the predicted bitrate in the corresponding step. For example, a positive number less than 1 is set as the target bitrate adjustment coefficient, such as 0.8. When determining the target bitrate of the content to be projected onto the screen, the predicted bitrate of the content to be projected onto the screen is directly multiplied by the coefficient 0.8 as the target bitrate.

[0105] In a possible implementation manner, step 103 above, determining the predicted bitrate of the content to be projected onto the screen according to the picture complexity of the content already projected onto the screen includes: step 1031 and step 1032. Step 1031, determining the target picture complexity of the content to be projected onto the screen according to the picture complexity of the content already projected onto the screen. For example, in the content already projected onto the screen in the most recent 1 minute, one frame of image is obtained per second, 60 frames of images are obtained, and the picture complexity of the content to be projected onto the screen is predicted based on the picture complexity of these 60 frames of images; for another example, in the content already projected onto the screen in the most recent 1 second, each frame of image is obtained according to the frame display rate. Assuming the frame rate is 60 Hz, then 60 consecutive frames of images that have been most recently projected are obtained, and the picture complexity of the content to be projected onto the screen is predicted based on the picture complexity of these 60 frames of images. Step 1032, determining the predicted bitrate of the content to be projected onto the screen according to the target picture complexity of the content to be projected onto the screen. On the premise of the same image quality, there is a positive correlation between the complexity of the image and the bitrate required for the image. After obtaining the picture complexity of the content to be projected onto the screen, according to the correlation between the picture complexity and the bitrate, the bitrate of the content to be projected onto the screen can be determined, and this bitrate is referred to as the predicted bitrate in the embodiments of the present application.

[0106] In a possible implementation manner, step 1031 above, determining the target picture complexity of the content to be projected onto the screen according to the picture complexity of the content already projected onto the screen includes: calculating the target picture complexity of the content to be projected onto the screen according to the following formula: C N+1 is the target picture complexity of the content to be projected onto the screen, N is the number of frames of the content already projected onto the screen in multiple frames, i is the serial number of the frame in multiple frames of the content already projected onto the screen, the value of i is positively correlated with the time series of multiple frames of the content already projected onto the screen, W i is the complexity weight coefficient of the i-th frame in multiple frames of the content already projected onto the screen, C i is the picture complexity of the i-th frame in multiple frames of the content already projected onto the screen; W i is positively correlated with i.

[0107] Specifically, the multi-frame already-screened content can be selected from the most recently screened content, and the target picture complexity of the content to be screened is determined based on the picture complexity of the selected multi-frame already-screened content. In one embodiment, for example, all 60 frames of the already-screened content in the most recent 1 second can be used as the multi-frame already-screened content on which step 1031 is based, N = 60, and the value of i is 1, 2, 3, …, 60. The value of i is positively correlated with the time sequence of the multi-frame already-screened content. That is to say, the larger the value of i, the later the time in the corresponding 60 frames, and C N+1 and C N can correspond to two adjacent frame pictures. The frame pictures closer to the content to be screened have a higher correlation with the content to be screened. Therefore, the picture complexity of the content to be screened can be determined based on multiple frames of already-screened content adjacent to the content to be screened. For this reason, in the calculation formula, the complexity weight coefficient with a larger i is set to be larger to improve the prediction accuracy of the picture complexity of the content to be screened. In another embodiment, for example, 1 frame can be selected every 1 second from the 360 frames of already-screened content in the most recent 1 minute, and 60 of them can be used as the multi-frame already-screened content on which step 1031 is based.

[0108] In a possible implementation manner, L i is the coding length corresponding to the i-th frame of already-screened content, and QP i is the coding parameter corresponding to the i-th frame of already-screened content, and QP i is usually a natural number from 0 to 51. The larger the value, the lower the image quality. The values corresponding to different frame pictures may be different, but QP i is a known parameter.

[0109] In addition, in addition to the method of determining the picture complexity of the already-screened content through the above formula, the picture complexity of the already-screened content can also be determined by other methods. For example, the picture complexity can be determined by the color distribution of the already-screened content, and the color distribution may be related to picture parameters such as the variance or standard deviation of RGB pixels. In the embodiments of the present application, the determination of the picture complexity of the already-screened content based on the formula of C i is used as an example for illustration.

[0110] In a possible implementation manner, the above step 1032, determining the predicted bit rate of the content to be screened according to the target picture complexity of the content to be screened, includes: calculating the coding length corresponding to the content to be screened according to the following formula; L N+1 is the coding length corresponding to the content to be screened, and QP N+1 is the coding parameter corresponding to the content to be screened, qp iis the weight coefficient of the encoding parameter of the i-th frame in the multi-frame content that has been projected onto the screen. qp i is positively correlated with i, and C N+1 is the target picture complexity of the content to be projected onto the screen; calculate the predicted bitrate of the content to be projected onto the screen according to the following formula: Predicted bitrate = L N+1 × frame rate, and the frame rate can be the screen data acquisition frequency of the content that has been projected onto the screen on the projection source device.

[0111] In a possible implementation manner, as Figure 6 shown, the above method further includes: Step 106, obtain the first application type weight, where the first application type weight is the application type weight corresponding to the current foreground application; Step 107, determine whether to perform bitrate adjustment according to the bitrate adjustment parameter, where the bitrate adjustment parameter includes the first application type weight. If so, execute Step 105, the process of encoding and wirelessly projecting the content to be projected onto the screen based on the target bitrate. If not, execute Step 108, encode and wirelessly project the content to be projected onto the screen based on the original bitrate. After Step 105, Step 106 can be executed again to determine whether to perform bitrate adjustment in the next cycle, that is, to achieve periodic dynamic bitrate adjustment. After Step 108, Step 106 can be executed again to determine whether to perform bitrate adjustment in the next cycle, that is, to achieve periodic dynamic bitrate adjustment.

[0112] Specifically, it can be determined whether to reduce the projection bitrate according to the current foreground application type. The first application type weight is used to represent the value of the current foreground application for bitrate adjustment. The greater the weight, the greater the possibility of determining to perform bitrate adjustment in Step 107. The current foreground application corresponds to the scenario of the content to be projected onto the screen. If in a relatively stable scenario, that is, the continuity of the image is relatively good and the probability of application switching is relatively low, then the application type weight corresponding to this application is relatively high; if in a scenario with a relatively high complexity, then the first application type weight corresponding to this application is relatively high, because the higher the bitrate, the more complex the scenario. According to Figure 5 shown, the higher the bitrate, the smaller the impact on the user when reducing the bitrate, that is, the greater the power consumption benefit of performing bitrate adjustment. Therefore, the first application type weight corresponding to this application is relatively high. According to the relatively high first application type weight, it is easier to determine to perform bitrate adjustment in Step 107. In this scenario, encoding and wirelessly projecting based on the dynamically reduced bitrate can, on the one hand, reduce power consumption, and on the other hand, have a relatively small impact on the user's perception. For some application types, there may be unstable or less complex scenarios. In these scenarios, reducing the bitrate has a relatively large impact on the user's perception. Therefore, it is easier to determine not to perform bitrate adjustment in Step 107, that is, to encode and wirelessly project the content to be projected onto the screen based on the original bitrate in Step 108.

[0113] It should be noted that in Figure 6 In the process shown, when it is determined to be in step 107, steps 102 to 105 are executed, that is, after determining the bitrate adjustment, the target bitrate is determined. However, the embodiments of the present application do not limit the order between steps 102 to 104 and step 107. That is to say, in other possible embodiments, the target bitrate can also be determined before determining whether to perform bitrate adjustment or during the process of determining bitrate adjustment. For whether to perform encoding and wireless screen mirroring based on the target bitrate, it needs to be triggered according to the result determined in step 107.

[0114] In a possible implementation manner, the bitrate is related to the resolution, and the bitrate is related to the quality of each pixel. The bitrate is approximately equal to the resolution × the quality of each pixel. Therefore, for the adjustment of the bitrate, the data amount of each pixel can be reduced while keeping the resolution unchanged, that is, the quality of each pixel is reduced. That is to say, step 105 includes determining the target quality of each pixel corresponding to the content to be screen mirrored based on the target bitrate of the content to be screen mirrored. The target quality of each pixel is less than the original quality of the corresponding pixel. Keep the resolution unchanged, and encode and wirelessly screen mirror the content to be screen mirrored based on the target quality of each pixel and the unchanged resolution. It can be understood that in other possible ways, the quality of each pixel and the resolution can also be reduced at the same time, and the content to be screen mirrored is encoded and wirelessly screen mirrored based on the reduced quality of each pixel and the reduced resolution.

[0115] In a possible implementation manner, the above step 105, encoding and wirelessly screen mirroring the content to be screen mirrored based on the target bitrate includes: determining the target resolution of the content to be screen mirrored based on the target bitrate of the content to be screen mirrored, and the target resolution of the content to be screen mirrored is lower than the original resolution of the content to be screen mirrored; encoding and wirelessly screen mirroring the content to be screen mirrored based on the target resolution.

[0116] Specifically, in most screen mirroring scenarios, the resolution of the picture captured by the screen mirroring source device is relatively high, and when it is displayed as the screen mirroring target device, the received data will be downsampled and then sent for display. Therefore, the resolution displayed by the screen mirroring target device is relatively low. For example, as Figure 7 shown Figure 7The size of the middle rectangle represents the resolution. The larger the rectangle, the higher the resolution. In Comparative Example 1, the resolution of the screen-casting source device's captured image is relatively high. Without changing the resolution, after encoding, transmission, and decoding, the resolution of the image displayed on the screen-casting target device is relatively low. In Comparative Example 2, the resolution of encoding, transmission, and decoding is lower than the resolution of the captured image, but the resolution of the image finally displayed on the screen-casting target device is even lower. In Comparative Example 3, the resolution of encoding, transmission, and decoding is the same as the resolution displayed on the screen-casting target device. In this way, the power consumption during the encoding, transmission, and decoding processes is the lowest, but the display effect is the same as that of the other two comparative examples. Therefore, in the embodiments of the present application, while keeping the quality of a single pixel point unchanged, the target resolution is determined by reducing the resolution to achieve bitrate adjustment. This way of reducing the bitrate results in a relatively small reduction in the quality of the image displayed on the screen-casting target device. Due to encoding limitations, there is a corresponding relationship between the encoding ratio and the resolution ratio under different encoding parameter QPs, as shown in Table 1 and Figure 8 as shown.

[0117]

[0118] Table 1 is a correspondence table between the resolution and the bitrate, Figure 8 is a schematic diagram of the relationship between the resolution and the bitrate corresponding to Table 1. Among them, both the bitrate and the resolution are expressed as ratios in percentage form. Table 1 and Figure 8 in addition to indicating the corresponding relationship between the resolution ratio and the bitrate ratio, also indicates the corresponding relationship between two reference ratios and the bitrate ratio. Among them, the single-sided ratio refers to the single-sided ratio of pixels. The corresponding relationship between the pixel area ratio and the bitrate ratio can be equivalently understood as the relationship between the ideal resolution and the bitrate. However, due to encoding limitations, the actual corresponding relationship between the resolution ratio and the bitrate is related to the encoding parameter QP. The relationship between the resolution ratio and the bitrate ratio under different encoding parameter QPs can be pre-stored in the screen-casting source device. During the process of determining the target resolution of the content to be screen-cast based on the target bitrate of the content to be screen-cast, the ratio of the reduced target bitrate to the original bitrate can be determined, that is, the reduced bitrate ratio. According to the relationship between the resolution ratio and the bitrate ratio corresponding to the current encoding parameter QP, the resolution ratio corresponding to the reduced bitrate ratio can be determined. Based on this resolution ratio, the corresponding target resolution can be determined, and then the content to be screen-cast can be encoded and wirelessly screen-cast based on this target resolution.

[0119] As shown in Table 2 and Figure 9 as shown.

[0120] Table 2

[0121]

[0122] Table 2 shows the temperature test data of the electronic device under three scenarios. Figure 9 Fig. Figure 9 shows the temperature rise curves of the electronic device under three scenarios. It can be seen that the temperature rises faster in the wireless screen mirroring scenario than in the local video playback scenario. And for the wireless screen mirroring method of the embodiment of the present application, the temperature rises slower than that of the wireless screen mirroring method without changing the bit rate. That is to say, the embodiment of the present application can improve the temperature rise of the screen mirroring source device caused by screen mirroring and extend the time of temperature rise.

[0123] As Figure 10 shown, the embodiment of the present application also provides a screen mirroring method, including:

[0124] Step 201, obtain a first application type weight, where the first application type weight is the application type weight corresponding to the current foreground application;

[0125] Step 202, determine whether to perform bit rate adjustment according to the bit rate adjustment parameter, where the bit rate adjustment parameter includes the first application type weight. If so, execute Step 203, encode and wirelessly screen mirror the content to be screen mirrored based on the target bit rate, where the target bit rate is lower than the original bit rate of the content to be screen mirrored. If not, execute Step 204, encode and wirelessly screen mirror the content to be screen mirrored based on the original bit rate.

[0126] Specifically, the execution subject of this screen mirroring method can be the screen mirroring source device. Step 201 can be the same as Step 106 in the above embodiment, Step 202 can be the same as Step 107 above, Step 204 can be the same as Step 108 above, and Step 203 can be similar to Step 105 above. The difference is that Step 203 does not limit the determination method of the target bit rate. The target bit rate determination method in the above embodiment can be applied, or other possible target bit rate determination methods can be applied, as long as the target bit rate is lower than the original bit rate.

[0127] In the screen mirroring method of the embodiments of the present application, the current foreground application corresponds to the scene of the content to be screen mirrored. If the scene is relatively stable, that is, the continuity of the image is relatively good and the probability of application switching is relatively low, then the application type weight corresponding to this application is relatively high; if the scene has a higher complexity, then the application type weight corresponding to this application is relatively high, because the higher the bit rate, the more complex the scene is, and the higher the bit rate, the smaller the impact on the user when reducing the bit rate, that is, the greater the power consumption benefit of bit rate adjustment. Therefore, the application type weight corresponding to this application is relatively high. According to the relatively high application type weight, it is easier to determine to perform bit rate adjustment. In this scenario, encoding and wireless screen mirroring are performed based on the reduced bit rate. On the one hand, power consumption can be reduced, and on the other hand, the impact on the user's perception is relatively small. For some application types, there may be unstable or less complex scenes. In these scenes, reducing the bit rate has a greater impact on the user's perception. Therefore, it is easier to determine not to perform bit rate adjustment, that is, encoding and wireless screen mirroring are performed on the content to be screen mirrored based on the original bit rate. In addition, the screen mirroring method in the embodiments of the present application does not need to wait until the temperature rises to the threshold before adjusting the bit rate, but dynamically adjusts the bit rate based on the foreground application type, which can improve the temperature rise of the screen mirroring source device due to screen mirroring, extend the time of temperature rise, and reduce the probability of device abnormality due to heat generation.

[0128] In a possible implementation manner, as Figure 6 and Figure 10 shown, step 107 or step 202, determining whether to perform bit rate adjustment according to the bit rate adjustment parameter includes: if the first application type weight is greater than the preset weight, determining to perform bit rate adjustment; if the first application type weight is not greater than the preset weight, determining not to perform bit rate adjustment. That is to say, in step 107 or 202, it is possible to directly determine whether to perform bit rate adjustment according to the first application type weight.

[0129] In a possible implementation manner, as Figure 6 and Figure 10 shown, the application type weight corresponding to the video or game application > the application type weight corresponding to other applications; the above step 107 or step 202, determining whether to perform bit rate adjustment according to the bit rate adjustment parameter includes: determining a bit rate adjustment index according to the first application type weight. If the necessity of bit rate adjustment characterized by the bit rate adjustment index meets the adjustment condition, determining to perform bit rate adjustment; if the necessity of bit rate adjustment characterized by the bit rate adjustment index does not meet the adjustment condition, determining not to perform bit rate adjustment. The necessity of bit rate adjustment characterized by the bit rate adjustment index is positively correlated with the first application type weight. The first application type weight is positively correlated with the power consumption benefit of reducing the bit rate in the scene. By dynamically adjusting the screen mirroring bit rate based on the application type, the user experience can be improved.

[0130] Specifically, the bitrate adjustment index is correlated with the weight of the first application type. For example, there may be a positive correlation between the two. In this case, if the bitrate adjustment index is greater than the adjustment threshold, that is, the necessity of bitrate adjustment represented by the bitrate adjustment index meets the adjustment condition, it is determined to perform bitrate adjustment. If the bitrate adjustment index is not greater than the adjustment threshold, that is, the necessity of bitrate adjustment represented by the bitrate adjustment index does not meet the adjustment condition, it is determined not to perform bitrate adjustment. There may be a negative correlation between the bitrate adjustment index and the weight of the first application type. In this case, if the necessity of bitrate adjustment represented by the bitrate adjustment index is not greater than the adjustment threshold, that is, the necessity of bitrate adjustment represented by the bitrate adjustment index meets the adjustment condition, it is determined to perform bitrate adjustment. If the bitrate adjustment index is greater than the adjustment threshold, that is, the necessity of bitrate adjustment represented by the bitrate adjustment index does not meet the adjustment condition, it is determined not to perform bitrate adjustment. For video applications and game applications, the frame rate is usually high and high-complexity images are likely to appear. The frame rate is usually stable at 30, 60, or 120 fps, and the correlation between consecutive images is relatively large. When these types of applications are projected as the foreground, the original power consumption is relatively large, and the power consumption benefit of reducing the bitrate is obvious. Therefore, the application type weight corresponding to video or game applications can be set to 1, that is, it has the highest priority in determining whether to perform bitrate adjustment. For other applications other than video and games, since the probability of high-complexity images appearing is relatively low, a lower application type weight can be set. For other applications, further classification can be carried out. For example, when playing a song, the music application will dynamically display lyrics, and the frame rate is usually stable at 30 or 60 fps. The image complexity is relatively small compared to games or videos, and the power consumption benefit of reducing the bitrate is medium and relatively stable. Therefore, the application type weight corresponding to the music application can be set to 0.5, that is, it has a medium priority in determining whether to perform bitrate adjustment. For other applications other than the music application, it refers to applications other than video applications, game applications, and music applications, such as instant messaging applications or memo applications. Taking the instant messaging application as an example, when in a static window, its frame rate is usually less than 5 fps, and the projection bitrate is relatively low. When performing functions such as sliding the window or refreshing other interfaces with pictures or videos, the instantaneous frame rate may quickly rise to 60 or 120 fps. It is not easy to match the response speed of reducing the bitrate with the change speed of the frame rate and bitrate of this application, and it is easy to affect the user's subjective feeling due to the reduction of the bitrate. Moreover, the overall bitrate of this type of application is relatively low, and the power consumption benefit of reducing the bitrate is not obvious. Therefore, the lowest application type weight is set. For example, the application type weight corresponding to other applications other than the music application can be set to 0. The weight of the first application type is positively correlated with the power consumption benefit of reducing the bitrate in the scenario. By dynamically adjusting the projection bitrate based on the application type, the user experience can be improved.

[0131] In a possible implementation, as Figure 6 and Figure 10As shown, the bitrate adjustment parameter further includes the frequency at which the current foreground application switches from the background to the foreground within a preset period. That is to say, in step 107 or 202, it is possible to determine whether to perform bitrate adjustment based on the first application type weight and the frequency at which the current foreground application switches from the background to the foreground within a preset period.

[0132] Specifically, the preset period is a recent period of time, such as the most recent 1 minute. The frequency at which the current foreground application switches from the background to the foreground within the preset period can characterize the user's historical operation behavior, and user operations usually have continuity and relevance. For example, some users are accustomed to switching to an instant messaging application to reply to messages while watching a video, while some users rarely switch applications while watching a video. Therefore, based on the user's historical operation behavior, it is possible to predict the probability that the user will switch the foreground application in the next period of time. If the probability that the user will switch the foreground application in the next step is high, it is more inclined not to perform bitrate adjustment, because if the bitrate of the foreground application after the switch decreases, it will have a great impact on the user's subjective feeling and easily reduce the user experience; if the probability that the user will switch the foreground application in the next step is low, it is more inclined to perform bitrate adjustment, because in a stable scenario, the reduction of the bitrate has a smaller impact on the user's perception, and the reduction of the bitrate is not likely to reduce the user experience. For example, if there is a foreground application switch within the most recent minute, then in the next 10 minutes, step 204 is executed and the bitrate is not reduced. Then, step 202 is executed again to determine whether to perform bitrate adjustment in the next cycle, that is, to achieve periodic dynamic bitrate adjustment; if there is no foreground application switch within the most recent minute, then in the next 10 minutes, step 203 is executed to reduce the bitrate and perform screen mirroring. Then, step 202 is executed again to determine whether to perform bitrate adjustment in the next cycle, that is, to achieve periodic dynamic bitrate adjustment. Determining whether to perform bitrate adjustment in combination with the switching frequency of the foreground application can further improve the user experience.

[0133] In a possible implementation manner, as Figure 6 and Figure 10 shown, the bitrate adjustment parameter further includes a second application type weight, and the second application type weight is the application type weight corresponding to the target application that is switched to the foreground application within the preset period, and the target application is an application other than the current foreground application.

[0134] Specifically, if a user frequently switches to an instant messaging application to reply to messages while watching a video in a recent period of time, it indicates that in the next period of time, the user has a relatively high probability of switching to the instant messaging application to reply to messages while watching the video. Since the instant messaging application and the video application have different application types, the impact of reducing the bitrate on the user experience also varies. Therefore, the necessity of bitrate adjustment is not only related to the switching frequency but also related to the application type weight corresponding to the target application. For example, if the target application that the user switches to in a recent period of time is a video application, its corresponding application type weight is relatively large, so it is more inclined to perform bitrate adjustment; if the target application that the user switches to in a recent period of time is, for example, an instant messaging application, its corresponding application type weight is relatively small, so it is more inclined not to perform bitrate adjustment. Determining whether to perform bitrate adjustment by combining the switching frequency of the foreground application and the application type of the switched-to target application can further improve the user experience.

[0135] In a possible implementation, as Figure 6 and Figure 10 shown, step 107 or 202, determining whether to perform bitrate adjustment according to the bitrate adjustment parameter includes: determining a bitrate adjustment index according to the first application type weight and the frequency at which the current foreground application switches from the background to the foreground within a preset period. If the necessity of bitrate adjustment characterized by the bitrate adjustment index meets the adjustment condition, it is determined to perform bitrate adjustment; if the necessity of bitrate adjustment characterized by the bitrate adjustment index does not meet the adjustment condition, it is determined not to perform bitrate adjustment. The necessity of bitrate adjustment characterized by the bitrate adjustment index is positively correlated with the first application type weight, and the necessity of bitrate adjustment characterized by the bitrate adjustment index is negatively correlated with the above frequency.

[0136] In a possible implementation, as Figure 6 and Figure 10 shown, step 107 or 202, determining whether to perform bitrate adjustment according to the bitrate adjustment parameter includes: determining a bitrate adjustment index according to the first application type weight and the second application type weight. If the necessity of bitrate adjustment characterized by the bitrate adjustment index meets the adjustment condition, it is determined to perform bitrate adjustment; if the necessity of bitrate adjustment characterized by the bitrate adjustment index does not meet the adjustment condition, it is determined not to perform bitrate adjustment. The necessity of bitrate adjustment characterized by the bitrate adjustment index is positively correlated with the first application type weight, and the necessity of bitrate adjustment characterized by the bitrate adjustment index is positively correlated with the second application type weight.

[0137] In a possible implementation, as Figure 6 and Figure 10As shown, step 107 or 202, determining whether to perform bitrate adjustment according to bitrate adjustment parameters includes: determining a bitrate adjustment index according to the second application type weight and the frequency at which the current foreground application switches from the background to the foreground within a preset period. If the necessity of bitrate adjustment characterized by the bitrate adjustment index meets the adjustment condition, it is determined to perform bitrate adjustment. If the necessity of bitrate adjustment characterized by the bitrate adjustment index does not meet the adjustment condition, it is determined not to perform bitrate adjustment. The necessity of bitrate adjustment characterized by the bitrate adjustment index is positively correlated with the second application type weight, and the necessity of bitrate adjustment characterized by the bitrate adjustment index is negatively correlated with the frequency.

[0138] In a possible implementation, as Figure 6 and Figure 10 shown, step 107 or 202, determining whether to perform bitrate adjustment according to bitrate adjustment parameters includes: determining a bitrate adjustment index according to the first application type weight, the second application type weight, and the frequency at which the current foreground application switches from the background to the foreground within a preset period. If the necessity of bitrate adjustment characterized by the bitrate adjustment index meets the adjustment condition, it is determined to perform bitrate adjustment. If the necessity of bitrate adjustment characterized by the bitrate adjustment index does not meet the adjustment condition, it is determined not to perform bitrate adjustment. The necessity of bitrate adjustment characterized by the bitrate adjustment index is positively correlated with the first application type weight, the necessity of bitrate adjustment characterized by the bitrate adjustment index is positively correlated with the second application type weight, and the necessity of bitrate adjustment characterized by the bitrate adjustment index is negatively correlated with the frequency.

[0139] Specifically, whether it is the current foreground application or the target application that has been recently switched, both may be the application corresponding to the next screen mirroring content, and both can reflect the next screen mirroring scenario. Therefore, it can be set that the necessity of bitrate adjustment characterized by the bitrate adjustment index is positively correlated with the first application type weight and the second application type weight. That is to say, the greater the application type weight of the application that has been switched to the foreground, the greater the probability that the bitrate of the next screen mirroring scenario is higher, and thus the more inclined to perform bitrate adjustment. And the higher the switching frequency, the lower the stability of the next screen mirroring scenario, and thus the more inclined not to perform bitrate adjustment. Determining whether to perform bitrate adjustment by combining the switching frequency of the foreground application, the application type of the switched target application, and the application type of the current foreground application can further improve the user experience.

[0140] In a possible implementation, determining whether to perform bitrate adjustment according to bitrate adjustment parameters includes: determining a bitrate adjustment index k according to the following formula: k = W app1 2 / (a × f / W app2 ), if k is greater than the threshold, it is determined to perform bitrate adjustment. If k is not greater than the threshold, it is determined not to perform bitrate adjustment. W app1is the weight of the first application type, f is the frequency at which the current foreground application switches from the background to the foreground within a preset period, and W app2 is the weight of the second application type, and a is a constant. Determining the bitrate adjustment index based on this formula can more accurately achieve dynamic adjustment of encoding.

[0141] Specifically, a×f / W app2 can be used to represent the probability of switching to an application type for which bitrate adjustment should not be performed in the next period of time. Suppose in the first scenario, the current foreground application is a game application, and its corresponding application type weight W app1 = 1, the user switches to the music application once in the last 1 minute, and the switching frequency of once per minute is set as f = 1 for example. The music application is the target application, and its corresponding application type weight W app2 = 0.5; Suppose the second scenario is similar to the first scenario, the only difference being that the user switches to the music application 5 times in the last 1 minute, and the switching frequency of 5 times per minute is set as f = 5 for example. Then, compared with the first scenario, the switching frequency in the second scenario is larger, so the bitrate adjustment index k is smaller and it is less likely to be greater than the threshold, that is, it is less likely to perform bitrate adjustment; Suppose the second scenario is similar to the first scenario, the only difference being that the current foreground application is an instant messaging application, and its corresponding application type weight W app1 = 0, the bitrate adjustment index k = 0, and the threshold is a positive number. k must be less than the threshold, so bitrate adjustment is not performed; Suppose the second scenario is similar to the first scenario, the only difference being that the target application is an instant messaging application, and its corresponding application type weight W app2 = 0, the bitrate adjustment index k = 0, and the threshold is a positive number. k must be less than the threshold, so bitrate adjustment is not performed. To ensure normal calculation, if there is no switch within the preset period, f and W app2 can be set to non-zero preset values. Determining the bitrate adjustment index based on this formula can more accurately achieve dynamic adjustment of the bitrate.

[0142] In a possible implementation manner, the above step 203, encoding and wirelessly casting the content to be cast based on the target bitrate includes: determining the target resolution of the content to be cast based on the target bitrate, and the target resolution of the content to be cast is lower than the original resolution of the content to be cast; encoding and wirelessly casting the content to be cast based on the target resolution. It is possible to minimize the adverse impact on the user experience while reducing the bitrate. The specific process and principle are the same as the relevant content of step 105 in the above embodiment, and will not be elaborated here.

[0143] As Figure 11 shown, the embodiment of the present application further provides a screen casting method, including:

[0144] Step 301: Determine whether to perform bitrate adjustment according to the frequency at which the current foreground application switches from the background to the foreground within a preset period. If so, execute Step 302: Encode and wirelessly cast the content to be cast based on the target bitrate, where the target bitrate is lower than the original bitrate of the content to be cast. If it is determined in Step 301 that the answer is no, execute Step 303: Encode and wirelessly cast the content to be cast based on the original bitrate.

[0145] Specifically, the execution entity of this screen mirroring method can be the screen mirroring source device. Step 301 can be the same as Step 107 above, Step 303 can be the same as Step 108 above, and Step 302 is similar to Step 105 above. The difference is that in Step 303, the determination method of the target bitrate is not restricted. The target bitrate determination method in the above embodiments can be applied, or other possible target bitrate determination methods can be applied, as long as the target bitrate is lower than the original bitrate.

[0146] The screen mirroring method according to the embodiments of the present application can predict the probability that the user will switch the foreground application in the next period of time based on the user's historical operation behavior. If the probability that the user will switch the foreground application in the next step is high, it is more inclined not to perform bitrate adjustment, because after switching to the foreground application, if the bitrate of the switched foreground application decreases, it will have a great impact on the user's subjective feeling and easily reduce the user experience. If the probability that the user will switch the foreground application in the next step is low, it is more inclined to perform bitrate adjustment, because in a stable scenario, the reduction of the bitrate has a small impact on the user's perception, and the reduction of the bitrate is not likely to reduce the user experience. Implementing dynamic bitrate adjustment according to the switching frequency of the foreground application can improve the user experience while reducing power consumption. In addition, the screen mirroring method in the embodiments of the present application does not need to wait until the temperature rises to the threshold before adjusting the bitrate, but realizes dynamic adjustment of the bitrate based on the switching frequency of the foreground application, which can improve the temperature rise of the screen mirroring source device caused by screen mirroring, extend the time of temperature rise, and reduce the probability of device abnormality caused by heat generation.

[0147] In a possible implementation, step 301 above, determining whether to perform bitrate adjustment based on the frequency of the current foreground application switching from the background to the foreground within a preset period includes: determining whether to perform bitrate adjustment based on the frequency of the current foreground application switching from the background to the foreground within a preset period and the second application type weight, where the second application type weight is the application type weight corresponding to the target application that is switched to the foreground application within the preset period, and the target application is an application other than the current foreground application. For example, if the target application switched to by the user in the recent period is a video application, its corresponding application type weight is relatively large, so it is more inclined to perform bitrate adjustment; if the target application switched to by the user in the recent period is, for example, an instant messaging application, its corresponding application type weight is relatively small, so it is more inclined not to perform bitrate adjustment. Determining whether to perform bitrate adjustment by combining the switching frequency of the foreground application and the application type of the switched-to target application can further improve the user experience.

[0148] In a possible implementation, the application type weight corresponding to a video or game application > the application type weight corresponding to other applications.

[0149] In a possible implementation, determining whether to perform bitrate adjustment based on the frequency of the current foreground application switching from the background to the foreground within a preset period and the second application type weight includes: determining a bitrate adjustment index based on the frequency of the current foreground application switching from the background to the foreground within a preset period and the second application type weight. If the bitrate adjustment necessity characterized by the bitrate adjustment index meets the adjustment condition, it is determined to perform bitrate adjustment. If the bitrate adjustment necessity characterized by the bitrate adjustment index does not meet the adjustment condition, it is determined not to perform bitrate adjustment. The bitrate adjustment necessity characterized by the bitrate adjustment index is positively correlated with the second application type weight and negatively correlated with the frequency. That is to say, the higher the frequency of switching from the background to the foreground within the preset period, the worse the stability of the scene to be projected, so the lower the bitrate adjustment necessity. The lower the switching frequency, the higher the stability of the scene to be projected, so the higher the bitrate adjustment necessity. The higher the application type weight of the target application, the greater the probability that the scene to be projected is a high bitrate, that is, the higher the bitrate adjustment necessity. The lower the application type weight of the target application, the smaller the probability that the scene to be projected is a high bitrate, that is, the lower the bitrate adjustment necessity.

[0150] In a possible implementation, the above-mentioned step 303, encoding and wireless mirroring the content to be mirrored based on the target bitrate includes: determining the target resolution of the content to be mirrored based on the target bitrate, where the target resolution of the content to be mirrored is lower than the original resolution of the content to be mirrored; encoding and wirelessly mirroring the content to be mirrored based on the target resolution. It is possible to minimize the adverse impact on the user experience while reducing the bitrate. The specific process and principle are the same as those in the above embodiments and will not be elaborated here.

[0151] The embodiments of the present application further provide an electronic device, including: a processor and a memory. The memory is used to store at least one instruction. When the instruction is loaded and executed by the processor, the electronic device executes the method of any of the above embodiments. The specific process and principle of this method are the same as those in the above embodiments and will not be elaborated here. The electronic device may specifically be Figure 2 the electronic device as shown.

[0152] The electronic device involved in the present application may be any product such as a smart TV, mobile phone, tablet computer, personal computer (PC), personal digital assistant (PDA), smart watch, wearable electronic device, augmented reality (AR) device, virtual reality (VR) device, in-vehicle device, drone device, smart car, smart speaker, robot, smart glasses, and so on.

[0153] The embodiments of the present application further provide a computer-readable storage medium, including a program or instruction. When the program or instruction runs on a computer, the method in any of the above embodiments is executed.

[0154] The embodiments of the present application further provide a computer program product. The computer program product includes executable instructions. When the executable instructions are executed on a computer, the computer executes the method in any of the above embodiments.

[0155] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in this application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from a website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer, or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk).

[0156] In the embodiments of the present application, "at least one" means one or more, and "a plurality" means two or more. "And / or" describes the association relationship of associated objects and indicates that three relationships can exist. For example, A and / or B can represent the cases of A existing alone, A and B existing simultaneously, and B existing alone. Where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after. "At least one of the following" and its similar expressions refer to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, and c can represent: a, b, c, a - b, a - c, b - c, or a - b - c, where a, b, and c can be single or multiple.

[0157] The above are only the preferred embodiments of the present application and are not used to limit the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included within the protection scope of the present application.

Claims

1. A screen mirroring method, characterized in that, it includes: Obtain a first application type weight, where the first application type weight is the application type weight corresponding to the current foreground application; Determine whether to perform bitrate adjustment according to the bitrate adjustment parameter, where the bitrate adjustment parameter includes the first application type weight. If so, encode and wirelessly mirror the content to be mirrored based on the target bitrate, and the target bitrate is lower than the original bitrate of the content to be mirrored. If not, encode and wirelessly mirror the content to be mirrored based on the original bitrate.

2. The method according to claim 1, characterized in that, The application type weight corresponding to a video or game application > the application type weight corresponding to other applications.

3. The method according to claim 1 or 2, characterized in that, The determining whether to perform bitrate adjustment according to the bitrate adjustment parameter includes: If the first application type weight is greater than the preset weight, determine to perform bitrate adjustment. If the first application type weight is not greater than the preset weight, determine not to perform bitrate adjustment.

4. The method according to claim 1, characterized in that, The bitrate adjustment parameter further includes the frequency at which the current foreground application switches from the background to the foreground within a preset period.

5. The method according to claim 5, characterized in that, The determining whether to perform bitrate adjustment according to the bitrate adjustment parameter includes: determining a bitrate adjustment index according to the bitrate adjustment parameter. If the bitrate adjustment necessity characterized by the bitrate adjustment index meets the adjustment condition, determine to perform bitrate adjustment. If the bitrate adjustment necessity characterized by the bitrate adjustment index does not meet the adjustment condition, determine not to perform bitrate adjustment. The bitrate adjustment necessity characterized by the bitrate adjustment index is positively correlated with the first application type weight, and the bitrate adjustment necessity characterized by the bitrate adjustment index is negatively correlated with the frequency.

6. The method according to claim 1 or 4, characterized in that, The bitrate adjustment parameter further includes a second application type weight, where the second application type weight is the application type weight corresponding to the target application that is switched to the foreground application within the preset period, and the target application is an application other than the current foreground application.

7. The method according to any one of claims 1 to 6, characterized in that, The encoding and wirelessly mirroring the content to be mirrored based on the target bitrate includes: Determine the target resolution of the content to be mirrored based on the target bitrate, and the target resolution of the content to be mirrored is lower than the original resolution of the content to be mirrored; Encode and wirelessly mirror the content to be mirrored based on the target resolution.

8. A screen mirroring method, characterized in that, it includes: Determine whether to perform bitrate adjustment according to the frequency at which the current foreground application switches from the background to the foreground within a preset period. If so, encode and wirelessly mirror the content to be mirrored based on the target bitrate, and the target bitrate is lower than the original bitrate of the content to be mirrored. If not, encode and wirelessly mirror the content to be mirrored based on the original bitrate.

9. The method according to claim 8, characterized in that, Determining whether to perform bitrate adjustment based on the frequency of the current foreground application switching from the background to the foreground within a preset period includes: determining whether to perform bitrate adjustment according to the frequency of the current foreground application switching from the background to the foreground within a preset period and a second application type weight, where the second application type weight is the application type weight corresponding to the target application that is switched to the foreground application within the preset period, and the target application is an application other than the current foreground application.

10. The method according to claim 9, wherein, The application type weight corresponding to a video or game application > the application type weight corresponding to other applications.

11. The method according to claim 9, wherein, Determining whether to perform bitrate adjustment according to the frequency of the current foreground application switching from the background to the foreground within a preset period and a second application type weight includes: determining a bitrate adjustment index according to the frequency of the current foreground application switching from the background to the foreground within a preset period and the second application type weight. If the necessity of bitrate adjustment characterized by the bitrate adjustment index meets the adjustment condition, it is determined to perform bitrate adjustment. If the necessity of bitrate adjustment characterized by the bitrate adjustment index does not meet the adjustment condition, it is determined not to perform bitrate adjustment. The necessity of bitrate adjustment characterized by the bitrate adjustment index is positively correlated with the second application type weight, and the necessity of bitrate adjustment characterized by the bitrate adjustment index is negatively correlated with the frequency.

12. The method according to any one of claims 8 to 11, wherein, Encoding and wirelessly casting the content to be cast based on the target bitrate includes: Determining the target resolution of the content to be cast based on the target bitrate, where the target resolution of the content to be cast is lower than the original resolution of the content to be cast; Encoding and wirelessly casting the content to be cast based on the target resolution.

13. An electronic device, wherein, including: A processor and a memory, where the memory is used to store at least one instruction, and when the instruction is loaded and executed by the processor, the electronic device executes the method according to any one of claims 1 to 12.

14. A computer-readable storage medium, wherein, including a program or instruction, when the program or instruction runs on a computer, the method according to any one of claims 1 to 12 is executed.

15. A computer program product, wherein, The computer program product contains executable instructions, and when the executable instructions are executed on a computer, the computer executes the method according to any one of claims 1 to 12.

Images