Mobile terminal screen streaming media adaptive transmission method and system

Abstract

The invention discloses a mobile terminal screen streaming media self-adaptive transmission method and system. The method comprises the following steps: acquiring display data of a display layer through a graphic synthesis service component; the display data comprises a display size, a display direction and a frame rate; setting encoding parameters of a hardware encoder according to display data of the display layer; acquiring current picture frame data from a graph buffer area managed by a graph synthesis service component, and performing compressed encoding by using a hardware encoder; and transmitting the coded data to a receiving end through a network. Wherein in the data transmission process after display data acquisition and encoding, the network state is monitored in real time, and the encoding parameters of the hardware encoder are dynamically adjusted according to the network state. According to the method, the CPU occupancy rate is effectively reduced, the coding efficiency is high, the delay is low, and the network adaptability is high.

Description

Technical Field

[0001] The present invention relates to the technical field of mobile terminal screen streaming media transmission. Specifically, it relates to a method and system for adaptive transmission of mobile terminal screen streaming media. Background Art

[0002] Currently, the transmission of mobile terminal screen streaming media mainly relies on traditional solutions, such as the MediaProjection API of the Android system, which obtains screen data through the system screenshot interface and then compresses and transmits it through software encoding (such as libjpeg-turbo). However, such solutions have the following defects:

[0003] 1. High CPU occupancy rate: Multiple memory copies are required for screenshot and software encoding, resulting in the CPU load exceeding 30%.

[0004] 2. High latency: The software encoding efficiency is low, and the end-to-end transmission latency is generally higher than 200 ms, making it difficult to meet the requirements of real-time interaction.

[0005] 3. Poor network adaptability: Fixed encoding parameters cannot dynamically adapt to bandwidth fluctuations, easily resulting in stuttering or image quality degradation. Summary of the Invention

[0006] In an embodiment of the present invention, a method and system for adaptive transmission of mobile terminal screen streaming media are provided to solve the problems in the prior art that multiple CPU memory copies must be performed, the software encoding efficiency is low, and there is a lack of a dynamic quality control mechanism.

[0007] To achieve the above object, on the one hand, the present invention provides a method for adaptive transmission of mobile terminal screen streaming media, the method comprising: S1. Obtain the display data of the display layer through a graphics composition service component; the display data includes the display size, display direction, and frame rate; S2. Set the encoding parameters of the hardware encoder according to the display data of the display layer; S3. Obtain the current frame data of the screen from the graphics buffer managed by the graphics composition service component, and compress and encode the current frame data of the screen using the hardware encoder; S4. Transmit the encoded data to the receiving end through the network; wherein, during the process of obtaining the display data and transmitting the encoded data, the network status is monitored in real time, and the encoding parameters of the hardware encoder are dynamically adjusted according to the network status.

[0008] Optionally, it further comprises: monitoring the display data of the display layer in real time. When the display data of the display layer changes, determine whether the changed display data indicates a change in the resolution of the screen image data of the display layer. If so, dynamically adjust the encoding parameters of the hardware encoder according to the resolution of the current screen image data of the display layer.

[0009] Optionally, the real-time monitoring of the network status and dynamically adjusting the encoding parameters of the hardware encoder according to the network status includes: real-time monitoring of the network status and generating a network quality evaluation result; the network status includes: network delay, packet loss rate and bandwidth utilization; dynamically adjusting the encoding parameters of the hardware encoder according to the network quality evaluation result; the encoding parameters include: compression rate and frame sampling rate.

[0010] Optionally, the dynamically adjusting the encoding parameters of the hardware encoder according to the network quality assessment result includes: when the network quality assessment result shows that the network status reaches a preset good state, reducing the compression rate of the hardware encoder and increasing the frame sampling rate of the hardware encoder; when the network quality assessment result shows that the network status is in a preset poor state, increasing the compression rate of the hardware encoder and reducing the frame sampling rate of the hardware encoder.

[0011] Optionally, the hardware encoder is a hardware encoder that comes with the Android platform, which is used to compress and encode the original RGB format picture frame data to generate data in PNG format or MJPG format.

[0012] On the other hand, the present invention provides a mobile terminal screen streaming media adaptive transmission system, which includes: an acquisition unit for acquiring display data of a display layer through a graphics synthesis service component; the display data includes display size, display direction and frame rate; a setting unit for setting encoding parameters of a hardware encoder according to the display data of the display layer; an encoding unit for acquiring current screen frame data from a graphics buffer managed by a graphics synthesis service component, and compressing and encoding the current screen frame data using a hardware encoder; a transmission unit for transmitting the encoded data to a receiving end through a network; a first adjustment unit for monitoring the network status in real time during the display data acquisition and encoded data transmission process, and dynamically adjusting the encoding parameters of the hardware encoder according to the network status.

[0013] Optionally, it also includes: a second adjustment unit, used to: monitor the display data of the display layer in real time, and when the display data of the display layer changes, determine whether the changed display data indicates that the resolution of the screen image data of the display layer has changed; if so, dynamically adjust the encoding parameters of the hardware encoder according to the current resolution of the screen image data of the display layer.

[0014] Optionally, the first adjustment unit includes: a generation subunit, used to monitor the network status in real time and generate a network quality assessment result; the network status includes: network delay, packet loss rate and bandwidth utilization; an adjustment subunit, used to dynamically adjust the encoding parameters of the hardware encoder according to the network quality assessment result; the encoding parameters include: compression rate and frame sampling rate.

[0015] Optionally, the adjustment subunit is used to: when the network quality assessment result shows that the network status has reached a preset good state, reduce the compression rate of the hardware encoder and increase the frame sampling rate of the hardware encoder; when the network quality assessment result shows that the network status is in a preset poor state, increase the compression rate of the hardware encoder and reduce the frame sampling rate of the hardware encoder.

[0016] Optionally, the hardware encoder is a hardware encoder that comes with the Android platform, which is used to compress and encode the original RGB format picture frame data to generate data in PNG format or MJPG format.

[0017] Beneficial effects of the present invention:

[0018] The present invention provides a method and system for adaptive transmission of streaming media on mobile terminal screens. The method directly obtains current screen frame data from a graphics buffer, skips screenshots, and performs hardware encoding on the current screen frame data, avoiding multiple memory copies. This is a "zero-copy mechanism," effectively reducing CPU usage (less than 10%) and power consumption. The method utilizes a hardware encoder in place of traditional software encoding. The hardware encoder, equipped with a specialized image compression instruction set and parallel operation logic units, can achieve image compression speeds 2 to 10 times faster than software encoding under the same conditions, thereby increasing the encoding rate. Under the same frame sampling rate, the hardware encoder only requires 3 to 5 ms to complete single-frame compression, while software encoding often requires more than 20 ms. Therefore, the method utilizes a hardware encoder to achieve an end-to-end transmission delay of less than 100 ms. In a mobile network environment with frequent bandwidth fluctuations, the method monitors the network status in real time and dynamically adjusts the encoding parameters of the hardware encoder based on the network status, thereby avoiding frequent freezes or sudden changes in the screen and improving the overall user viewing experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a flow chart of a method for adaptively transmitting streaming media on a mobile terminal screen provided by an embodiment of the present invention;

[0020] Figure 2 is a flowchart of dynamically adjusting the encoding parameters of the hardware encoder according to the network status provided by an embodiment of the present invention;

[0021] Figure 3 This is a structural diagram of a mobile terminal screen streaming media adaptive transmission system provided by an embodiment of the present invention;

[0022] Figure 4 It is a structural diagram of the first adjustment unit provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0023] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0024] Figure 1 It is a flowchart of a method for adaptively transmitting screen streaming media of a mobile terminal according to an embodiment of the present invention. As Figure 1 shown, the method includes:

[0025] S1. Obtain the display data of the display layer through the graphics composition service component; the display data includes the display size, display orientation, and frame rate.

[0026] Among them, the graphics composition service component is SurfaceFlinger.

[0027] In an optional implementation manner, during the system initialization phase, an application or service obtains the display data of the display layer through SurfaceFlinger. The process of obtaining the display data of the display layer generally includes that the application or service obtains a reference to the SurfaceFlinger service through the Binder mechanism and calls its publicly available interface method to obtain the display data of the display layer; or the application or service directly accesses the device file related to the display data (such as / dev / graphics / fb) to request the latest display information (i.e., the display data, which refers to a set of key parameters managed by SurfaceFlinger in the Android platform and provided externally to describe the display state of the display layer; the display data includes the display size, display orientation, and frame rate).

[0028] Obtain the display size:

[0029] SurfaceFlinger is responsible for managing the composition of all display layers on the screen. Therefore, calling its interface can return the current display size (screen resolution), that is, the width and height of the screen (in pixels). After the system calls the interface of SurfaceFlinger, SurfaceFlinger will return a structure or data object containing the display size, and this data is passed to the image acquisition module so that the image acquisition module can set the boundary of the image acquisition area according to the display size, ensuring that the subsequent captured pictures match the actual display size, thereby avoiding problems such as image cropping or stretching caused by inconsistent sizes.

[0030] Obtain the display orientation:

[0031] The display orientation refers to the rotation state of the current screen (which can be represented by the rotation angle, such as 0°, 90°, 180°, or 270°), reflecting the physical placement of the device or the user's usage habits. Through the internally managed display configuration, SurfaceFlinger can return the current rotation angle information. The obtained display orientation data is used to guide the image acquisition and subsequent processing modules. For example, in landscape or portrait mode, the acquisition parameters are adjusted to ensure that the captured image is in the same orientation as the user's actual viewing.

[0032] Obtain the frame rate:

[0033] The frame rate, which is the screen refresh rate, refers to the number of times the screen is updated per unit time, usually expressed in Hz. SurfaceFlinger can return the refresh rate of the current display, and this information is crucial for the real-time performance and smoothness of the video stream. In the image acquisition and encoding processes, the frame rate data is used to guide the setting of the acquisition frequency to ensure that the captured video stream matches the display update cycle, thereby avoiding screen jitter or latency caused by inappropriate acquisition frequencies.

[0034] S2. Set the encoding parameters of the hardware encoder according to the display data of the display layer.

[0035] In an optional implementation manner,

[0036] First, parse the display data:

[0037] Display size: The system obtains the current display size of the display layer, i.e., width and height, from SurfaceFlinger or a device file (such as / dev / graphics / fb), which determines the basic resolution of the image.

[0038] Display orientation: After obtaining the current display orientation information (such as 0°, 90°, 180°, or 270°), the system adjusts the image rotation parameters during encoding to ensure that the encoded image is in the same orientation as the actual display.

[0039] Frame rate: The refresh rate (in Hz) returned by the display layer is used to set the frame rate of image acquisition and encoding to ensure the continuity and smoothness of the video stream.

[0040] The encoding parameters of the hardware encoder include: compression ratio and frame sampling rate.

[0041] The system internally pre-sets a mapping formula or decision tree. By inputting the display size, display orientation, and frame rate data, the optimal compression ratio and frame sampling rate are calculated.

[0042] (1) Determine the compression ratio parameter

[0043] Influence of the display size:

[0044] A larger display size generally means the image contains more details. To maintain high image quality, the system tends to set a lower compression rate to reduce detail loss. A smaller display size allows for a higher compression rate, further compressing the image data to reduce the data volume.

[0045] Adjustment of display direction:

[0046] Different display orientations may change the aspect ratio of the image. If the aspect ratio is greater than 1.5 and the orientation is 0°, the standard compression rate is used. If the aspect ratio is less than 1.0 and the orientation is 90° or 270°, the compression rate is reduced by 10% to preserve more image structural details.

[0047] Frame rate trade-offs:

[0048] At high frame rates, the amount of data increases significantly. The system may increase the compression rate appropriately while maintaining a certain level of image quality to reduce the amount of data transmitted per second. At lower frame rates, a lower compression rate can be used to ensure more complete image detail in each frame.

[0049] (2) Determine the frame sampling rate parameters

[0050] The system directly uses the current display refresh rate as the initial frame sampling rate. The system adjusts the frame sampling rate appropriately according to the refresh rate changes to keep it consistent with the screen update rhythm and avoid lag.

[0051] S3. Obtain the current picture frame data from the graphics buffer managed by the graphics composition service component, and compress and encode the current picture frame data using a hardware encoder.

[0052] The system establishes a connection with the SurfaceFlinger service through Binder communication, and directly accesses the graphics buffer managed by SurfaceFlinger through the standard interface (specifically, the standard underlying interface mConsumer->lockNextBuffer() of the image acquisition module is used to capture the current frame data); since the captured frame data are all original GRB data, the data volume will be very large, so encoding processing is required.

[0053] The current picture frame data is compressed and encoded using a hardware encoder; the hardware encoder is a hardware encoder that comes with the Android platform and is used to compress and encode the original RGB data to generate PNG or MJPG format.

[0054] To avoid multiple memory copies by the CPU during data transmission, the system directly obtains the current frame data of the screen from the graphics buffer, skips the screenshot, and performs hardware encoding on the current frame data without passing through the CPU memory transfer, thus avoiding multiple memory copies. This is the 'zero-copy mechanism', which effectively reduces the CPU occupancy rate (below 10%) and reduces power consumption. This hardware encoder is usually integrated in the GPU (such as Mali or Adreno) and is specifically used for efficient compression to generate JPEG, MJPG, or PNG formats.

[0055] In the previous steps, appropriate encoding parameters, including the compression ratio and frame sampling rate, have been calculated based on the display size, display orientation, and frame rate. The hardware encoder compresses the original RGB data in real time according to these parameters: when the compression ratio is set low, the image details are maximally retained; when the compression ratio is high, the image data is compressed more deeply to reduce the data volume, thus meeting the network transmission requirements.

[0056] Upon receiving the current frame data of the screen, the hardware encoder immediately starts the compression process, converting the original RGB data into a predetermined compressed format (such as MJPG or PNG). During this process, the encoder utilizes the hardware acceleration feature for fast parallel processing, greatly shortening the encoding time and simultaneously reducing the CPU occupancy rate to below 10%.

[0057] S4. Transmit the encoded data to the receiving end through the network.

[0058] The compressed data output from the hardware encoder is finally transmitted to the receiving end efficiently and stably through the network interface after a series of processes such as data buffering, segmentation, encapsulation, protocol selection, error control, and secure transmission. After receiving the data, the receiving end needs to perform decoding processing and transmit the decoded frame data of the screen for real-time display.

[0059] Among them, during the process of obtaining display data and transmitting the encoded data, the network status is monitored in real time, and the encoding parameters of the hardware encoder are dynamically adjusted according to the network status.

[0060] In an optional embodiment, Figure 2 is a flowchart of dynamically adjusting the encoding parameters of the hardware encoder according to the network status provided by an embodiment of the present invention; as Figure 2 shown, the real-time monitoring of the network status and dynamically adjusting the encoding parameters of the hardware encoder according to the network status include:

[0061] S41. Real-time monitor the network status and generate a network quality evaluation result; the network status includes: network latency, packet loss rate, and bandwidth utilization rate,

[0062] Network latency: The system measures the round-trip time (RTT) of data packets periodically or through event triggering to determine the current network response speed.

[0063] Packet loss rate: The system monitors the sending and receiving of data packets within a certain time window and calculates the proportion of lost packets, which is the packet loss rate. As an important indicator of network stability, the packet loss rate can reflect whether the network is congested or unstable.

[0064] Bandwidth Utilization: The system measures the ratio of the actual data transmission rate to the network's maximum available bandwidth, which is the bandwidth utilization rate. This parameter helps determine whether the current network is under high load and thus assess the network's transmission capacity.

[0065] The above indicators are combined to calculate a network quality score using pre-set mapping rules or dynamic algorithms. For example, each network indicator is weighted as W1 to W3, and combined with the scoring function Q = W1 × (1 / network delay) + W2 × (1 / packet loss rate) + W3 × bandwidth utilization, the network quality score Q is calculated. This score can be used to categorize network status into different levels, such as "good" or "poor," providing a basis for subsequent parameter adjustments.

[0066] S42. Dynamically adjust the encoding parameters of the hardware encoder according to the network quality evaluation result; the encoding parameters include: compression rate and frame sampling rate.

[0067] In an optional embodiment, when the network quality assessment result shows that the network status has reached a preset good state, the compression rate of the hardware encoder is reduced and the frame sampling rate of the hardware encoder is increased;

[0068] When the network quality evaluation result shows that the network status is in a preset poor status, the compression rate of the hardware encoder is increased and the frame sampling rate of the hardware encoder is reduced.

[0069] Specifically, compression ratio adjustment:

[0070] When the network is in good condition: When the network latency is low, the packet loss rate is low, and the bandwidth is sufficient, the system selects a lower compression rate to ensure that image details are preserved and high image quality is presented.

[0071] Poor network conditions: When increased latency, packet loss, or high bandwidth utilization are detected, the system increases compression to reduce data size and transmission load. This higher compression rate may result in a loss of image detail, but it ensures a consistent and smooth video stream.

[0072] Frame sampling rate adjustment:

[0073] When the network status is good: The system can maintain or increase the frame sampling rate, so that more frames are captured and encoded per second, thus achieving a smoother video playback effect.

[0074] When the network status is poor: The system will reduce the frame sampling rate, reduce the number of frames encoded and transmitted per second, so as to reduce the data volume, thereby reducing the network pressure and avoiding jams caused by data congestion.

[0075] The network monitoring module continuously feeds back the real-time network quality assessment results to the central control unit; the central control unit classifies the network status according to the preset mapping rules or dynamic algorithms, and then determines the optimal compression ratio and frame sampling rate. Subsequently, the central control unit issues new encoding parameters to the hardware encoder through an interface or command, and the hardware encoder immediately adjusts its working mode to adapt to the current network condition. This process constitutes a closed-loop control system, and the system continuously monitors the network and transmission effect to ensure that the encoding parameters are always in the best state.

[0076] The method of the present invention realizes an end-to-end transmission delay of <100ms by directly accessing the Android graphics buffer data, improving the real-time performance of the image; by adopting the hardware encoding zero-copy mechanism, it effectively reduces the CPU occupancy rate (below 10%) and reduces power consumption; real-time network monitoring and dynamic encoding parameter adjustment ensure the stability and high picture quality of the streaming media transmission in the environment of 1 - 10Mbps network bandwidth fluctuation.

[0077] In an optional embodiment, the method further includes:

[0078] Real-time monitor the display data of the display layer. When the display data of the display layer changes, determine whether the changed display data indicates a change in the resolution of the screen image data of the display layer. If so, dynamically adjust the encoding parameters of the hardware encoder according to the resolution of the current screen image data of the display layer.

[0079] The system continuously monitors the data of the display layer in SurfaceFlinger or device files (such as / dev / graphics / fb) by registering a listening callback or using a timed polling method. This data includes display size, display orientation, frame rate, etc. After obtaining new display data each time, the system compares it with the previously saved display state. When it is detected that the display size (i.e., resolution) is inconsistent with the previous data, it is determined that the resolution has changed.

[0080] When the resolution changes, the encoding parameters of the hardware encoder are dynamically adjusted. Specifically, when the resolution decreases, increase the compression ratio of the hardware encoder and decrease the frame sampling rate of the hardware encoder; when the resolution increases, decrease the compression ratio of the hardware encoder and increase the frame sampling rate of the hardware encoder.

[0081] Specifically, for example, when changing from 1920*1080 to 1280*720 (or from 1280*720 to 1920*1080), the hardware encoder part will be notified to dynamically adjust the parameters of the hardware encoder to adapt to the new resolution, so as to ensure the image quality and smoothness.

[0082] When the resolution drops from 1920×1080 to 1280×720, the system may increase the compression ratio, that is, further compress the image data without sacrificing the basic smoothness of the picture, and reduce the data volume per frame. Since the volume of a single frame of data decreases at a lower resolution, the system may appropriately reduce the sampling rate to balance processing resources, or remain relatively stable to ensure video continuity.

[0083] When the resolution rises from 1280×720 to 1920×1080, in order to retain more image details, the system will reduce the compression ratio, that is, reduce the degree of data compression, so that the details of the high-resolution image can be presented more completely. The system may need to increase the frame sampling rate to fully capture more details and dynamic information, ensuring the coherence and high-quality display of video playback.

[0084] The system can dynamically adjust the parameters of the hardware encoder when detecting a change in the display resolution, so as to adapt to the new resolution environment, ensuring both the smoothness of video stream transmission and maintaining a high image quality.

[0085] When the change trends of display data (such as resolution) and network status (such as bandwidth) are contradictory, a multi-factor decision-making mechanism needs to be introduced to coordinate and optimize the setting of encoding parameters.

[0086] The multi-factor decision-making mechanism is to quantify the changes in network status and display data, and assign a weight value α to the network status and a weight value β to the display data; calculate the comprehensive result = α×quantified network status change + β×quantified display data change; if the comprehensive result is greater than the preset threshold, reduce the compression ratio of the hardware encoder and increase the frame sampling rate of the hardware encoder; if the comprehensive result is less than or equal to the preset threshold, increase the compression ratio of the hardware encoder and reduce the frame sampling rate of the hardware encoder.

[0087] Figure 3 It is a schematic structural diagram of a mobile terminal screen streaming media adaptive transmission system provided by an embodiment of the present invention, as Figure 3 shown, the system includes:

[0088] An acquisition unit 201, configured to obtain display data of the display layer through a graphics composition service component; the display data includes display size, display direction, and frame rate;

[0089] A setting unit 202, configured to set encoding parameters of a hardware encoder according to display data of the display layer;

[0090] An encoding unit 203, configured to obtain current frame data of a picture from a graphic buffer managed by a graphic composition service component, and perform compression encoding on the current frame data of the picture by using the hardware encoder;

[0091] A transmission unit 204, configured to transmit the encoded data to a receiving end through a network;

[0092] A first adjustment unit 205, configured to monitor a network state in real time during a process of obtaining display data and transmitting the encoded data, and dynamically adjust the encoding parameters of the hardware encoder according to the network state.

[0093] In an optional embodiment, the system further includes: a second adjustment unit, configured to: monitor the display data of the display layer in real time, and when the display data of the display layer changes, determine whether the changed display data indicates that a resolution of screen image data of the display layer changes, and if so, dynamically adjust the encoding parameters of the hardware encoder according to the resolution of the current screen image data of the display layer.

[0094] In an optional embodiment, Figure 4 is a schematic structural diagram of the first adjustment unit provided by an embodiment of the present invention; as Figure 4 shown, the first adjustment unit 205 includes:

[0095] A generation subunit 2051, configured to monitor the network state in real time and generate a network quality evaluation result; the network state includes: network latency, packet loss rate, and bandwidth utilization rate;

[0096] An adjustment subunit 2052, configured to dynamically adjust the encoding parameters of the hardware encoder according to the network quality evaluation result; the encoding parameters include: compression ratio and frame sampling rate.

[0097] In an optional embodiment, the adjustment subunit 2052 is configured to:

[0098] when the network quality evaluation result shows that the network state reaches a preset good state, reduce the compression ratio of the hardware encoder and increase the frame sampling rate of the hardware encoder;

[0099] when the network quality evaluation result shows that the network state is in a preset poor state, increase the compression ratio of the hardware encoder and reduce the frame sampling rate of the hardware encoder.

[0100] In an optional embodiment, the hardware encoder is a hardware encoder that comes with the Android platform, which is used to compress and encode the original RGB format picture frame data to generate data in PNG format or MJPG format.

[0101] Beneficial effects of the present invention:

[0102] The present invention provides a method and system for adaptive transmission of streaming media on a mobile terminal screen, wherein the method directly obtains the current screen frame data from the graphics buffer, skips screenshots, and performs hardware encoding on the current screen frame data to avoid multiple memory copies, which is a "zero copy mechanism" and effectively reduces the CPU occupancy rate (less than 10%) and reduces power consumption; the method uses a hardware encoder to replace traditional software encoding. The hardware encoder has a special image compression instruction set and a parallel operation logic unit. Compared with software encoding, it can achieve an image compression speed of 2 to 10 times higher under the same conditions, that is, improve the encoding rate; under the same frame sampling rate, the hardware encoder It only takes 3 to 5 ms to complete single-frame compression, while software encoding often takes more than 20 ms. Therefore, this method uses a hardware encoder to achieve an end-to-end transmission delay of <100 ms. In a mobile network environment with frequent bandwidth fluctuations, this method monitors the network status in real time and dynamically adjusts the encoding parameters of the hardware encoder according to the network status to avoid frequent freezes or sudden changes in the picture, thereby improving the user's overall viewing experience. This method monitors the display data of the display layer in real time and dynamically adjusts the encoding parameters of the hardware encoder according to the resolution of the screen image data of the current display layer to avoid frequent freezes or sudden changes in the picture, thereby improving the user's overall viewing experience.

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for adaptively transmitting screen streaming media of a mobile terminal, characterized in that, include: S1. Obtain display data of the display layer through the graphics composition service component; the display data includes display size, display direction and frame rate; S2. Setting encoding parameters of the hardware encoder according to the display data of the display layer; S3. Obtaining current picture frame data from the graphics buffer managed by the graphics composition service component, and compressing and encoding the current picture frame data using a hardware encoder; S4, transmitting the encoded data to the receiving end through the network; In the process of display data acquisition and encoded data transmission, the network status is monitored in real time, and the encoding parameters of the hardware encoder are dynamically adjusted according to the network status.

2. The method according to claim 1, wherein Also includes: Monitor the display data of the display layer in real time. When the display data of the display layer changes, determine whether the changed display data indicates that the resolution of the screen image data of the display layer has changed. If so, dynamically adjust the encoding parameters of the hardware encoder according to the current resolution of the screen image data of the display layer.

3. The method according to claim 1 or 2, characterized in that, The real-time monitoring of the network status and the dynamic adjustment of the encoding parameters of the hardware encoder according to the network status include: Monitor network status in real time and generate network quality assessment results; the network status includes: network delay, packet loss rate and bandwidth utilization; The encoding parameters of the hardware encoder are dynamically adjusted according to the network quality evaluation result; the encoding parameters include: compression rate and frame sampling rate.

4. The method according to claim 3, wherein The dynamically adjusting the encoding parameters of the hardware encoder according to the network quality evaluation result includes: When the network quality assessment result shows that the network status reaches a preset good status, the compression rate of the hardware encoder is reduced and the frame sampling rate of the hardware encoder is increased; When the network quality assessment result shows that the network status is in a preset poor state, the compression rate of the hardware encoder is increased and the frame sampling rate of the hardware encoder is reduced.

5. The method according to claim 4, characterized in that: The hardware encoder is a built-in hardware encoder of the Android platform, which is used to compress and encode the original RGB format picture frame data to generate data in PNG format or MJPG format.

6. A mobile terminal screen streaming media adaptive transmission system, characterized in that, include: An acquisition unit, configured to acquire display data of a display layer through a graphics composition service component; the display data includes display size, display direction, and frame rate; A setting unit, configured to set encoding parameters of a hardware encoder according to display data of the display layer; The encoding unit is used to obtain the current picture frame data from the graphics buffer managed by the graphics composition service component and compress and encode the current picture frame data using a hardware encoder; A transmission unit, configured to transmit the encoded data to a receiving end via a network; The first adjustment unit is used to monitor the network status in real time during the process of display data acquisition and encoded data transmission, and dynamically adjust the encoding parameters of the hardware encoder according to the network status.

7. The system according to claim 6, characterized in that, Also includes: The second adjustment unit is used for: Monitor the display data of the display layer in real time. When the display data of the display layer changes, determine whether the changed display data indicates that the resolution of the screen image data of the display layer has changed. If so, dynamically adjust the encoding parameters of the hardware encoder according to the current resolution of the screen image data of the display layer.

8. The system according to claim 6 or 7, characterized in that, The first adjustment unit includes: The generation subunit is used to monitor the network status in real time and generate network quality assessment results; the network status includes: network delay, packet loss rate and bandwidth utilization; The adjustment subunit is used to dynamically adjust the encoding parameters of the hardware encoder according to the network quality evaluation result; the encoding parameters include: compression rate and frame sampling rate.

9. The system according to claim 8, wherein, The adjustment subunit is used for: When the network quality assessment result shows that the network status reaches a preset good status, the compression rate of the hardware encoder is reduced and the frame sampling rate of the hardware encoder is increased; When the network quality evaluation result shows that the network status is in a preset poor status, the compression rate of the hardware encoder is increased and the frame sampling rate of the hardware encoder is reduced.

10. The system according to claim 6, characterized in that: The hardware encoder is a built-in hardware encoder of the Android platform, which is used to compress and encode the original RGB format picture frame data to generate data in PNG format or MJPG format.

Images