Multimedia data processing method and system for same-screen interaction of cloud mobile phone, computer equipment and storage medium

Abstract

The invention discloses a multimedia data processing method for same-screen interaction of a cloud mobile phone. The multimedia data processing method comprises the following steps: the cloud mobile phone obtains multimedia data requests of all terminal equipment participating in same-screen interaction; allocating a first dynamic transmission parameter to the terminal device based on a hardware constraint condition of the terminal device; allocating a second dynamic transmission parameter to the terminal device based on a network constraint condition of the terminal device; fusing the multimedia data request, the first dynamic transmission parameter and the second dynamic transmission parameter to generate a data processing task corresponding to the terminal equipment; and the cloud mobile phone executes the data processing task and performs multimedia data interaction with the terminal equipment. Based on the invention, the cloud mobile phone can select the optimal multimedia data processing mode based on the multimedia data request of the terminal equipment and the hardware constraint condition and the network constraint condition of the terminal equipment, so that smooth multimedia data transmission service can be provided for the terminal equipment, and the user experience is ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of cloud mobile phones, in particular to a multimedia data processing method and system for cloud mobile phone screen interaction, computer equipment and storage medium. BACKGROUND

[0002] A cloud mobile phone is a smart phone application cloud platform based on the powerful virtual application of cloud computing and the mobile Internet. Based on the outstanding advantages of cloud computing in large-scale computing, powerful storage capacity, virtualization application, etc., the cloud platform provides diversified and all-round virtual mobile phone applications and services for smart phone users, fully supports the display, download, application and management services of various mobile application programs, and outstandingly realizes all-weather cloud intelligent hosting application, so that users can also host various application programs to the cloud in the state of being free from mobile terminals, maintain all-weather online state, and fully realize the virtualization of smart phone applications.

[0003] Screen interaction technology is a technology that allows multiple users to interact, collaborate or compete in real time on the same screen, and is widely used in education, entertainment, meetings, exhibitions and other scenarios. The core is to realize low-latency, high-synchronization multi-user operation experience. In the prior art, screen interaction technology mainly relies on a local network, i.e. all terminal devices participating in screen interaction need to be connected in the same local area network. In actual use, the flexibility is poor, and the performance of the home network is limited, so the user experience of screen interaction is also poor. For example, in the home entertainment scenario, multiple users connect through Wi-Fi to play screen games. Due to the limitations of Wi-Fi bandwidth and the differences in device performance, some users may experience video lag and audio out of sync, resulting in a fragmented experience.

[0004] Using a cloud mobile phone can effectively improve the user experience of screen interaction technology. However, because multiple different types of terminal devices may participate in screen interaction, the hardware performance and network performance of different terminal devices may be different, and the data transmitted during screen interaction is mainly multimedia data such as video data and audio data, which has certain requirements for the hardware performance and network performance of terminal devices. When using a cloud mobile phone for screen interaction, it is difficult for the cloud mobile phone to accurately provide good multimedia data transmission services for different types of terminal devices. SUMMARY

[0005] In order to solve the problems in the prior art, the present application provides a multimedia data processing method and system for cloud mobile phone screen interaction, computer equipment and storage medium. The cloud mobile phone can select the optimal multimedia data processing mode based on the multimedia data request of the terminal device, the hardware constraint condition and the network constraint condition of the terminal device, ensure that the terminal device can be provided with smooth multimedia data transmission services, and ensure user experience.

[0006] To achieve the above object, the specific scheme adopted by the present application is:

[0007] A multimedia data processing method for cloud phone screen interaction, comprising the following steps:

[0008] The cloud phone acquires multimedia data requests of all terminal devices participating in the screen interaction;

[0009] The first dynamic transmission parameter is allocated to the terminal device based on the hardware constraint condition of the terminal device;

[0010] The second dynamic transmission parameter is allocated to the terminal device based on the network constraint condition of the terminal device;

[0011] The data processing task corresponding to the terminal device is generated by fusing the multimedia data request, the first dynamic transmission parameter and the second dynamic transmission parameter;

[0012] The data processing task is executed by the cloud phone, and the multimedia data interaction is performed with the terminal device.

[0013] Preferably, the multimedia data request is a data uplink request or a data parallel request, when the multimedia data request is the data uplink request, the terminal device requests to send the multimedia data to the cloud phone, when the multimedia data request is the data downlink request, the terminal device requests to acquire the multimedia data from the cloud phone, and when the multimedia data request is the data parallel request, the terminal device requests to send the multimedia data to the cloud phone while acquiring the multimedia data from the cloud phone.

[0014] Preferably, the data uplink request and the data parallel request each include at least one data type, and the data type is audio data, video data or operation data.

[0015] Preferably, the data uplink request and the data parallel request each include a data standard corresponding to the data type, and the data standard includes data capacity and data quality.

[0016] Preferably, the method for allocating the first dynamic transmission parameter to the terminal device based on the hardware constraint condition of the terminal device comprises:

[0017] The cloud phone analyzes the multimedia data request to acquire the data type and the data standard;

[0018] The cloud phone issues a hardware query instruction to the terminal device;

[0019] The terminal device sends its hardware parameters to the cloud phone based on the hardware query instruction;

[0020] The cloud phone allocates the first dynamic transmission parameter to the terminal device based on the hardware parameters of the terminal device.

[0021] Preferably, after obtaining the hardware parameters of the terminal device, the cloud phone saves the hardware parameters based on a preset lifecycle.

[0022] Preferably, the method for allocating a second dynamic transmission parameter to a terminal device based on the network constraints of the terminal device includes:

[0023] The cloud phone sends test data packets to the terminal device;

[0024] After receiving the test data packet, the terminal device sends a return data packet to the cloud phone;

[0025] The cloud phone assigns a second dynamic transmission parameter to the terminal device based on the transmission process of test data packets and return data packets.

[0026] A multimedia data processing system for screen sharing interaction of cloud phones, used to implement the aforementioned multimedia data processing method for screen sharing interaction of cloud phones, the system comprising:

[0027] The data acquisition module is used to enable the cloud phone to acquire multimedia data requests from all terminal devices participating in the screen sharing interaction;

[0028] The parameter design module is used to allocate a first dynamic transmission parameter to the terminal device based on the hardware constraints of the terminal device, and to allocate a second dynamic transmission parameter to the terminal device based on the network constraints of the terminal device.

[0029] The task generation module is used to integrate multimedia data requests, first dynamic transmission parameters, and second dynamic transmission parameters to generate data processing tasks corresponding to the terminal device.

[0030] Computer equipment, including:

[0031] Memory, used to store computer programs;

[0032] A processor is used to execute the computer program to implement the above-described multimedia data processing method for screen sharing interaction on cloud phones.

[0033] A storage medium for storing a computer program that, when executed, implements the aforementioned multimedia data processing method for screen sharing and interaction on a cloud phone.

[0034] In the process of multi-terminal device screen-sharing interaction based on cloud phones using this invention, the cloud phone can select the optimal multimedia data processing method based on the multimedia data requests of the terminal devices, the hardware constraints of the terminal devices, and the network constraints, so as to ensure that it can provide smooth multimedia data transmission services to the terminal devices and guarantee user experience. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 This is a flowchart of the data processing method of the present invention;

[0037] Figure 2 This is a flowchart for allocating the first dynamic transmission parameters;

[0038] Figure 3 This is a flowchart for allocating the second dynamic transmission parameters;

[0039] Figure 4 This is a structural block diagram of the data processing system of the present invention. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] like Figure 1 As shown, a multimedia data processing method for screen sharing interaction of cloud phones includes S1 to S5.

[0042] S1. The cloud phone acquires multimedia data requests from all participating terminal devices in the screen-sharing interaction. When multiple terminal devices interact on the cloud phone, their multimedia data requirements differ. For example, smartphones may need to transmit audio data, video data, game data, or mixed data containing various types of data, while products like smart speakers may only need to transmit audio data. Therefore, it is necessary to actively acquire the multimedia data requests from the terminal devices and then select and provide the correct multimedia data to each terminal device based on its request.

[0043] Furthermore, multimedia data requests can be either uplink requests or parallel data requests. When a multimedia data request is an uplink request, the terminal device requests to send multimedia data to the cloud phone. When a multimedia data request is a downlink request, the terminal device requests to retrieve multimedia data from the cloud phone. When a multimedia data request is a parallel data request, the terminal device requests to send multimedia data to the cloud phone while simultaneously retrieving multimedia data from the cloud phone. Specifically, for terminal devices such as smart speakers or surveillance cameras, most can only upload multimedia data and cannot retrieve and play multimedia data from the cloud phone. For terminal devices such as smartphones and tablets, they can both retrieve and play multimedia data from the cloud phone and simultaneously collect and upload multimedia data to the cloud phone. Therefore, different types of multimedia data requests need to be pre-configured so that the terminal device can better establish a connection with the cloud phone and transmit multimedia data. In this invention, multimedia data requests are divided into two categories: uplink requests and parallel data requests. The appropriate type of multimedia data request can be selected according to the specific situation of the terminal device.

[0044] Considering the varying requirements of different types of multimedia data on network performance between terminal devices and cloud phones, as well as the data processing performance of the terminal devices—for example, video data demands higher network and data processing performance—it is necessary to determine the data types of multimedia data required by the terminal devices to enable better screen-sharing interaction. Therefore, both data uplink requests and data parallel requests must include at least one data type, which can be audio data, video data, or operational data. Operational data refers to system command data and response data generated by user operations on the terminal device. For example, when a user uses a smartphone in conjunction with a cloud phone to run a cloud game, the game program runs on the cloud phone, while the user's operations are performed on the smartphone. The user's operations generate command data on the smartphone, which is then uploaded to the cloud phone, processed by the game program, and generates response data. This response data is then sent to the smartphone, which responds to the user's actions based on the response data. For cloud gaming applications, in addition to transmitting operational data, video and audio data also need to be transmitted synchronously. Therefore, when a smartphone runs a cloud game, it should send a data parallel request, and the data type should simultaneously include audio data, video data, and operational data. For example, when users use smartphones, cloud phones, and surveillance cameras to achieve screen mirroring, users may need to adjust the surveillance camera, such as rotating the camera's angle, zooming in and out of the video feed, and controlling the camera to take photos. In this case, the surveillance camera should also send parallel data requests, and the data types should include audio data, video data, and operation data simultaneously.

[0045] Different data types consume different amounts of cloud phone resources. For example, video data requires more storage and data processing resources than audio data. Therefore, to enable cloud phones to better transmit multimedia data with terminal devices, both uplink and parallel data requests include data standards corresponding to the data type. These standards include data capacity and data quality. Data capacity refers to the size of the multimedia data, while data quality has different meanings for different data types. For example, for video data, data quality includes resolution, bitrate, and frame rate; for audio data, data quality includes sampling rate and bitrate.

[0046] Based on S1, after the cloud phone receives a multimedia data request from the terminal device, it first parses whether the multimedia data request is an uplink request or a parallel request, and then parses the data type and data standard. In this way, it can call appropriate resources according to the request type, data type and data standard, and perform appropriate processing on the multimedia data, thereby better transmitting multimedia data with the terminal device.

[0047] S2. Allocate first dynamic transmission parameters to the terminal device based on the hardware constraints of the terminal device. More specifically, the method for allocating first dynamic transmission parameters to the terminal device based on the hardware constraints of the terminal device includes S21 to S23, such as... Figure 2 As shown.

[0048] S21. The cloud phone parses multimedia data requests to obtain data types and data standards.

[0049] S22, The cloud phone sends a hardware query command to the terminal device.

[0050] S23. The terminal device sends its own hardware parameters to the cloud phone based on the hardware query command.

[0051] As previously explained, different data types and standards place varying demands on the hardware and network performance of terminal devices. To enable better screen mirroring interaction and for cloud phones to provide multimedia data mirroring services to terminal devices, adjustments need to be made based on the actual conditions of the terminal devices. Cloud phones send hardware query commands to terminal devices, which in turn trigger responses, allowing the cloud phone to assess the actual hardware and network performance of the terminal devices. Specifically, a hardware query command can include multiple instruction units, each used to query at least one piece of hardware. For example, a hardware query command might include a processor instruction unit, an internal memory instruction unit, an external memory instruction unit, a display instruction unit, and a network module instruction unit. The processor instruction unit queries the terminal device's processor, the internal memory instruction unit queries the terminal device's internal memory, the external memory instruction unit queries the terminal device's external memory, the display instruction unit queries the terminal device's display screen, and the network module instruction unit queries the terminal device's network module. More specifically, the hardware parameters of a terminal device's processor may include the processor model, operating frequency, and power supply status; the hardware parameters of internal memory may include memory capacity and frequency; the hardware parameters of external memory may include hard disk capacity and read / write speed; the hardware parameters of the display screen may include resolution, color gamut, and refresh rate; and the hardware parameters of the network module may include model and maximum transmission rate. All these hardware parameters are existing technologies in this field, and their definitions and data ranges will not be elaborated upon here.

[0052] Considering that terminal devices typically have a long lifespan for users, meaning users will use a single terminal device for extended periods, and consequently, a user may use the same terminal device multiple times for screen mirroring interactions, a solution is proposed. To avoid the cloud phone sending hardware query commands to the terminal device every time a screen mirroring interaction occurs, the cloud phone, after obtaining the terminal device's hardware parameters, saves these parameters based on a preset lifespan. In this way, after obtaining the terminal device's hardware parameters using the hardware query command, the cloud phone saves the hardware parameters for a certain period. During this period, if the terminal device performs screen mirroring interaction again, the cloud phone can directly access the corresponding hardware parameters, thus avoiding repeated queries of the terminal device's hardware parameters. This reduces the load on both the cloud phone and the terminal device. Especially for terminal devices with limited hardware performance, this method ensures more stable operation, preventing lag or even crashes, and guaranteeing a better user experience.

[0053] Furthermore, the lifecycle of hardware parameters is divided into three types: the first is short-term caching, with a duration of 1 hour, which is suitable for devices that frequently participate in screen sharing, such as cloud gaming devices; the second is medium-term storage, with a duration of 7 days, which is suitable for devices that are used occasionally, such as smart speakers; and the third is long-term archiving, with a duration of 30 days, which is suitable for analyzing historical data and optimizing future task allocation.

[0054] Furthermore, when a change in hardware parameters is detected to exceed a threshold, such as a 10% decrease in processor performance or a 20% fluctuation in network bandwidth, the cloud phone proactively triggers a re-query mechanism to terminate the lifecycle of the hardware parameters.

[0055] In one embodiment of the present invention, the formula for calculating the first dynamic transmission parameter (HDP) is as follows:

[0056]

[0057] Where P represents processor performance, which can be measured in GHz, and M represents memory capacity, which can be measured in GB. This indicates the processor performance of the actual terminal device. Indicates the processor performance of the terminal device requested by the standard; This indicates the actual memory capacity of the terminal device. This represents the memory capacity of the terminal device requested by the standard. α and β are weighting coefficients; for example, α=0.6, β=0.4.

[0058] S24. The cloud phone allocates a first dynamic transmission parameter to the terminal device based on the terminal device's hardware parameters. Specifically, after the terminal device feeds back its hardware parameters to the cloud phone, the cloud phone can determine whether the terminal device's hardware performance matches the data standard based on the received hardware parameters. If the terminal device's hardware performance can achieve the transmission and processing of multimedia data based on the data standard, then the hardware performance matches the data standard, and the cloud phone uses the data standard as the first dynamic transmission parameter. If the terminal device's hardware performance cannot achieve the transmission and processing of multimedia data based on the data standard, then the hardware performance does not match the data standard, and the cloud phone needs to adjust the first dynamic transmission parameter based on the data standard.

[0059] The method for determining the first dynamic transmission parameters based on data standards by a cloud phone includes the following steps: First, the cloud phone compresses the data standard based on the hardware performance of the terminal device, particularly compressing the data quality based on the processor's hardware parameters, so that the terminal device's hardware performance can achieve the transmission and processing of multimedia data based on the compressed data standard. Second, the cloud phone further compresses the data standard based on the terminal device's hardware performance, particularly compressing the data capacity based on the hardware parameters of internal and external memory, so that the terminal device can allocate sufficient cache space for multimedia data processing. Finally, the cloud phone performs redundant compression on the data standard after the two compressions, because the terminal device may perform other tasks besides screen sharing, resulting in limited available hardware performance. Therefore, redundant compression is necessary to enable the terminal device to process multimedia data more stably. Furthermore, if the user grants the cloud phone sufficient permissions to fully manage the operation of other tasks on the terminal device during screen sharing, the actual quality that the user can choose can be left unset, and the cloud phone can control the terminal device entirely based on the first dynamic transmission parameters.

[0060] S3. Allocate second dynamic transmission parameters to the terminal device based on the network constraints of the terminal device. More specifically, the method for allocating second dynamic transmission parameters to the terminal device based on the network constraints of the terminal device includes S31 to S33, such as... Figure 3 As shown.

[0061] S31. The cloud phone sends a test data packet to the terminal device.

[0062] S32. After receiving the test data packet, the terminal device sends a return data packet to the cloud phone.

[0063] S33, the cloud phone allocates a second dynamic transmission parameter to the terminal device based on the transmission process of test data packets and return data packets.

[0064] Both test and return data packets can include a sending timestamp. After receiving the test data packet, the terminal device calculates a first transmission rate based on the reception time, the sending timestamp, and the data size of the test data packet. After receiving the return data packet, the cloud phone calculates a second transmission rate based on the reception time, the sending timestamp, and the data size of the return data packet. Then, the cloud phone calculates the terminal device's current actual network performance based on the average of the first and second transmission rates. Afterwards, the cloud phone optimizes the data based on the terminal device's actual network performance to obtain the second dynamic transmission parameters.

[0065] In one embodiment of the present invention, the test data packet is transmitted using the UDP protocol, is 1MB in size, and contains random data and a timestamp. The test frequency is once every 10 minutes, lasting for 3 seconds, and the following metrics are calculated based on the test data packet:

[0066] Bandwidth = DataSize / RTT;

[0067] Jitter = max(RTT1, RTT2, ..., RTT) n )-min(RTT1,RTT2,……,RTT n );

[0068] Where DataSize is the test data packet size and RTT is the round-trip time.

[0069] More specifically, the method for generating the second dynamic transmission parameters by the cloud phone includes: First, the cloud phone compresses the data standard based on the actual network performance of the terminal device; second, based on the compressed data capacity and the total duration of the multimedia data, the data quality is compressed. By reducing the data quality, the amount of data that the terminal device needs to transmit and process per unit time is reduced, thereby achieving smoother multimedia data transmission; third, redundant compression of the data quality is performed, and the data capacity is adjusted. Although the compressed data quality and data capacity can match the actual network performance of the terminal device, network transmission itself is unstable. When data fluctuations occur, transmission may still be interrupted. Therefore, redundant compression is required to further compress the data quality and data capacity, thereby leaving an adjustable margin for network fluctuations; finally, based on the redundantly compressed data quality, at least two actual quality values ​​are set for users to choose from. Users can flexibly choose according to the actual situation of the terminal device during screen sharing interaction to ensure user experience.

[0070] For example, the actual quality can include three types: the first is low latency mode, which prioritizes reducing resolution and frame rate to ensure operation response speed; the second is high definition mode, which prioritizes increasing resolution and can tolerate slight latency; the third is custom mode, which allows users to manually set resolution, bitrate and frame rate, where the resolution options can be 480p, 720p or 1080p, the bitrate options can be 1Mbps or 5Mbps, and the frame rate options can be 15fps or 30fps.

[0071] In one embodiment of the present invention, the second dynamic transmission parameter (NDP) is calculated based on the round-trip time (RTT) and packet loss rate (PLR) of the network test data packets, and the calculation formula is as follows:

[0072]

[0073] Where γ and δ are weighting coefficients, for example γ=0.7, δ=0.3.

[0074] Furthermore, when a user switches modes, the cloud phone recalculates HDP and NDP in real time and generates new data processing tasks to ensure seamless parameter adjustments.

[0075] S4. A data processing task corresponding to the terminal device is generated by integrating the multimedia data request, the first dynamic transmission parameter, and the second dynamic transmission parameter. Specifically, if the first and second dynamic transmission parameters are consistent, the data processing task can be determined based on either the first or second dynamic transmission parameter; otherwise, the data processing task that minimizes the load on the terminal device is selected. The generated data processing task is mainly used to control the cloud host to perform operations such as parsing, processing, and generating multimedia data. The specific processing method is determined according to the data type and the selected first or second dynamic transmission parameter. The processing method is existing technology and will not be described in detail here.

[0076] When generating data processing tasks, the cloud phone generates a global optimization scheme based on the HDP and NDP of all terminal devices, and can select an appropriate method according to multimedia data requests. For example, it can use load balancing to allocate more resources to high-priority terminal devices. It can also use resource reservation to reserve minimum bandwidth for critical operations such as cloud gaming command transmission. When resource requirements of multiple devices conflict, the cloud phone uses a priority queue to dynamically adjust the task order to ensure that critical tasks are executed first.

[0077] In one embodiment of the present invention, the cloud phone predicts optimal dynamic transmission parameters based on historical hardware parameters and network performance data using a machine learning model, and then generates data processing tasks based on the optimal dynamic transmission parameters. The machine learning model can be an existing model, such as the Transformer model, which will not be elaborated upon here.

[0078] S5: The cloud phone performs data processing tasks and interacts with the terminal device for multimedia data.

[0079] Considering that many users possess similar terminal devices, and many of these devices are even the same model, to further enhance the user experience, if the hardware parameters received by the cloud phone include the terminal device's model number, the cloud phone will store the terminal device's model number when storing the hardware parameters. When other users use the same model of terminal device for screen mirroring, the cloud phone can directly call the first dynamic transmission parameters corresponding to that model of terminal device, without needing to regenerate the first dynamic transmission parameters. This reduces user waiting time and lowers the load on both the terminal device and the cloud phone.

[0080] To ensure data security, the cloud phone and terminal device use the TLS 1.3 protocol for encrypted communication, guaranteeing the confidentiality and integrity of multimedia data. Terminal devices must be authenticated via OAuth 2.0 before access. Furthermore, the hardware parameters of the terminal device are stored in an encrypted database, accessible only to the cloud phone's core services, and are periodically subject to differential privacy processing to prevent data leakage.

[0081] like Figure 2 As shown, the present invention also provides a multimedia data processing system for cloud phone screen sharing interaction, which is used to implement the above-mentioned multimedia data processing method for cloud phone screen sharing interaction. The system includes a data acquisition module, a parameter design module, and a task generation module.

[0082] The data acquisition module is used to enable cloud phones to acquire multimedia data requests from all terminal devices participating in screen sharing.

[0083] The parameter design module is used to allocate a first dynamic transmission parameter to the terminal device based on the hardware constraints of the terminal device, and to allocate a second dynamic transmission parameter to the terminal device based on the network constraints of the terminal device.

[0084] The task generation module is used to integrate multimedia data requests, first dynamic transmission parameters, and second dynamic transmission parameters to generate data processing tasks corresponding to the terminal device.

[0085] Furthermore, the system also includes a multi-device priority management submodule, which is used to allocate resource priorities based on the type of terminal device and user permissions.

[0086] The present invention further provides a computer device, including a memory and a processor.

[0087] Memory is used to store computer programs.

[0088] A processor is used to execute the computer program to implement the above-described multimedia data processing method for screen sharing interaction on cloud phones.

[0089] Finally, the present invention provides a storage medium for storing a computer program, which, when executed, implements the above-described multimedia data processing method for screen sharing interaction on cloud phones.

[0090] It should be understood that the processor in the embodiments of the present invention can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0091] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0092] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage system such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0093] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0094] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0095] In the embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0096] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer system (which may be a personal computer, server, or network system, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0097] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0098] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multimedia data processing method for screen mirroring interaction on cloud phones, characterized in that, Includes the following steps: The cloud phone obtains multimedia data requests from all terminal devices participating in the screen sharing interaction; The first dynamic transmission parameter is assigned to the terminal device based on the hardware constraints of the terminal device. The second dynamic transmission parameter is assigned to the terminal device based on the network constraints of the terminal device. The data processing task corresponding to the terminal device is generated by integrating multimedia data requests, first dynamic transmission parameters, and second dynamic transmission parameters. The cloud phone performs data processing tasks and interacts with the terminal device for multimedia data.

2. The multimedia data processing method for screen sharing interaction in cloud phones as described in claim 1, characterized in that, Multimedia data requests can be either uplink requests or parallel requests. When a multimedia data request is an uplink request, the terminal device requests to send multimedia data to the cloud phone. When a multimedia data request is a downlink request, the terminal device requests to retrieve multimedia data from the cloud phone. When a multimedia data request is a parallel request, the terminal device requests to send multimedia data to the cloud phone while simultaneously retrieving multimedia data from the cloud phone.

3. A multimedia data processing method for screen sharing interaction on cloud phones as described in claim 2, characterized in that, Both data uplink requests and data parallel requests include at least one data type, which can be audio data, video data, or operational data.

4. A multimedia data processing method for screen sharing interaction on cloud phones as described in claim 3, characterized in that, Both data uplink requests and data parallel requests include data standards corresponding to the data type, which include data capacity and data quality.

5. A multimedia data processing method for screen sharing interaction in cloud phones as described in claim 4, characterized in that, The method for allocating first dynamic transmission parameters to a terminal device based on the hardware constraints of the terminal device includes: Cloud phones parse multimedia data requests to obtain data types and standards; Cloud phones send hardware query commands to terminal devices; The terminal device sends its own hardware parameters to the cloud phone based on a hardware query command. Cloud phones allocate the first dynamic transmission parameters to the terminal device based on the terminal device's hardware parameters.

6. A multimedia data processing method for screen sharing interaction in cloud phones as described in claim 5, characterized in that, After obtaining the hardware parameters of the terminal device, the cloud phone saves the hardware parameters based on a preset lifecycle.

7. A multimedia data processing method for screen sharing interaction on cloud phones as described in claim 5, characterized in that, Methods for allocating second dynamic transmission parameters to terminal devices based on network constraints include: The cloud phone sends test data packets to the terminal device; After receiving the test data packet, the terminal device sends a return data packet to the cloud phone; The cloud phone assigns a second dynamic transmission parameter to the terminal device based on the transmission process of test data packets and return data packets.

8. A multimedia data processing system for screen mirroring interaction on cloud phones, characterized in that, The system is used to implement a multimedia data processing method for screen sharing interaction on cloud phones as described in any one of claims 1-7, the system comprising: The data acquisition module is used to enable the cloud phone to acquire multimedia data requests from all terminal devices participating in the screen sharing interaction; The parameter design module is used to allocate a first dynamic transmission parameter to the terminal device based on the hardware constraints of the terminal device, and to allocate a second dynamic transmission parameter to the terminal device based on the network constraints of the terminal device. The task generation module is used to integrate multimedia data requests, first dynamic transmission parameters, and second dynamic transmission parameters to generate data processing tasks corresponding to the terminal device.

9. A computer device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement a multimedia data processing method for screen sharing interaction of cloud phones as described in any one of claims 1-7.

10. A storage medium, characterized in that, Used to store a computer program, which, when executed, implements a multimedia data processing method for screen sharing interaction of cloud phones as described in any one of claims 1-7.

Images