Media data transmission control method and apparatus, and storage medium

Abstract

The embodiment of the present application discloses a media data transmission control method, device and storage medium. The above method comprises the following steps: receiving media data from a media data server; forwarding the media data to at least one first client corresponding to the first controller; obtaining state information fed back by a target client, the target client being any client in the at least one first client; and executing a control strategy determined based on the state information, the control strategy being used to control the media data transmitted between the first controller and the target client. The client of the embodiment of the present application can interact with the corresponding controller, so that the controller can control the media data transmitted between the client and the controller without affecting other clients. The clients are highly decoupled, the control path is shorter, the network resources are fully saved, the control sensitivity is improved, and the communication quality is improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of computer, and particularly relate to a media data transmission control method and device and a storage medium. BACKGROUND

[0002] With the development of wireless communication technology, multimedia interaction has become an important channel for people to exchange information. However, wireless networks generally have problems such as signal attenuation and network congestion, which can easily cause network packet loss, delay or jitter, resulting in problems such as discontinuity and lag in multimedia interaction, and reducing the interaction quality. In order to improve the interaction quality, the related technology usually controls the client participating in the multimedia interaction in a C2C (Client to Client) mode, but this control method has a long control path, which reduces the control quality and also wastes network resources. SUMMARY

[0003] In order to shorten the control path, save network resources, and improve control quality, embodiments of the present application provide a media data transmission control method and device and a storage medium.

[0004] In one aspect, a media data transmission control method is provided, applied to a first controller, and the method comprises:

[0005] receiving media data from a media data server;

[0006] forwarding the media data to at least one first client corresponding to the first controller;

[0007] obtaining state information fed back by a target client, the target client being any client in the at least one first client;

[0008] executing a control strategy determined based on the state information, the control strategy being used to control media data transmitted between the first controller and the target client.

[0009] In another aspect, a media data transmission control method is provided, applied to a first controller, and the method comprises:

[0010] receiving media data sent by a target client, the target client being any first client corresponding to the first controller;

[0011] sending the media data to a media data server to trigger the media data server to forward the media data;

[0012] detecting the media data to obtain state information;

[0013] execute a control strategy determined based on the state information, the control strategy being used to control media data transmitted between the first controller and the target client.

[0014] In another aspect, an embodiment of the present application provides a media data transmission control device, which is applied to a first controller, and the device comprises:

[0015] a first media data receiving module, configured to receive media data from a media data server;

[0016] a downlink sending module, configured to forward the media data to at least one first client corresponding to the first controller;

[0017] a feedback receiving module, configured to acquire state information fed back by a target client, the target client being any one of the at least one first client;

[0018] a first control module, configured to execute a control strategy determined based on the state information, the control strategy being used to control media data transmitted between the first controller and the target client.

[0019] In another aspect, an embodiment of the present application provides a media data transmission control device, which is applied to a first controller, and the device comprises:

[0020] a second media data receiving module, configured to receive media data sent by a target client, the target client being any one of first clients corresponding to the first controller;

[0021] an uplink sending module, configured to send the media data to a media data server, so as to trigger the media data server to forward the media data;

[0022] a state detecting module, configured to detect the media data to obtain state information;

[0023] a second control module, configured to execute a control strategy determined based on the state information, the control strategy being used to control media data transmitted between the first controller and the target client.

[0024] In another aspect, an embodiment of the present application provides a computer readable storage medium, characterized in that the computer readable storage medium stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by a processor to implement the above-mentioned media data transmission control method.

[0025] In another aspect, an electronic device is provided, and has at least one processor and a memory communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor, and the at least one processor implements the media data transmission control method described above by executing the instructions stored in the memory.

[0026] Embodiments of the present application provide a media data transmission control method, device and storage medium. In the embodiments of the present application, a client can interact with a corresponding controller, so that the controller can control the media data transmitted between the client and the controller without affecting other clients. The clients are highly decoupled, the control path is shorter, network resources are fully saved, control sensitivity is improved, and thus the communication quality is improved. BRIEF DESCRIPTION OF DRAWINGS

[0027] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the related art, the drawings needed to be used in the embodiments or the related art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0028] Figure 1 is a media data packet loss control schematic diagram in a two-end communication scenario provided by the embodiments of the present application;

[0029] Figure 2 is a media data packet loss control schematic diagram in a multi-end communication scenario provided by the embodiments of the present application;

[0030] Figure 3 is a feasible implementation framework schematic diagram of implementing the media data transmission control method provided by the embodiments of the present application;

[0031] Figure 4 is an application scenario schematic diagram of the implementation framework provided by the embodiments of the present application;

[0032] Figure 5 is a media data transmission control method flowchart provided by the embodiments of the present application;

[0033] Figure 6 is a schematic diagram of one application scenario of the media data transmission control method provided by the embodiments of the present application;

[0034] Figure 7 is another application scenario schematic diagram of the media data transmission control method provided by the embodiments of the present application;

[0035] Figure 8is a flowchart of a media data transmission control method in an uplink scenario provided by an embodiment of the present application;

[0036] Figure 9 is a schematic diagram of a media data transmission control method provided by an embodiment of the present application;

[0037] Figure 10 is a block diagram of a media data transmission control device provided by an embodiment of the present application;

[0038] Figure 11 is another block diagram of a media data transmission control device provided by an embodiment of the present application;

[0039] Figure 12 is a hardware structure schematic diagram of a device for implementing the method provided by an embodiment of the present application. DETAILED DESCRIPTION

[0040] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0041] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or server including a series of steps or units does not necessarily have to include only those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to the process, method, product or device.

[0042] In order to make the purposes, technical solutions and advantages of the present application disclosed more clear, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and do not limit the present application.

[0043] Hereinafter, the terms "first", "second", "third", "fourth", "fifth", "sixth", "seventh" and "eighth" are used only for descriptive purposes and are not to be construed as indicating or implying relative importance or a specific number of technical features indicated. Thus, features defined with "first", "second", "third", "fourth", "fifth", "sixth", "seventh" and "eighth" can explicitly or implicitly include one or more of the features. In the description of the embodiments, unless otherwise specified, the meaning of "a plurality of" is two or more. In order to facilitate understanding of the technical solutions of the embodiments of the present application and the technical effects generated thereby, the present application first explains the related professional terms:

[0044] C2C: Client to Client, client to client. In the embodiments of the present application, it represents a communication architecture of client to client. Under the C2C communication architecture, the two clients communicating with each other affect each other, and if it is multi-end communication, each client participating in the communication will also be affected, so it is a highly coupled communication architecture of each client.

[0045] S2C: Server to Client, server to client. In the embodiments of the present application, it represents a communication architecture of server to client. Under the S2C communication architecture, the client only interacts with the controller directly connected for communication, which can be understood as a kind of control server. Under this communication architecture, whether it is double-end communication or multi-end communication, it will not affect other clients, so it is a decoupled communication architecture of the client.

[0046] RTP: Real-time Transport Protocol, real-time transport protocol. The RTP protocol specifies the standard packet format for transmitting audio and video over the Internet. It was originally designed as a multicast protocol, but later used in many unicast applications. The RTP protocol is commonly used in streaming media systems (in conjunction with the RTSP protocol (Real Time Streaming Protocol)), video conferencing and push-to-talk (Push to Talk) systems, making it a technical basis for the IP (Internet Protocol) phone industry. The RTP protocol can be used with the RTP control protocol RTCP, and it is created on the UDP (User Datagram Protocol) protocol.

[0047] RTCP: Real-time Transport Control Protocol or RTP Control Protocol. RTCP collects statistics of the associated media connection, such as: number of transmitted bytes, number of transmitted packets, number of lost packets, Jitter (clock jitter), one-way and two-way network delay, etc. Network applications can use the statistics of RTCP to control the quality of transmission, such as limiting information flow or using a smaller codec when the network bandwidth is heavily loaded.

[0048] In the related art, transmission control needs to be performed in a weak network environment, and the transmission control is usually in a C2C mode. Embodiments of the present application do not limit the control object, such as packet loss control, delay control, and jitter control. The following briefly introduces a general process of controlling various control objects in a C2C mode in a two-end communication and multi-end communication scenario.

[0049] Taking two-end communication in a C2C mode as an example, the receiving client generally performs detection on the received media data and feeds back the detection result to the sending client, and the sending client can adjust the sending strategy according to the detection result. Taking packet loss control as an example, please refer to Figure 1 , which shows a media data packet loss control diagram in a two-end communication scenario. Client B is the receiving client and client A is the sending client for explaining the transmission control in a C2C mode. Client B as the receiving client can obtain media data from the media forwarding server, and the media data is transmitted to the media forwarding server by the sending client (client A). The transmission of the media data is based on the RTP protocol. Client B detects the media data, and if packet loss is found, it can feed back the control signaling related to the packet loss control to the media forwarding server based on the RTCP protocol. The media forwarding server feeds back the control signaling to client A, and client A compensates for the packet loss according to the control signaling, such as increasing redundant media data. The redundant media data can be determined based on the forward error correction (FEC) technology or the automatic repeat request (ARQ), and embodiments of the present application do not repeat the description.

[0050] Taking multi-end communication in a C2C mode as an example, a receiving client generally performs detection on the received media data and feeds back the detection result to the sending client, and the sending client can adjust the sending strategy according to the detection result, and all clients that can receive the media data sent by the sending client will be affected by the sending strategy adjustment. Still taking packet loss control as an example, please refer to Figure 2Fig. 1 shows a schematic diagram of media data packet loss control in a multi-party communication scenario. Take client a and client b as receiving clients, and client c as a sending client as an example. Client a as a receiving client can obtain media data from a media forwarding server, which is transmitted to the media forwarding server by the sending client (client c). The transmission of the media data is based on the RTP protocol. Client a probes the media data, and if it finds packet loss, it can feed back relevant control signaling for controlling packet loss to the media forwarding server based on the RTCP protocol. The media forwarding server feeds back the control signaling to client c, and client c compensates for packet loss according to the control signaling and increases redundant media data. The redundant media data is transmitted to all receiving clients relative to client c without distinction, that is, client a and client b both receive the redundant media data.

[0051] Of course, the above receiving client and sending client are relative concepts, and in some scenarios at some moments, they can be interchangeable, which is not described in detail herein.

[0052] As can be seen from the above, in the C2C mode, the media data server almost does not control the media transmission process, but instead, the control right is decentralized to the clients, thereby causing high coupling between the clients, and the clients can affect each other. Take Figure 1 For example, client B finds packet loss, and first needs to inform client A via the media forwarding server, and then needs client A to make control adjustments to increase redundant media data, and then transmit the redundant media data to client B via the media forwarding server, which obviously causes a long control path, thereby causing waste of transmission time, reducing control sensitivity, and additionally consuming more network resources. Take Figure 2 For example, in a multi-party communication scenario, the problem of a long control path is consistent with that in a two-party communication, and detection of packet loss by one receiving client in a multi-party communication process will cause all receiving clients to correspondingly receive redundant media data, which adversely affects other clients, and further exacerbates waste of transmission time and consumption of network resources.

[0053] The high coupling between the clients in the C2C mode causes a long control path, and causes waste of transmission time and consumption of network resources. In order to solve this technical problem, the embodiments of the present application provide a media data transmission control method which can decouple the clients, shorten the control path, improve control flexibility, shorten transmission time, and save network resources.

[0054] The method provided by the embodiments of the present application can be related to a blockchain, that is, the method provided by the embodiments of the present application can be implemented based on a blockchain, or the data involved in the method provided by the embodiments of the present application can be stored based on a blockchain, or the execution subject of the method provided by the embodiments of the present application can be located in a blockchain. The blockchain is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, and encryption algorithm. The blockchain (Blockchain) is essentially a decentralized database, and is a series of data blocks associated using a cryptographic method, each data block containing information of a batch of network transactions, for verifying the validity (anti-fake) of the information and generating the next block. The blockchain can include a blockchain underlying platform, a platform product service layer, and an application service layer.

[0055] The blockchain underlying platform can include user management, basic services, smart contracts, and operation monitoring processing modules. The user management module is responsible for identity information management of all blockchain participants, including maintaining public and private key generation (account management), key management, and maintenance of the correspondence between real identities and blockchain addresses (permission management), and in the case of authorization, supervising and auditing the transaction of certain real identities, providing risk control rule configuration (risk audit); the basic service module is deployed on all blockchain node devices to verify the validity of business requests, and after consensus, the valid requests are recorded on the storage. For a new business request, the basic service first performs interface adaptation analysis and authentication processing (interface adaptation), then encrypts the business information through a consensus algorithm (consensus management), and after encryption, the complete and consistent information is transmitted to the shared ledger (network communication) and recorded and stored; the smart contract module is responsible for contract registration and issuance, contract triggering and contract execution. Developers can define contract logic through a certain programming language, publish it to the blockchain (contract registration), call keys or other events to trigger execution according to the logic of the contract terms, complete the contract logic, and also provide contract upgrade and cancellation functions; the operation monitoring module is mainly responsible for deployment, configuration modification, contract setting, cloud adaptation during product release, and real-time state visualization output during product operation, such as alarm, monitoring network conditions, and monitoring node device health status.

[0056] The platform product service layer provides basic capabilities and implementation frameworks for typical applications. Developers can stack business characteristics based on these basic capabilities to complete blockchain implementation of business logic. The application service layer provides application services based on the blockchain solution for business participants to use.

[0057] Please refer to Figure 3 , Figure 3 is a feasible implementation framework diagram for implementing the media data transmission control method provided by the embodiments of the present application, as shown in Figure 3As shown, the implementation framework can at least include a first client 01, a first controller 02, a second client 03, a second controller 04, and a media data server 05. One first controller 02 can control at least one first client 01, one second controller 04 can control at least one second client 03, and the second controller 04 can be one or more. The first client 01 and the first controller 02 can communicate with each other, the second controller 04 and the second client 03 can also communicate with each other, and the first controller 02 and the second controller 04 can both communicate with the media data server 05. In the embodiments of the present application, there can be a predetermined association between each first client 01 and each second client 03, such as participating in an audio conference, a video conference, or a multimedia call at the same time, or being located in the same live room. Figure 4 An application scenario of the implementation framework is shown, which is a multi-person conversation scenario, and each client participating in the multi-person conversation is located in the implementation framework. Among them, the client corresponding to "I" can be the target client in the following.

[0058] In the embodiments of the present application, the first client can be understood as a media data receiving end, and the second client can be understood as a media data sending end. Of course, the media data receiving end and the media data sending end are a functional definition. In some cases, the first client can also send media data, thereby becoming a media data sending end, and the second client can also receive media data, thereby becoming a media data receiving end.

[0059] In the embodiments of the present application, the first controller controls the first client through bidirectional interaction with the first client, and this control process does not affect the media sending server or other clients. Similarly, the second controller controls the second client through bidirectional interaction with the second client, thereby realizing the decoupling between the clients by setting the first controller and / or the second controller. In the embodiments of the present application, the first controller and the second controller can be independently set in the form of independent physical components, or can be integrated in the media data server. The form of the first controller and the second controller is not limited.

[0060] The first client 01 or the second client 03 described above can be a mobile terminal, a desktop computer, a tablet computer, a notebook computer, a digital assistant, a smart wearable device, and various physical devices that can have communication and navigation capabilities. It can also include software running in a physical device. The media data server 05 described above can include a standalone server, a distributed server, or a server cluster composed of multiple servers. The media data server 05 described above can include a network communication unit, a processor, a memory, and the like.

[0061] The following introduces a media data transmission control method of an embodiment of the present application. The method can be implemented by an electronic device, which can be the first controller 02 in the implementation framework. Figure 5 A flowchart of a media data transmission control method provided by an embodiment of the present application is shown. The embodiment provides the method operation steps as described above in the embodiment or flowchart, but more or fewer operation steps can be included based on conventional or non-inventive labor. The order of steps listed in the embodiment is only one of the many execution orders, and does not represent the only execution order. In actual system or server product execution, the method can be executed in sequence or in parallel (for example, in a parallel processor or multi-threaded processing environment), and the method can include:

[0062] S101. Receiving media data from a media data server.

[0063] For example, steps S101 to S107 describe the media data transmission control method from the perspective of a downstream scenario. The first controller 02 obtains media data from the media data server 05 and distributes the media data to each first client 01 in communication connection with the first controller 02. For ease of description, in the embodiment of the present application, all the first clients 01 are in communication connection with the first controller 02, and any controller other than the first controller 02 is a second controller 04, and all the second clients 03 are in communication connection with the second controller 04. Figure 3 S103. Forwarding the media data to at least one first client corresponding to the first controller.

[0064] The embodiment of the present application does not limit the number of first clients sent by the first controller. For example, the first controller can send media data to all first clients in connection therewith, or can send media data to one or more first clients in connection therewith.

[0065] S105. Obtaining state information fed back by the target client, the target client being any client in the at least one first client.

[0066] The embodiment of the present application does not limit the state information. The state information includes at least one of the following: packet loss state information, jitter state information, bandwidth state information, delay state information, and parameter control information.

[0067] For example, the first controller 02 obtains state information fed back by the target client 01, and the target client 01 is any first client in the at least one first client 01.

[0068] Figure 6 ​This diagram illustrates one application scenario of the media data transmission control method provided in this application. A first controller CRA is connected to a single first client CA1, and a second controller CRB is connected to a single second client CB1. The first controller CRA sends media data obtained from a media data server to the first client CA1. This media data can also be sent from the second client CB1 to the media data server via the second controller CRB. This application scenario typically enables two-way calls, two-way audio calls, or two-way video calls. Figure 6 In the scenario shown, the target client is the first client CA1, and the first client CA1 feeds back the acquired status information to the first controller CRA.

[0069] Please refer to Figure 7 This diagram illustrates another application scenario of the media data transmission control method provided in this application. A first controller CRA connects to two first clients, CA1 and CA2. A second controller CRB connects to a single second client, CB1, and a second controller CRC connects to a single second client, CC1. The first controller CRA sends media data obtained from the media data server to the first clients CA1 and CA2. If the media data is sent from the second client CB1 to the media data server via the second controller CRB, the media data is also transmitted to the second client CC1 via the second controller CRC. This application scenario typically enables multi-terminal calls, multi-terminal audio calls, or multi-terminal video calls. Figure 7 In the scenario shown, the target client can be either the first client ca1 or the first client ca2. Taking the first client ca1 as an example, the first client ca1 will feed back the acquired status information to the first controller cra.

[0070] S107. Execute the control strategy determined based on the above-mentioned state information, the control strategy being used to control the media data transmitted between the first controller and the target client.

[0071] by Figure 6 Taking the scenario shown as an example, the first client CA1 (target client) feeds back the acquired status information to the first controller CRA. The first controller CRA can then determine the corresponding control strategy and adjust the communication process between the first client CA1 and the first controller CRA based on the control strategy. For example, it can adjust the communication parameters or send additional redundant media data, or adjust the parameters related to communication based on the parameter adjustment information in the status information, such as the redundancy rate or buffer size in the first controller.

[0072] by Figure 7Taking the scenario shown as an example, both the first client ca1 and the first client ca2 receive media data. If the first client ca1 (the target client) feeds back the acquired status information to the first controller cra, the first controller cra can determine the corresponding control strategy. Based on this control strategy, the communication process between the first client ca1 and the first controller cra is adjusted without affecting the first client ca2 or any other clients besides the first client ca1.

[0073] This application does not limit the method for determining the control strategy; it can be determined based on a preset control scheme and state information. In one embodiment, when the state information is packet loss state information, executing the control strategy determined based on the state information includes:

[0074] S1071. Determine the transmission control strategy under packet loss conditions.

[0075] The embodiments of this application do not limit the transmission control strategy under packet loss conditions, which can be a forward error correction (FEC) strategy or an automatic repeat request (ARQ) strategy.

[0076] S1073. Based on the above transmission control strategy and the above packet loss status information, determine the redundant media data.

[0077] S1075. Send the aforementioned redundant media data to the target client.

[0078] The redundant media data can be determined based on packet loss status information and transmission control strategy. This application embodiment does not limit the determination method. The redundant information is sent to the target client and not to other clients. That is, only the client that is determined to have packet loss is compensated for data, and other clients are not affected.

[0079] by Figure 6 For example, if the first client CA1 reports packet loss, the first controller CRA will directly send redundant media data to the first client CA1 without affecting other clients (the second client CB1). Figure 7 For example, if the first client ca1 reports packet loss status information, the first controller cra will directly send redundant media data to the first client ca1 without affecting other clients (first client ca2, second client cb1, and second client cc1).

[0080] In another embodiment, when the state information is jitter state information, bandwidth state information, or delay state information, the executing a control strategy determined based on the state information comprises:

[0081] S1072. Determining a transmission control strategy corresponding to the state information.

[0082] The jitter state information, the bandwidth state information, or the delay state information corresponds to different transmission control strategies respectively, and specific transmission control strategies are not limited in the embodiments of the present application.

[0083] S1074. Adjusting a transmission control parameter according to the transmission control strategy and the state information.

[0084] In the present application, specific schemes of adjusting the transmission control parameter according to the transmission control strategy and the state information are not limited. For example, if the jitter state information indicates that the media data generates jitter and the jitter exceeds a preset first threshold, the buffer parameter corresponding to the target client in the first controller can be adjusted to improve the buffer parameter to suppress the jitter. For another example, if the bandwidth state information indicates that the bandwidth is less than a preset second threshold, the frequency of sending data by the first controller to the target client or the compression rate of the media data can be adjusted. For another example, if the delay state information indicates that the delay time exceeds a preset third threshold, the bandwidth configuration parameter corresponding to the target client in the first controller can be adjusted to reduce the delay. The first threshold, the second threshold, or the third threshold is not limited in the embodiments of the present application, and can be configured autonomously according to actual conditions.

[0085] S1076. Sending the media data to be sent to the target client based on the adjusted transmission control parameter.

[0086] The transmission control method in the embodiments of the present application can be widely applied in two-end communication or multi-end communication, and the purpose of communication is not limited in the present application, for example, audio communication or video communication. Taking multi-end communication as an example, in the architecture shown in FIG. 1, the media data can be sent by any second client to trigger the media data server to send the media data to the first type of client and the second type of client, the first type of client refers to other second clients except the second client sending the media data, and the second type of client refers to all first clients corresponding to the first controller. Figure 3

[0087] ​In the media data transmission control method provided in the embodiments of the present application, the client can interact with the corresponding controller, so that the controller can control the media data transmitted between the client and the controller without affecting other clients, which is a S2C transmission control mode. In this mode, the clients are not affected by each other and are highly decoupled. Compared with the C2C transmission control mode, the control path is shorter, the network resources are fully saved, the control sensitivity is improved, and the communication quality is improved.

[0088] The media data transmission control method in the embodiments of the present application is described in detail below in the following uplink scenario. Please refer to Figure 8 which shows a flowchart of the media data transmission control method in the uplink scenario provided by the embodiments of the present application, still applied to the first controller, and the method comprises:

[0089] S201. receiving media data sent by a target client; the target client is any first client corresponding to the first controller.

[0090] S203. sending the media data to a media data server to trigger the media data server to forward the media data.

[0091] S205. detecting the media data to obtain state information.

[0092] S207. executing a control strategy determined based on the state information, the control strategy being used to control the media data transmitted between the first controller and the target client.

[0093] Please refer to Figure 9 which shows a schematic diagram of the media data transmission control method in the uplink scenario provided by the embodiments of the present application, Figure 9 In the uplink scenario, the first client ca1 (target client) sends media data to the first controller cra, the first controller cra detects state information, and controls the communication process between the first client ca1 and the first controller cra according to the state information. The media data transmission control method in the uplink scenario is based on the same inventive concept as described above, and will not be described again here. The media data transmission control method in the uplink scenario can be applied to a two-end communication scenario or a multi-end communication scenario. Taking the multi-end communication scenario as an example, the target client can send the media data to the media data server via the first controller to trigger the media data server to forward the media data to the third type of client and the fourth type of client. The third type of client refers to all second clients having a preset association relationship with the target client, and the fourth type of client refers to other first clients corresponding to the first controller except the target client.

[0094] The present application emphasizes that,Figure 3 The first controller of the implementation framework in the method for controlling media data transmission in the uplink scenario or the downlink scenario can execute the method for controlling media data transmission in the uplink scenario or the downlink scenario, or can have the ability to execute the method for controlling media data transmission in the uplink scenario and the downlink scenario. For example, the first controller can execute the method for controlling media data transmission in the uplink scenario at a first time, and can execute the method for controlling media data transmission in the downlink scenario at a second time.

[0095] The embodiment of the present application further discloses a method for controlling media data transmission, as shown in Figure 10 The method is applied to a first controller, and the method comprises the steps of:

[0096] The first media data receiving module 101 is configured to receive media data from a media data server.

[0097] The downlink sending module 103 is configured to forward the media data to at least one first client corresponding to the first controller.

[0098] The feedback receiving module 105 is configured to obtain state information fed back by a target client, wherein the target client is any client in the at least one first client.

[0099] The first control module 107 is configured to execute a control strategy determined based on the state information, and the control strategy is used to control media data transmitted between the first controller and the target client.

[0100] Specifically, the embodiment of the present application discloses a method for controlling media data transmission and a corresponding device, and the method and the device are both based on the same inventive concept. For details, refer to the method embodiment, which will not be described here.

[0101] The embodiment of the present application further discloses a method for controlling media data transmission, as shown in Figure 11 The method is applied to a first controller, and the method comprises the steps of:

[0102] The second media data receiving module 201 is configured to receive media data sent by a target client, wherein the target client is any first client corresponding to the first controller.

[0103] The uplink sending module 203 is configured to send the media data to a media data server, so as to trigger the media data server to forward the media data.

[0104] The state detecting module 205 is configured to detect the media data to obtain state information.

[0105] The second control module 207 is configured to execute a control strategy determined based on the state information, and the control strategy is used to control media data transmitted between the first controller and the target client.

[0106] Specifically, the embodiments of the present application disclose a media data transmission control device and a corresponding method embodiment of the uplink scenario, both of which are based on the same inventive concept. For details, please refer to the method embodiment, which will not be described here.

[0107] The embodiments of the present application also provide a computer program product or a computer program, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to enable the computer device to execute the media data transmission control device of the uplink scenario or the downlink scenario.

[0108] The embodiments of the present application also provide a computer readable storage medium, which can store a plurality of instructions. The instructions can be adapted to be loaded by a processor and execute the media data transmission control device of the uplink scenario or the downlink scenario described above.

[0109] Further, Figure 12 A hardware structure schematic diagram of a device for implementing the method provided by the embodiments of the present application is shown, and the device can participate in constituting or containing the device or system provided by the embodiments of the present application. As shown in the figure, Figure 12 The device 10 can include one or more processors 102 (the processor 102 can include but is not limited to a microprocessor MCU or a programmable logic device FPGA processing device), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it can also include a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which can be included as one of the ports of the I / O interface), a network interface, a power supply and / or a camera. Those skilled in the art can understand that Figure 12 The structure shown in the figure is only a schematic, and it does not limit the structure of the above-mentioned electronic device. For example, the device 10 can include more or less components than those shown in the figure, or have a different configuration from that shown in the figure. Figure 12 For example, the device 10 can include more or less components than those shown in the figure, or have a different configuration from that shown in the figure. Figure 12 For example, the device 10 can include more or less components than those shown in the figure, or have a different configuration from that shown in the figure.

[0110] It should be noted that the one or more processors 102 and / or other data processing circuitry described above can be generally referred to herein as "data processing circuitry." The data processing circuitry can be embodied in whole or in part as software, hardware, firmware, or any combination thereof. In addition, the data processing circuitry can be a single standalone processing module or incorporated in whole or in part within any of the other elements of the device 10 (or mobile device). As referred to in the embodiments of the present application, the data processing circuitry functions as a processor to control, for example, the selection of the variable resistance terminal path connected to the interface.

[0111] The memory 104 can be used to store software programs of application software and modules, such as the program instructions / data storage means corresponding to the method described above in the embodiments of the present application. The processor 102 can execute various functional applications and data processing by running the software programs and modules stored in the memory 104, i.e., implement the media data transmission control method in the uplink scenario or downlink scenario described above. The memory 104 can include a high-speed random access memory, and can further include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory 104 can further include a memory disposed remotely with respect to the processor 102, which can be connected to the device 10 through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0112] The transmission device 106 is configured to receive or send data via a network. Specific examples of the network can include a wireless network provided by a communication provider of the device 10. In one example, the transmission device 106 includes a network interface controller (NIC) that can be connected to other network devices through a base station to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module configured to communicate with the Internet in a wireless manner.

[0113] The display can be, for example, a touch screen type liquid crystal display (LCD) that can enable a user to interact with the user interface of the device 10 (or mobile device).

[0114] It should be noted that the above-mentioned sequence of the embodiments of the present application is only for description, and does not represent the advantages and disadvantages of the embodiments. And the above-mentioned embodiments of the present application are described. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be executed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multi-task processing and parallel processing are possible or can be advantageous.

[0115] Each of the embodiments in the embodiments of the present application is described in a progressive manner, and the same and similar parts between each embodiment can be referred to each other. Each embodiment focuses on the difference from other embodiments. Especially, for the device and server embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiments.

[0116] A person of ordinary skill in the art can understand that all or part of the steps of the above-mentioned embodiments can be completed by hardware, or by program to instruct relevant hardware to complete. The above-mentioned program can be stored in a computer readable storage medium, and the above-mentioned storage medium can be a read-only memory, a magnetic disk or an optical disk, etc.

[0117] The above-mentioned is only the preferred embodiment of the present application, and does not limit the present application. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A media data transmission method, characterized by, The method is applied to an implementation framework, and the implementation framework comprises a plurality of media data sending ends, a first controller, a plurality of media data receiving ends, a second controller, and a media data server. The first controller controls the plurality of media data sending ends, and the second controller controls the plurality of media data receiving ends. The method comprises the following steps: The first controller receives media data sent by a target client, the target client being any media data sending end. The first controller detects the media data to obtain state information, and executes a control strategy determined based on the state information. The control strategy is only used for adjusting a communication process between the first controller and the target client, and does not affect other media data sending ends except the target client. The media data server forwards the media data to the second controller, the second controller sends the media data to a corresponding media data receiving end, and the media data server sends the media data to other media data sending ends except the target client through the first controller.

2. The method of claim 1, wherein, The state information comprises at least one of the following: packet loss state information, jitter state information, bandwidth state information, delay state information, and parameter control information.

3. The method of claim 2, wherein, In a case where the state information is packet loss state information, the execution of the control strategy determined based on the state information comprises the following steps: determining a transmission control strategy under a packet loss state; determining redundant media data according to the transmission control strategy and the packet loss state information; sending the redundant media data to the target client.

4. The method of claim 2, wherein, In a case where the state information is jitter state information, bandwidth state information, or delay state information, the execution of the control strategy determined based on the state information comprises the following steps: determining a transmission control strategy corresponding to the state information; adjusting a transmission control parameter according to the transmission control strategy and the state information; sending media data to be sent to the target client based on the adjusted transmission control parameter.

5. A computer readable storage medium, characterized in that, The computer readable storage medium stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the media data sending method in any one of claims 1 to 4.

Images