Distributed streaming media system for cross-device multimedia circulation

Abstract

The invention provides a distributed streaming media system for cross-device multimedia circulation, which relates to the technical field of inter-device communication and comprises an application end, a software development kit SDK module, an inter-process communication IPC module, a shared memory module and a streaming media service module. The SDK module and the streaming media service module are independent, and communication between the application end and the streaming media service module is performed through the IPC module and the shared memory module according to the platform of the current equipment, so that the problem of poor adaptability between different platforms is avoided, and the efficiency of multimedia circulation between the equipment is improved.

Description

Technical Field

[0001] The present application relates to the technical field of inter-device communication, and in particular to a distributed streaming media system method, apparatus, electronic device and storage medium for cross-device multimedia streaming. Background Art

[0002] In recent years, with the rapid development of high-speed networks, high-bandwidth storage devices and media coding and compression technology, it has become possible to provide real-time streaming media services through the Internet. Streaming media technology is widely used in all aspects of Internet information services such as multimedia news release, online live broadcast, online advertising, e-commerce, video on demand, distance education, telemedicine, online TV stations and real-time video conferencing.

[0003] In the related art, due to the different data transmission modes in devices of different platforms, when multimedia streaming services are provided between devices, the adaptability between different platforms is poor, resulting in low efficiency of multimedia streaming between devices. Summary of the invention

[0004] The present application aims to solve one of the technical problems in the related art at least to some extent.

[0005] To this end, the first objective of the present application is to propose a distributed streaming media system for cross-device multimedia streaming.

[0006] The second objective of the present application is to provide an electronic device.

[0007] The third object of the present application is to provide a computer-readable storage medium.

[0008] A fourth object of the present application is to provide a computer program product.

[0009] To achieve the above-mentioned purpose, the first embodiment of the present application proposes a distributed streaming media system for cross-device multimedia streaming, including:

[0010] Application side, software development kit SDK module, inter-process communication IPC module, shared memory module, streaming media service module;

[0011] The application end is used to call the SDK module request to request a streaming media service;

[0012] The SDK module is used to generate request information according to the request of the application end, and send the request information to the streaming media service module;

[0013] The streaming media service module is used to perform streaming media services according to the request information, communicate with the SDK module, and perform multimedia streaming services with other devices through streaming media channels.

[0014] Optionally, the streaming media service module includes:

[0015] Streaming media control module, used to control the relevant processes of streaming media services;

[0016] Streaming media module, used to perform multimedia streaming services across devices;

[0017] A streaming media service control module, used to control the relevant processes of the IPC module, the shared memory module and the streaming media module;

[0018] An inter-process communication IPC module, used for transmitting control data between the SDK module and the streaming media service module according to the platform of the current device;

[0019] The shared memory module is used to transmit media data between the SDK module and the streaming media service module according to the platform of the current device.

[0020] Optionally, the SDK module includes:

[0021] An application program interface API module, including a plurality of callable APIs, for accepting calls from the application end to generate request information;

[0022] SDK control module, used for controlling the initialization and deinitialization process of the IPC module and the shared memory module;

[0023] An IPC communication module, used for communicating with the IPC module;

[0024] A shared memory communication module is used for performing data transmission with the shared memory module.

[0025] Optionally, the IPC module includes:

[0026] An IPC framework module, including an API corresponding to IPC, for establishing an IPC process to send the request information or control data to the streaming media service module;

[0027] The first platform adaptation module includes at least one plug-in corresponding to the platform, and is used to receive a call from the IPC framework to execute an IPC communication process corresponding to the current device.

[0028] Optionally, the shared memory module includes:

[0029] A sharing module, used for writing the media data sent by the writing end into the shared memory area, or reading the media data in the shared memory area and sending it to the reading end;

[0030] The synchronization module is used to notify the writing end to perform a writing operation, or to notify the reading end to perform a reading operation.

[0031] The second platform adaptation module includes at least one plug-in corresponding to the platform, and is used to execute the shared memory process corresponding to the current device.

[0032] Optionally, the writing end is the application end, and the receiving end is the streaming media service module; or,

[0033] The writing end is the streaming media service module, and the receiving end is the application end.

[0034] Optionally, the relevant process includes:

[0035] Initialization, deinitialization, startup.

[0036] To achieve the above-mentioned purpose, a second aspect of the present application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0037] The memory stores computer-executable instructions;

[0038] The processor executes the computer-executable instructions stored in the memory to implement the system as described in any one of the first aspects.

[0039] To achieve the above-mentioned purpose, the third aspect of the present application proposes a computer-readable storage medium, in which computer-readable storage medium is stored computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement a system as described in any one of the first aspects.

[0040] To achieve the above-mentioned purpose, the fourth aspect of the present application provides a computer program product, which, when executed by a processor, implements the system described in any one of the first aspects.

[0041] The distributed streaming media system method, apparatus, electronic device and storage medium for cross-device multimedia flow provided by the present application separate the SDK module and the streaming media service module, and communicate between the application end and the streaming media service module according to the platform of the current device through the IPC module and the shared memory module, thereby avoiding the problem of poor adaptability between different platforms and improving the efficiency of multimedia flow between devices.

[0042] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0044] Figure 1 A schematic diagram of the structure of a distributed streaming media system for multi-media streaming across devices provided in an embodiment of the present application;

[0045] Figure 2 A schematic diagram of the structure of a distributed streaming media system for multi-media streaming across devices provided in an embodiment of the present application;

[0046] Figure 3 A schematic diagram of the structure of a distributed streaming media system for multi-media streaming across devices provided in an embodiment of the present application;

[0047] Figure 4 A schematic diagram of the structure of a distributed streaming media system for multi-media streaming across devices provided in an embodiment of the present application;

[0048] Figure 5 A schematic diagram of the structure of a distributed streaming media system for cross-device multimedia streaming provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0050] In modern society, users usually have multiple devices, such as smartphones, tablets, TVs, and computers, etc. Users want to be able to share and transmit media streams, such as audio, video, and images, between these devices to achieve multi-device interoperability, seamless viewing experience, and cross-device streaming.

[0051] Currently, major manufacturers have their own streaming media solutions for cross-device multimedia streaming, which mainly include media data generation / consumption, media data transmission, media data transmission, and media data security technology, based on which the sharing of multimedia data across devices is achieved.

[0052] In one possible embodiment, integrated streaming media is used: for different services on a single end, the streaming capability needs to be integrated with the entire streaming media solution, but only point-to-point services can be implemented, and multimedia streaming from one device to multiple devices cannot be implemented.

[0053] Cross-device multimedia streaming involves multiple platforms and multiple devices. Different platforms require different SDKs, which have poor universality. Each business that requires cross-device multimedia streaming requires high adaptation and maintenance costs.

[0054] To address this problem, the present application provides a distributed streaming media system method for multi-device multimedia streaming. Figure 1 The following is a schematic diagram of the structure of a distributed streaming media system for multimedia streaming across devices provided by an embodiment of the present application. Figure 1 As shown, the system includes:

[0055] An application end 10 , a software development kit (SDK) module 20 , and a streaming media service module 50 ; the streaming media service module 50 includes an inter-process communication (IPC) module 30 and a shared memory module 40 .

[0056] The application end 10 is used to call the SDK module 20 to request a streaming media service;

[0057] The SDK module 20 is used to generate request information according to the request of the application end, and send the request information to the streaming media service module 50;

[0058] The streaming media service module is used to perform streaming media service according to the request information, communicate with the IPC module 30 and the shared memory module 40 in the streaming media service module 50, and perform multimedia streaming service with other devices through the streaming media channel;

[0059] The IPC module 30 is used to transmit control data between the SDK module 20 and the streaming service module 50 according to the platform of the current device;

[0060] The shared memory module 40 is used to transmit media data between the SDK module 20 and the streaming service module 50 according to the platform of the current device.

[0061] In this embodiment, the system is configured on each terminal, and each service in the application end that needs to use the streaming capability needs to integrate the streaming SDK. For a terminal, the SDK module 20 and the streaming media service module 50 therein realize media control and media data transmission through IPC communication and shared memory. For communication between two terminals, the two streaming media service modules corresponding to the two terminals realize device control and media data streaming through network communication, and a distributed architecture for cross-device streaming is realized through IPC communication and shared memory.

[0062] In a possible embodiment, the application end of device A requests to execute service A, and the specific service of service A is to transfer the video data captured by device A to device B. Then the application end of device A executes service A, calls the SDK module to transmit control data to the streaming media service module of device A, and the streaming media service module executes the service, collects the video data captured by the camera, and sends the video data to the streaming media service module of device B through network communication. The streaming media service module of device B writes the video data into the shared memory through the shared memory module in device B, and the application end in device B executes service A to read the specified location in the shared memory and obtain the video data, thereby completing the flow of multimedia data across devices.

[0063] This embodiment provides another distributed streaming media system for multimedia streaming across devices. Figure 2 The following is a schematic diagram of the structure of a distributed streaming media system for multimedia streaming across devices provided by an embodiment of the present application. Figure 2 As shown, the SDK module 20 includes:

[0064] An application programming interface (API) module 21, including a plurality of callable APIs, for accepting calls from the application end to generate request information;

[0065] SDK control module 22, used to control the initialization and deinitialization process of the IPC module and the shared memory module;

[0066] IPC communication module 23, used to communicate with the IPC module;

[0067] The shared memory communication module 24 is used for performing data transmission with the shared memory module.

[0068] In this embodiment, the application side needs to integrate SDK to use the cross-device streaming capability. The SDK framework is mainly divided into three layers: an application program interface module 21 (MiPlayCastSDKProxy), which mainly includes the SDK API layer; an SDK control module 22 (MiPlayCastSDKControl), which is mainly used to control the initialization and deinitialization of the IPC module and the shared memory module; an IPC communication module 23 (MiPlayCastIPC), which is mainly used to communicate and transfer media data between the SDK and the streaming media service module 50 through shared memory.

[0069] This embodiment provides another distributed streaming media system for multimedia streaming across devices. Figure 3The following is a schematic diagram of the structure of a distributed streaming media system for multimedia streaming across devices provided by an embodiment of the present application. Figure 3 As shown, the IPC module 30 includes:

[0070] An IPC framework module 31, including an API corresponding to IPC, for establishing an IPC process to send the request information or control data to the streaming media service module;

[0071] The first platform adaptation module 32 includes at least one plug-in corresponding to the platform, and is used to receive a call from the IPC framework to execute an IPC communication process corresponding to the current device.

[0072] In this embodiment, IPC is used for communication between SDK and streaming media service module 50. Through IPC communication, SDK module 20 can control streaming media service module 50 to perform a series of tasks. The IPC framework is mainly divided into two layers: IPC framework module 31 (MiPlayCast IPC Proxy), which mainly includes the API of IPC module. The interface in this module is a universal interface and is applicable to IPC communication in any platform; the other layer is the first platform adaptation module 32 (platform adaptation layer), which contains plug-ins for multiple platforms. These plug-ins are used to implement IPC communication of the corresponding platform. Optionally, the platform includes: Windows platform / Android platform / Linux platform, then the first platform adaptation module 32 includes Windows IPC module, Android IPC module, and Linux IPC module.

[0073] This embodiment provides another distributed streaming media system for multi-media streaming across devices. Figure 4 The following is a schematic diagram of the structure of a distributed streaming media system for multimedia streaming across devices provided by an embodiment of the present application. Figure 4 As shown, the memory sharing module 40 includes:

[0074] The sharing module 41 is used to write the media data sent by the writing end into the shared memory area, or read the media data in the shared memory area and send it to the reading end;

[0075] The synchronization module 42 is used to notify the writing end to perform a writing operation, or to notify the reading end to perform a reading operation.

[0076] The second platform adaptation module 43 includes at least one plug-in corresponding to a platform, and is used to execute a shared memory process corresponding to the current device.

[0077] In this embodiment, shared memory is used for SDK and runtime media data transmission. IPC communication is suitable for the transmission of smaller volume data, so the IPC module 30 is used to transmit control data. However, for media data (such as video, audio, etc.), its volume is large, and it is suitable to be transmitted in the form of shared memory. Shared memory allows two or more processes to access the same block of memory, just like the malloc() function returns a pointer to the same physical memory area to different processes. When a process changes the content in this address, other processes will notice the change.

[0078] The shared memory framework is mainly divided into two layers: the first layer includes a sharing module 41 (MiPlayCast shared memory Proxy) and a synchronization module 42 (MiPlayCast shared memory SyncProxy), which mainly includes the API of the shared memory module. The interface in this module is a universal interface and is applicable to the shared memory process in any platform. The synchronization module 42 is used to coordinate the actions between the writing end that writes media data into the memory and the reading end that reads the media data in the memory, and notifies the reading end to read after the writing is completed, or notifies the writing end to write after the reading is completed to avoid conflicts.

[0079] Another layer is the second platform adaptation module 43 (platform adaptation layer), which includes plug-ins for multiple platforms, and these plug-ins are used to implement the shared memory process of the corresponding platform. Optionally, the platform includes: Windows platform / Android platform / Linux platform, then the second platform adaptation module 42 includes Windows shared memory data module, Android shared memory data module, Linux shared memory data module; Windows shared memory Sync module, Android shared memory Sync module, Linux shared memory Sync module.

[0080] Optionally, the writing end is the application end, and the receiving end is the streaming media service module; or,

[0081] The writing end is the streaming media service module, and the receiving end is the application end.

[0082] This embodiment provides another distributed streaming media system for multimedia streaming across devices. Figure 5 The following is a schematic diagram of the structure of a distributed streaming media system for multimedia streaming across devices provided by an embodiment of the present application. Figure 5 As shown, the streaming media service module 50, in addition to the IPC module 30 and the shared memory module 40, also includes:

[0083] A streaming media control module 51, used to control the relevant processes of the streaming media service;

[0084] A streaming media module 52, for performing multimedia streaming services across devices;

[0085] The streaming media service control module 53 is used to control the relevant processes of the IPC module, the shared memory module and the streaming media module.

[0086] Optionally, the relevant process includes:

[0087] Initialization, deinitialization, startup.

[0088] In the initialization process, resources need to be configured to provide a basis for the service. In the deinitialization process, the configured resources need to be released to facilitate other services to call resources and avoid resource occupation. Optionally, initialization is performed before the service starts, and deinitialization is performed after the service is completed.

[0089] Optional resources include: memory, threads, etc.

[0090] Optionally, the submodules included in the streaming media module 52 are used to implement functions such as media acquisition, media playback, media encoding, and media decoding.

[0091] In a possible embodiment, after the flow starts, the business party calls the API in the SDK to start the flow. During the flow, the streaming media can be controlled (start, pause, fast forward). After the flow ends, the API is called to end the process, and deinitialization is performed after the end.

[0092] In order to implement the above embodiments, the present application also proposes an electronic device, comprising: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the system provided by the above embodiments.

[0093] In order to implement the above embodiments, the present application also proposes a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the system provided by the above embodiments.

[0094] In order to implement the above embodiments, the present application also proposes a computer program product, including a computer program, which implements the system provided by the above embodiments when executed by a processor.

[0095] The collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in this application are in compliance with relevant laws and regulations and do not violate public order and good morals.

[0096] It should be noted that personal information from users should be collected for legitimate and reasonable purposes and should not be shared or sold outside of these legitimate uses. In addition, such collection / sharing should be carried out after receiving the user's informed consent, including but not limited to notifying the user to read the user agreement / user notice and sign the agreement / authorization including authorization of relevant user information before the user uses the function. In addition, any necessary steps should be taken to protect and safeguard access to such personal information data and ensure that others who have access to personal information data comply with its privacy policy and procedures.

[0097] The present application is expected to provide an implementation scheme for users to selectively block the use or access of personal information data. That is, the present disclosure is expected to provide hardware and / or software to prevent or block access to such personal information data. Once the personal information data is no longer needed, the risk can be minimized by limiting data collection and deleting the data. In addition, when applicable, such personal information is de-identified to protect the privacy of the user.

[0098] In the description of the aforementioned embodiments, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0099] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0100] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0101] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0102] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0103] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0104] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0105] The storage medium mentioned above may be a read-only memory, a disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A distributed streaming media system for multimedia streaming across devices, characterized in that: include: Application side, software development kit SDK module, streaming media service module; The application end is used to call the SDK module request to request a streaming media service; The SDK module is used to generate request information according to the request of the application end, and send the request information to the streaming media service module; The streaming media service module is used to perform streaming media services according to the request information, communicate with the SDK module, and perform multimedia streaming services with other devices through streaming media channels.

2. The system according to claim 1, characterized in that The streaming media service module includes: Streaming media control module, used to control the relevant processes of streaming media services; Streaming media module, used to perform multimedia streaming services across devices; A streaming media service control module, used to control the relevant processes of the IPC module, the shared memory module and the streaming media module; An inter-process communication IPC module, used for transmitting control data between the SDK module and the streaming media service module according to the platform of the current device; The shared memory module is used to transmit media data between the SDK module and the streaming media service module according to the platform of the current device.

3. The system according to claim 2, characterized in that The SDK module includes: An application program interface API module, including a plurality of callable APIs, for accepting calls from the application end to generate request information; SDK control module, used for controlling the initialization and deinitialization process of the IPC module and the shared memory module; An IPC communication module, used for communicating with the IPC module; A shared memory communication module is used for performing data transmission with the shared memory module.

4. The system according to claim 2, characterized in that The IPC module includes: An IPC framework module, including an API corresponding to IPC, for establishing an IPC process to send the request information or control data to the streaming media service module; The first platform adaptation module includes at least one plug-in corresponding to the platform, and is used to receive a call from the IPC framework to execute an IPC communication process corresponding to the current device.

5. The system according to claim 2, characterized in that The shared memory module comprises: A sharing module, used for writing the media data sent by the writing end into the shared memory area, or reading the media data in the shared memory area and sending it to the reading end; The synchronization module is used to notify the writing end to perform a writing operation, or to notify the reading end to perform a reading operation. The second platform adaptation module includes at least one plug-in corresponding to the platform, and is used to execute the shared memory process corresponding to the current device.

6. The system according to claim 5, characterized in that The writing end is the application end, and the receiving end is the streaming media service module; or, The writing end is the streaming media service module, and the receiving end is the application end.

7. The system according to claim 2, characterized in that The relevant processes include: Initialization, deinitialization, startup.

8. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the system according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the system according to any one of claims 1 to 7 when executed by a processor.

10. A computer program product, characterized in that The invention comprises a computer program, which implements the system according to any one of claims 1 to 7 when being executed by a processor.

Images