A SIP-based synthetic media playback method and apparatus

By combining multiple media data streams into a single media data stream using the Media Synthesis Service Unit, the problem of limited media data playback channels in the SIP protocol is solved, achieving efficient media data transmission and playback and improving network bandwidth utilization.

CN116389798BActive Publication Date: 2026-11-10BEIJING FEIXUN DIGITAL TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202211725697.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-12-30
Publication Date
2026-11-10
Estimated Expiration
2042-12-30

AI Technical Summary

Technical Problem

In the existing SIP protocol, each client session can only replay one channel of media data, and the applicability of media data replay is limited when network bandwidth is restricted, making it impossible to fully utilize the rendering window.

Method used

The media compositing service unit combines multiple media data streams into a single media data stream, merging multiple high-bitrate media streams into a single low-bitrate media stream, which is then decoded and displayed on the client side. The media compositing service unit also sends the synthesized historical media data to the media processing service unit, thereby increasing the number of media playback channels and transmission efficiency.

Benefits of technology

It expands the applicability of media data playback, makes full use of the rendering window, effectively improves network bandwidth utilization, and realizes the composite playback of multiple media data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116389798B_ABST
    Figure CN116389798B_ABST
Patent Text Reader

Abstract

This application discloses a SIP-based method and apparatus for playing composite media. The method may include the following steps: when the media compositing service unit receives a first query request, the media compositing service unit obtains the composite playback identifier from the first query request and generates a first message request; the media compositing service unit sends the first message request to the signaling control service unit, the first message request being used to query media composite playback information; the media compositing service unit sends the composited historical media data to the media processing service unit; the media processing service unit sends the composited historical media data to the client; the client plays the composited historical media data. This method can increase the number of media playback channels, improve transmission efficiency, expand the applicability of media data playback, fully utilize the rendering window, and effectively improve network bandwidth utilization.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to a SIP-based method and apparatus for playing composite media, a communication system, a computer-readable storage medium, and a computer program. Background Technology

[0002] SIP (Session Initiation Protocol) is a text-based application-layer signaling control protocol for multi-party multimedia communication. Independent of the underlying transport protocol, it is used to establish, modify, and terminate two-party or multi-party multimedia sessions over IP networks. It is a signaling protocol for implementing real-time communication applications in IP-based networks. SIP messages have various applications; media playback is a commonly used function in audio and video systems.

[0003] From the signaling process of the client initiating audio and video playback, it can be seen that each client session can only play back one channel of media data. Playing back one channel of media data requires one rendering window. However, the number of rendering windows is fixed. For example, if there are only 16 rendering windows, then a maximum of 16 channels of media data can be played back. If the network bandwidth is limited, the number of channels that can be played back will be even less, and the applicable scope of media data playback is relatively small. Summary of the Invention

[0004] This application provides a SIP-based composite media playback method and apparatus, communication system, computer-readable storage medium, and computer program, which can increase the number of media playback channels, improve transmission efficiency, expand the applicability of media data playback, make full use of the rendering window, and effectively improve network bandwidth utilization.

[0005] In a first aspect, embodiments of this application provide a SIP-based method for synthesizing media playback, applied to a communication system, the communication system comprising a client, a signaling control service unit, a media processing service unit, and a media synthesis service unit; including:

[0006] When the media compositing service unit receives the first query request, the media compositing service unit obtains the compositing playback identifier in the first query request and generates a first MESSAGE request; wherein, the first MESSAGE request contains a message element Query;

[0007] The media synthesis service unit sends a first MESSAGE request to the signaling control service unit. The first MESSAGE request is used to query media synthesis playback information. After receiving the first MESSAGE request, the signaling control service unit obtains the synthesis playback information according to the synthesis playback identifier of the first MESSAGE request, generates a second MESSAGE request, and sends it to the media synthesis service unit. The second MESSAGE request includes a message element Response.

[0008] After the handshake operation of the signaling control service unit, media processing service unit and media synthesis service unit is completed, the media synthesis service unit sends the synthesized historical media data to the media processing service unit; the media processing service unit is used to send the synthesized historical media data to the client; the client is used to play the synthesized historical media data.

[0009] By implementing the embodiments of this application, the media synthesis service unit can synthesize multiple media data into a single media data, and merge multiple high-bitrate media streams into a single low-bitrate media stream. This can increase the number of media playback channels, improve transmission efficiency, and expand the applicability of media data playback. The media synthesis service unit sends the synthesized historical media data to the media processing service unit, and then forwards it to the client for decoding and display, making full use of the rendering window and effectively improving network bandwidth utilization.

[0010] In one possible implementation, the media synthesis service unit sends the synthesized historical media data to the media processing service unit, including:

[0011] The media compositing service unit reads the split-screen file based on the compositing playback information;

[0012] The media synthesis service unit performs video decoding on the split-screen file to obtain decoded YUV data;

[0013] The media synthesis service unit performs a splicing operation on the decoded YUV data to obtain spliced ​​YUV data;

[0014] The media compositing service unit encodes the spliced ​​YUV data to obtain the synthesized historical media data;

[0015] The media synthesis service unit encapsulates the synthesized historical media data into RTP data packets and transmits them to the media processing service unit.

[0016] By implementing the embodiments of this application, multiple high-bitrate media streams can be merged into a single low-bitrate media stream, thereby improving transmission efficiency.

[0017] In one possible implementation, the client is configured to send an initial INVITE request carrying SDP to the signaling control service unit; the signaling control service unit is configured to send a first query request to the media synthesis service unit when it determines that the s field value in the initial INVITE request is a media synthesis playback request.

[0018] In one possible implementation, when the media compositing service unit receives the first query request, the media compositing service unit obtains the compositing playback identifier in the first query request and generates a first MESSAGE request, including:

[0019] When the media compositing service unit receives the first query request, it determines whether the first query request is a media compositing playback request based on the s field value in the first query request. If so, it extracts the u field value in the first query request as a compositing playback identifier and generates a first MESSAGE request based on the compositing playback identifier.

[0020] In one possible implementation, the method further includes:

[0021] When the signaling control service unit receives the INFO message sent by the signaling control service unit, the media synthesis service unit parses the playback control command in the INFO message;

[0022] The media compositing service unit executes corresponding playback control operations on the synthesized historical media data according to the playback control instructions.

[0023] Secondly, embodiments of this application provide a SIP-based composite media playback device applied to a communication system, the communication system including a client, a signaling control service unit, a media processing service unit, and a media synthesis service unit; including:

[0024] The first MESSAGE request generation module is used to generate a first MESSAGE request when the media compositing service unit obtains the first query request, and the media compositing service unit obtains the compositing playback identifier in the first query request; wherein, the first MESSAGE request includes the message element Query.

[0025] The first MESSAGE request sending module is used by the media synthesis service unit to send a first MESSAGE request to the signaling control service unit. The first MESSAGE request is used to query media synthesis playback information. The signaling control service unit is used to obtain synthesis playback information according to the synthesis playback identifier of the first MESSAGE request after receiving the first MESSAGE request, generate a second MESSAGE request and send it to the media synthesis service unit. The second MESSAGE request includes a message element Response.

[0026] The media data sending module is configured to, after the handshake operation between the signaling control service unit, the media processing service unit, and the media synthesis service unit is completed, have the media synthesis service unit send synthesized historical media data to the media processing service unit; the media processing service unit is configured to send the synthesized historical media data to the client; and the client is configured to play the synthesized historical media data.

[0027] Thirdly, embodiments of this application provide a communication system including a processor and a memory interconnected thereto. The memory is used to store a computer program that supports the communication system in executing the above-described method. The computer program includes program instructions, and the processor is configured to invoke the program instructions to execute the method described in the first aspect.

[0028] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, the computer program including program instructions, which, when executed by a processor, cause the processor to perform the method described in the first aspect.

[0029] Fifthly, embodiments of this application also provide a computer program, the computer program including program instructions, which, when executed by a processor, cause the processor to perform the method described in the first aspect. Attached Figure Description

[0030] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below.

[0031] Figure 1 This is a schematic diagram illustrating an application scenario provided in an embodiment of this application;

[0032] Figure 2 This is a flowchart illustrating the steps of an embodiment of a SIP-based synthetic media playback method provided in this application.

[0033] Figure 3 This is a flowchart of a data transmission step provided in an embodiment of this application;

[0034] Figure 4 This is a flowchart of the playback control operation execution steps provided in an embodiment of this application;

[0035] Figure 5 This is a timing diagram of a media composition and playback operation provided in an embodiment of this application;

[0036] Figure 6 This is a structural block diagram of an embodiment of a SIP-based synthetic media playback device provided in this application.

[0037] Figure 7 This is a schematic diagram of the structure of a communication system provided in an embodiment of this application. Detailed Implementation

[0038] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0039] The terms "first" and "second," etc., used in the specification and drawings of this application are used to distinguish different objects or to differentiate different treatments of the same object, rather than to describe a specific order of objects. Furthermore, the terms "comprising" and "having," and any variations thereof, mentioned in the description of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices. It should be noted that in the embodiments of this application, words such as "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design method described as "exemplarily" or "for example" in the embodiments of this application should not be construed as being superior or more advantageous than other embodiments or design solutions. Specifically, the use of words such as "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner. In the embodiments of this application, "A and / or B" means both A and B, or A or B. "A, and / or B, and / or C" means any one of A, B, and C, or means any two of A, B, and C, or means A, B, and C.

[0040] To facilitate a better understanding of this application, the application scenarios of the method described in this application are introduced below:

[0041] See Figure 1 This application provides a schematic diagram of an application scenario. For example... Figure 1As shown, the embodiments of this application can be applied to a communication system. The components in the communication system may include a client, a signaling control service unit, a media processing service unit, and a media synthesis service unit. The client can be interconnected with the signaling control service unit, and the signaling control service unit can be interconnected with the media processing service unit and the media synthesis service unit, respectively. In another case, the media processing service unit and the media synthesis service unit can also be directly interconnected with the client to form a complete data transmission link.

[0042] The media compositing service unit is used to read video files, decode, splice, and encode video files, and registers with the signaling server.

[0043] Reference Figure 2 This document illustrates a flowchart of a SIP-based synthetic media playback method according to an embodiment of this application, which may specifically include the following steps:

[0044] Step 101: When the media compositing service unit obtains the first query request, the media compositing service unit obtains the compositing playback identifier in the first query request and generates a first MESSAGE request; wherein, the first MESSAGE request contains a message element Query.

[0045] In this embodiment of the application, a media compositing service unit is added. This media compositing service unit combines and transmits media data according to requirements. For example, it can provide three split-screen styles. The media compositing service unit accesses media files through the mounted directory and reads data according to the file information corresponding to each split-screen.

[0046] In a preferred embodiment of the application, when the media compositing service unit obtains the first query request, the media compositing service unit obtains the compositing playback identifier in the first query request to generate a first MESSAGE request, which includes: when the media compositing service unit obtains the first query request, the media compositing service unit determines whether the first query request is a media compositing playback request based on the s field value in the first query request; if so, the media compositing service unit extracts the u field value in the first query request as the compositing playback identifier and generates a first MESSAGE request based on the compositing playback identifier.

[0047] In other words, the value of the s field in the extended SDP in the first query request can be used to determine whether it is a media composition playback request. For example, "MixPlayback" can be preset as the value of the s field to identify it as a media composition playback request. If the value of the s field is the preset information, the value of the u field in the first query request is extracted and used as the composition playback identifier, i.e., the composition playback ID, for querying composition playback information.

[0048] Prior to the steps described above, the client may also send an initial INVITE request carrying SDP (Session Description Protocol) to the signaling control service unit; the signaling control service unit is used to send a first query request to the media synthesis service unit when it determines that the s field value in the initial INVITE request is a media synthesis and playback request; that is, the client first establishes a connection with the signaling control service unit to confirm whether a media synthesis and playback operation is required.

[0049] The first MESSAGE request is a request to query media synthesis playback information. It is a MESSAGE method based on the SIP protocol and can be extended in the form of XML. The XML element is defined as the root element, which contains the query message element Query, making it easier to control the transmission of information.

[0050] Step 102: The media synthesis service unit sends a first MESSAGE request to the signaling control service unit. The first MESSAGE request is used to query media synthesis playback information. After receiving the first MESSAGE request, the signaling control service unit obtains the synthesis playback information according to the synthesis playback identifier of the first MESSAGE request, generates a second MESSAGE request, and sends it to the media synthesis service unit. The second MESSAGE request includes a message element Response.

[0051] After the media synthesis service unit sends the first MESSAGE request, the signaling control service unit can obtain the synthesis playback information based on the synthesis playback identifier of the first MESSAGE request, generate the second MESSAGE request, and send the second MESSAGE request to the media synthesis service unit.

[0052] Specifically, the second MESSAGE request is a query for the media synthesis playback message response. It is also a MESSAGE method based on the SIP protocol, extended in the form of XML, defining the XML element as the root element, which contains the response message element Response, to facilitate the control of information transmission.

[0053] It should be noted that the content of the first MESSAGE request and the message body of the first MESSAGE request can be preset, and this application embodiment does not impose too many restrictions on this.

[0054] Step 103: After the handshake operation of the signaling control service unit, the media processing service unit, and the media synthesis service unit is completed, the media synthesis service unit sends the synthesized historical media data to the media processing service unit; the media processing service unit is used to send the synthesized historical media data to the client; the client is used to play the synthesized historical media data.

[0055] In a preferred embodiment of the application, reference is made to... Figure 3 The flowchart illustrates a data transmission step according to an embodiment of this application, wherein the media synthesis service unit sends synthesized historical media data to the media processing service unit, including the following sub-steps:

[0056] Sub-step 11: The media compositing service unit reads the split-screen file based on the compositing playback information;

[0057] Sub-step 12: The media synthesis service unit performs video decoding on the split-screen file to obtain decoded YUV data;

[0058] Sub-step 13: The media synthesis service unit performs a splicing operation on the decoded YUV data to obtain spliced ​​YUV data;

[0059] Sub-step 14: The media synthesis service unit encodes the spliced ​​YUV data to obtain synthesized historical media data;

[0060] Sub-step 15: The media synthesis service unit encapsulates the synthesized historical media data into RTP data packets and transmits them to the media processing service unit.

[0061] The media compositing service unit can read multiple split-screen files based on the compositing playback information, and then perform operations such as decoding, splicing, and encoding to obtain the compositing historical media data. The compositing historical media data is then transmitted to the media processing service unit, merging multiple high-bitrate media streams into a single low-bitrate media stream, thereby improving transmission efficiency.

[0062] In this embodiment, multiple historical media data are encoded into a single media stream by the media synthesis service unit and sent to the media processing service unit, and then forwarded to the client for decoding and display. This can increase the number of media playback channels, improve transmission efficiency, expand the applicability of media data playback, make full use of the rendering window, and effectively improve network bandwidth utilization.

[0063] In a preferred embodiment of the application, reference is made to... Figure 4 The diagram illustrates a flowchart of playback control operation execution steps according to an embodiment of this application. The method includes the following sub-steps:

[0064] Sub-step 21: When the INFO message sent by the signaling control service unit is received, the media synthesis service unit parses the playback control command in the INFO message;

[0065] Sub-step 22: The media synthesis service unit performs corresponding playback control operations on the synthesized historical media data according to the playback control command.

[0066] Specifically, the INFO message may include video pause, play, fast forward, slow down, and random drag-and-drop operations. The media compositing service unit can parse the playback control instructions in the INFO message and control the playback operation of the video file according to the instructions.

[0067] To enable those skilled in the art to better understand the embodiments of this application, the following specific examples are provided for illustration;

[0068] Specifically, in the extended SIP INVITE message, the s field in the extended SDP is valued as “MixPlayback”, which is used to identify a media composition playback request, and the u field is filled with a composition playback ID, which is used to query composition playback information.

[0069] Specifically, the request to query media synthesis playback information is based on the SIP protocol's MESSAGE method. To facilitate the transmission of control information, XML is used for extension. XML elements are predefined, and the root element contains a query message element, where the message body is an internal private message body.

[0070] In the Query message, playbackid is the value of the u field in the SDP of the INVITE message. The signaling server obtains media synthesis and playback information based on playbackid.

[0071] Specifically, querying media synthesis playback message responses is based on the SIP protocol's MESSAGE method, extended with XML, and predefined XML elements. The root element contains a response message element, where the message body is an internal private message body; the meanings of the fields are shown in Table 1.

[0072]

[0073]

[0074] Table 1: Response Fields for Synthetic Playback Information Query

[0075] When the client initiates media composition and playback, it synchronizes the composition and playback information (i.e., the values ​​of each field in Table 1) to the aforementioned business units. The sequence diagram of the client-initiated media composition and playback operation is as follows: Figure 5 As shown, it includes the following steps:

[0076] 1. The client sends an INVITE request to the signaling control service unit. The request contains the target address and carries an SDP message body. The 's' field in the SDP message body is "MixPlayback", which means mixed playback.

[0077] 2. The signaling control service unit sends an INVITE request (without an SDP message body) to the media processing service unit.

[0078] 3. The media processing service unit returns a 200 OK response, carrying an SDP message body. The SDP message body describes the IP address, port, media format, and other information of the media stream received by the media processing service unit.

[0079] 4. The signaling control service unit initiates an INVITE request to the media synthesis service unit. The request carries the 200 OK response message body from the media processing service unit in message 3, and modifies the s field to “MixPlayback” to represent composite playback.

[0080] 5. The media synthesis service unit obtains the sipid from the INVITE and generates a query message containing the message element Query. It then sends a MESSAGE request to the signaling control service unit to query the media synthesis playback information.

[0081] 6. The signaling control service unit returns a 200 OK response.

[0082] 7. The signaling control service unit obtains the composite playback information based on sipid, generates a response message containing the message element Response, and sends a MESSAGE request to the media composite service unit.

[0083] 8. The Media Composition Service Unit returns a 200 OK response.

[0084] 9. The Media Synthesis Service Unit returns a 200 OK response, carrying an SDP message body, and has one-way media transmission attributes.

[0085] 10. The signaling control service unit sends an ACK request to the media processing service unit, carrying an SDP message body, and has unidirectional media transmission attributes.

[0086] 11. The signaling control service unit sends an ACK request (without carrying an SDP message body) to the media synthesis service unit.

[0087] 12. The signaling control service unit initiates an INVITE request to the media processing service unit, carrying an SDP message body, and has one-way media reception attributes.

[0088] 13. The Media Processing Service Unit returns a 200 OK response to the Signaling Control Service Unit, carrying an SDP message body, and has the attribute of one-way media transmission.

[0089] 14. The signaling control service unit sends a 200 OK response to the client, carrying an SDP message body, and has one-way media transmission attributes.

[0090] 15. The client sends an ACK request to the signaling control service unit (without carrying an SDP message body).

[0091] 16. The signaling control service unit sends an ACK request to the media processing service unit (without carrying an SDP message body).

[0092] 17. When all split-screen files have played to the end, the Media Composition Service Unit sends a file end notification to the Signaling Control Service Unit.

[0093] 18. The signaling control service unit forwards the file end notification to the client.

[0094] 19. During playback, the client controls playback by sending in-session INFO messages to the signaling control service unit, including operations such as pausing, playing, fast forward, slow down, and random drag-and-drop playback of the video.

[0095] 20. The signaling control service unit replies with a 200 OK response to the client.

[0096] 21. After receiving message 19, the signaling control service unit forwards it to the media synthesis service unit.

[0097] 22. The Media Synthesis Service Unit replies with a 200 OK response to the Signaling Control Service Unit.

[0098] 23. The client sends a BYE request to the signaling control service unit to disconnect the session established by messages 1, 14, and 15.

[0099] 24. After receiving the BYE request, the signaling control service unit replies with a 200 OK response, and the session is closed;

[0100] 25. The signaling control service unit sends a BYE request to the media processing service unit to disconnect the session established by messages 12, 13, and 16.

[0101] 26. After receiving the BYE request, the Media Processing Service Unit replies with a 200 OK response, and the session is closed.

[0102] 27. The signaling control service unit sends a BYE request to the media processing service unit to disconnect the session established by messages 2, 3, and 10.

[0103] 28. After receiving the BYE request, the Media Processing Service Unit replies with a 200 OK response, and the session is closed.

[0104] 29. The signaling control service unit sends a BYE request to the media synthesis service unit to disconnect the session established by messages 4, 9, and 11.

[0105] After receiving the BYE request, the media compositing service unit replies with a 200 OK response, and the session ends.

[0106] Compared with the prior art, the technical solution of this application embodiment replaces the media storage service unit with a media synthesis service unit, adding a multi-channel media data synthesis function; in terms of the interaction process, this application embodiment differs from the prior art in many ways. For example, in the prior art, when the client sends an INVITE request to the signaling control service unit, the s field in the SDP message body is "Playback", representing historical playback, while in the corresponding SDP message body of this application embodiment, the s field is "MixPlayback", representing synthesized playback;

[0107] Furthermore, in the prior art, when the signaling control service unit initiates an INVITE request to the media storage service unit, the request carries a 200 OK response message body from the media processing service unit. In this response message body, the 's' field is modified to 'Playback' to represent historical playback, the 'u' field represents the URL of the playback file, and the 't' field represents the playback time period. However, in this embodiment, the signaling control service unit initiates an INVITE request to the media synthesis service unit, and the request carries a 200 OK response message body from the media processing service unit. In this response message body, the 's' field is modified to 'MixPlayback' to represent composite playback.

[0108] In existing technologies, the media storage service unit and the signaling control service unit both return a 180 Ringing response. However, in this embodiment, the media service unit obtains the sipid from the INVITE and generates a query message containing the message element Query, then sends a MESSAGE request to the signaling control service unit to query media synthesis and playback information. The signaling control service unit obtains the synthesis and playback information based on the sipid, generates a reply message containing the message element Response, and sends a MESSAGE request to the media synthesis service unit. The two message elements are used for obtaining media synthesis and playback information and retrieving synthesis and playback information, respectively. Compared to the simple reply in existing technologies, this embodiment performs corresponding information interaction settings according to playback requirements, improving information interaction efficiency.

[0109] On the other hand, in this embodiment of the application, the client performs playback control by sending an in-session INFO message to the signaling control service unit, including operations such as pausing, playing, fast-forwarding, slowing down, and random drag-and-drop playback of the video.

[0110] In other words, this application embodiment not only realizes the function of storing media data, but also realizes the function of combining multiple media data into a single media data. The client can play back multiple media data, save transmission bandwidth, and make full use of the rendering window.

[0111] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this application are not limited to the described order of actions, because according to the embodiments of this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of this application.

[0112] Reference Figure 6 This diagram illustrates a structural block diagram of a SIP-based composite media playback device according to an embodiment of this application. The device is applied to a communication system, which includes a client, a signaling control service unit, a media processing service unit, and a media synthesis service unit. Specifically, it may include the following modules:

[0113] The first MESSAGE request generation module 301 is used to generate a first MESSAGE request when the media compositing service unit obtains the first query request, by obtaining the compositing playback identifier in the first query request; wherein, the first MESSAGE request includes a message element Query.

[0114] The first MESSAGE request sending module 302 is used by the media synthesis service unit to send a first MESSAGE request to the signaling control service unit. The first MESSAGE request is used to query media synthesis playback information. The signaling control service unit is used to obtain synthesis playback information according to the synthesis playback identifier of the first MESSAGE request after receiving the first MESSAGE request, generate a second MESSAGE request and send it to the media synthesis service unit. The second MESSAGE request includes a message element Response.

[0115] The media data sending module 303 is used to send the synthesized historical media data to the media processing service unit after the handshake operation of the signaling control service unit, the media processing service unit and the media synthesis service unit is completed; the media processing service unit is used to send the synthesized historical media data to the client; the client is used to play the synthesized historical media data.

[0116] Preferably, the media data sending module includes:

[0117] The reading submodule is used by the media compositing service unit to read the split-screen file based on the compositing playback information.

[0118] The decoding operation submodule is used by the media synthesis service unit to perform video decoding operation on the split-screen file to obtain decoded YUV data.

[0119] The splicing operation submodule is used by the media synthesis service unit to splice the decoded YUV data to obtain spliced ​​YUV data.

[0120] The encoding submodule is used by the media synthesis service unit to encode the spliced ​​YUV data to obtain synthesized historical media data;

[0121] The encapsulation submodule is used by the media synthesis service unit to encapsulate the synthesized historical media data into RTP data packets and transmit them to the media processing service unit.

[0122] Preferably, the client is used to send an initial INVITE request carrying SDP to the signaling control service unit; the signaling control service unit is used to send a first query request to the media synthesis service unit when it determines that the value of the s field in the initial INVITE request is a media synthesis playback request.

[0123] Preferably, when the media compositing service unit receives the first query request, the first MESSAGE request generation module includes:

[0124] The first MESSAGE request generation submodule is used to determine whether the first query request is a media synthesis playback request based on the s field value in the first query request when the media synthesis service unit obtains the first query request. If it is, the media synthesis service unit extracts the u field value in the first query request as a synthesis playback identifier and generates the first MESSAGE request based on the synthesis playback identifier.

[0125] Preferably, the device further includes:

[0126] The playback control instruction parsing module is used to parse the playback control instruction in the INFO message sent by the signaling control service unit when the media synthesis service unit receives the INFO message.

[0127] The playback control operation module is used by the media synthesis service unit to perform corresponding playback control operations on the synthesized historical media data according to the playback control instructions.

[0128] The modules in the aforementioned SIP-based composite media playback device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of the communication system in hardware form or independent of it, or stored in the memory of the communication system in software form, so that the processor can call and execute the corresponding operations of each module.

[0129] The SIP-based composite media playback device provided above can be used to execute the SIP-based composite media playback method provided in any of the above embodiments, and has corresponding functions and beneficial effects.

[0130] Based on the same inventive concept as the above method, embodiments of this application provide a communication system, the internal structure of which can be shown as follows: Figure 7 As shown, the communication system includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a SIP-based synthetic media playback method. The display screen can be a liquid crystal display (LCD) or an e-ink display. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the system casing, or an external keyboard, touchpad, or mouse.

[0131] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the communication system to which the present application is applied. A specific communication system may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0132] This application also provides a computer-readable storage medium for storing a computer program that causes a control device to perform some or all of the steps of any of the SIP-based synthetic media playback methods described in the above method embodiments.

[0133] This application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program operable to cause an electronic device to perform some or all of the steps of any of the SIP-based synthetic media playback methods described in the above method embodiments.

[0134] It is understood that those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in the various embodiments of this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0135] Those skilled in the art will appreciate that the functionality described by the various illustrative logic blocks, modules, and algorithmic steps disclosed in connection with the various embodiments of this application can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality described by the various illustrative logic blocks, modules, and steps can be stored or transmitted as one or more instructions or codes on a computer-readable medium and executed by a hardware-based processing unit. The computer-readable medium may comprise a computer-readable storage medium, which corresponds to a tangible medium, such as a data storage medium, or a communication medium that includes any medium that facilitates the transfer of a computer program from one place to another (e.g., according to a communication protocol). In this way, the computer-readable medium may substantially correspond to (1) a non-transitory tangible computer-readable storage medium, or (2) a communication medium, such as a signal or carrier wave. The data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementing the techniques described in this application. The computer program product may comprise a computer-readable medium.

[0136] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units, i.e., they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0137] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0138] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or communication system, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0139] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for playing synthesized media based on SIP, characterized in that, Applied to a communication system, the communication system includes a client, a signaling control service unit, a media processing service unit, and a media synthesis service unit; including: The client is used to send an initial INVITE request carrying SDP to the signaling control service unit; the signaling control service unit is used to send a first query request to the media synthesis service unit when it determines that the s field value in the initial INVITE request is a media synthesis playback request. When the media synthesis service unit obtains the first query request, the media synthesis service unit determines whether the first query request is a media synthesis playback request based on the s field value in the first query request. If it is, the u field value in the first query request is extracted as a synthesis playback identifier, and a first MESSAGE request is generated based on the synthesis playback identifier. The first MESSAGE request contains a message element Query. The media synthesis service unit sends a first MESSAGE request to the signaling control service unit. The first MESSAGE request is used to query media synthesis playback information. After receiving the first MESSAGE request, the signaling control service unit obtains the synthesis playback information according to the synthesis playback identifier of the first MESSAGE request, generates a second MESSAGE request, and sends it to the media synthesis service unit. The second MESSAGE request includes a message element Response. After the handshake operation of the signaling control service unit, media processing service unit, and media synthesis service unit is completed, the media synthesis service unit reads the split-screen file according to the synthesis playback information; the media synthesis service unit performs video decoding on the split-screen file to obtain decoded YUV data; the media synthesis service unit performs splicing operation on the decoded YUV data to obtain spliced ​​YUV data; the media synthesis service unit encodes the spliced ​​YUV data to obtain synthesized historical media data; the media synthesis service unit encapsulates the synthesized historical media data into RTP data packets and transmits them to the media processing service unit; the media processing service unit is used to send the synthesized historical media data to the client; the client is used to play the synthesized historical media data.

2. The method as described in claim 1, characterized in that, The method further includes: When the signaling control service unit receives the INFO message sent by the signaling control service unit, the media synthesis service unit parses the playback control command in the INFO message; The media compositing service unit performs corresponding playback control operations on the synthesized historical media data according to the playback control instructions.

3. A SIP-based synthetic media playback device, characterized in that, Applied to a communication system, the communication system includes a client, a signaling control service unit, a media processing service unit, and a media synthesis service unit; including: The client is used to send an initial INVITE request carrying SDP to the signaling control service unit; the signaling control service unit is used to send a first query request to the media synthesis service unit when it determines that the s field value in the initial INVITE request is a media synthesis playback request. The first MESSAGE request generation module is used to determine whether the first query request is a media synthesis playback request based on the s field value in the first query request when the media synthesis service unit obtains the first query request. If it is, the media synthesis service unit extracts the u field value in the first query request as a synthesis playback identifier and generates a first MESSAGE request based on the synthesis playback identifier. The first MESSAGE request includes a message element Query. The first MESSAGE request sending module is used by the media synthesis service unit to send a first MESSAGE request to the signaling control service unit. The first MESSAGE request is used to query media synthesis playback information. The signaling control service unit is used to obtain synthesis playback information according to the synthesis playback identifier of the first MESSAGE request after receiving the first MESSAGE request, generate a second MESSAGE request and send it to the media synthesis service unit. The second MESSAGE request includes a message element Response. The media data sending module is used to send synthesized historical media data to the media processing service unit after the handshake operation of the signaling control service unit, media processing service unit, and media synthesis service unit is completed; the media processing service unit is used to send the synthesized historical media data to the client; the client is used to play the synthesized historical media data; the media data sending module includes: a reading submodule, used by the media synthesis service unit to read the split-screen file according to the synthesis playback information; a decoding operation submodule, used by the media synthesis service unit to perform video decoding operation on the split-screen file to obtain decoded YUV data; a splicing operation submodule, used by the media synthesis service unit to splice the decoded YUV data to obtain spliced ​​YUV data; an encoding submodule, used by the media synthesis service unit to encode the spliced ​​YUV data to obtain synthesized historical media data; and an encapsulation submodule, used by the media synthesis service unit to encapsulate the synthesized historical media data into RTP data packets and transmit them to the media processing service unit.

4. A communication system, characterized in that, The device includes a processor and a memory interconnected thereto, wherein the memory is used to store a computer program, the computer program including program instructions, and the processor is configured to invoke the program instructions to perform the method as described in any one of claims 1-2.

5. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method as described in any one of claims 1-2.

6. A computer program product comprising program instructions that, when executed by a processor, cause the processor to perform the method as described in any one of claims 1-2.

Citation Information

Patent Citations

  • Method for processing video data in video monitoring system

    CN103929623A

  • Multi-source multi-protocol video fusion transmission switching system and method

    CN110519641A