Multimedia data stream processing method and apparatus, electronic device, and storage medium

Abstract

Embodiments of the present disclosure provide a multimedia data stream processing method and apparatus, an electronic device, and a storage medium. The method comprises: acquiring a multimedia data stream; and when a target request is received, sending the acquired multimedia data stream to a target device, such that the target device performs rendering processing on the multimedia data stream on the basis of the target request, so as to obtain a multimedia data stream to be pushed.

Description

Multimedia data stream processing method, device, electronic device and storage medium

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed on December 29, 2023, with application number 202311868465.4 and invention name “Multimedia data stream processing method, device, electronic device and storage medium”. The entire contents of that application are incorporated by reference into this application. Technical Field

[0003] The embodiments of the present disclosure relate to the field of data processing technology, and in particular to a method, device, electronic device, and storage medium for processing multimedia data streams. Background Art

[0004] With the rapid development of computer and internet technologies, a wide variety of service scenarios have emerged. In these service scenarios, multiple scenarios are often associated with them to enrich their functionality and user interaction methods. For example, by inputting an interaction request associated with a live broadcast page, the live broadcast page will render the corresponding effect.

[0005] Summary of the Invention

[0006] The present disclosure provides a multimedia data stream processing method, device, electronic device and storage medium, so as to effectively render the picture content without requiring the performance of the terminal device, thereby improving the universality and efficiency of video processing.

[0007] In a first aspect, an embodiment of the present disclosure provides a method for processing a multimedia data stream, the method comprising:

[0008] Acquiring a multimedia data stream, wherein the multimedia data stream includes an audio stream and / or a video stream;

[0009] When a target request is received, the acquired multimedia data stream is sent to a target device, so that the target device performs rendering processing on the multimedia data stream based on the target request to obtain a multimedia data stream to be pushed.

[0010] In a second aspect, an embodiment of the present disclosure further provides a multimedia data stream processing device, the device comprising:

[0011] A data stream acquisition module, configured to acquire a multimedia data stream, wherein the multimedia data stream includes an audio stream and / or a video stream;

[0012] The data stream sending module is used to send the acquired multimedia data stream to the target device when a target request is received, so that the target device renders the multimedia data stream based on the target request to obtain the multimedia data stream to be pushed.

[0013] In a third aspect, an embodiment of the present disclosure further provides an electronic device, the electronic device comprising:

[0014] one or more processors;

[0015] a storage device for storing one or more programs,

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the multimedia data stream processing method as described in any one of the embodiments of the present disclosure.

[0017] In a fourth aspect, an embodiment of the present disclosure further provides a storage medium comprising computer-executable instructions, which, when executed by a computer processor, are used to execute the multimedia data stream processing method as described in any one of the embodiments of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale.

[0019] FIG1 is a flow chart of a multimedia data stream processing method provided by an embodiment of the present disclosure;

[0020] FIG2 is a flow chart of another multimedia data stream processing method provided by an embodiment of the present disclosure;

[0021] FIG3 is a flow chart of a multimedia data stream processing method provided by an embodiment of the present disclosure;

[0022] FIG4 is a schematic structural diagram of a multimedia data stream processing device provided by an embodiment of the present disclosure;

[0023] FIG5 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0025] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0026] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to." The term "based on" means "based, at least in part, on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one additional embodiment," and the term "some embodiments" means "at least some embodiments." Other terms are defined in the following description.

[0027] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0028] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0029] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0030] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0031] For example, in response to a user's active request, a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operations of the disclosed technical solution based on the prompt message.

[0032] As an optional but non-limiting implementation, in response to receiving a user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also contain a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0033] It is understandable that the above notification and user authorization process are merely illustrative and do not limit the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0034] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) must comply with the requirements of relevant laws, regulations and relevant provisions.

[0035] Typically, live page rendering relies primarily on processing by the client to which the live page belongs. This places high demands on the performance of the terminal device, resulting in high video processing costs and poor universality.

[0036] Before introducing this technical solution, an example application scenario can be provided. This technical solution can be applied to any scenario requiring real-time rendering of multimedia data streams, such as live broadcast scenarios, real-time video processing scenarios, or real-time audio processing scenarios. To more clearly illustrate this technical solution, the execution process of this technical solution will be described using a live broadcast scenario as an example. For other scenarios involving real-time data stream rendering, please refer to the multimedia data stream rendering process in a live broadcast scenario. For example, during a live broadcast, the live broadcast client can obtain and process the multimedia data stream of the host user. However, when the host user or the user watching the live broadcast triggers certain interactive controls on the live broadcast page, if the processing content corresponding to these interactive controls is complex and time-consuming, the live broadcast client still needs to handle the processing. This places high performance requirements on the live broadcast client, that is, it is highly dependent on the live broadcast client's hardware. Furthermore, if the live broadcast client's performance is not high, there may be problems such as page freezes, delays, and content lag, which in turn leads to poor live broadcast visual quality.

[0037] At this time, based on the technical solution of the embodiment of the present disclosure, the live broadcast picture can be obtained, and then, when it is detected that the interactive request associated with the live broadcast picture is a target request, the obtained live broadcast picture can be sent to a target device with better performance that belongs to the same local area network as the live broadcast client, so that the target device can render and process the obtained live broadcast picture based on the target request.

[0038] Figure 1 is a flow chart of a multimedia data stream processing method provided by an embodiment of the present disclosure. The embodiment of the present disclosure is applicable to any scenario that requires real-time rendering of data streams. The method can be executed by a multimedia data stream processing device, which can be implemented in the form of software and / or hardware. Optionally, it can be implemented by an electronic device, which can be a mobile terminal, PC or server, etc.

[0039] As shown in FIG1 , the method of this embodiment may specifically include:

[0040] S110: Acquire multimedia data stream.

[0041] It should be noted that the devices implementing the technical solutions provided by the embodiments of this disclosure may include at least two devices. One of these two devices may be a device for acquiring multimedia data streams, and the other may be a device for processing the acquired multimedia data streams. To enhance universality, the data stream acquisition device may optionally be a terminal device. The device for processing the multimedia data stream may be a high-performance device belonging to the same local area network, such as a terminal device or a server.

[0042] Among them, the multimedia data stream can be a data stream composed of at least one media type (such as images, text, audio, video, etc.). In this embodiment, the multimedia data stream can include an audio stream and / or a video stream. The audio stream can be understood as a data stream composed of audio. The video stream can be understood as a data stream composed of video. It should be noted that in a live broadcast scenario, the multimedia data stream can be the live broadcast picture and audio information presented simultaneously on the live broadcast page; in a real-time audio processing scenario, the multimedia data stream can be audio information.

[0043] S120 . When a target request is received, the acquired multimedia data stream is sent to a target device, so that the target device performs rendering processing on the multimedia data stream based on the target request to obtain a multimedia data stream to be pushed.

[0044] In this embodiment, when presenting a live page corresponding to a multimedia data stream, the host user or viewing user can trigger a corresponding operation on the live page. Furthermore, a request can be generated based on the corresponding operation, and the generated request can be used as a target request. In other words, the target request acting on the multimedia data stream display page can be continuously or periodically detected. Furthermore, upon receiving the target request, an effect corresponding to the target request can be rendered on the display page. Typically, the screen content required to render the target request is relatively time-consuming, so the multimedia data stream can be sent to the target device so that another device with higher performance can perform the rendering process.

[0045] Among them, the target request can be understood as an interactive request associated with the multimedia data stream. Generally, in the multimedia data stream display process, it may include the end to which the multimedia data stream belongs and the multimedia data stream viewing end. Both ends can input requests for the page displaying the multimedia data stream, and at least part of the requests input by at least one of the two ends for the multimedia data stream display page can be determined as target requests. Optionally, the target request includes a request from the first user and / or the second user to act on the live broadcast page. The target request can include multiple types, and optionally, it can include at least one of a rendering special effect request, a resource transfer request, and a live broadcast interaction request.

[0046] The first user may be any user associated with the multimedia data stream display process. Optionally, the first user may be the host of the live broadcast room. The second user may also be any user associated with the multimedia data stream display process. For example, the second user may be a user whose live broadcast page of the live broadcast room corresponding to the first user is displayed on the display interface of the corresponding terminal, such as an audience watching the live broadcast of the host. The live broadcast page is a page that presents the multimedia data stream. The live broadcast page can be understood as a page that can be presented on the display interface of the associated terminal during the live broadcast process, and the page may include the live broadcast data stream corresponding to the first user or the second user.

[0047] Among them, the rendering special effect request can be understood as a request to perform special effect processing on the content in the live broadcast page or a request to display special effects in the live broadcast page. Exemplarily, the rendering special effect request can be a corresponding request when a hair accessory special effect control is triggered.

[0048] The resource transfer request can be understood as a request to transfer the associated interactive services and resources, and optionally, a request corresponding to sending a gift. The resource transfer request can be a request to transfer resources from a resource sender to a resource receiver. The resource can be an interactive gift, a virtual resource, etc. In actual applications, when a trigger operation of an associated control for resource transfer is detected, the trigger operation can be responded to and a preset animation or preset image corresponding to the associated control can be rendered on the live broadcast page.

[0049] A live interactive request can be understood as a request generated during a live broadcast by an anchor when interacting with an audience, or when an anchor interacts with another anchor, or when an audience interacts with another audience. For example, a live interactive request can include posting a comment.

[0050] It should be noted that the request objects corresponding to rendering special effects requests, resource transfer requests, and live interactive requests can be at least one of the first and second users. In other words, both the first and second users can send rendering special effects requests, resource transfer requests, and live interactive requests. When the corresponding target request is received, the corresponding information is rendered on the live page. The benefits of this setting are: improving the performance of the application software in the interactive request rendering capability, expanding the scope of application of interactive requests that can be rendered, and enhancing the user experience and interactive experience.

[0051] In this embodiment, when a target request is received, the acquired multimedia data stream is sent to the target device, so that the target device renders the multimedia data stream based on the target request to obtain the multimedia data stream to be pushed.

[0052] The target device can be understood as an electronic device associated with the client to which the multimedia data stream belongs and having higher performance than that client. Optionally, the target device is a device that meets a preset processing condition. The preset processing condition can be that the device has higher performance than the client to which the associated multimedia data stream belongs. In this embodiment, the host terminal that obtains the multimedia data stream and the target device correspond to the same local area network, that is, the distance between the client and the target device can be within the coverage range of the corresponding local area network. The target device can be any electronic device with higher performance than the client to which the multimedia data stream belongs. Optionally, the target device can be a server, a PC, or the cloud. The multimedia data stream to be pushed can be understood as the data stream to be pushed after rendering the multimedia data stream. It should be noted that when this method is applied to the host terminal, the multimedia data stream to be pushed is the data stream to be pushed to the host terminal and / or the viewer terminal. That is, when the multimedia data stream to be pushed is obtained, the multimedia data stream to be pushed can be pushed to the host terminal and / or the viewer terminal according to the corresponding push method.

[0053] It should be noted that in the related art, when a target request is received, the client to which the multimedia data stream belongs usually renders the acquired multimedia data stream based on the target request, and displays the rendered multimedia data stream on the live broadcast page of the corresponding client. However, under normal circumstances, the client devices to which the multimedia data stream belongs are mostly mobile terminals, which have certain limitations and have slow rendering, resulting in problems such as freezes, delays, and lags in the screen. Based on this, the multimedia data stream that needs to be rendered can be sent to a target device with higher performance, so that the target device renders the multimedia data stream based on the target request. Thereby, the effect of video rendering can be achieved for various types of terminal devices, the universality of the video rendering process is improved, and the processing efficiency and rendering effect of the multimedia data stream are improved.

[0054] In actual applications, during the process of presenting a multimedia data stream on a live page, multiple interaction requests associated with the live page may be generated. Among these interaction requests, there may be requests that match the target request, and there may also be requests that do not match the target request. Furthermore, when an interaction request associated with a live page is received, the interaction request can be detected to determine whether the interaction request matches the target request. Furthermore, if it is determined that the target request has been received, the acquired multimedia data stream can be sent to the target device so that the target device can render and process the received multimedia data stream based on the target request.

[0055] In this embodiment, the acquired multimedia data stream is sent to the target device so that the target device renders the multimedia data stream based on the target request, including: sending the acquired multimedia data stream and the target request to the target device so that the target device calls the corresponding rendering data packet based on the target request, and renders the multimedia data stream based on the rendering data packet.

[0056] Among them, the rendering data packet may include data associated with the rendering process. In actual applications, after determining multiple target requests, the rendering process data corresponding to each target request can be set separately and encapsulated into a data packet, and the rendering data packet corresponding to each target request can be obtained. Furthermore, after determining each target request, an association relationship between the target request and the rendering data packet corresponding to the corresponding target request can be established, and the rendering data packet can be stored in the target device. Furthermore, after the target device receives the target request sent by the client to which the multimedia data stream belongs, the corresponding rendering data packet can be retrieved according to the target request and the pre-established association relationship.

[0057] In actual applications, when a target request is received, the acquired multimedia data stream and the received target request can be sent to the target device. Furthermore, when the target device receives the multimedia data stream and the target request, the rendering data packet corresponding to the target request can be retrieved based on the received target request and the pre-established data packet association relationship. Afterwards, the received multimedia data stream can be rendered according to the retrieved rendering data packet to obtain the multimedia data stream to be pushed. Furthermore, the multimedia data stream to be pushed can be processed according to actual needs. The advantage of such a setting is that it improves the data processing efficiency and processing quality of interactive data streams with higher performance requirements, and ensures the real-time rendering of the multimedia data stream, thereby improving the rendering and display effect of the multimedia video stream.

[0058] Exemplarily, the rendering process can be explained by taking the target request as a rendering special effects request and the obtained multimedia data stream as a live broadcast screen as an example: the live broadcast screen and the hair accessory special effects rendering request corresponding to the triggered hair accessory special effects can be sent to the target device. Then, after the target device receives the live broadcast screen and the hair accessory special effects rendering request, the rendering data packet corresponding to the hair accessory special effects rendering request can be retrieved, and the live broadcast screen can be rendered based on the retrieved rendering data packet. In this way, the multimedia data stream to be pushed can be obtained, that is, the live broadcast screen of the rendered hair accessory special effects.

[0059] The technical solution of the embodiment of the present disclosure obtains a multimedia data stream, and further, upon receiving a target request, sends the obtained multimedia data stream to a target device, so that the target device renders and processes the multimedia data stream based on the target request, thereby obtaining the multimedia data stream to be pushed. This solves the problem in the related art that when rendering a live page, the performance requirements for the terminal device are relatively high, resulting in high video processing costs and poor universality. It achieves the effect of effectively rendering the picture content without requiring the performance of the terminal device, thereby improving the universality and efficiency of video processing.

[0060] Figure 2 is a flow chart illustrating another multimedia data stream processing method provided by an embodiment of the present disclosure. The technical solution of this embodiment, based on the above-described embodiment, after obtaining the multimedia data stream to be pushed, can push the multimedia data stream to be pushed based on the push streaming method corresponding to the target device. Technical features that are identical or similar to those of the above-described embodiments are not further described here.

[0061] As shown in FIG2 , the method of this embodiment may specifically include:

[0062] S210: Acquire multimedia data stream.

[0063] S220 . When a target request is received, the acquired multimedia data stream is sent to a target device, so that the target device performs rendering processing on the multimedia data stream based on the target request to obtain a multimedia data stream to be pushed.

[0064] S230: Push the multimedia data stream to be pushed based on the streaming mode corresponding to the target device.

[0065] In actual applications, in order to display the multimedia data stream to be pushed on the corresponding client, after obtaining the multimedia data stream to be pushed, the multimedia data stream to be pushed can be pushed to the corresponding client.

[0066] In this embodiment, the push-streaming method can be used to indicate the process of pushing the multimedia data stream to be pushed to the corresponding server, that is, the method of executing the push-streaming. Those skilled in the art will understand that push-streaming refers to the process of transmitting packaged content to the server. Push-streaming is a real-time streaming media transmission technology that allows users to perform online audio and video transmission, thereby allowing other users to watch specified video streams and audio streams through the network. Push-streaming methods can include multiple, optional methods, including direct push-streaming methods and indirect push-streaming methods. Among them, the direct push-streaming method can be understood as pushing the multimedia data stream to be pushed directly to the corresponding server. The indirect push-streaming method can be understood as first sending the multimedia data stream to be pushed to the client to which the multimedia data stream belongs, and then pushing the multimedia data stream to be pushed to the corresponding server based on the client.

[0067] It should be noted that streaming requires a high network connection. If the network is unstable, the streaming effect may be poor. Therefore, the streaming mode setting can be associated with the local area network (LAN) to which the target device belongs. If the LAN network environment is relatively stable, the streaming operation can be performed directly, that is, the streaming mode can be set to direct. If the LAN network environment is poor or the target device is not connected to the network, direct streaming will not produce good results. Therefore, the streaming mode can be set to indirect.

[0068] It should also be noted that the setting of the streaming mode can also be associated with actual needs, that is, the user can set the corresponding streaming mode by inputting the streaming mode setting operation, so that the target device pushes the corresponding multimedia data stream according to the streaming mode set by the user.

[0069] In actual applications, after obtaining the multimedia data stream to be pushed, a push mode corresponding to the target device can be determined, and then the multimedia data stream to be pushed can be pushed according to the push mode.

[0070] It should be noted that there may be at least two streaming push modes. The following will describe the process of pushing the multimedia data stream to be pushed under these two streaming push modes respectively.

[0071] Optionally, the streaming method includes a direct streaming method, which pushes the multimedia data stream to be pushed based on the streaming method corresponding to the target device, including: sending the multimedia data stream to be pushed to the target server or streaming server based on the target device, so that the target server or streaming server pushes the multimedia data stream to be pushed to the anchor end and / or the audience end.

[0072] The target server can be understood as a program that serves the client associated with the multimedia data stream. The device running the target server can be referred to as the target server. The push server can be understood as a server that deploys a Content Delivery Network (CDN) interface. The host can be the client of the user to whom the multimedia data stream belongs. The viewer can be the client of the user viewing the multimedia data stream.

[0073] In actual applications, when the streaming mode corresponding to the target device is a direct streaming mode, it can be indicated that the multimedia data stream to be pushed can be directly pushed to the corresponding server based on the target device. Specifically, after obtaining the multimedia data stream to be pushed, the multimedia data stream to be pushed can be sent to the target server or the streaming service based on the target device. Furthermore, the multimedia data stream to be pushed can be pushed to the anchor terminal and / or the audience terminal based on the target server or the streaming service, so as to display the multimedia data stream to be pushed based on the display interface corresponding to each client. It should be noted that, when the streaming mode is a direct streaming mode, in order to enable the client that sends the multimedia data stream to the target device to also receive the multimedia data stream to be pushed, the target server or the streaming service can push the multimedia data stream to be pushed to the client of the user to whom the multimedia data stream belongs, that is, the anchor terminal.

[0074] For example, continuing to refer to the above example, after obtaining the live broadcast picture of the rendered hair accessory special effects, the live broadcast picture can be sent to the target server or the streaming server based on the target device, and then, the live broadcast picture can be pushed to the anchor end and / or the audience end based on the target server or the streaming server, so that the live broadcast page of the rendered hair accessory special effects can be displayed on the display interface of the live broadcast viewing users and the anchor's client.

[0075] It should be noted that the benefit of pushing the multimedia data stream to be pushed based on the direct streaming method is that it realizes the effect of directly pushing the data stream to be pushed to the server based on the data stream rendering device, improves the data rendering efficiency and rendering effect, and improves the streaming efficiency and the client's display effect of the data stream.

[0076] Optionally, the streaming method includes an indirect streaming method, which pushes the multimedia data stream to be pushed based on the streaming method set in the target device, including: receiving and displaying the multimedia data stream to be pushed sent by the target device, and sending the multimedia data stream to be pushed to the target server or streaming server, so that the target server or streaming server pushes the multimedia data stream to be pushed to the audience end.

[0077] In actual applications, when the streaming mode corresponding to the target device is an indirect streaming mode, it means that the multimedia data stream to be pushed cannot be directly pushed to the corresponding server end, and the multimedia data stream to be pushed needs to be sent to the device end that can perform streaming first, so that the device end can push the received multimedia data stream to the corresponding server end. Specifically, after obtaining the multimedia data stream to be pushed, the multimedia data stream to be pushed can be fed back to the client to which the multimedia data stream belongs, that is, the client that sends the acquired multimedia data stream. Furthermore, the multimedia data stream to be pushed sent by the target device can be displayed based on the client, and the received multimedia data stream can be sent to the target server end or streaming server end based on the client. Furthermore, the multimedia data stream to be pushed can be pushed to the viewer end based on the target server end or streaming server end, so as to display the received multimedia data stream based on the display interface corresponding to each viewer end.

[0078] For example, referring to the previous example, after obtaining a live broadcast image with a hair accessory special effect, the live broadcast image can be fed back to the host's client based on the target device, so that the live broadcast image with the hair accessory special effect can be displayed based on the display interface of the host's client. Furthermore, the live broadcast image can be sent to the target server or streaming server based on the host's client, and further, the live broadcast image can be pushed to the client of the user watching the live broadcast based on the target server or streaming server, so that the live broadcast page with the hair accessory special effect can be displayed on the display interface of the client of the user watching the live broadcast.

[0079] It should be noted that the benefit of pushing the multimedia data stream to be pushed based on the indirect push streaming method is that it realizes the effect of pushing the data stream to be pushed based on the client to which the data stream belongs, and improves the display effect of the data stream on the data stream viewing end.

[0080] The technical solution of the embodiment of the present disclosure obtains a multimedia data stream, and then, upon receiving a target request, sends the obtained multimedia data stream to a target device, so that the target device renders and processes the multimedia data stream based on the target request to obtain the multimedia data stream to be pushed. Furthermore, the multimedia data stream to be pushed is pushed based on a push streaming method corresponding to the target device, thereby achieving effective rendering of screen content without requiring performance of the terminal device, and achieving effective push streaming of the rendered data stream to display the rendered data stream, thereby improving the push streaming efficiency and display effect of the data stream.

[0081] FIG3 is a flow chart of a multimedia data stream processing method provided by an embodiment of the present disclosure. This embodiment of the present disclosure is an optional embodiment of the above-mentioned disclosed embodiments. The multimedia data stream processing method can be described by taking the multimedia data stream as a live broadcast image, and the received target requests as a request to render hair accessory special effects and a request to receive an interactive gift as an example. As shown in FIG3, the execution process of the method provided by the embodiment of the present disclosure can be:

[0082] During the process of the live broadcast screen being displayed by the client (e.g., mobile phone) to which the user (anchor and / or audience) belongs, the live broadcast screen can be obtained. Furthermore, when a request for rendering hair accessory special effects is received (updating hair accessory special effects), the acquired live broadcast screen and the triggered hair accessory special effects can be sent to a target device with higher performance (e.g., a PC or cloud). Furthermore, time-consuming operations such as part recognition can be performed on the live broadcast screen based on the target device, and the received hair accessory special effects can be rendered into the live broadcast screen, so as to render the hair accessory special effects onto the hair included in the live broadcast screen. Furthermore, if an interactive gift reception request is received again (interactive gift is received), the acquired live broadcast screen and the triggered interactive gift can be sent to the target device. Furthermore, the preset animation corresponding to the received interactive gift can be rendered onto the live broadcast screen, and a live broadcast screen with rendered hair accessory special effects and interactive gift preset animation can be obtained. Furthermore, a pre-set streaming method can be determined. If the streaming method is direct streaming, the live video to be pushed can be sent to the target server or CDN, and then pushed to the host and audience based on the target server or CDN. If the streaming method is indirect streaming, the live video to be pushed can be fed back to the host, and then displayed based on the client. Furthermore, the live video to be pushed can be sent to the target server or CDN based on the client, and then pushed to the audience based on the target server or CDN.

[0083] The technical solution of the embodiment of the present disclosure obtains a multimedia data stream, and further, upon receiving a target request, sends the obtained multimedia data stream to a target device, so that the target device renders and processes the multimedia data stream based on the target request, thereby obtaining the multimedia data stream to be pushed. This solves the problem in the related art that when rendering a live page, the performance requirements for the terminal device are relatively high, resulting in high video processing costs and poor universality. It achieves the effect of effectively rendering the picture content without requiring the performance of the terminal device, thereby improving the universality and efficiency of video processing.

[0084] FIG4 is a schematic structural diagram of a multimedia data stream processing device provided by an embodiment of the present disclosure. As shown in FIG4 , the device includes: a data stream acquisition module 310 and a data stream sending module 320 .

[0085] Among them, the data stream acquisition module 310 is used to obtain multimedia data streams, wherein the multimedia data streams include audio streams and / or video streams; the data stream sending module 320 is used to send the acquired multimedia data streams to the target device when a target request is received, so that the target device renders the multimedia data stream based on the target request to obtain the multimedia data stream to be pushed.

[0086] On the basis of the above-mentioned optional technical solutions, optionally, the device is configured at the host end, and the multimedia data stream to be pushed is a data stream to be pushed to the host end and / or the viewer end.

[0087] Based on the above optional technical solutions, optionally, the target request includes at least one of the following: a rendering special effect request, a resource transfer request, and a live broadcast interaction request.

[0088] Based on the above-mentioned optional technical solutions, optionally, the data stream sending module 320 is specifically used to send the acquired multimedia data stream and the target request to the target device, so that the target device calls the corresponding rendering data packet based on the target request, and renders the multimedia data stream based on the rendering data packet.

[0089] Based on the above optional technical solutions, optionally, the device includes: a data stream pushing module.

[0090] The data stream pushing module is used to push the multimedia data stream to be pushed based on the streaming mode corresponding to the target device after obtaining the multimedia data stream to be pushed.

[0091] On the basis of the above-mentioned optional technical solutions, optionally, the streaming mode includes a direct streaming mode, and the data stream pushing module includes: a direct streaming unit.

[0092] The direct streaming unit is used to send the multimedia data stream to be pushed to the target server or streaming server based on the target device, so that the target server or streaming server pushes the multimedia data stream to be pushed to the anchor terminal and / or the viewer terminal.

[0093] On the basis of the above-mentioned optional technical solutions, optionally, the streaming mode includes an indirect streaming mode, and the data stream pushing module includes: an indirect streaming unit.

[0094] The indirect streaming unit is used to receive and display the multimedia data stream to be pushed sent by the target device, and send the multimedia data stream to be pushed to the target server or streaming server, so that the target server or streaming server pushes the multimedia data stream to be pushed to the audience end.

[0095] Based on the above optional technical solutions, optionally, the anchor terminal that obtains the multimedia data stream and the target device correspond to the same local area network.

[0096] Based on the above optional technical solutions, optionally, the target device is a device that meets preset processing conditions.

[0097] The technical solution of the embodiment of the present disclosure obtains a multimedia data stream, and further, upon receiving a target request, sends the obtained multimedia data stream to a target device, so that the target device renders and processes the multimedia data stream based on the target request, thereby obtaining the multimedia data stream to be pushed. This solves the problem in the related art that when rendering a live page, the performance requirements for the terminal device are relatively high, resulting in high video processing costs and poor universality. It achieves the effect of effectively rendering the picture content without requiring the performance of the terminal device, thereby improving the universality and efficiency of video processing.

[0098] The multimedia data stream processing device provided in the embodiments of the present disclosure can execute the multimedia data stream processing method provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0099] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present disclosure.

[0100] FIG5 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Referring to FIG5 , a schematic diagram of the structure of an electronic device (such as a terminal device or server in FIG5 ) 500 suitable for implementing an embodiment of the present disclosure is shown below. The terminal device in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. The electronic device shown in FIG5 is merely an example and should not impose any limitations on the functions and scope of use of the embodiments of the present disclosure.

[0101] As shown in FIG5 , the electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. Various programs and data required for the operation of the electronic device 500 are also stored in the RAM 503. The processing device 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An edit / output (I / O) interface 505 is also connected to the bus 504.

[0102] Typically, the following devices may be connected to the I / O interface 505: an input device 506 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 508 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 509. The communication device 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. Although FIG5 shows the electronic device 500 with various devices, it should be understood that not all of the devices shown are required to be implemented or present. More or fewer devices may alternatively be implemented or present.

[0103] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0104] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0105] The electronic device provided in the embodiment of the present disclosure and the multimedia data stream processing method provided in the above embodiment belong to the same inventive concept. For technical details not fully described in the embodiment of the present disclosure, please refer to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0106] An embodiment of the present disclosure provides a computer storage medium on which a computer program is stored. When the program is executed by a processor, the multimedia data stream processing method provided by the above embodiment is implemented.

[0107] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0108] In some embodiments, the client and server can communicate using any currently known or later developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or later developed network.

[0109] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0110] The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: obtains a multimedia data stream, wherein the multimedia data stream includes an audio stream and / or a video stream; and upon receiving a target request, sends the obtained multimedia data stream to a target device, so that the target device renders and processes the multimedia data stream based on the target request to obtain the multimedia data stream to be pushed.

[0111] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0112] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0113] The units involved in the embodiments described in this disclosure may be implemented in software or hardware. In some cases, the name of a unit does not limit the unit itself. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses."

[0114] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0115] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0116] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0117] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0118] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. A method for processing multimedia data stream, comprising: Obtaining a multimedia data stream, wherein the multimedia data stream includes an audio stream and / or a video stream; and When a target request is received, sending the obtained multimedia data stream to a target device, so that the target device performs rendering processing on the multimedia data stream based on the target request to obtain a multimedia data stream to be pushed.

2. The method according to claim 1, wherein the method is applied to a host side, and the multimedia data stream to be pushed is a data stream to be pushed to the host side and / or the viewer side.

3. The method according to claim 1, wherein the target request includes at least one of the following: Rendering effect request, resource transfer request, live interaction request.

4. The method according to claim 1, wherein the sending the obtained multimedia data stream to a target device, so that the target device performs rendering processing on the multimedia data stream based on the target request, includes: Sending the obtained multimedia data stream and the target request to the target device, so that the target device retrieves a corresponding rendering data packet based on the target request and performs rendering processing on the multimedia data stream based on the rendering data packet.

5. The method according to claim 1, further comprising, after obtaining the multimedia data stream to be pushed: Pushing the multimedia data stream to be pushed based on the push mode corresponding to the target device.

6. The method according to claim 5, wherein the push mode includes a direct push mode, and the pushing the multimedia data stream to be pushed based on the push mode corresponding to the target device includes: Sending the multimedia data stream to be pushed to a target server or a push server based on the target device, so that the target server or the push server pushes the multimedia data stream to be pushed to the host side and / or the viewer side.

7. The method according to claim 5, wherein the push mode includes an indirect push mode, and the pushing the multimedia data stream to be pushed based on the push mode set in the target device includes: Receiving and displaying the multimedia data stream to be pushed sent by the target device, and sending the multimedia data stream to be pushed to a target server or a push server, so that the target server or the push server pushes the multimedia data stream to be pushed to the viewer side.

8. The method according to claim 1, wherein the host side for obtaining the multimedia data stream and the target device correspond to the same local area network.

9. The method according to claim 1, wherein the target device is a device that meets a preset processing condition.

10. A multimedia data stream processing apparatus, comprising: A data stream obtaining module, configured to obtain a multimedia data stream, wherein the multimedia data stream includes an audio stream and / or a video stream; and A data stream sending module, configured to send the obtained multimedia data stream to a target device when a target request is received, so that the target device performs rendering processing on the multimedia data stream based on the target request to obtain a multimedia data stream to be pushed.

11. An electronic device, comprising: One or more processors; A storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, enabling the one or more processors to implement the multimedia data stream processing method according to any one of claims 1-9.

12. A storage medium containing computer-executable instructions, wherein the computer-executable instructions are used to execute the multimedia data stream processing method according to any one of claims 1-9 when executed by a computer processor.

Images