Multimedia data stream processing method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a multimedia data stream processing method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a multimedia data stream; and under the condition that the target request is received, sending the obtained multimedia data stream to the target equipment, so that the target equipment performs rendering processing on the multimedia data stream based on the target request to obtain a multimedia data stream to be pushed. According to the technical scheme provided by the embodiment of the invention, the effect of improving the video processing universality is realized.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the technical field of data processing, and in particular, to a method, apparatus, electronic device, and storage medium for processing multimedia data streams. Background Art

[0002] With the rapid development of computer technology and Internet technology, various service scenarios emerge in an endless stream. In service scenarios, the scene functions and the interaction methods with users are often enriched through various associated scene services. For example, by inputting an interaction request associated with a live broadcast page, the live broadcast page can be rendered to display an effect corresponding to the interaction request.

[0003] Generally, the rendering of a live broadcast page mainly depends on the client to which the live broadcast page belongs for processing. At this time, the performance requirements for the terminal device are relatively high, resulting in problems such as high video processing costs and poor universality. Summary of the Invention

[0004] The present disclosure provides a method, apparatus, electronic device, and storage medium for processing multimedia data streams, so as to effectively render the picture content under the condition of having no requirements for the performance of the terminal device, and improve the universality and efficiency of video processing.

[0005] In a first aspect, an embodiment of the present disclosure provides a method for processing a multimedia data stream, the method including:

[0006] Obtain a multimedia data stream, where the multimedia data stream includes an audio stream and / or a video stream;

[0007] When a target request is received, send the obtained multimedia data stream to a target device, so that the target device performs rendering processing on the multimedia data stream based on the target request to obtain a multimedia data stream to be pushed.

[0008] In a second aspect, an embodiment of the present disclosure further provides a multimedia data stream processing apparatus, the apparatus including:

[0009] A data stream obtaining module, configured to obtain a multimedia data stream, where the multimedia data stream includes an audio stream and / or a video stream;

[0010] A data stream sending module, configured to send the obtained multimedia data stream to a target device when a target request is received, so that the target device performs rendering processing on the multimedia data stream based on the target request to obtain a multimedia data stream to be pushed.

[0011] In a third aspect, an embodiment of the present disclosure further provides an electronic device, the electronic device including:

[0012] One or more processors;

[0013] A storage device for storing one or more programs,

[0014] When the one or more programs are executed by the one or more processors, the one or more processors implement the multimedia data stream processing method as described in any one of the embodiments of the present disclosure.

[0015] In a fourth aspect, an embodiment of the present disclosure further provides a storage medium containing computer-executable instructions, and the computer-executable instructions are used to execute the multimedia data stream processing method as described in any one of the embodiments of the present disclosure when executed by a computer processor.

[0016] The technical solution of the embodiment of the present disclosure obtains a multimedia data stream. Further, in the case of receiving a target request, the obtained multimedia data stream is sent to a target device, so that the target device performs rendering processing on the multimedia data stream based on the target request to obtain a multimedia data stream to be pushed. Thus, the picture content can be effectively rendered, and the universality and efficiency of video processing can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In combination with the drawings and with reference to the following specific embodiments, the above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic and the original elements and elements are not necessarily drawn to scale.

[0018] Figure 1 It is a flowchart of a multimedia data stream processing method provided by an embodiment of the present disclosure;

[0019] Figure 2 It is a flowchart of another multimedia data stream processing method provided by an embodiment of the present disclosure;

[0020] Figure 3 It is a flowchart of a multimedia data stream processing method provided by an embodiment of the present disclosure;

[0021] Figure 4 It is a structural schematic diagram of a multimedia data stream processing device provided by an embodiment of the present disclosure;

[0022] Figure 5 It is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0023] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Instead, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not used to limit the protection scope of the present disclosure.

[0024] It should be understood that the various steps recited in the method embodiments of the present disclosure can be executed in a different order and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this regard.

[0025] As used herein, the term "comprising" and its variations are open-ended, i.e., "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0026] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependent relationships.

[0027] It should be noted that the modifications of "one" and "plural" mentioned in the present disclosure are illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly specified in the context, it should be understood as "one or more".

[0028] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only for illustrative purposes and are not used to limit the scope of these messages or information.

[0029] It can be understood that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, usage scopes, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0030] For example, when responding to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested by the user will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, an application program, a server or a storage medium that executes the operation of the technical solution of the present disclosure according to the prompt message.

[0031] As an optional but non-limiting implementation manner, in response to receiving an active request from a user, the manner of sending a prompt message to the user may be, for example, a pop-up window manner, and the prompt message may be presented in text in the pop-up window. In addition, the pop-up window may also carry selection controls for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0032] It can be understood that the above notification and user authorization acquisition process is only illustrative and does not constitute a limitation on the implementation manner of the present disclosure. Other manners that comply with relevant laws and regulations can also be applied to the implementation manner of the present disclosure.

[0033] It can be understood that the data involved in the present technical solution (including but not limited to the data itself, the acquisition or use of the data) should comply with the requirements of corresponding laws, regulations and related regulations.

[0034] Before introducing the present technical solution, an exemplary description of the application scenario can be given first. This technical solution can be applied to any scenario that requires real-time rendering processing of multimedia data streams, such as a live broadcast scenario, a video real-time processing scenario or an audio real-time processing scenario, etc. To introduce the present technical solution more clearly, the execution process of the present technical solution can be described by taking the live broadcast scenario as an example. For other data stream real-time rendering processing scenarios, reference can be made to the multimedia data stream rendering processing process in the live broadcast scenario. Exemplarily, during a live broadcast, the multimedia data stream of the host user can be obtained based on a live broadcast client and processed. However, when the host user or the user watching the live broadcast triggers some interactive controls on the live broadcast page, and the processing content corresponding to the interactive controls is relatively complex and time-consuming, it is still necessary to process based on the live broadcast client. At this time, the performance requirements for the live broadcast client are relatively high, that is, highly dependent on the hardware device of the live broadcast client. Further, if the performance of the live broadcast client is not high, there will be problems such as page freezing, latency and content lag of the live broadcast page, which will further cause problems with the live broadcast picture effect.

[0035] At this time, based on the technical solution of the embodiment of the present disclosure, the live broadcast picture can be obtained. Further, when it is detected that the interaction request associated with the live broadcast picture is a target request, the obtained live broadcast picture can be sent to a target device with better performance belonging to the same local area network as the live broadcast client, so that the target device renders and processes the obtained live broadcast picture based on the target request.

[0036] Figure 1The figure is a schematic flowchart of a method for processing multimedia data streams provided by an embodiment of the present disclosure. The embodiment of the present disclosure is applicable to any scenario that requires real-time rendering of data streams. This method can be executed by a multimedia data stream processing device, which can be implemented in the form of software and / or hardware. Optionally, it can be implemented by an electronic device, which can be a mobile terminal, a PC, or a server, etc.

[0037] As Figure 1 shown, the method of this embodiment specifically may include:

[0038] S110. Obtain a multimedia data stream.

[0039] It should be noted that the device for executing the technical solution provided by the embodiment of the present disclosure may include at least two devices. Among them, one of the two devices can be a device for obtaining a multimedia data stream, and the other can be a device for processing the obtained multimedia data stream. To improve universality, optionally, the data stream acquisition device can be a terminal device. The device for processing the multimedia data stream can be a device with higher processing performance belonging to the same local area network, such as a terminal device or a server.

[0040] Among them, the multimedia data stream can be a data stream composed of at least one media type (such as, image, text, audio, video, etc.). In this embodiment, the multimedia data stream can include an audio stream and / or a video stream. The audio stream can be understood as a data stream composed of audio. The video stream can be understood as a data stream composed of video. It should be noted that in a live broadcast scenario, the multimedia data stream can be the live broadcast picture and audio information presented simultaneously on the live broadcast page; in an audio real-time processing scenario, the multimedia data stream can be audio information.

[0041] S120. When a target request is received, send the obtained multimedia data stream to a target device, so that the target device performs rendering processing on the multimedia data stream based on the target request to obtain a multimedia data stream to be pushed.

[0042] In this embodiment, when presenting a live broadcast page corresponding to a multimedia data stream, the host user or the viewing user can trigger corresponding operations on the live broadcast page. Furthermore, a request can be generated based on the corresponding operations, and the generated request can be used as the target request. That is to say, the target request acting on the multimedia data stream display page can be detected continuously or periodically. Furthermore, when the target request is received, an effect corresponding to the target request can be rendered on the display page. Usually, the picture content required to be rendered by the target request at this time is relatively time-consuming, and the multimedia data stream can be sent to the target device so that another device with higher performance can perform rendering processing.

[0043] Among them, the target request can be understood as an interaction request associated with the multimedia data stream. Generally, during the display process of the multimedia data stream, there may be an end where the multimedia data stream belongs and an end that watches the multimedia data stream. Both ends can input requests for the page that displays the multimedia data stream. At least some of the requests input by at least one of the two ends for the multimedia data stream display page can be determined as the target request. Optionally, the target request includes requests made by the first user and / or the second user on the live page. The target request can be of various types. Optionally, it can include at least one of a rendering special effect request, a resource transfer request, and a live interaction request.

[0044] Among them, the first user can be any user associated with the display process of the multimedia data stream. Optionally, the first user can be the host in the live broadcast room. The second user can also be any user associated with the display process of the multimedia data stream. For example, the second user can be a user who displays the live page of the live broadcast room corresponding to the first user on the display interface of the affiliated terminal, such as an audience watching the live broadcast of the host. The live page is the page that presents the multimedia data stream. The live page can be understood as the page that can be presented in the associated terminal display interface during the live broadcast process, and this page can include the live data stream corresponding to the first user or the second user.

[0045] Among them, the rendering special effect request can be understood as a request to perform special effect processing on the content in the live page or a request to display special effects in the live page. Exemplarily, the rendering special effect request can be the corresponding request when triggering the hair accessory special effect control.

[0046] Among them, the resource transfer request can be understood as a request for the associated interactive service and resources to be transferred. Optionally, it is a request corresponding to sending gifts. The resource transfer request can be a request to transfer resources from the resource sender to the resource receiver. The resources can be interactive gifts, virtual resources, etc. In practical applications, when detecting a trigger operation on the associated control for resource transfer, the trigger operation can be responded to, and a preset animation or preset image corresponding to the associated control can be rendered on the live page.

[0047] Among them, the live interaction request can be understood as a request generated during the interaction between the host side and the audience side, or between the host side and the host side, or between the audience side and the audience side during the host's live broadcast. Exemplarily, the live interaction request can include posting comments.

[0048] It should be noted that the request objects corresponding to the rendering effect request, the resource transfer request, and the live interaction request can be at least one of the first user and the second user. That is to say, both the first user and the second user can send the rendering effect request, the resource transfer request, and the live interaction request to render the corresponding information on the live page when receiving the corresponding target request. The advantage of this setting is as follows: It improves the performance of the application software in the rendering ability of interaction requests, enhances the applicable range of renderable interaction requests, and improves the user's usage experience and interaction experience.

[0049] In this embodiment, when receiving the target request, the obtained multimedia data stream is sent to the target device, so that the target device performs rendering processing on the multimedia data stream based on the target request to obtain the multimedia data stream to be pushed.

[0050] Among them, the target device can be understood as an electronic device associated with the client to which the multimedia data stream belongs and having higher performance than the client. Optionally, the target device is a device that meets the preset processing conditions. The preset processing conditions can be that the device performance is higher than that of the client to which the associated multimedia data stream belongs. In this embodiment, the host end that obtains the multimedia data stream and the target device correspond to the same local area network, that is, the distance between the client and the target device can be within the coverage range of the local area network. The target device can be any electronic device with higher performance than the client to which the multimedia data stream belongs. Optionally, the target device can be a server, a PC, or the cloud, etc. The multimedia data stream to be pushed can be understood as the data stream obtained after rendering processing on the multimedia data stream and to be pushed. It should be noted that when this method is applied to the host end, the multimedia data stream to be pushed is the data stream to be pushed to the host end and / or the audience end. That is, when the multimedia data stream to be pushed is obtained, the multimedia data stream to be pushed can be pushed to the host end and / or the audience end according to the corresponding streaming method.

[0051] It should be noted that in the related art, when receiving the target request, usually the multimedia data stream is rendered on the client to which it belongs based on the target request, and the rendered multimedia data stream is displayed on the live page of the corresponding client. However, usually, most of the client devices to which the multimedia data stream belongs are mobile terminals, and they have certain limitations, such as slow rendering, resulting in problems such as stuttering, delay, and lag of the picture. Based on this, the multimedia data stream that needs to be rendered can be sent to a target device with higher performance, so that the target device performs rendering processing on the multimedia data stream based on the target request. Thus, the effect of video rendering can be achieved for various types of terminal devices, the universality of the video rendering process is improved, and the processing efficiency and rendering effect of the multimedia data stream are improved.

[0052] In practical applications, during the process of presenting a multimedia data stream on a live broadcast page, multiple interaction requests associated with the live broadcast page may be generated. Among these interaction requests, there may be requests that match the target request, or there may be requests that do not match the target request. Furthermore, when an interaction request associated with the live broadcast page is received, the interaction request can be detected to determine whether it matches the target request. Further, in the case where the target request is determined to be received, the obtained multimedia data stream can be sent to the target device so that the target device can perform rendering processing on the received multimedia data stream based on the target request.

[0053] In this embodiment, sending the obtained multimedia data stream to the target device so that the target device performs rendering processing on the multimedia data stream based on the target request includes: sending the obtained multimedia data stream and the target request to the target device so that the target device retrieves the corresponding rendering data packet based on the target request and performs rendering processing on the multimedia data stream based on the rendering data packet.

[0054] Among them, the rendering data packet may include data associated with the rendering process. In practical applications, after multiple target requests are determined, the rendering process data corresponding to each target request can be set separately and encapsulated into a data packet, and thus the rendering data packet corresponding to each target request can be obtained. Further, after each target request is determined, an association relationship can be established between the target request and the corresponding rendering data packet, and the rendering data packet can be stored in the target device. Furthermore, after the target device receives the target request sent by the client to which the multimedia data stream belongs, the corresponding rendering data packet can be retrieved according to the target request and the pre-established association relationship.

[0055] In practical applications, in the case where the target request is received, the obtained multimedia data stream and the received target request can be sent to the target device. Furthermore, in the case where the target device receives the multimedia data stream and the target request, the rendering data packet corresponding to the target request can be retrieved according to the received target request and the pre-established data packet association relationship. After that, the received multimedia data stream can be rendered according to the retrieved rendering data packet, and thus the multimedia data stream to be pushed can be obtained. Furthermore, the multimedia data stream to be pushed can be processed according to actual requirements. The advantage of such a setting is that: it realizes the effect of improving the data processing efficiency and processing quality of interaction data streams with high performance requirements, and moreover, ensures the real-time rendering of the multimedia data stream and improves the rendering display effect of the multimedia video stream.

[0056] Exemplarily, taking the target request as a rendering special effect request and the acquired multimedia data stream as a live broadcast screen as an example to illustrate the rendering process: The live broadcast screen and the hair accessory special effect rendering request corresponding to the triggered hair accessory special effect can be sent to the target device. Further, after the target device receives the live broadcast screen and the hair accessory special effect rendering request, it can retrieve the rendering data packet corresponding to the hair accessory special effect rendering request, and perform rendering processing on the live broadcast screen based on the retrieved rendering data packet. Thus, the multimedia data stream to be pushed can be obtained, that is, the live broadcast screen with the hair accessory special effect rendered.

[0057] The technical solution of the embodiments of the present disclosure, by acquiring the multimedia data stream, and further, in the case of receiving the target request, sending the acquired multimedia data stream to the target device, so that the target device performs rendering processing on the multimedia data stream based on the target request to obtain the multimedia data stream to be pushed, solves the problems in the related art that the performance requirements for the terminal device are relatively high when rendering the live page, resulting in relatively high video processing costs and poor universality, and realizes that under the condition of having no requirements for the performance of the terminal device, the picture content can be effectively rendered, improving the universality and efficiency of video processing.

[0058] Figure 2 It is a schematic flowchart of another method for processing multimedia data stream provided by the embodiments of the present disclosure. Based on the above embodiments, after obtaining the multimedia data stream to be pushed, the technical solution of this embodiment can push the multimedia data stream to be pushed based on the push stream mode corresponding to the target device. Among them, the same or similar technical features as those in the foregoing embodiments will not be described in detail here.

[0059] As Figure 2 shown, the method of this embodiment may specifically include:

[0060] S210. Acquire the multimedia data stream.

[0061] S220. In the case of receiving the target request, send the acquired multimedia data stream to the target device, so that the target device performs rendering processing on the multimedia data stream based on the target request to obtain the multimedia data stream to be pushed.

[0062] S230. Push the multimedia data stream to be pushed based on the push stream mode corresponding to the target device.

[0063] In practical applications, in order to display the multimedia data stream to be pushed on the corresponding client, after obtaining the multimedia data stream to be pushed, the multimedia data stream to be pushed can also be pushed to push the multimedia data stream to be pushed to the corresponding client.

[0064] In this embodiment, the streaming method can be used to indicate the process of pushing the multimedia data stream to be pushed to the corresponding server, that is, the method of performing streaming. Those skilled in the art can understand that streaming refers to the process of transmitting the encapsulated content to the server. Streaming is a real-time streaming media transmission technology that allows users to perform online audio and video transmission, enabling other users to watch the specified video stream and audio stream through the network. The streaming method can include various types. Optionally, it includes the direct streaming method, the indirect streaming method, etc. Among them, the direct streaming method can be understood as directly pushing the multimedia data stream to be pushed to the corresponding server. The indirect streaming method can be understood as first sending the multimedia data stream to be pushed to the client to which the multimedia data stream belongs, and then pushing the multimedia data stream to be pushed to the corresponding server based on this client.

[0065] It should be noted that streaming has relatively high requirements for the network. If the network is unstable, the streaming effect may deteriorate. Therefore, the setting of the streaming method can be associated with the local area network to which the target device belongs. If the network environment of this local area network is relatively stable, the streaming operation can be directly performed, that is, the streaming method can be set to the direct streaming method; if the network environment of this local area network is poor or the target device is in an unconnected state, if the streaming operation is directly performed, the streaming effect is poor. Therefore, the streaming method can be set to the indirect streaming method.

[0066] It should also be noted that the setting of the streaming method can also be associated with the actual requirements, that is, the user can set the corresponding streaming method by inputting the streaming method setting operation, so that the target device pushes the corresponding multimedia data stream according to the streaming method set by the user.

[0067] In practical applications, after obtaining the multimedia data stream to be pushed, the streaming method corresponding to the target device can be determined. Furthermore, the multimedia data stream to be pushed can be processed by streaming according to this streaming method.

[0068] It should be noted that the streaming method can include at least two types. The following can respectively describe the process of pushing the multimedia data stream to be pushed under these two streaming methods.

[0069] Optionally, the streaming method includes the direct streaming method. Pushing the multimedia data stream to be pushed based on the streaming method corresponding to the target device includes: sending the multimedia data stream to be pushed to the target server or the streaming server based on the target device, so that the target server or the streaming server pushes the multimedia data stream to be pushed to the host side and / or the audience side.

[0070] Among them, the target server can be understood as a program that serves the client associated with the multimedia data stream. The device running the target server can be called the target server. The push stream server can be understood as a server that deploys the Content Delivery Network (CDN) interface. The host client can be the client of the user to whom the multimedia data stream belongs. The viewer client can be the client of the user who watches the multimedia data stream.

[0071] In practical applications, when the push stream method corresponding to the target device is the direct push stream method, it can be indicated that the multimedia data stream to be pushed can be directly pushed to the corresponding server based on the target device. Specifically, after obtaining the multimedia data stream to be pushed, the multimedia data stream to be pushed can be sent to the target server or the push stream server based on the target device. Furthermore, the multimedia data stream to be pushed can be pushed to the host client and / or the viewer client based on the target server or the push stream server, so as to display the multimedia data stream to be pushed based on the display interfaces corresponding to each client. It should be noted that when the push stream method is the direct push stream method, in order to enable the client that sends the multimedia data stream to the target device to also receive the multimedia data stream to be pushed, the target server or the push stream server can be made to push the multimedia data stream to be pushed to the client of the user to whom the multimedia data stream belongs, that is, the host client.

[0072] Exemplarily, continuing with the above example, after obtaining the live broadcast screen with the rendered hairpin special effect, the live broadcast screen can be sent to the target server or the push stream server based on the target device. Furthermore, the live broadcast screen can be pushed to the host client and / or the viewer client based on the target server or the push stream server, so that the live broadcast page with the rendered hairpin special effect can be displayed on the display interfaces of the live broadcast viewing user and the host's client.

[0073] It should be noted that the advantage of pushing the multimedia data stream to be pushed based on the direct push stream method is that it realizes the effect of directly pushing the data stream to be pushed to the server based on the data stream rendering device, improves the data rendering efficiency and rendering effect, and improves the push stream efficiency and the display effect of the data stream on the client.

[0074] Optionally, the push stream method includes an indirect push stream method. Pushing the multimedia data stream to be pushed based on the push stream method set in the target device includes: receiving and displaying the multimedia data stream to be pushed sent by the target device, and sending the multimedia data stream to be pushed to the target server or the push stream server, so that the target server or the push stream server pushes the multimedia data stream to be pushed to the viewer client.

[0075] In practical applications, when the push stream mode corresponding to the target device is the indirect push stream mode, it indicates that the multimedia data stream to be pushed cannot be directly pushed to the corresponding server. Instead, the multimedia data stream to be pushed needs to be sent to the device end capable of pushing the stream first, so that the device end can push the received multimedia data stream to the corresponding server. Specifically, after obtaining the multimedia data stream to be pushed, the multimedia data stream to be pushed can be fed back to the client to which the multimedia data stream belongs, that is, the client that has sent the obtained multimedia data stream. Furthermore, based on this client, the multimedia data stream to be pushed sent by the target device can be displayed, and based on this client, the received multimedia data stream can be sent to the target server or the push stream server. Furthermore, based on the target server or the push stream server, the multimedia data stream to be pushed can be pushed to the viewer end, so that the received multimedia data stream can be displayed based on the display interface corresponding to each viewer end.

[0076] Exemplarily, continuing to refer to the above example, after obtaining the live broadcast screen with the rendered hairpin special effect, the live broadcast screen can be fed back to the host's client based on the target device, so that the live broadcast screen with the rendered hairpin special effect can be displayed based on the display interface of the host client. Furthermore, based on the host client, the live broadcast screen can be sent to the target server or the push stream server. Furthermore, based on the target server or the push stream server, the live broadcast screen can be pushed to the client of the live broadcast viewing user, so that the live broadcast page with the rendered hairpin special effect can be displayed on the display interface of the client to which the live broadcast viewing user belongs.

[0077] It should be noted that the advantage of pushing the multimedia data stream to be pushed based on the indirect push stream mode is that it realizes the effect of pushing the data stream to be pushed based on the client to which the data stream belongs, and improves the display effect of the data stream on the data stream viewing end.

[0078] The technical solution of the embodiments of the present disclosure obtains the multimedia data stream, and then, when receiving the target request, sends the obtained multimedia data stream to the target device, so that the target device performs rendering processing on the multimedia data stream based on the target request to obtain the multimedia data stream to be pushed. Further, the multimedia data stream to be pushed is pushed based on the push stream mode corresponding to the target device, realizing that under the condition of having no requirements for the performance of the terminal device, the screen content can be effectively rendered, and realizing the effect of effectively pushing the rendered data stream to display the rendered data stream, improving the push stream efficiency and display effect of the data stream.

[0079] Figure 3Schematic flowchart of a multimedia data stream processing method provided by an embodiment of the present disclosure. The embodiment of the present disclosure is an optional embodiment of the above-mentioned various disclosure embodiments. Taking the multimedia data stream as a live broadcast screen and the received target request as a rendering hairpin special effect request and an interactive gift receiving request as an example, the multimedia data stream processing method will be described. As Figure 3 shown, the execution process of the method provided by the embodiment of the present disclosure may be as follows:

[0080] During the process of displaying the live broadcast screen on the client (such as a mobile phone) of the user (the anchor and / or the audience), the live broadcast screen can be obtained. Furthermore, in the case of receiving a rendering hairpin special effect request (updating the hairpin special effect), the obtained live broadcast screen and the triggered hairpin special effect can be sent to a target device with higher performance (such as a PC or the cloud, etc.). Furthermore, time-consuming operations such as part recognition can be performed on the live broadcast screen based on the target device, and the received hairpin special effect can be rendered onto the live broadcast screen to render the hairpin special effect onto the hair included in the live broadcast screen. Further, if an interactive gift receiving request (receiving an interactive gift) is received again, the obtained live broadcast screen and the triggered interactive gift can be sent to the target device. Furthermore, the preset animation corresponding to the received interactive gift can be rendered onto the live broadcast screen, and the live broadcast screen with the rendered hairpin special effect and the interactive gift preset animation can be obtained. Further, the preset push stream method can be determined. If the push stream method is a direct push stream method, the live broadcast screen to be pushed can be sent to the target server or the CDN end to push the live broadcast screen to be pushed to the anchor end and the audience end based on the target server or the CDN end; if the push stream method is an indirect push stream method, the live broadcast screen to be pushed can be fed back to the anchor end to display the live broadcast screen based on this client. Furthermore, the live broadcast screen to be pushed can be sent to the target server or the CDN end based on this client to push the live broadcast screen to be pushed to the audience end based on the target server or the CDN end.

[0081] The technical solution of the embodiment of the present disclosure obtains a multimedia data stream. Further, in the case of receiving a target request, the obtained multimedia data stream is sent to a target device, so that the target device performs rendering processing on the multimedia data stream based on the target request to obtain the multimedia data stream to be pushed, solving the problem in the related art that the performance requirements for the terminal device are relatively high when rendering the live broadcast page, resulting in higher video processing costs and poor universality. It realizes that under the condition of no requirements for the performance of the terminal device, the content of the screen can be effectively rendered, improving the universality and efficiency of video processing.

[0082] Figure 4 Schematic structural diagram of a multimedia data stream processing device provided by an embodiment of the present disclosure, as Figure 4As shown, the device includes: a data stream acquisition module 310 and a data stream sending module 320.

[0083] Among them, the data stream acquisition module 310 is used to acquire a multimedia data stream, where the multimedia data stream includes an audio stream and / or a video stream; the data stream sending module 320 is used to, upon receiving a target request, send the acquired multimedia data stream to a target device, so that the target device performs rendering processing on the multimedia data stream based on the target request to obtain a multimedia data stream to be pushed.

[0084] Based on the above optional technical solutions, optionally, the device is configured at the host end, and the multimedia data stream to be pushed is a data stream to be pushed to the host end and / or the viewer end.

[0085] Based on the above optional technical solutions, optionally, the target request includes at least one of the following: a rendering effect request, a resource transfer request, and a live interaction request.

[0086] Based on the above optional technical solutions, optionally, the data stream sending module 320 is specifically configured to send the acquired multimedia data stream and the target request to the target device, so that the target device retrieves a corresponding rendering data packet based on the target request and performs rendering processing on the multimedia data stream based on the rendering data packet.

[0087] Based on the above optional technical solutions, optionally, the device includes: a data stream pushing module.

[0088] The data stream pushing module is used to, after obtaining the multimedia data stream to be pushed, push the multimedia data stream to be pushed based on the push mode corresponding to the target device.

[0089] Based on the above optional technical solutions, optionally, the push mode includes a direct push mode, and the data stream pushing module includes: a direct push unit.

[0090] The direct push unit is used to send the multimedia data stream to be pushed to a target server or a push server based on the target device, so that the target server or the push server pushes the multimedia data stream to be pushed to the host end and / or the viewer end.

[0091] Based on the above optional technical solutions, optionally, the push mode includes an indirect push mode, and the data stream pushing module includes: an indirect push unit.

[0092] An indirect streaming unit is configured to receive and display the multimedia data stream to be pushed sent by the target device, and send the multimedia data stream to be pushed to the target server or the streaming server, so that the target server or the streaming server pushes the multimedia data stream to be pushed to the viewer side.

[0093] Based on the above optional technical solutions, optionally, the host end for obtaining the multimedia data stream and the target device correspond to the same local area network.

[0094] Based on the above optional technical solutions, optionally, the target device is a device that meets the preset processing conditions.

[0095] The technical solution of the embodiment of the present disclosure obtains a multimedia data stream. Further, in the case of receiving a target request, the obtained multimedia data stream is sent to a target device, so that the target device performs rendering processing on the multimedia data stream based on the target request to obtain a multimedia data stream to be pushed, which solves the problem that in the related art, the performance requirements for the terminal device are relatively high when rendering a live page, resulting in a relatively high video processing cost and poor universality. It realizes that under the condition of having no requirements for the performance of the terminal device, the picture content can be effectively rendered, improving the universality and efficiency of video processing.

[0096] The multimedia data stream processing device provided by the embodiment of the present disclosure can execute the multimedia data stream processing method provided by any embodiment of the present disclosure, and has corresponding functional modules and beneficial effects for executing the method.

[0097] It should be noted that the various units and modules included in the above device are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of the functional units are only for the convenience of mutual distinction and do not limit the protection scope of the embodiment of the present disclosure.

[0098] Figure 5 It is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. The following refers to Figure 5 , which shows a schematic structural diagram of an electronic device (such as Figure 5 the terminal device or the server in) 500 suitable for implementing the embodiment of the present disclosure. The terminal device in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), PMPs (Portable Multimedia Players), in-vehicle terminals (such as in-vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5The illustrated electronic device is merely an example and should not impose any limitation on the functions and usage scope of the embodiments of the present disclosure.

[0099] As Figure 5 shown, the electronic device 500 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 501, which may perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other through a bus 504. An editing / output (I / O) interface 505 is also connected to the bus 504.

[0100] Generally, the following devices may be connected to the I / O interface 505: an input device 506 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 508 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 509. The communication device 509 may allow the electronic device 500 to communicate with other devices wirelessly or wirelessly to exchange data. Although Figure 5 the electronic device 500 with various devices is shown, it should be understood that it is not required to implement or include all the shown devices. More or fewer devices may be implemented or included alternatively.

[0101] In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, the embodiments of the present disclosure include a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes program codes for performing the methods shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network through the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are executed.

[0102] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only for illustrative purposes and do not limit the scope of these messages or information.

[0103] The electronic device provided in the embodiments of the present disclosure and the multimedia data stream processing method provided in the above embodiments belong to the same inventive concept. The technical details not described in detail in the embodiments of the present disclosure may be referred to in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0104] An embodiment of the present disclosure provides a computer storage medium, on which a computer program is stored, and when the program is executed by a processor, the multimedia data stream processing method provided in the above embodiment is implemented.

[0105] It should be noted that the computer-readable medium in the present disclosure may be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which computer-readable program code is carried. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, and the computer-readable signal medium may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on a computer-readable medium may be transmitted by any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0106] In some embodiments, the client and the server may communicate using any currently known or future-developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with digital data communication in any form or medium (for example, a communication network). Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), the Internet (for example, the Internet), and end-to-end networks (for example, ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0107] The above computer-readable medium may be included in the above electronic device; or may exist separately and not be assembled into the electronic device.

[0108] The above computer-readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: obtain a multimedia data stream, where the multimedia data stream includes an audio stream and / or a video stream; and in response to receiving a target request, send the obtained multimedia data stream to a target device so that the target device performs rendering processing on the multimedia data stream based on the target request to obtain a multimedia data stream to be pushed.

[0109] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages or combinations thereof. The programming languages include, but are not limited to, object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0110] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system for performing the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0111] The units described in the embodiments of the present disclosure may be implemented in software or in hardware. In some cases, the name of the unit does not constitute a limitation on the unit itself. For example, the first acquisition unit may also be described as "the unit for acquiring at least two Internet protocol addresses".

[0112] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. By way of example, and without limitation, the types of hardware logic components that may be used include: Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), Systems on Chip (SOCs), Complex Programmable Logic Devices (CPLDs), and the like.

[0113] In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a Read-Only Memory (ROM), an Erasable Programmable Read-Only Memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0114] The foregoing description is only a preferred embodiment of the present disclosure and an illustration of the applied technical principles. Those skilled in the art should understand that the scope of the disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosure concept. For example, the technical solutions formed by the mutual replacement of the above features and the technical features (but not limited to) having similar functions disclosed in the present disclosure.

[0115] Moreover, although the operations are depicted in a particular order, this should not be understood as requiring that the operations be performed in the particular order shown or in sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present disclosure. Certain features described in the context of separate embodiments may also be implemented combinatorially in a single embodiment. Conversely, the various features described in the context of a single embodiment may also be implemented separately or in any suitable sub-combination in multiple embodiments.

[0116] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. On the contrary, the specific features and acts described above are merely example forms for implementing the claims.

Claims

1. A method for processing multimedia data streams, characterized in that, Including: Obtain a multimedia data stream, where the multimedia data stream includes an audio stream and / or a video stream; When a target request is received, send the obtained multimedia data stream to a target device, so that the target device performs rendering processing on the multimedia data stream based on the target request to obtain a multimedia data stream to be pushed.

2. The method according to claim 1, wherein The method is applied to the host side, and the multimedia data stream to be pushed is a data stream to be pushed to the host side and / or the viewer side.

3. The method according to claim 1, wherein The target request includes at least one of the following: A rendering effect request, a resource transfer request, a live interaction request.

4. The method according to claim 1, wherein The sending the obtained multimedia data stream to a target device so that the target device performs rendering processing on the multimedia data stream based on the target request includes: Send the obtained multimedia data stream and the target request to the target device, so that the target device retrieves a corresponding rendering data packet based on the target request, and performs rendering processing on the multimedia data stream based on the rendering data packet.

5. The method according to claim 1, wherein After obtaining the multimedia data stream to be pushed, the method further includes: Push the multimedia data stream to be pushed based on the push mode corresponding to the target device.

6. The method according to claim 5, wherein The push mode includes a direct push mode, and the pushing the multimedia data stream to be pushed based on the push mode corresponding to the target device includes: Based on the target device, send the multimedia data stream to be pushed to a target server or a push server, so that the target server or the push server pushes the multimedia data stream to be pushed to the host side and / or the viewer side.

7. The method according to claim 5, wherein The push mode includes an indirect push mode, and the pushing the multimedia data stream to be pushed based on the push mode set in the target device includes: Receive and display the multimedia data stream to be pushed sent by the target device, and send the multimedia data stream to be pushed to a target server or a push server, so that the target server or the push server pushes the multimedia data stream to be pushed to the viewer side.

8. The method according to claim 1, characterized in that, The host side that obtains the multimedia data stream and the target device correspond to the same local area network.

9. The method according to claim 1, characterized in that, The target device is a device that meets preset processing conditions.

10. A multimedia data stream processing device, characterized in that, Including: A data stream acquisition module, configured to acquire a multimedia data stream, where the multimedia data stream includes an audio stream and / or a video stream; A data stream sending module, configured to send the acquired multimedia data stream to a target device when a target request is received, so that the target device performs rendering processing on the multimedia data stream based on the target request to obtain a multimedia data stream to be pushed.

11. An electronic device, characterized in that, The electronic device includes: One or more processors; A storage device, configured to store one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the multimedia data stream processing method according to any one of claims 1-9.

12. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions are used to execute the multimedia data stream processing method according to any one of claims 1-9 when executed by a computer processor.

Images