Multimedia data stream processing method, device, electronic device, storage medium, and program

Abstract

An embodiment of the present disclosure provides a multimedia data stream processing method, device, electronic device, and storage medium, in which the method includes: acquiring a multimedia data stream; and, when a target request is received, sending the acquired multimedia data stream to a target device, so that the target device performs a rendering process on the multimedia data stream according to the target request, to obtain a multimedia data stream to be pushed.

Description

[Technical Field]

[0001] [CROSS-REFERENCE TO RELATED APPLICATIONS] This application claims priority to a Chinese patent application filed on December 29, 2023, bearing application number 202311868465.4 and entitled "Multimedia data stream processing method, device, electronic device and storage medium," the entire contents of which are incorporated herein by reference.

[0002] TECHNICAL FIELD The embodiments of the present disclosure relate to the field of data processing technology, and in particular to a method, device, electronic device and storage medium for processing multimedia data streams. [Background technology]

[0003] With the rapid development of computer technology and Internet technology, various service scenarios are emerging one after another. In service scenarios, many related scenario services are often used to enrich the scenario functions and user interaction methods. For example, by inputting a related interaction request on a live streaming page, the live streaming page can render an effect corresponding to the interaction request. Summary of the Invention

[0004] The present disclosure provides a multimedia data stream processing method, device, electronic device, and storage medium to achieve the effect of effectively rendering screen content without requiring high performance from the terminal device, thereby improving the versatility and efficiency of video processing.

[0005] In a first aspect, an embodiment of the present disclosure provides a multimedia data stream processing method, the method comprising: obtaining a multimedia data stream including an audio stream and / or a video stream; When a target request is received, the acquired multimedia data stream is sent to the target device, and the target device performs a rendering process on the multimedia data stream based on the target request to obtain a multimedia data stream to be pushed.

[0006] In a second aspect, an embodiment of the present disclosure further provides a multimedia data stream processing device, the device comprising: a data stream acquisition module for acquiring a multimedia data stream including an audio stream and / or a video stream; and a data stream sending module for sending the acquired multimedia data stream to a target device when a target request is received, so that the target device performs rendering processing on the multimedia data stream based on the target request, thereby obtaining a multimedia data stream to be pushed.

[0007] In a third aspect, an embodiment of the present disclosure further provides an electronic device, the electronic device comprising: one or more processors; a storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, they cause the one or more processors to implement the multimedia data stream processing method according to any one of the embodiments of the present disclosure.

[0008] In a fourth aspect, an embodiment of the present disclosure further provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the multimedia data stream processing method described in any one of the embodiments of the present disclosure. [Brief explanation of the drawings]

[0009] These and other features, advantages, and aspects of each embodiment of the present disclosure will become more apparent with reference to the following specific embodiments in conjunction with the drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are illustrative and that elements and components are not necessarily drawn to scale. [Figure 1] 1 is a flowchart of a method for processing a multimedia data stream according to an embodiment of the present disclosure. [Figure 2] 10 is a flowchart of another multimedia data stream processing method according to an embodiment of the present disclosure. [Figure 3] 1 is a flowchart of a method for processing a multimedia data stream according to an embodiment of the present disclosure. [Figure 4] 1 is a structural schematic diagram of a multimedia data stream processing device according to an embodiment of the present disclosure; [Figure 5] 1 is a structural schematic diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, the embodiments of the present disclosure will be described in more detail with reference to the drawings. Although the drawings show several embodiments of the present disclosure, it should be understood that the present disclosure can be realized in various forms and should not be construed as being limited to the embodiments described herein, but rather that these embodiments are provided for a clearer and more complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are merely illustrative and are not intended to limit the scope of protection of the present disclosure.

[0011] It should be understood that the steps recited in the method embodiments of the present disclosure may be performed in different orders and / or in parallel, and that method embodiments may include additional steps and / or omit performing steps as shown, and the scope of the present disclosure is not limited in this respect.

[0012] As used herein, the term "comprises" and variations thereof are open-ended inclusions of "including, but not limited to." The term "based on" means "based at least in part on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one other embodiment," and the term "some embodiments" means "at least some embodiments." Relevant definitions of other terms are provided below.

[0013] It should be noted that the concepts of "first," "second," etc. mentioned in this disclosure are merely intended to distinguish between different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their mutual dependencies.

[0014] It should be noted that the modifications "one" and "multiple" referred to in this disclosure are intended to be exemplary rather than limiting, and that those skilled in the art would understand that "one or more" should be understood unless the context clearly indicates otherwise.

[0015] The names of messages or information exchanged between devices in the embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0016] As can be understood, before using the technical solutions disclosed in each embodiment of the present disclosure, the type, scope of use, and usage scenarios of the personal information related to the present disclosure should be properly informed to the user in accordance with relevant laws and regulations, and permission should be obtained from the user.

[0017] For example, in response to receiving a user's voluntary request, prompt information is sent to the user to clearly inform the user that the requested operation requires the acquisition and use of the user's personal information, so that the user can choose whether to provide personal information to software or hardware, such as an electronic device, an application program, a server, or a storage medium, that performs the operation of the technical solution of the present disclosure based on the prompt information.

[0018] As an optional but non-limiting implementation, the method of sending prompt information to the user in response to receiving a voluntary request from the user may be, for example, a pop-up window, or the prompt information may be displayed in text form in the pop-up window, which may further include a selection control for allowing the user to select whether to "agree" or "disagree" to providing personal information to the electronic device.

[0019] As can be appreciated, the above notification and user permission acquisition process is merely exemplary and does not limit the implementation of the present disclosure, and other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0020] As can be understood, the data related to the technical solution (including but not limited to the data itself, the acquisition or use of the data) should comply with the requirements of applicable laws, regulations and related provisions.

[0021] Generally, the rendering of a live streaming page is mainly dependent on the processing performed by the client to which the live streaming page belongs, which results in high requirements for the performance of the terminal device, high video processing costs, and poor versatility.

[0022] Before introducing the technical solution, an application scenario may be described as an example. The technical solution can be used in any scenario requiring real-time rendering of a multimedia data stream, such as a live streaming scenario, a video real-time processing scenario, or an audio real-time processing scenario. To more clearly introduce the technical solution, the implementation process of the technical solution can be described using a live streaming scenario as an example. For other data stream real-time rendering scenarios, the multimedia data stream rendering process in the live streaming scenario can be referenced. For example, in a live streaming process, a live streaming client can be used to obtain and process a live stream user's multimedia data stream. However, when a live stream user or a user watching a live stream triggers some interaction controls on the live streaming page, if the processing content corresponding to the interaction controls is complex and time-consuming, the live streaming client still needs to process it. This places high demands on the performance of the live streaming client, i.e., it is highly dependent on the hardware of the live streaming client. Furthermore, if the performance of the live streaming client is not high, problems such as stuttering, delays, and content lag on the live streaming page may occur, leading to undesirable live streaming screen effects.

[0023] At this time, based on the technical proposal of the embodiment of the present disclosure, a live streaming screen can be obtained. Furthermore, if it is detected that the interaction request related to the live streaming screen is a target request, the already obtained live streaming screen can be sent to a target device with good performance that belongs to the same local area network as the live streaming client, and the target device can perform rendering processing on the already obtained live streaming screen based on the target request.

[0024] FIG. 1 is a flowchart of a multimedia data stream processing method according to an embodiment of the present disclosure, which is applicable to any scene that requires real-time rendering of a data stream, and the method may be executed by a multimedia data stream processing device, which may be implemented in the form of software and / or hardware, and optionally implemented by an electronic device, which may be a mobile terminal, a PC, a server, etc.

[0025] As shown in FIG. 1, the method of this embodiment may specifically include:

[0026] S110, obtaining a multimedia data stream;

[0027] The technical solution according to the embodiment of the present disclosure may include at least two devices. Here, one of the two devices may be a device for acquiring a multimedia data stream, and the other may be a device for processing the already acquired multimedia data stream. For improved versatility, optionally, the data stream acquiring device may be a terminal device. The device for processing the multimedia data stream may be a device with high processing performance, such as a terminal device or a server, belonging to the same local area network.

[0028] Here, the multimedia data stream may be a data stream consisting of at least one media type (e.g., image, text, audio, video, etc.). In this embodiment, the multimedia data stream may include an audio stream and / or a video stream. An audio stream may be understood as a data stream consisting of audio. A video stream may be understood as a data stream consisting of video. Note that in a live streaming scenario, the multimedia data stream may be a live streaming screen and audio information simultaneously displayed on a live streaming page, and in an audio real-time processing scenario, the multimedia data stream may be audio information.

[0029] S120, if a target request is received, send the acquired multimedia data stream to the target device, and the target device performs rendering processing on the multimedia data stream according to the target request to obtain a multimedia data stream to be pushed.

[0030] In this embodiment, when a live streaming page corresponding to a multimedia data stream is displayed, a live streamer user or a viewer user can trigger a corresponding operation on the live streaming page. Furthermore, a request can be generated based on the corresponding operation, and the generated request can be a target request. That is, target requests acting on the multimedia data stream display page can be continuously or periodically detected. Furthermore, when a target request is received, an effect corresponding to the target request can be rendered on the display page. Generally, the screen content that needs to be rendered in response to the target request takes time, so the multimedia data stream can be sent to the target device and rendered by another device with higher performance.

[0031] Here, the target request may be understood as an interaction request related to the multimedia data stream. Generally, in a multimedia data stream display process, it may include a side to which the multimedia data stream belongs and a side to watch the multimedia data stream, both of which can input requests to a page displaying the multimedia data stream, and at least some of the requests input by at least one of the two sides to the multimedia data stream display page can be determined as target requests. Optionally, the target request includes a request for a first user and / or a second user to act on the live broadcast page. The target request may include multiple ones, and optionally may include at least one of a rendering effect request, a resource transition request, and a live interaction request.

[0032] Here, the first user may be any user associated with the multimedia data stream display process. Optionally, the first user may be a live streamer of a live room. The second user may be any user associated with the multimedia data stream display process, for example, the second user may be a user who displays a live streaming page of a live room corresponding to the first user on the terminal display interface to which the second user belongs, for example, a viewer who watches the live streaming of the live streamer. The live streaming page is a page that displays a multimedia data stream. The live streaming page may be understood as a page that can be displayed on a related terminal display interface in the live streaming process, and the page may include a live streaming data stream corresponding to the first user or the second user.

[0033] Here, the rendering effect request may be understood as a request to perform effect processing on content on the live streaming page or a request to display an effect on the live streaming page. For example, the rendering effect request may be a request corresponding to triggering a hair accessory effect control.

[0034] Here, the resource transfer request may be understood as a request for related interaction services and resources to be transferred, optionally corresponding to gift sending. The resource transfer request may be a request for transferring resources from a resource sender to a resource receiver, and the resources may be interaction gifts, virtual resources, etc. In practical application, when a trigger operation on a control related to resource transfer is detected, a preset animation or a preset image corresponding to the related control can be rendered on the live streaming page in response to the trigger operation.

[0035] Here, the live interaction request may be understood as a request generated when a live streamer and a viewer interact with each other in a live streaming process, or a request generated when a live streamer and a viewer interact with each other, or a request generated when a viewer interacts with each other. For example, the live interaction request may include posting a comment.

[0036] The request target corresponding to the rendering effect request, resource transition request, and live interaction request may be at least one of the first user and the second user, that is, either the first user or the second user can send the rendering effect request, resource transition request, and live interaction request, and when the corresponding target request is received, the corresponding information is rendered on the live streaming page. The advantage of this setting is that it improves the performance of the application software related to the interaction request rendering capability, expands the application range of the renderable interaction requests, and improves the user's usage experience and interaction experience.

[0037] In this embodiment, when a target request is received, the acquired multimedia data stream is sent to the target device, and the target device performs rendering processing on the multimedia data stream according to the target request to obtain the multimedia data stream to be pushed.

[0038] Here, the target device may be understood as an electronic device related to a client to which a multimedia data stream belongs and having higher performance than the client. Optionally, the target device is a device that satisfies a preset processing condition. The preset processing condition may be that the device performance is higher than that of the client to which the associated multimedia data stream belongs. In this embodiment, the driver side acquiring the multimedia data stream and the target device correspond to the same local area network, i.e., the distance between the client and the target device may be within a coverage range corresponding to the local area network. The target device may be any electronic device having higher performance than the client to which the multimedia data stream belongs. Optionally, the target device may be a server, a PC, a cloud, etc. The multimedia data stream to be pushed is obtained after performing a rendering process on the multimedia data stream and may be understood as the data stream to be pushed. It should be noted that when the method is used on the live stream side, the multimedia data stream to be pushed is a data stream to be pushed to the live stream side and / or the viewer side, that is, when a multimedia data stream to be pushed is obtained, the multimedia data stream to be pushed can be pushed to the live stream side and / or the viewer side based on the corresponding stream push method.

[0039]

[0005] In addition, in related technologies, when a target request is received, the client to which the multimedia data stream belongs generally performs rendering processing on the multimedia data stream that has already been acquired based on the target request, and displays the rendered multimedia data stream on the corresponding client's live streaming page. However, generally, most of the client devices to which the multimedia data stream belongs are mobile terminals, which have certain limitations, causing problems such as slow rendering, stuttering, delays, and lags on the screen. Based on this, the multimedia data stream that requires rendering processing can be sent to a high-performance target device, and the target device can perform rendering processing on the multimedia data stream based on the target request. This achieves the effect of being able to perform video rendering on various types of terminal devices, improves the versatility of the video rendering process, and improves the processing efficiency and rendering effect of the multimedia data stream.

[0040] In practical application, during the process of displaying a multimedia data stream on a live streaming page, multiple interaction requests related to the live streaming page may occur, and among these interaction requests, some may match the target request and some may not. Furthermore, when an interaction request related to the live streaming page is received, the interaction request can be detected to determine whether it matches the target request. Furthermore, when it is determined that the target request is received, the previously acquired multimedia data stream can be sent to the target device, so that the target device can perform rendering processing on the received multimedia data stream according to the target request.

[0041] In this embodiment, sending the acquired multimedia data stream to the target device and having the target device perform rendering processing on the multimedia data stream based on the target request includes sending the acquired multimedia data stream and the target request to the target device, having the target device call the corresponding rendering data packet based on the target request, and performing rendering processing on the multimedia data stream based on the rendering data packet.

[0042] Here, the rendering data packet may include data related to the rendering process. In practical application, after determining multiple target requests, rendering process data corresponding to each target request can be set and packaged into a data packet, so as to obtain a rendering data packet corresponding to each target request. Furthermore, after determining each target request, an association relationship between the target request and the rendering data packet corresponding to the target request can be established, and the rendering data packet can be stored in the target device. Furthermore, after the target device receives a target request sent by a client to which the multimedia data stream belongs, it can retrieve the corresponding rendering data packet based on the association relationship previously established with the target request.

[0043] In practical application, when a target request is received, the previously acquired multimedia data stream and the received target request can be sent to the target device. Furthermore, when the target device receives the multimedia data stream and the target request, it can call up a rendering data packet corresponding to the target request based on the received target request and the pre-established data packet association relationship. Then, it can perform rendering processing on the received multimedia data stream based on the called rendering data packet, thereby obtaining the multimedia data stream to be pushed. Furthermore, it can process the multimedia data stream to be pushed according to actual demand. The advantages of this configuration are that it improves the data processing efficiency and processing quality of interaction data streams with high performance demands, guarantees the real-time rendering of multimedia data streams, and improves the rendering display effect of multimedia video streams.

[0044] For example, the rendering process can be described by taking the example where the target request is a rendering effect request and the already-obtained multimedia data stream is a live streaming screen. A hair accessory effect rendering request corresponding to the live streaming screen and the triggered hair accessory effect can be sent to the target device. After the target device receives the live streaming screen and the hair accessory effect rendering request, it can retrieve a rendering data packet corresponding to the hair accessory effect rendering request and perform rendering processing on the live streaming screen based on the retrieved rendering data packet, thereby obtaining the multimedia data stream to be pushed, i.e., the live streaming screen with the hair accessory effect rendered.

[0045] In the technical solution of the embodiments of the present disclosure, a multimedia data stream is acquired, and when a target request is received, the acquired multimedia data stream is sent to a target device, and the target device performs rendering processing on the multimedia data stream based on the target request, thereby obtaining the multimedia data stream to be pushed. This solves the problem that, in related technologies, when rendering on a live streaming page, high requirements are placed on the performance of the terminal device, high video processing costs are incurred, and versatility is poor. Instead, the technical solution achieves the effect of effectively rendering screen content under conditions that do not require the performance of the terminal device, thereby improving the versatility and efficiency of video processing.

[0046] 2 is a flowchart of another multimedia data stream processing method according to an embodiment of the present disclosure. Based on the above embodiment, the technical solution of this embodiment can obtain the multimedia data stream to be pushed, and then push the multimedia data stream to be pushed according to a stream push method corresponding to the target device. Here, technical features that are the same or similar to those of the above embodiment will not be described again here.

[0047] As shown in FIG. 2, the method of this embodiment may specifically include:

[0048] S210, obtaining a multimedia data stream;

[0049] S220, if a target request is received, send the acquired multimedia data stream to the target device, and the target device performs rendering processing on the multimedia data stream according to the target request to obtain a multimedia data stream to be pushed.

[0050] S230, pushing the multimedia data stream to be pushed to the target device according to a stream push method corresponding to the target device.

[0051] In practical applications, after obtaining the multimedia data stream to be pushed, stream push can be performed on the multimedia data stream to be pushed so that the multimedia data stream to be pushed can be displayed on the corresponding client.

[0052] In this embodiment, the stream push method may be used to indicate a process of pushing a multimedia data stream to a corresponding service side, i.e., a method of performing stream push. As will be understood by those skilled in the art, stream push refers to a process of transmitting packaged content to a server. Stream push is a real-time streaming media transmission technology that enables users to perform online audio-video transmission, allowing other users to view designated video and audio streams over a network. The stream push method may include multiple methods, optionally including a direct stream push method and an indirect stream push method. Here, the direct stream push method may be understood as directly pushing a multimedia data stream to a corresponding service side. The indirect stream push method may be understood as first transmitting a multimedia data stream to a client to which the multimedia data stream belongs, and then pushing the multimedia data stream to a corresponding service side based on the client.

[0053] In addition, since stream push places high demands on the network and the stream push effect may be poor if the network is unstable, the stream push mode setting may be related to the local area network to which the target device belongs. If the network environment of the local area network is stable, the stream push operation can be performed directly, that is, the stream push mode can be set to the direct stream push mode. However, if the network environment of the local area network is poor or the target device is not connected to the network, the stream push effect will be poor if the stream push operation is performed directly. Therefore, the stream push mode can be set to the indirect stream push mode.

[0054] Furthermore, the setting of the stream push method may be related to actual needs, that is, the user can set a corresponding stream push method by inputting a stream push method setting operation, and the target device can push the corresponding multimedia data stream according to the stream push method set by the user.

[0055] In practical application, after obtaining the multimedia data stream to be pushed, a stream push method corresponding to the target device can be determined, and then a stream push process can be performed on the multimedia data stream to be pushed according to the stream push method.

[0056] It should be noted that the stream push method may include at least two methods, and the processes for pushing the multimedia data streams to be pushed in these two stream push methods will be described below respectively.

[0057] Optionally, the stream push method includes a direct stream push method, and pushing the multimedia data stream to be pushed based on the stream push method corresponding to the target device includes sending the multimedia data stream to be pushed to the target service side or the stream push service side based on the target device, and the target service side or the stream push service side pushing the multimedia data stream to be pushed to the live stream side and / or the viewer side.

[0058] Here, the target service side may be understood as a program that provides a service to a client related to a multimedia data stream. A device that runs the target service side may be called a target server. The stream push service side may be understood as a service side that deploys a Content Delivery Network (CDN) interface. The live stream side may be a client of a user to which the multimedia data stream belongs. The viewer side may be a client of a user viewing the multimedia data stream.

[0059] In practical application, when the stream push method corresponding to the target device is a direct stream push method, it can be indicated that the multimedia data stream to be pushed can be directly pushed to the corresponding service side based on the target device. Specifically, after obtaining the multimedia data stream to be pushed, the multimedia data stream to be pushed can be sent to the target service side or the stream push service side based on the target device. Furthermore, the multimedia data stream to be pushed can be pushed to the driver side and / or viewer side based on the target service side or the stream push service side, and the multimedia data stream to be pushed can be displayed based on a display interface corresponding to each client. Furthermore, when the stream push method is a direct stream push method, the target service side or the stream push service side can push the multimedia data stream to the client of the user to which the multimedia data stream belongs, i.e., the driver side, so that the client transmitting the multimedia data stream to the target device can also receive the multimedia data stream to be pushed.

[0060] For example, continuing to refer to the above example, after receiving the live streaming screen with the hair accessory effect rendered, the live streaming screen can be sent to the target service side or the stream push service side based on the target device, and further, the live streaming screen can be pushed to the live streamer side and / or the viewer side based on the target service side or the stream push service side, so that the live streaming page with the hair accessory effect rendered can be displayed on the display interface of the client to which the live streaming viewing user and the live streamer belong.

[0061] In addition, the advantage of pushing the multimedia data stream to be pushed based on the direct stream push method is that it achieves the effect of directly pushing the data stream to be pushed based on the data stream rendering device to the service side, thereby improving the data rendering efficiency and rendering effect, and also improving the stream push efficiency and the display effect of the data stream by the client.

[0062] Optionally, the stream push method includes an indirect stream push method, and pushing the multimedia data stream to be pushed based on the stream push method set in the target device includes receiving and displaying the multimedia data stream to be pushed sent by the target device, sending the multimedia data stream to be pushed to the target service side or the stream push service side, and the target service side or the stream push service side pushing the multimedia data stream to be pushed to the viewer side.

[0063] In practical applications, if the stream push method corresponding to the target device is an indirect stream push method, the multimedia data stream to be pushed cannot be directly pushed to the corresponding service side. Instead, the multimedia data stream to be pushed must first be sent to a device side capable of stream push, so that the device side can push the received multimedia data stream to the corresponding service side. Specifically, after obtaining the multimedia data stream to be pushed, the multimedia data stream to be pushed can be fed back to the client to which the multimedia data stream belongs, i.e., the client that sent the multimedia data stream that has already been obtained. Furthermore, the multimedia data stream to be pushed sent by the target device can be displayed based on the client, and the received multimedia data stream can be sent to the target service side or the stream push service side based on the client. Furthermore, the multimedia data stream to be pushed can be pushed to the viewer side based on the target service side or the stream push service side, and the received multimedia data stream can be displayed based on the display interface corresponding to each viewer side.

[0064] For example, continuing to refer to the above example, after obtaining the live streaming screen with the hair accessory effect rendered, the live streaming screen can be fed back to the live streaming client based on the target device, and the live streaming screen with the hair accessory effect rendered can be displayed based on the display interface of the live streaming client. Furthermore, the live streaming screen can be sent to the target service side or the stream push service side based on the live streaming client, and the live streaming screen can be pushed to the client of the live streaming viewing user based on the target service side or the stream push service side, so that the live streaming page with the hair accessory effect rendered can be displayed on the display interface of the client to which the live streaming viewing user belongs.

[0065] The advantage of pushing a multimedia data stream based on the indirect stream push method is that it achieves the effect of performing stream push on a data stream based on the client to which the data stream belongs, thereby improving the display effect of the data stream on the data stream viewing side.

[0066] In the technical solution of the embodiments of the present disclosure, a multimedia data stream is obtained, and then, when a target request is received, the obtained multimedia data stream is sent to the target device, and the target device performs rendering processing on the multimedia data stream according to the target request, thereby obtaining the multimedia data stream to be pushed. Furthermore, the multimedia data stream to be pushed is pushed according to a stream push method corresponding to the target device, thereby realizing effective rendering of screen content without requiring the performance of the terminal device, and achieving the effect of effectively stream pushing the rendered data stream and displaying the rendered data stream, thereby improving the stream push efficiency and display effect of the data stream.

[0067] 3 is a flowchart of a multimedia data stream processing method according to an embodiment of the present disclosure. This embodiment of the present disclosure is an optional embodiment of each of the above-described embodiments. The multimedia data stream processing method can be described by taking an example in which the multimedia data stream is a live broadcast screen and the received target requests are a hair accessory effect rendering request and an interaction gift receiving request. As shown in FIG. 3, the execution process of the method according to the embodiment of the present disclosure may be as follows:

[0068] In the process of displaying a live streaming screen on a client (e.g., a mobile phone) to which a user (a live streamer and / or a viewer) belongs, the live streaming screen can be acquired. Furthermore, when a hair accessory effect rendering request is received (hair accessory effect update), the already acquired live streaming screen and the triggered hair accessory effect can be sent to a high-performance target device (e.g., a PC or the cloud). Furthermore, time-consuming operations such as body part identification can be performed on the live streaming screen based on the target device, and the received hair accessory effect can be rendered on the hair included in the live streaming screen. Furthermore, when an interaction gift reception request is received again (an interaction gift is received), the already acquired live streaming screen and the triggered interaction gift can be sent to the target device. Furthermore, a preset animation corresponding to the received interaction gift can be rendered on the live streaming screen, thereby obtaining a live streaming screen rendered with the preset animation of the hair accessory effect and the interaction gift. Furthermore, a preset stream push method can be determined. If the stream push method is a direct stream push method, the live streaming screen to be pushed can be sent to the target service side or CDN side, and the live streaming screen to be pushed can be pushed to the live streamer side and the viewer side based on the target service side or CDN side. If the stream push method is an indirect stream push method, the live streaming screen to be pushed can be fed back to the live streamer side, and the live streaming screen can be displayed based on the client. Furthermore, the live streaming screen to be pushed can be sent to the target service side or CDN side based on the client, and the live streaming screen to be pushed can be pushed to the viewer side based on the target service side or CDN side.

[0069] In the technical solution of the embodiments of the present disclosure, a multimedia data stream is acquired, and when a target request is received, the acquired multimedia data stream is sent to a target device, and the target device performs rendering processing on the multimedia data stream based on the target request, thereby obtaining the multimedia data stream to be pushed. This solves the problem that, in related technologies, when rendering on a live streaming page, high requirements are placed on the performance of the terminal device, high video processing costs are incurred, and versatility is poor. Instead, the technical solution achieves the effect of effectively rendering screen content under conditions that do not require the performance of the terminal device, thereby improving the versatility and efficiency of video processing.

[0070] FIG. 4 is a structural schematic diagram of a multimedia data stream processing device according to an embodiment of the present disclosure. As shown in FIG. 4, the device includes: a data stream obtaining module 310 and a data stream transmitting module 320 .

[0071] Here, the data stream acquisition module 310 is used to acquire a multimedia data stream including an audio stream and / or a video stream, and the data stream transmission module 320 is used to transmit the acquired multimedia data stream to a target device when a target request is received, so that the target device performs rendering processing on the multimedia data stream based on the target request, and obtains the multimedia data stream to be pushed.

[0072] In each of the above optional technical solutions, optionally, the device is located on the live stream side, and the multimedia data stream to be pushed is a data stream to be pushed to the live stream side and / or the viewer side.

[0073] In each of the above optional technical solutions, optionally, the target request includes at least one of a rendering effect request, a resource migration request, and a live interaction request.

[0074] In each of the above optional technical solutions, optionally, the data stream sending module 320 is specifically used to send the acquired multimedia data stream and the target request to the target device, so that the target device can call up a corresponding rendering data packet based on the target request and perform rendering processing on the multimedia data stream based on the rendering data packet.

[0075] In each of the above optional technical solutions, optionally, the device includes a data stream push module.

[0076] The data stream push module is used to obtain the multimedia data stream to be pushed, and then push the multimedia data stream to be pushed according to a stream push method corresponding to the target device.

[0077] In each of the above optional technical solutions, optionally, the stream push method includes a direct stream push method, and the data stream push module includes a direct stream push unit.

[0078] The direct stream push unit is used to send the multimedia data stream to be pushed to a target service side or a stream push service side based on the target device, and the target service side or the stream push service side pushes the multimedia data stream to be pushed to a live stream side and / or a viewer side.

[0079] In each of the above optional technical solutions, optionally, the stream push method includes an indirect stream push method, and the data stream push module includes an indirect stream push unit.

[0080] The indirect stream push unit is used to receive and display the multimedia data stream to be pushed sent by the target device, and to send the multimedia data stream to be pushed to the target service side or the stream push service side, so that the target service side or the stream push service side pushes the multimedia data stream to be pushed to the viewer side.

[0081] In the above-mentioned optional technical solutions, optionally, the driver side that acquires the multimedia data stream and the target device correspond to the same local area network.

[0082] In each of the above optional technical solutions, optionally, the target device is a device that satisfies a preset processing condition.

[0083] In the technical solution of the embodiments of the present disclosure, a multimedia data stream is acquired, and when a target request is received, the acquired multimedia data stream is sent to a target device, and the target device performs rendering processing on the multimedia data stream based on the target request, thereby obtaining the multimedia data stream to be pushed. This solves the problem that, in related technologies, when rendering on a live streaming page, high requirements are placed on the performance of the terminal device, high video processing costs are incurred, and versatility is poor. Instead, the technical solution achieves the effect of effectively rendering screen content under conditions that do not require the performance of the terminal device, thereby improving the versatility and efficiency of video processing.

[0084] The multimedia data stream processing device according to the embodiments of the present disclosure can execute the multimedia data stream processing method according to any embodiment of the present disclosure, and has corresponding functional modules and beneficial effects for executing the method.

[0085] It should be noted that the units and modules included in the above device are merely divided based on functional logic, and are not limited to the above divisions as long as they can achieve the corresponding functions. Furthermore, the specific names of the functional units are merely for the convenience of mutual division, and are not intended to limit the scope of protection of the embodiments of the present disclosure.

[0086] FIG. 5 is a structural schematic diagram of an electronic device according to an embodiment of the present disclosure. Referring now to FIG. 5, which shows a structural schematic diagram of an electronic device (e.g., a terminal device or server in FIG. 5) 500 suitable for implementing an embodiment of the present disclosure. The terminal device in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), etc., and fixed terminals such as digital TVs and desktop computers, etc. The electronic device shown in FIG. 5 is merely an example and does not impose any limitations on the functionality and scope of use of the embodiment of the present disclosure.

[0087] 5, electronic device 500 may include a processing unit (e.g., a central processing unit, a graphics processor, etc.) 501, which may perform various appropriate operations and processes based on programs stored in read-only memory (ROM) 502 or programs loaded from storage device 508 into random access memory (RAM) 503. RAM 503 further stores various programs and data necessary for the operation of electronic device 500. Processing unit 501, ROM 502, and RAM 503 are connected to each other via bus 504. Edit / output (I / O) interface 505 is also connected to bus 504.

[0088] Typically, devices such as input devices 506 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 507 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 508 including, for example, a magnetic tape, hard disk, etc.; and communication devices 509 can be connected to the I / O interface 505. The communication devices 509 can allow the electronic device 500 to exchange data with other devices through wireless or wired communication. While FIG. 5 illustrates the electronic device 500 with various devices, it should be understood that it is not required to implement or include all of the devices shown. More or fewer devices may alternatively be implemented or included.

[0089] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product, which includes a computer program stored on a non-transitory computer-readable medium, the computer program including program code for performing the methods shown in the flowcharts. In such embodiments, the computer program may be downloaded and installed from a network via the communication device 509, or may be installed from the storage device 508, or may be installed from the ROM 502. When the computer program is executed by the processing device 501, the above-described functions defined by the methods of the embodiments of the present disclosure are performed.

[0090] The names of messages or information exchanged between devices in the embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0091] The electronic device according to the embodiments of the present disclosure belongs to the same inventive concept as the multimedia data stream processing method according to the above embodiments, and the technical details not described in detail in the embodiments of the present disclosure can be referred to the above embodiments, and the present embodiment has the same beneficial effects as the above embodiments.

[0092] An embodiment of the present disclosure provides a computer storage medium having a computer program stored thereon, which, when executed by a processor, implements the multimedia data stream processing method according to the above embodiment.

[0093] It should be noted that the computer-readable medium of the present disclosure may be a computer-readable signal medium, a computer-readable storage medium, or any combination of the above. The computer-readable storage medium may be, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of the computer-readable storage medium may include, but are not limited to, an electrical connection having one or more conductors, a portable computer magnetic disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact magnetic disk read-only memory (CD-ROM), an optical memory device, a magnetic memory device, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium may include a propagated data signal, either in baseband or as part of a carrier, bearing computer-readable program code. Such propagated data signals may take various forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may be any computer-readable medium other than a computer-readable storage medium, which is capable of transmitting, propagating, or transmitting a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted over any suitable medium, including, but not limited to, electrical wire, optical cable, RF (radio frequency), etc., or any suitable combination of the above.

[0094] In some embodiments, clients and servers may communicate using any network protocol now known or hereafter developed, such as HTTP (HyperText Transfer Protocol), and may interconnect with any form or medium of digital data communication (e.g., a communications network). Examples of communications networks include local area networks ("LANs"), wide area networks ("WANs"), the World Wide Web (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), and any networks now known or hereafter developed.

[0095] The computer-readable medium may be included in the electronic device, or may be a standalone entity not integrated into the electronic device.

[0096] The computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the following: acquire a multimedia data stream including an audio stream and / or a video stream, and, upon receiving a target request, transmit the acquired multimedia data stream to a target device, which performs a rendering process on the multimedia data stream based on the target request, thereby obtaining a multimedia data stream to be pushed.

[0097] Computer program code for carrying out the operations of the present disclosure can be organized in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, and the like, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may run entirely on the user computer, partially on the user computer, as a separate software package, partially on the user computer and partially on a remote computer, or entirely on a remote computer or server. When referring to a remote computer, the remote computer may be connected to the user computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider).

[0098] The flowcharts and block diagrams in the drawings illustrate system architecture, functions, and operations that can be implemented according to the systems, methods, and computer program products of the embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams may represent a module, program segment, or portion of code, which includes one or more executable instructions for implementing the specified logical function. It should also be noted that in some implementations, the functions noted in the blocks may occur in a different order than that noted in the drawings. For example, two blocks shown in succession may actually be executed substantially in parallel, or in some cases, may be executed in the reverse order, as determined by the functionality involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified function or operation, or by a combination of dedicated hardware and computer instructions.

[0099] The units mentioned in the description of the embodiments of the present disclosure may be implemented in a software manner or a hardware manner, and the names of the units may not necessarily limit the units themselves, for example, a first acquiring unit may be described as "a unit for acquiring at least two Internet Protocol addresses."

[0100] The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, typical types of hardware logic components that may be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), etc.

[0101] In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the above. More specific examples of machine-readable storage media include one or more wire-based electrical connections, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0102] The above description merely describes preferred embodiments of the present disclosure and the technical principles employed therein. Those skilled in the art should understand that the scope of the present disclosure is not limited to the technical solution formed by a specific combination of the above technical features, but should encompass other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the concept disclosed above. For example, a technical solution formed by mutually replacing the above features with technical features having similar functions disclosed in the present disclosure (but not limited to these).

[0103] It should be noted that although operations are described using a particular sequence, this should not be understood as requiring that these operations be performed in the particular sequence or order shown. In certain environments, multitasking and parallel processing may be advantageous. Similarly, although the above discussion includes several specific implementation details, these should not be understood as limiting the scope of the present disclosure. Some features that are described in the context of a single embodiment may also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment may also be implemented in multiple embodiments alone or in any suitable subcombination.

[0104] Although the present subject matter has been described employing language specific to structural features and / or logical operations of a method, it should be understood that the subject matter defined in the appended claims is not limited to the specific features or operations described above. Rather, the specific features and operations described above are merely example forms of implementing the claims.

Claims

1. A method for processing a multimedia data stream, comprising: obtaining a multimedia data stream including an audio stream and / or a video stream; A multimedia data stream processing method including: when a target request is received, sending the acquired multimedia data stream to a target device, and the target device performing a rendering process on the multimedia data stream based on the target request, thereby obtaining a multimedia data stream to be pushed.

2. 2. The method of claim 1, wherein the method is used on a live stream side, and the multimedia data stream to be pushed is a data stream to be pushed on a live stream side and / or a viewer side.

3. The target request: The method of claim 1 , comprising at least one of a rendering effect request, a resource transition request, and a live interaction request.

4. transmitting the acquired multimedia data stream to a target device, and causing the target device to perform rendering processing on the multimedia data stream based on the target request; 2. The method of claim 1, further comprising: transmitting the acquired multimedia data stream and the target request to the target device; and having the target device call a corresponding rendering data packet based on the target request; and performing rendering processing on the multimedia data stream based on the rendering data packet.

5. After obtaining the multimedia data stream to be pushed, The method of claim 1 , further comprising: pushing the multimedia data stream to be pushed based on a stream push method corresponding to the target device.

6. The stream push method includes a direct stream push method, and the step of pushing the multimedia data stream to the target device based on the stream push method corresponding to the target device includes: The method of claim 5, further comprising transmitting the multimedia data stream to be pushed to a target service side or a stream push service side based on the target device, and the target service side or the stream push service side pushing the multimedia data stream to be pushed to a live stream side and / or a viewer side.

7. the stream push method includes an indirect stream push method, and the step of pushing the multimedia data stream to be pushed based on the stream push method set in the target device includes:

6. The method of claim 5, comprising receiving and displaying the multimedia data stream to be pushed sent by the target device, sending the multimedia data stream to be pushed to a target service side or a stream push service side, and the target service side or the stream push service side pushing the multimedia data stream to be pushed to a viewer side.

8. The method of claim 1 , wherein the driver side and the target device acquiring the multimedia data stream correspond to the same local area network.

9. The method of claim 1 , wherein the target device is a device that satisfies a preset processing condition.

10. A multimedia data stream processing device, comprising: a data stream acquisition module for acquiring a multimedia data stream including an audio stream and / or a video stream; a data stream transmission module for transmitting the acquired multimedia data stream to a target device when a target request is received, so that the target device performs rendering processing on the multimedia data stream based on the target request, thereby obtaining a multimedia data stream to be pushed.

11. An electronic device, one or more processors; a storage device for storing one or more programs; An electronic device that, when the one or more programs are executed by the one or more processors, causes the one or more processors to implement the multimedia data stream processing method according to any one of claims 1 to 9.

12. A storage medium containing computer-executable instructions, the computer-executable instructions being used to perform the multimedia data stream processing method of any one of claims 1 to 9 when executed by a computer processor.

Images