Live broadcast processing method and device, and storage medium

Abstract

Embodiments of the present disclosure provide a live processing method and device and a storage medium. A host terminal device adds supplementary enhancement information in live stream data, wherein the supplementary enhancement information includes first layout information of a live room interface, and adds preset coding information in a live room interface image of the live stream data. The preset coding information can include second layout information of the live room interface. In this way, the audience terminal device has two ways to obtain the layout information of the live room interface. When the supplementary enhancement information is lost, the second layout information of the live room interface is obtained according to the preset coding information in the live room interface image, so that the audience terminal device can stably obtain the layout information of the live room interface and timely perceive the change of the layout of the live room interface, and corresponding layout control is performed, thereby ensuring user experience.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the technical field of computer and network communication, and particularly relate to a live broadcast processing method and device and storage medium. BACKGROUND

[0002] In a network live broadcast scenario, the layout of a live broadcast room interface can change. For example, in a live broadcast intercom scenario, a host and an intercom guest in the live broadcast room can have real-time audio and video calls, and the content of the audio and video calls can be pushed to a viewer terminal device after being combined by multiple streams. However, as the number of intercom hosts and guests changes, the layout of the live broadcast room interface also changes, and the picture content of the viewer terminal device also changes.

[0003] Since the viewer terminal device needs to load some interface components according to the layout of the live broadcast room interface, it is necessary to perceive the change of the layout of the live broadcast room interface. In the prior art, supplemental enhancement information (SEI) is usually carried in live broadcast stream data to deliver the change of the layout of the live broadcast room interface.

[0004] However, the supplemental enhancement information carried in the live broadcast stream data is easy to be lost, which causes the viewer terminal device to be unable to perceive the change of the layout of the live broadcast room interface, and affects the user experience. SUMMARY

[0005] Embodiments of the present disclosure provide a live broadcast processing method and device and storage medium to ensure that the viewer terminal device can stably obtain the layout information of the live broadcast room interface and timely perceive the change of the layout of the live broadcast room interface.

[0006] In a first aspect, embodiments of the present disclosure provide a live broadcast processing method applied to a viewer terminal device, and the method comprises:

[0007] In the case that no supplemental enhancement information is included in live broadcast stream data, a live broadcast room interface image is obtained, wherein the supplemental enhancement information includes first layout information of the live broadcast room interface;

[0008] Second layout information of the live broadcast room interface is obtained according to preset encoding information included in the live broadcast room interface image;

[0009] The live broadcast room interface is controlled according to the second layout information.

[0010] In a second aspect, embodiments of the present disclosure provide a live broadcast processing method applied to a host terminal device, and the method comprises:

[0011] Supplemental enhancement information is added in live broadcast stream data of a live broadcast room, wherein the supplemental enhancement information includes first layout information of the live broadcast room interface;

[0012] adding preset coding information in each frame of the live room interface image in the live streaming data, wherein the preset coding information comprises second layout information of the live room interface;

[0013] pushing the live streaming data to a server, so that a viewer-side device obtains the live streaming data from the server for playing.

[0014] In a third aspect, the embodiments of the present disclosure provide a live processing device, comprising:

[0015] an image obtaining unit configured to obtain a live room interface image in a case where the live streaming data does not comprise supplementary enhancement information, wherein the supplementary enhancement information comprises first layout information of the live room interface;

[0016] a parsing unit configured to obtain second layout information of the live room interface according to preset coding information comprised in the live room interface image;

[0017] a layout control unit configured to perform layout control on the live room interface according to the second layout information.

[0018] In a fourth aspect, the embodiments of the present disclosure provide a live processing device, comprising:

[0019] a data processing unit configured to add supplementary enhancement information in live streaming data of a live room, wherein the supplementary enhancement information comprises first layout information of the live room interface;

[0020] a synthesizing unit configured to add preset coding information in each frame of the live room interface image in the live streaming data, wherein the preset coding information comprises second layout information of the live room interface;

[0021] a communication unit configured to push the live streaming data to a server, so that a viewer-side device obtains the live streaming data from the server for playing.

[0022] In a fifth aspect, the embodiments of the present disclosure provide an electronic device, comprising at least one processor and a memory;

[0023] the memory stores computer-executed instructions;

[0024] the at least one processor executes the computer-executed instructions stored in the memory, so that the at least one processor performs the live processing method as described in the above first aspect and various possible designs of the first aspect.

[0025] In a sixth aspect, the embodiments of the present disclosure provide a computer readable storage medium, wherein computer execution instructions are stored in the computer readable storage medium, and when a processor executes the computer execution instructions, the live processing method in the first aspect and various possible designs of the first aspect are implemented.

[0026] In a seventh aspect, the embodiments of the present disclosure provide a computer program product, comprising computer execution instructions, and when a processor executes the computer execution instructions, the live processing method in the first aspect and various possible designs of the first aspect are implemented.

[0027] The live processing method, device and storage medium provided by the embodiments of the present disclosure add the supplementary enhancement information in the live stream data by the anchor terminal device, the first layout information of the live room interface is included in the supplementary enhancement information, and the preset coding information is added in the live room interface image of the live stream data. The preset coding information can include the second layout information of the live room interface. In this way, the audience terminal device has two ways to obtain the layout information of the live room interface. When the supplementary enhancement information is lost, the second layout information of the live room interface is obtained according to the preset coding information in the live room interface image, so that the audience terminal device can stably obtain the layout information of the live room interface, timely perceive the change of the live room interface layout, and perform corresponding layout control, thereby ensuring the user experience. BRIEF DESCRIPTION OF DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the present disclosure, and other drawings can also be obtained by those skilled in the art without creative labor.

[0029] Figure 1 An example diagram of the layout of the live room interface in the prior art;

[0030] Figure 2 A live processing method flowchart provided by an embodiment of the present disclosure;

[0031] Figure 3 A live processing method flowchart provided by another embodiment of the present disclosure;

[0032] Figure 4 A live processing method flowchart provided by another embodiment of the present disclosure;

[0033] Figure 5 A structural block diagram of a live processing device provided by an embodiment of the present disclosure;

[0034] Figure 6A structural block diagram of a live processing device provided for another embodiment of the present disclosure is shown in FIG. 1.

[0035] Figure 7 A hardware structure schematic diagram of an electronic device provided for an embodiment of the present disclosure is shown in FIG. 1. DETAILED DESCRIPTION

[0036] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are some but not all of the embodiments of the present disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present disclosure.

[0037] The blind watermark function refers to adding a watermark in an invisible form to a picture, such that the watermark does not affect the visual effect of the picture, and meanwhile ensures the originality of the picture. By extracting the blind watermark, the invisible watermark is extracted to verify the ownership of the picture. One method of adding a blind watermark is a method of adding a digital watermark in a frequency domain, which refers to transforming an image to a frequency domain (wavelet domain) through a certain transformation means (Fourier transform, discrete cosine transform, wavelet transform, etc.), adding a watermark to the image in the frequency domain, and then converting the image to a spatial domain through inverse transformation. The frequency domain blind watermark has stronger concealment and higher attack resistance, and the watermark content can still be parsed after the picture is compressed, cropped, and scaled.

[0038] Supplemental Enhancement Information (SEI) is text data inserted into an audio and video stream to convey additional information, which can be accurately synchronized with the related audio and video content and reaches a client through a transmission link.

[0039] In a network live streaming scenario, the layout of a live streaming room interface can change. For example, in a live streaming and microphone connection scenario, the anchor and microphone guest in the live streaming room can have real-time audio and video calls, and the content of the audio and video calls can be pushed to the audience end device after multi-stream merging. However, as the number of microphone anchors and guests changes, the layout of the live streaming room interface also changes, and the picture content of the audience end device also changes accordingly, as shown in FIG. 1. As the number of microphone anchors and guests changes, different layouts of the live streaming room interface can be presented. Figure 1

[0040] ​Since the audience-side device needs to load some interface components according to the live room interface layout, such as the anchor avatar, name, and live PK score of each live anchor in the live room, the audience-side device needs to perceive the change of the live room interface layout. In the prior art, the change of the live room interface layout is usually transmitted to the audience-side device by carrying supplemental enhancement information SEI in the live stream data. The SEI is aligned with the video frame, and the use of SEI to perceive the change of the video picture is the most timely and accurate.

[0041] As supplemental enhancement information in H.264 and H.265 encoding formats, SEI is easily lost in the transmission link of the stream media, especially in the transcoding process. As a data transmission link, the stability is relatively poor, and there are problems such as complete loss of SEI and SEI truncation causing failure to parse. At present, the live and live broadcast business strongly depends on SEI to transmit information and state, and the stability of the SEI link is poor, which causes the audience-side device to be unable to perceive the change of the live room interface layout, thereby affecting the user experience.

[0042] To solve the above technical problems, the present disclosure provides a live processing method. The anchor-side device adds supplemental enhancement information in the live stream data, and the supplemental enhancement information includes first layout information of a live room interface. Meanwhile, preset encoding information is added in the live room interface image of the live stream data. The preset encoding information can include second layout information of the live room interface. In this way, the audience-side device has two ways to obtain the layout information of the live room interface. The layout information of the live room interface is preferentially obtained from the supplemental enhancement information. When the supplemental enhancement information is lost, the layout information of the live room interface can be obtained through the preset encoding information in the live room interface image. In this way, it is ensured that the audience-side device can stably obtain the layout information of the live room interface, perceive the change of the live room interface layout in a timely manner, and then perform corresponding layout control, thereby ensuring the user experience.

[0043] The live processing method of the present disclosure is applicable to a live scene. The live scene includes an anchor-side device, a server, and an audience-side device. The anchor-side device pushes the live stream data to the server, and the audience-side device pulls and plays the live stream data from the server. The specific process is not described here.

[0044] It should be noted that the user information and data involved in the present application are all information and data authorized by the user or fully authorized by all parties. The collection, use, and processing of related data need to comply with relevant laws, regulations, and standards of relevant countries and regions, and provide corresponding operation portals for users to choose authorization or refusal.

[0045] The live processing method of the present disclosure will be described in detail below with reference to specific embodiments.

[0046] Reference Figure 2 ,Figure 2 A live processing method flowchart is provided for an embodiment of the present disclosure. The method of the embodiment can be applied in a viewer-side device in a live scene, and the live processing method comprises:

[0047] S201, acquiring a live room interface image in a case where supplemental enhancement information is not included in live stream data; wherein the supplemental enhancement information includes first layout information of the live room interface.

[0048] In the embodiment, the live stream data of the live room carries supplemental enhancement information, and the supplemental enhancement information includes first layout information of the live room interface. Since the supplemental enhancement information is synchronized with the live stream data, the first layout information of the live room interface can be the layout information of the live room interface of the current frame of the live stream data. In a live room mic-in scene, the layout information of the live room interface includes position information of video pictures of each mic-in anchor, such as coordinate information (such as coordinates of four corners of the video picture) of the video picture of each mic-in anchor. The viewer-side device can load interface components on the live room interface according to the layout information, such as loading anchor avatars, names, mic-in PK scores and the like on the video pictures of each mic-in anchor according to the coordinate information of the video picture of each mic-in anchor.

[0049] However, the supplemental enhancement information may be lost in the transcoding process of the live stream data. For example, the viewer-side device switches the live definition, and transcodes the live stream data of the blue light into live stream data of high definition or live stream data of low definition, which may cause the loss of the supplemental enhancement information. Or when the viewer-side device performs other compression, cropping, scaling and the like on the live stream data, the supplemental enhancement information may also be lost, so that the viewer-side device cannot perceive the layout change of the live room interface, especially the number of mic-in anchors in the live room changes, the position and size of the video picture of each mic-in anchor in the live room interface change and the like, which causes the layout error of the live room interface of the viewer-side device.

[0050] Therefore, in the embodiment, the viewer-side device can detect whether the live stream data of the live room includes supplemental enhancement information, wherein the supplemental enhancement information includes first layout information of the live room interface. If the live stream data includes supplemental enhancement information, the first layout information of the live room interface can be acquired from the supplemental enhancement information to control the layout of the live room interface according to the first layout information, including loading interface components, updating interface components and the like. If the live stream data does not include supplemental enhancement information, S201 can be performed.

[0051] It should be noted that in the embodiment, the audience-side device can detect in real time whether the live streaming data of the live room includes the supplemental enhancement information; or, considering that the live streaming data loses the supplemental enhancement information due to transcoding or the like, the supplemental enhancement information is not included in the live streaming data all the time, and therefore the audience-side device only needs to detect whether the live streaming data includes the supplemental enhancement information, and does not need to detect again whether the live streaming data includes the supplemental enhancement information in the future; or, the audience-side device can detect whether the live streaming data includes the supplemental enhancement information once after entering the live room, or detect whether the live streaming data includes the supplemental enhancement information once after switching the definition each time.

[0052] In the embodiment, after it is detected that the live streaming data does not include the supplemental enhancement information, the live room interface image is acquired in real time or periodically. Optionally, the live room interface image can be acquired by means of taking a screenshot of the player, and of course can be acquired by other manners, which are not limited herein. In addition, when the live room interface image is acquired periodically, the acquisition period can be set according to actual requirements. For example, the acquisition period can be 1 second. The shorter the acquisition period is, the more sensitive the perception of the layout change of the live room interface is, but more resources of the audience-side device are occupied.

[0053] S202, acquiring second layout information of the live room interface according to preset encoding information included in the live room interface image.

[0054] In the embodiment, the preset encoding information can be pre-encoded into each frame (or every several frames) of the live room interface image, wherein the preset encoding information includes the second layout information of the live room interface, or the second layout information of the live room interface can be acquired according to the preset encoding information. The preset encoding information is invisible and does not affect the user to watch the live broadcast.

[0055] Optionally, the preset encoding information can be pre-encoded into the live room interface image, which can be adding a blind watermark into each frame of the live room interface image, or adding a blind watermark every several frames, the second layout information of the live room interface included in the blind watermark is added into each frame of the live room interface image by the anchor-side device.

[0056] Of course, other known methods of adding invisible information into the image can also be used in the embodiment, and are not limited to the blind watermark method.

[0057] The second layout information can be layout information of the live room interface of the current frame (i.e., the same as the first layout information). Considering that preset encoding information (such as a blind watermark) can carry limited information, the second layout information can also be identification information corresponding to the layout information of the live room interface of the current frame, and a mapping relationship between the identification information and the layout information is maintained at the server. The audience-side device can query the corresponding layout information from the server according to the identification information. Optionally, when the anchor-side device determines that the layout information of the live room interface of the current frame changes, the anchor-side device can update the layout information of the live room interface of the current frame and update the corresponding identification information, and then upload the updated layout information of the live room interface of the current frame and the corresponding identification information to the server for querying by the audience-side device.

[0058] S203, performing layout control on the live room interface according to the second layout information.

[0059] In this embodiment, after the audience-side device obtains the second layout information of the live room interface included in the preset encoding information, the audience-side device can perform UI layout control on the live room interface according to the second layout information.

[0060] Optionally, if the second layout information changes, it indicates that the layout of the live room interface changes, for example, the number of anchors in the live room changes, the positions and sizes of the video pictures of the anchors in the live room in the live room interface change, and the like. The second layout information after the change can be used to change the layout of the live room interface, for example, loading the anchor avatar, name, and PK score of a new anchor, deleting the anchor avatar, name, and PK score of an anchor who exits the live room, changing the positions of the anchor avatar, name, and PK score, and the like.

[0061] If the second layout information is identification information corresponding to the layout information of the live room interface of the current frame, when the second layout information changes, the audience-side device can request the layout information corresponding to the changed identification information from the server, so as to obtain the layout information of the changed live room interface, i.e., the latest layout information of the live room interface, and realize timely perception of the layout change of the live room interface.

[0062] Optionally, if the second layout information does not change, it indicates that the layout of the live room interface does not change, and the audience-side device can control the live room interface to remain unchanged.

[0063] The live processing method provided in the embodiment, the audience terminal device detects whether the live stream data of the live room includes supplementary enhancement information, wherein the supplementary enhancement information includes first layout information of a live room interface; in the case that the live stream data does not include the supplementary enhancement information, the live room interface image is acquired; the second layout information of the live room interface is acquired according to preset coding information included in the live room interface image; and the layout of the live room interface is controlled according to the second layout information. In the embodiment, the second layout information of the live room interface is acquired through the preset coding information included in the live room interface image as a backup link of the supplementary enhancement information. When the supplementary enhancement information is lost, the second layout information of the live room interface can be acquired by analyzing the preset coding information in the live room interface image, so that the audience terminal device can stably acquire the layout information of the live room interface, timely perceive the change of the layout of the live room interface, and perform corresponding layout control, thereby ensuring the user experience.

[0064] On the basis of any of the above embodiments, the preset coding information can be pre-encoded into a preset area of the live room interface image. In particular, if the preset coding information adopts blind watermarking, the blind watermarking is synthesized into the preset area of the live room interface image. The preset area can be a background area of the live room interface image, for example Figure 1 the background area above and / or below the live room interface image (the gray areas above and below), wherein no video of any host is displayed in the background area, but the blind watermarking can be synthesized in the background area.

[0065] Further, the blind watermarking included in the live room interface image is analyzed to acquire the second layout information of the live room interface included in the blind watermarking. Specifically, the method can include the following steps.

[0066] The blind watermarking included in the preset area of the live room interface image is analyzed to acquire the second layout information of the live room interface included in the blind watermarking.

[0067] In the embodiment, since the blind watermarking is synthesized into the preset area of the live room interface image, when the blind watermarking is analyzed, the second layout information included in the blind watermarking can be acquired by analyzing only the preset area of the live room interface image. Optionally, the preset area image can be intercepted from the live room interface image, and the second layout information included in the blind watermarking can be acquired by analyzing the preset area image. Optionally, when the live room interface image is acquired, only the preset area image in the live room interface image can be acquired.

[0068] On the basis of any of the foregoing embodiments, considering that adding a blind watermark in the image frequency domain is more resistant to attacks and has better stability, and the blind watermark content can still be parsed after compression, cropping and scaling, in the present embodiment, the blind watermark can be synthesized into the live room interface image in the frequency domain, therefore, the blind watermark included in the live room interface image is parsed to obtain the second layout information of the live room interface included in the blind watermark, which can specifically include:

[0069] The live room interface image is transformed into the frequency domain, and the blind watermark included in the live room interface image is parsed in the frequency domain to obtain the second layout information of the live room interface included in the blind watermark.

[0070] In the present embodiment, the transformation means adopted for transforming the live room interface image into the frequency domain is not limited, and can be Fourier transform, discrete cosine transform, wavelet transform, etc., and can be consistent with the transformation means adopted when the blind watermark is added.

[0071] Optionally, if the blind watermark is synthesized into the preset region of the live room interface image, only the preset region image of the live room interface image needs to be transformed into the frequency domain, and the blind watermark included in the preset region image is parsed in the frequency domain to obtain the second layout information of the live room interface included in the blind watermark.

[0072] On the basis of any of the foregoing embodiments, the live processing method provided in the present embodiment, as shown in Figure 3 includes the following steps:

[0073] S301, when entering the live room, obtaining the latest layout information of the live room interface from the server, and loading interface components in the live room interface according to the latest layout information of the live room interface to perform initial layout on the live room interface;

[0074] In the present embodiment, when the audience terminal device enters the live room, the latest layout information of the live room interface can be obtained from the server first, and the latest layout information of the live room interface can be included in the live room information (Room information) or be the latest layout information requested from the server separately.

[0075] Loading interface components in the live room interface according to the latest layout information of the live room interface, including but not limited to loading the components such as the anchor avatar, name and mic-PK score of each mic anchor in the live room at appropriate positions, so as to complete the initial layout of the live room interface.

[0076] S302, detecting whether the live stream data of the live room includes supplementary enhancement information, wherein the first layout information of the live room interface is included in the supplementary enhancement information.

[0077] Please refer to the above embodiment, which will not be repeated here, wherein if it is detected that the live streaming data includes the supplementary enhancement information, S303 is performed, otherwise S304 is performed; optionally, since the audience-side device has just entered the live room, a certain time is required for the downloading and decoding of the live streaming data and other processing procedures, therefore, a preset time period (for example, N seconds) can be waited before detecting whether the live streaming data of the live room includes the supplementary enhancement information;

[0078] S303, if it is detected that the live streaming data of the live room includes the supplementary enhancement information, first layout information of the live room interface included in the supplementary enhancement information is acquired;

[0079] Further, according to the first layout information, S306 is performed to control the layout of the live room interface;

[0080] S304, if it is detected that the live streaming data does not include the supplementary enhancement information, live room interface images are acquired in real time or periodically;

[0081] S305, the blind watermark included in the live room interface image is analyzed to acquire second layout information of the live room interface included in the blind watermark;

[0082] Further, according to the second layout information, S306 is performed to control the layout of the live room interface;

[0083] S306, the layout of the live room interface is controlled.

[0084] It should be noted that if other known methods are used to add preset encoding information to the live room interface image, the process of analyzing the blind watermark in the above steps is replaced accordingly.

[0085] The live processing method provided in this embodiment has two ways for the audience-side device to acquire the layout information of the live room interface, and the first layout information of the live room interface is preferentially acquired from the supplementary enhancement information. When the supplementary enhancement information is lost, the second layout information of the live room interface in the blind watermark can be acquired by analyzing the blind watermark in the live room interface image, so that the audience-side device can stably acquire the layout information of the live room interface and timely perceive the change of the layout of the live room interface, and then perform corresponding layout control, thereby ensuring the user experience.

[0086] Reference Figure 4 , Figure 4 A live processing method flow diagram is provided for an embodiment of the present disclosure. The method of this embodiment can be applied to a host-side device in a live scene, and the live processing method includes:

[0087] S401, adding supplementary enhancement information in the live streaming data of the live room, wherein the supplementary enhancement information comprises first layout information of the live room interface.

[0088] In this embodiment, the anchor-side device can obtain the first layout information of the live room interface, add the first layout information to the live streaming data as the supplementary enhancement information, and optionally, since the supplementary enhancement information is synchronized with the live streaming data, the first layout information of the live room interface can be the layout information of the live room interface of the current frame of the live streaming data. Optionally, in the live room microphone connection scene, the layout information of the live room interface comprises position information of video pictures of each microphone connection anchor, for example, coordinate information of the video pictures of each microphone connection anchor (such as coordinates of four corners of the video picture).

[0089] Optionally, the anchor-side device can add the supplementary enhancement information in every frame of the live streaming data, or add the supplementary enhancement information at intervals of several frames, or add the supplementary enhancement information only when the layout of the live room interface changes and the first layout information is updated.

[0090] S402, adding preset encoding information in each frame of the live room interface image of the live streaming data, wherein the preset encoding information comprises second layout information of the live room interface.

[0091] In this embodiment, the anchor-side device can add the second layout information of the live room interface to each frame of the live room interface image of the live streaming data through the preset encoding information.

[0092] Optionally, the anchor-side device can synthesize the second layout information of the live room interface into each frame of the live room interface image of the live streaming data through blind watermarking (or add blind watermarking at intervals of several frames). Of course, other known methods of adding invisible information to an image can also be used, and the present embodiment is not limited in this regard.

[0093] The second layout information can be the layout information of the live room interface of the current frame (i.e. the same as the first layout information); considering that the preset encoding information (such as blind watermarking) can carry limited information, the second layout information can also be identification information corresponding to the layout information of the live room interface of the current frame.

[0094] Optionally, the second layout information of the live room interface can be synthesized into a preset region of each frame of the live room interface image of the live streaming data through blind watermarking in this embodiment. The preset region can be a background region of the live room interface image, for example Figure 1 the background region above and / or below the live room interface image (the gray regions above and below), wherein no video picture of any anchor is displayed in the background region, but blind watermarking can be synthesized in the background region.

[0095] Specifically, a preset region image of the live room interface image is acquired first (if the preset region image does not change, the preset region image does not need to be acquired repeatedly), the second layout information is synthesized into the preset region image by blind watermark (if the second layout information does not change, the process of synthesizing the blind watermark into the preset region image does not need to be executed repeatedly), and then the synthesized preset region image is synthesized into each frame of the live room interface image.

[0096] Optionally, considering that adding the blind watermark in the frequency domain is more resistant to attacks and more stable, in the embodiment, the blind watermark can be synthesized into the live room interface image in the frequency domain. Specifically, each frame of the live room interface image is transformed into the frequency domain, the second layout information is synthesized into each frame of the live room interface image in the frequency domain by blind watermark, and then the synthesized each frame of the live room interface image is converted into the spatial domain. The transformation means for transforming the live room interface image into the frequency domain is not limited, which can be Fourier transform, discrete cosine transform, wavelet transform, etc. The inverse transformation of the above transformation means can be used to convert the synthesized each frame of the live room interface image into the spatial domain.

[0097] On the basis of the above embodiment, if the blind watermark is synthesized into a preset region of the live room interface image, the preset region image of the live room interface image is acquired first (if the preset region image does not change, the preset region image does not need to be acquired repeatedly); the preset region image is transformed into the frequency domain, the second layout information is synthesized into the preset region image in the frequency domain by blind watermark, and then the synthesized preset region image is converted into the spatial domain (if the second layout information does not change, the process of synthesizing the blind watermark into the preset region image does not need to be executed repeatedly); and the synthesized preset region image is synthesized into each frame of the live room interface image in the spatial domain.

[0098] Taking the background region above and / or below the live room interface image as an example, the background region image is acquired first (if the background region does not change, the background region image does not need to be acquired repeatedly); the background region image is transformed into the frequency domain, the second layout information is synthesized into the background region image in the frequency domain by blind watermark, and then the synthesized background region image is converted into the spatial domain (if the second layout information does not change, the process of synthesizing the blind watermark into the background region image does not need to be executed repeatedly); and the synthesized background region image is synthesized into each frame of the live room interface image in the spatial domain.

[0099] It should be noted that S401 and S402 are not limited to the order of execution, and preferably, S401 and S402 are executed simultaneously.

[0100] S403, push the live streaming data to a server, so that a viewer-side device obtains the live streaming data from the server for playing.

[0101] In the embodiment, the anchor-side device performs a live streaming data pushing process after completing the above process, which is not described herein again. The viewer-side device performs a live streaming data pulling process, and after pulling, the live streaming processing method of the viewer-side device in the above embodiment can be executed.

[0102] The live processing method provided in the embodiment includes: adding, by an anchor-side device, supplementary enhancement information in live streaming data in a live room, wherein the supplementary enhancement information includes first layout information of a live room interface; adding preset coding information in each live room interface image of the live streaming data, wherein the preset coding information includes second layout information of the live room interface; and pushing the live streaming data to a server, so that a viewer-side device obtains the live streaming data from the server for playing. The anchor-side device provides two ways for the viewer-side device to obtain the layout information of the live room interface, so that the viewer-side device can stably obtain the layout information of the live room interface and timely perceive the change of the layout of the live room interface, and then performs corresponding layout control, thereby ensuring user experience.

[0103] On the basis of any of the above embodiments, when the layout of the live room interface of the current frame changes, for example, the number of anchors in the live room changes, or the position and size of the video of each anchor in the live room interface change, the layout information of the live room interface of the current frame changes, and the anchor-side device needs to update the first layout information and the second layout information, so as to ensure that the supplementary enhancement information and the preset coding information (such as blind watermarking) in the live streaming data also change synchronously, thereby enabling the viewer-side device to perceive the change of the layout of the live room interface.

[0104] If the second layout information is identification information corresponding to the layout information of the live room interface of the current frame, when the layout information of the live room interface of the current frame changes, the second layout information is updated, that is, the identification information is updated, and then the updated second layout information and the changed layout information of the live room interface of the current frame are uploaded to the server, and the server maintains the mapping relationship between the identification information and the layout information, so that the viewer-side device can query the corresponding layout information from the server according to the identification information after obtaining the identification information from the preset coding information (such as blind watermarking).

[0105] The live processing method of the viewer-side device corresponding to the above embodiment includes: Figure 5 A structural block diagram of a live processing device provided in the embodiment is shown. For ease of illustration, only parts related to the embodiment of the disclosure are shown. For details, refer to the description of the live processing method of the viewer-side device. Figure 5The live processing device 500 comprises an image acquisition unit 501, an analysis unit 502, and a layout control unit 503.

[0106] The image acquisition unit 501 is configured to acquire a live room interface image in a case where the live stream data does not comprise supplementary enhancement information.

[0107] The analysis unit 502 is configured to acquire second layout information of the live room interface according to preset encoding information comprised in the live room interface image.

[0108] The layout control unit 503 is configured to perform layout control on the live room interface according to the second layout information.

[0109] In one or more embodiments of the present disclosure, the device can further comprise a detection unit configured to detect whether the live stream data of the live room comprises supplementary enhancement information, wherein the supplementary enhancement information comprises first layout information of the live room interface.

[0110] In one or more embodiments of the present disclosure, when performing layout control on the live room interface according to the second layout information, the layout control unit 503 is configured to:

[0111] If the second layout information changes, the layout control unit 503 is configured to perform layout change on the live room interface according to the changed second layout information.

[0112] In one or more embodiments of the present disclosure, the first layout information is layout information of the live room interface of a current frame.

[0113] The second layout information is layout information of the live room interface of a current frame, or the second layout information is identification information corresponding to the layout information of the live room interface of the current frame.

[0114] In one or more embodiments of the present disclosure, if the second layout information is identification information corresponding to the layout information of the live room interface of the current frame, when performing layout change on the live room interface according to the changed second layout information, the layout control unit 503 is configured to:

[0115] If the identification information changes, the layout control unit 503 is configured to request layout information corresponding to the changed identification information from a server, wherein the server stores a mapping relationship between different identification information and different layout information.

[0116] The layout control unit 503 is configured to perform layout change on the live room interface according to the layout information corresponding to the changed identification information.

[0117] In one or more embodiments of the present disclosure, the preset encoding information included in the live room interface image is a blind watermark synthesized in the live room interface image, and the blind watermark includes the second layout information of the live room interface.

[0118] The analysis unit 502 is configured to:

[0119] analyze the blind watermark included in the live room interface image to obtain the second layout information of the live room interface.

[0120] In one or more embodiments of the present disclosure, when the analysis unit 502 analyzes the blind watermark included in the live room interface image to obtain the second layout information of the live room interface included in the blind watermark, the analysis unit 502 is configured to:

[0121] transform the live room interface image to the frequency domain, analyze the blind watermark included in the live room interface image in the frequency domain, and obtain the second layout information of the live room interface included in the blind watermark.

[0122] In one or more embodiments of the present disclosure, when the analysis unit 502 analyzes the blind watermark included in the live room interface image to obtain the second layout information of the live room interface included in the blind watermark, the analysis unit 502 is configured to:

[0123] analyze the blind watermark included in a preset region of the live room interface image to obtain the second layout information of the live room interface included in the blind watermark.

[0124] In one or more embodiments of the present disclosure, the layout control unit 503 is further configured to:

[0125] When entering the live room, obtain the latest layout information of the live room interface from the server, load interface components in the live room interface according to the latest layout information of the live room interface, and perform initial layout on the live room interface.

[0126] The live processing device provided in the embodiment can be used to execute the technical solutions of the method embodiments on the audience side of the device, and has similar implementation principles and technical effects. Details are not described herein.

[0127] The live processing method on the host side of the device corresponding to the above embodiment, Figure 6 FIG. 1 is a structural block diagram of a live processing device provided in an embodiment of the present disclosure. For ease of illustration, only parts related to the embodiments of the present disclosure are shown. Details can be referred to Figure 6The live processing device 600 comprises a data processing unit 601, a synthesis unit 602, and a communication unit 603.

[0128] The data processing unit 601 is configured to add supplementary enhancement information in live stream data of a live room, wherein the supplementary enhancement information comprises first layout information of a live room interface.

[0129] The synthesis unit 602 is configured to add preset encoding information in each frame of live room interface image of the live stream data, wherein the preset encoding information comprises second layout information of the live room interface.

[0130] The communication unit 603 is configured to push the live stream data to a server, so that a viewer terminal device obtains the live stream data from the server for playing.

[0131] In one or more embodiments of the present disclosure, the first layout information is layout information of the live room interface of the current frame.

[0132] The second layout information is layout information of the live room interface of the current frame, or the second layout information is identification information corresponding to the layout information of the live room interface of the current frame.

[0133] In one or more embodiments of the present disclosure, if the second layout information is identification information corresponding to the layout information of the live room interface of the current frame,

[0134] The data processing unit 601 is further configured to update the second layout information when the layout information of the live room interface of the current frame changes.

[0135] The communication unit 603 is further configured to upload the updated second layout information and the changed layout information of the live room interface of the current frame to the server.

[0136] In one or more embodiments of the present disclosure, when the data processing unit 601 updates the second layout information when the layout information of the live room interface of the current frame changes,

[0137] When the number of anchors in the live room changes, or the position and size of the video picture of each anchor in the live room interface change, it is determined that the layout information of the live room interface of the current frame changes, and the second layout information is updated.

[0138] In one or more embodiments of the present disclosure, when the synthesis unit 602 adds preset encoding information in each frame of live room interface image of the live stream data,

[0139] The second layout information of the live room interface is synthesized into each frame of live room interface image of the live stream data through blind watermarking.

[0140] In one or more embodiments of the present disclosure, when the second layout information of the live room interface is synthesized into each frame of live room interface image of the live stream data through blind watermarking, the synthesis unit 602 is configured to:

[0141] The second layout information of the live room interface is synthesized into a preset region of each frame of live room interface image of the live stream data through blind watermarking.

[0142] In one or more embodiments of the present disclosure, when the second layout information of the live room interface is synthesized into each frame of live room interface image of the live stream data through blind watermarking, the synthesis unit 602 is configured to:

[0143] Each frame of live room interface image is transformed into a frequency domain, the second layout information is synthesized into each frame of live room interface image in the frequency domain through blind watermarking, and the synthesized each frame of live room interface image is converted into a spatial domain.

[0144] In one or more embodiments of the present disclosure, when each frame of live room interface image is transformed into a frequency domain, the second layout information is synthesized into each frame of live room interface image in the frequency domain through blind watermarking, and the synthesized each frame of live room interface image is converted into a spatial domain, the synthesis unit 602 is configured to:

[0145] A preset region image of the live room interface image is obtained, the preset region image is transformed into a frequency domain, the second layout information is synthesized into the preset region image in the frequency domain through blind watermarking, the synthesized preset region image is converted into a spatial domain, and the synthesized preset region image is synthesized into each frame of live room interface image in the spatial domain.

[0146] The live processing device provided in the embodiment can be used to execute the technical solutions of the method embodiments on the side of the anchor terminal device, and has similar implementation principles and technical effects. Details are not described herein.

[0147] Reference Figure 7The diagram illustrates a structural schematic of an electronic device 700 suitable for implementing embodiments of the present disclosure. The electronic device 700 can be a terminal device or a server. The terminal device can include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, personal digital assistants (PDAs), portable Android devices (PADs), portable media players (PMPs), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 7 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0148] like Figure 7 As shown, the electronic device 700 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. The RAM 703 also stores various programs and data required for the operation of the electronic device 700. The processing unit 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0149] Typically, the following devices can be connected to I / O interface 705: input devices 706 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 707 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 708 including, for example, magnetic tapes, hard disks, etc.; and communication devices 709. Communication device 709 allows electronic device 700 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 7 An electronic device 700 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0150] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for executing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network by the communication device 709, or installed from the storage device 708, or installed from the ROM 702. When the computer program is executed by the processing device 701, the live processing method of the viewer-side device or the host-side device defined in the method of the embodiments of the present disclosure is executed.

[0151] It should be noted that the computer readable medium described above in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination thereof. The computer readable storage medium may, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination thereof. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, device or apparatus. In the present disclosure, the computer readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer readable program code. Such a propagated data signal can take on many forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination thereof. The computer readable signal medium can also be any computer readable medium that is not a computer readable storage medium and that can transmit, propagate or transport program for use by or in connection with an instruction execution system, device or apparatus. The program code contained on the computer readable medium can be transmitted by any suitable medium, including but not limited to a wire, cable, optical fiber, RF (radio frequency), or any suitable combination thereof.

[0152] The computer readable medium described above can be included in the electronic device described above; or can exist separately from the electronic device and not be assembled into the electronic device.

[0153] The computer readable medium described above carries one or more programs, which, when executed by the electronic device, cause the electronic device to execute the method shown in the embodiments described above.

[0154] Computer program code for carrying out operations of the present disclosure can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0155] The flow diagrams and the block diagrams in the drawings are meant as possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flow diagrams and the block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flow diagrams, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or combinations of special purpose hardware and computer instructions.

[0156] The units described in the embodiments of the present disclosure can be implemented by software, or by hardware. In some cases, the name of the unit does not constitute a limitation on the unit itself. For example, the first obtaining unit can also be described as a unit for obtaining at least two Internet protocol addresses.

[0157] The functions described above in the detailed description of embodiments of the present disclosure can be implemented in at least in part by one or more hardware logic components. For example, and without limitation, illustrative types of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Program-specific Integrated Circuits (ASICs), Program-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

[0158] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0159] In a first aspect, according to one or more embodiments of the present disclosure, a live processing method is provided, comprising:

[0160] In the case where the supplementary enhancement information is not included in the live streaming data, a live room interface image is acquired; wherein the supplementary enhancement information includes first layout information of the live room interface;

[0161] According to preset encoding information included in the live room interface image, second layout information of the live room interface is acquired;

[0162] According to the second layout information, the live room interface is controlled in layout.

[0163] According to one or more embodiments of the present disclosure, the controlling the live room interface in layout according to the second layout information comprises:

[0164] If the second layout information changes, the live room interface is changed in layout according to the changed second layout information.

[0165] According to one or more embodiments of the present disclosure, the first layout information is layout information of the live room interface of a current frame;

[0166] The second layout information is layout information of the live room interface of the current frame, or the second layout information is identification information corresponding to the layout information of the live room interface of the current frame.

[0167] According to one or more embodiments of the present disclosure, if the second layout information is identification information corresponding to the layout information of the live room interface of the current frame, the changing the live room interface in layout according to the changed second layout information comprises:

[0168] If the identification information changes, the server is requested for layout information corresponding to the changed identification information, wherein the server stores a mapping relationship between different identification information and different layout information.

[0169] The live room interface is laid out according to the layout information corresponding to the changed identification information.

[0170] According to one or more embodiments of the present disclosure, the preset encoding information included in the live room interface image is a blind watermark synthesized in the live room interface image, and the blind watermark includes second layout information of the live room interface.

[0171] The second layout information of the live room interface is obtained according to the preset encoding information included in the live room interface image, including:

[0172] The blind watermark included in the live room interface image is analyzed to obtain the second layout information of the live room interface.

[0173] According to one or more embodiments of the present disclosure, the blind watermark included in the live room interface image is analyzed to obtain the second layout information of the live room interface included in the blind watermark, including:

[0174] The live room interface image is transformed to a frequency domain, and the blind watermark included in the live room interface image is analyzed in the frequency domain to obtain the second layout information of the live room interface included in the blind watermark.

[0175] According to one or more embodiments of the present disclosure, the blind watermark included in the live room interface image is analyzed to obtain the second layout information of the live room interface included in the blind watermark, including:

[0176] The blind watermark included in the preset region of the live room interface image is analyzed to obtain the second layout information of the live room interface included in the blind watermark.

[0177] According to one or more embodiments of the present disclosure, before detecting whether the live stream data of the live room includes the supplemental enhancement information, the method further includes:

[0178] When entering the live room, the latest layout information of the live room interface is obtained from the server, and interface components are loaded in the live room interface according to the latest layout information of the live room interface to perform initial layout on the live room interface.

[0179] In a second aspect, according to one or more embodiments of the present disclosure, a live processing method is provided, applied to a host terminal device, and the method includes:

[0180] adding supplementary enhancement information in the live streaming data of the live room, wherein the supplementary enhancement information comprises first layout information of the live room interface;

[0181] adding preset encoding information in each live room interface image of the live streaming data, wherein the preset encoding information comprises second layout information of the live room interface;

[0182] pushing the live streaming data to a server, so that a viewer-side device obtains the live streaming data from the server for playing.

[0183] According to one or more embodiments of the present disclosure, the first layout information is layout information of the live room interface of the current frame;

[0184] The second layout information is layout information of the live room interface of the current frame, or the second layout information is identification information corresponding to the layout information of the live room interface of the current frame.

[0185] According to one or more embodiments of the present disclosure, if the second layout information is identification information corresponding to the layout information of the live room interface of the current frame, the method further comprises:

[0186] updating the second layout information when the layout information of the live room interface of the current frame changes;

[0187] uploading the updated second layout information and the changed layout information of the live room interface of the current frame to the server.

[0188] According to one or more embodiments of the present disclosure, the updating of the second layout information when the layout information of the live room interface of the current frame changes comprises:

[0189] When the number of anchors in the live room changes, or the positions and sizes of the video pictures of each anchor in the live room interface change, it is determined that the layout information of the live room interface of the current frame changes, and the second layout information is updated.

[0190] According to one or more embodiments of the present disclosure, the adding of the preset encoding information in each live room interface image of the live streaming data comprises:

[0191] compositing the second layout information of the live room interface into each live room interface image of the live streaming data by blind watermarking.

[0192] According to one or more embodiments of the present disclosure, the compositing of the second layout information of the live room interface into each live room interface image of the live streaming data by blind watermarking comprises:

[0193] The second layout information of the live room interface is synthesized into a preset region of each frame of live room interface image of the live stream data through blind watermarking.

[0194] According to one or more embodiments of the present disclosure, the synthesizing of the second layout information of the live room interface into each frame of live room interface image of the live stream data through blind watermarking comprises:

[0195] Each frame of live room interface image is transformed into a frequency domain, the second layout information is synthesized into each frame of live room interface image in the frequency domain through blind watermarking, and the synthesized each frame of live room interface image is converted into a spatial domain.

[0196] According to one or more embodiments of the present disclosure, the transforming of each frame of live room interface image into a frequency domain, the synthesizing of the second layout information into each frame of live room interface image in the frequency domain through blind watermarking, and the converting of the synthesized each frame of live room interface image into a spatial domain comprise:

[0197] A preset region image of the live room interface image is acquired, the preset region image is transformed into a frequency domain, the second layout information is synthesized into the preset region image in the frequency domain through blind watermarking, the synthesized preset region image is converted into a spatial domain, and the synthesized preset region image is synthesized into each frame of live room interface image in the spatial domain.

[0198] In a third aspect, according to one or more embodiments of the present disclosure, a live processing device is provided, comprising:

[0199] An image acquisition unit is configured to acquire a live room interface image in a case where the live stream data does not include supplemental enhancement information, wherein the supplemental enhancement information includes first layout information of the live room interface.

[0200] An analysis unit is configured to acquire second layout information of the live room interface according to preset encoding information included in the live room interface image.

[0201] A layout control unit is configured to perform layout control on the live room interface according to the second layout information.

[0202] According to one or more embodiments of the present disclosure, when the layout control unit performs layout control on the live room interface according to the second layout information, the layout control unit is configured to:

[0203] If the second layout information changes, the layout control unit is configured to perform layout change on the live room interface according to the changed second layout information.

[0204] According to one or more embodiments of the present disclosure, the first layout information is layout information of a live room interface of a current frame.

[0205] The second layout information is layout information of a live room interface of a current frame, or the second layout information is identification information corresponding to the layout information of the live room interface of the current frame.

[0206] According to one or more embodiments of the present disclosure, if the second layout information is identification information corresponding to the layout information of the live room interface of the current frame, if the second layout information changes, the layout control unit is configured to:

[0207] If the identification information changes, the layout control unit is configured to request, from a server, layout information corresponding to the changed identification information, wherein the server stores a mapping relationship between different identification information and different layout information.

[0208] The layout control unit is configured to perform layout change on the live room interface according to the layout information corresponding to the changed identification information.

[0209] According to one or more embodiments of the present disclosure, the preset encoding information included in the live room interface image is a blind watermark synthesized in the live room interface image, and the blind watermark includes second layout information of the live room interface.

[0210] The layout control unit is configured to obtain the second layout information of the live room interface according to the preset encoding information included in the live room interface image, including:

[0211] The layout control unit is configured to parse the blind watermark included in the live room interface image to obtain the second layout information of the live room interface.

[0212] According to one or more embodiments of the present disclosure, the parsing unit is configured to parse the blind watermark included in the live room interface image to obtain the second layout information of the live room interface included in the blind watermark, including:

[0213] The layout control unit is configured to transform the live room interface image to a frequency domain, parse the blind watermark included in the live room interface image in the frequency domain, and obtain the second layout information of the live room interface included in the blind watermark.

[0214] According to one or more embodiments of the present disclosure, the parsing unit is configured to parse the blind watermark included in the live room interface image to obtain the second layout information of the live room interface included in the blind watermark, including:

[0215] The blind watermark included in the preset region of the live room interface image is analyzed to obtain second layout information of the live room interface included in the blind watermark.

[0216] According to one or more embodiments of the present disclosure, the layout control unit is further configured to:

[0217] When entering the live room, the latest layout information of the live room interface is obtained from the server, and interface components are loaded in the live room interface according to the latest layout information of the live room interface to perform initial layout on the live room interface.

[0218] In a fourth aspect, according to one or more embodiments of the present disclosure, a live processing device is provided, comprising:

[0219] A data processing unit is configured to add supplemental enhancement information in live stream data of a live room, wherein the supplemental enhancement information includes first layout information of the live room interface.

[0220] A synthesis unit is configured to add preset encoding information in each frame of live room interface image of the live stream data, wherein the preset encoding information includes second layout information of the live room interface.

[0221] A communication unit is configured to push the live stream data to a server, so that a viewer terminal device obtains the live stream data from the server for playing.

[0222] According to one or more embodiments of the present disclosure, the first layout information is layout information of the live room interface of the current frame.

[0223] The second layout information is layout information of the live room interface of the current frame, or the second layout information is identification information corresponding to the layout information of the live room interface of the current frame.

[0224] According to one or more embodiments of the present disclosure, if the second layout information is identification information corresponding to the layout information of the live room interface of the current frame;

[0225] The data processing unit is further configured to update the second layout information when the layout information of the live room interface of the current frame changes.

[0226] The communication unit is further configured to upload the updated second layout information and the changed layout information of the live room interface of the current frame to the server.

[0227] According to one or more embodiments of the present disclosure, when the data processing unit updates the second layout information when the layout information of the live room interface of the current frame changes, the data processing unit is configured to:

[0228] When the number of the online anchors in the live broadcast room changes, or the position and size of the video picture of each online anchor in the live broadcast room interface changes, it is determined that the layout information of the live broadcast room interface of the current frame changes, and the second layout information is updated.

[0229] According to one or more embodiments of the present disclosure, the synthesis unit, when adding the preset encoding information into each frame of the live broadcast room interface image of the live broadcast stream data, is configured to:

[0230] The second layout information of the live broadcast room interface is synthesized into each frame of the live broadcast room interface image of the live broadcast stream data by blind watermarking.

[0231] According to one or more embodiments of the present disclosure, the synthesis unit, when synthesizing the second layout information of the live broadcast room interface into each frame of the live broadcast room interface image of the live broadcast stream data by blind watermarking, is configured to:

[0232] The second layout information of the live broadcast room interface is synthesized into a preset region of each frame of the live broadcast room interface image of the live broadcast stream data by blind watermarking.

[0233] According to one or more embodiments of the present disclosure, the synthesis unit, when synthesizing the second layout information of the live broadcast room interface into each frame of the live broadcast room interface image of the live broadcast stream data by blind watermarking, is configured to:

[0234] Each frame of the live broadcast room interface image is transformed into a frequency domain, the second layout information is synthesized into each frame of the live broadcast room interface image in the frequency domain by blind watermarking, and the synthesized each frame of the live broadcast room interface image is converted into a spatial domain.

[0235] According to one or more embodiments of the present disclosure, the synthesis unit, when transforming each frame of the live broadcast room interface image into a frequency domain, synthesizing the second layout information into each frame of the live broadcast room interface image in the frequency domain by blind watermarking, and converting the synthesized each frame of the live broadcast room interface image into a spatial domain, is configured to:

[0236] A preset region image of the live broadcast room interface image is obtained, the preset region image is transformed into a frequency domain, the second layout information is synthesized into the preset region image in the frequency domain by blind watermarking, the synthesized preset region image is converted into a spatial domain, and the synthesized preset region image is synthesized into each frame of the live broadcast room interface image in the spatial domain.

[0237] In a fifth aspect, according to one or more embodiments of the present disclosure, an electronic device is provided, including at least one processor and a memory;

[0238] The memory stores computer execution instructions;

[0239] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the live processing method as described in the first aspect and various possible designs of the first aspect.

[0240] In a sixth aspect, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, and the computer-readable storage medium has stored therein computer-executable instructions that, when executed by a processor, implement the live processing method as described in the first aspect and various possible designs of the first aspect.

[0241] In a seventh aspect, according to one or more embodiments of the present disclosure, a computer program product is provided, and the computer program product includes computer-executable instructions that, when executed by a processor, implement the live processing method as described in the first aspect and various possible designs of the first aspect.

[0242] The above description merely describes preferred embodiments of the present disclosure and the technical principles of the application. It should be understood by those skilled in the art that the disclosed scope of the present disclosure is not limited to the technical solutions formed by the specific combinations of the above technical features, and also covers other technical solutions formed by the combinations of the above technical features or equivalent features without departing from the above disclosed concept. For example, the technical solutions formed by the mutual replacement of the above features and the technical features disclosed in the present disclosure (but not limited to) having similar functions.

[0243] In addition, although each operation is described in a particular order, this should not be understood as requiring the operations to be performed in the particular order shown or in a sequential order. In certain circumstances, multitasking and parallel processing can be advantageous. Similarly, although several implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Certain features described in the context of separate embodiments can also be combined in a single embodiment. Conversely, various features described in the context of a single embodiment can also be separated and implemented in multiple embodiments. The various features described in the context of separate embodiments can also be combined in a single embodiment.

[0244] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A live processing method, characterized by, Applied to a viewer terminal device, the method comprises: In the case that the supplementary enhancement information is not included in the live streaming data, acquiring a live room interface image; wherein the first layout information of the live room interface is included in the supplementary enhancement information; According to the preset encoding information included in the live room interface image, acquiring the second layout information of the live room interface; According to the second layout information, performing layout control on the live room interface.

2. The method of claim 1, wherein, The layout control on the live room interface according to the second layout information comprises: If the second layout information changes, performing layout change on the live room interface according to the changed second layout information.

3. The method of claim 2, wherein, The first layout information is the layout information of the live room interface of the current frame; The second layout information is the layout information of the live room interface of the current frame, or the second layout information is the identification information corresponding to the layout information of the live room interface of the current frame.

4. The method of claim 3, wherein, If the second layout information is the identification information corresponding to the layout information of the live room interface of the current frame, then if the second layout information changes, performing layout change on the live room interface according to the changed second layout information comprises: If the identification information changes, requesting the layout information corresponding to the changed identification information from the server; wherein the mapping relationship between different identification information and different layout information is stored in the server; According to the layout information corresponding to the changed identification information, performing layout change on the live room interface.

5. The method of claim 1, wherein, The preset encoding information included in the live room interface image is a blind watermark synthesized in the live room interface image, and the blind watermark includes the second layout information of the live room interface; According to the preset encoding information included in the live room interface image, acquiring the second layout information of the live room interface comprises: Analyzing the blind watermark included in the live room interface image to acquire the second layout information of the live room interface.

6. The method of claim 5, wherein, The analysis of the blind watermark included in the live room interface image to acquire the second layout information of the live room interface comprises: Transforming the live room interface image to the frequency domain, and analyzing the blind watermark included in the live room interface image in the frequency domain to acquire the second layout information of the live room interface.

7. The method according to claim 5 or 6, characterized in that, The analysis of the blind watermark included in the live room interface image to acquire the second layout information of the live room interface comprises: Analyzing the blind watermark included in the preset region of the live room interface image to acquire the second layout information of the live room interface.

8. A live processing method, characterized by, Applied to a host terminal device, the method comprises: Adding supplementary enhancement information in the live streaming data of the live room, wherein the first layout information of the live room interface is included in the supplementary enhancement information; Adding preset encoding information in each frame of the live room interface image of the live streaming data, wherein the preset encoding information includes the second layout information of the live room interface; Pushing the live streaming data to the server, so that the viewer terminal device acquires the live streaming data from the server for playing.

9. The method of claim 8, wherein, The first layout information is the layout information of the live room interface of the current frame; The second layout information is the layout information of the live room interface of the current frame, or the second layout information is the identification information corresponding to the layout information of the live room interface of the current frame. The second layout information is layout information of a live room interface of a current frame, or the second layout information is identification information corresponding to the layout information of the live room interface of the current frame.

10. The method of claim 9, wherein, If the second layout information is identification information corresponding to the layout information of the live room interface of the current frame, the method further includes: updating the second layout information when layout information of the live room interface of the current frame changes; uploading the updated second layout information and the changed layout information of the live room interface of the current frame to the server.

11. The method of claim 10, wherein, The updating of the second layout information when the layout information of the live room interface of the current frame changes includes: when the number of anchors in the live room changes, or the position and size of the video of each anchor in the live room interface change, it is determined that the layout information of the live room interface of the current frame changes, and the second layout information is updated.

12. The method of claim 8, wherein, The adding of the preset encoding information in each frame of the live room interface image of the live stream data includes: The second layout information of the live room interface is synthesized into each frame of the live room interface image of the live stream data by blind watermarking.

13. The method of claim 12, wherein, The synthesizing of the second layout information of the live room interface into each frame of the live room interface image of the live stream data by blind watermarking includes: The second layout information of the live room interface is synthesized into a preset region of each frame of the live room interface image of the live stream data by blind watermarking.

14. The method according to claim 12 or 13, characterized in that, The synthesizing of the second layout information of the live room interface into each frame of the live room interface image of the live stream data by blind watermarking includes: Each frame of the live room interface image is transformed into a frequency domain, the second layout information is synthesized into each frame of the live room interface image in the frequency domain by blind watermarking, and then the synthesized each frame of the live room interface image is converted into a spatial domain.

15. The method of claim 14, wherein, The synthesizing of the second layout information of the live room interface into each frame of the live room interface image of the live stream data by blind watermarking includes: A preset region image of the live room interface image is obtained, the preset region image is transformed into a frequency domain, the second layout information is synthesized into the preset region image in the frequency domain by blind watermarking, and then the synthesized preset region image is converted into a spatial domain, and the synthesized preset region image is synthesized into each frame of the live room interface image in the spatial domain.

16. A live processing device, comprising: It includes: An image acquisition unit is configured to acquire a live room interface image in the case that the live stream data does not include supplemental enhancement information. The supplemental enhancement information includes first layout information of a live room interface. An analysis unit is configured to acquire second layout information of the live room interface according to preset encoding information included in the live room interface image. A layout control unit is configured to perform layout control on the live room interface according to the second layout information.

17. A live processing device, comprising: It includes: A data processing unit is configured to add supplemental enhancement information in live stream data of a live room, wherein the supplemental enhancement information includes first layout information of a live room interface. The synthesis unit is configured to add preset encoding information in each live room interface image of the live streaming data, where the preset encoding information comprises second layout information of the live room interface. The communication unit is configured to push the live streaming data to a server, so that a viewer terminal device obtains the live streaming data from the server for playing.

18. An electronic device, comprising: Comprising: at least one processor and a memory; the memory stores computer-executable instructions; the at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the method according to any one of claims 1-15.

19. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the method according to any one of claims 1-15 is implemented.

20. A computer program product, characterised in that, The computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the method according to any one of claims 1-15 is implemented.

Images