Live streaming room display method and device, storage medium and electronic device

Abstract

The present disclosure relates to a live room display method and device, a storage medium and an electronic device, and relates to the technical field of network live streaming. The method comprises: collecting live stage data of a target live room, and processing the live stage data to obtain an arrangement rule of audience seats included in the live stage data; based on the arrangement rule, matching a synthesis template of a virtual audience sticker corresponding to the audience seats for the target live room from a preset sticker database; synthesizing the virtual audience sticker based on a preset original three-dimensional virtual image of a virtual object and the synthesis template; when it is detected that the target live room starts live streaming, synthesizing the virtual audience sticker and a position where the audience seats in the target live room are located to obtain a synthesis video stream, and pushing the synthesis video stream to a terminal device to display a live room picture in the synthesis video stream on the terminal device. The present disclosure solves the problem of poor display effect of a live room.

Description

Technical Field

[0001] This disclosure relates to the field of live streaming technology, and more specifically, to a method for displaying a live streaming room, a device for displaying a live streaming room, a computer-readable storage medium, and an electronic device. Background Technology

[0002] AR (Augmented Reality) effects during live streaming can effectively enhance the overall effect and interactivity of the program, and improve the user's viewing and interactive experience.

[0003] Currently, the only technology to address the lack of a live audience is to present a virtual audience through video recordings or images. Since it's impossible to show a three-dimensional audience on-site using cameras, the live broadcast's presentation quality is poor.

[0004] It should be noted that the information in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this disclosure is to provide a method for displaying a live streaming room, a device for displaying a live streaming room, a computer-readable storage medium, and an electronic device, thereby overcoming, to at least some extent, the problem of poor display effects in live streaming rooms caused by limitations and defects in related technologies.

[0006] According to one aspect of this disclosure, a method for displaying a live streaming room is provided, comprising:

[0007] Collect live stage data from the target live streaming room and process the live stage data to obtain the arrangement rules of the audience seats included in the live stage data;

[0008] Based on the arrangement rules, a composite template of virtual audience images corresponding to the audience seats is matched from a preset image database for the target live streaming room;

[0009] Based on the original 3D virtual image of the preset virtual object and the synthesis template, a virtual audience texture is synthesized;

[0010] When the target live streaming room is detected to have started live streaming, the virtual audience texture and the location of the audience seats in the target live streaming room are combined to obtain a composite video stream, and the composite video stream is pushed to the terminal device to display the live streaming room screen in the composite video stream on the terminal device.

[0011] In an example embodiment of the present disclosure, the audience seats include audience seats in the same preset area or audience seats in different preset areas.

[0012] In an example embodiment of the present disclosure, when the audience seats are audience seats in the same preset area, the processing of the live stage data includes obtaining the arrangement rule of the audience seats included in the live stage data, including:

[0013] performing image recognition on the live stage data to obtain the number of rows and columns of the audience seats included in the live stage data, and the position coordinate points of each seat in the audience seats in the target live room;

[0014] calculating the horizontal position interval between two horizontally adjacent seats and the vertical position interval between two vertically adjacent seats according to the position coordinate points of each seat in the target live room;

[0015] obtaining the arrangement rule of the audience seats included in the live stage data according to the number of rows and columns of the audience seats, and the horizontal position interval and the vertical position interval.

[0016] In an example embodiment of the present disclosure, when the audience seats are audience seats in different preset areas, the processing of the live stage data includes obtaining the arrangement rule of the audience seats included in the live stage data, including:

[0017] performing image recognition on the live stage data to obtain the number of regions of the audience seats included in the live stage data, and the number of rows and columns of the audience seats included in each region, and the position coordinate points of each seat in the audience seats included in the region in the target live room;

[0018] calculating the horizontal position interval between two horizontally adjacent seats and the vertical position interval between two vertically adjacent seats according to the position coordinate points of each seat included in each region in the target live room;

[0019] obtaining the sub-arrangement rule of the audience seats in each region according to the number of rows and columns of the audience seats included in each region, and the horizontal position interval and the vertical position interval, and obtaining the arrangement rule of the audience seats included in the live stage data according to the sub-arrangement rule of the audience seats in each region.

[0020] In an example embodiment of the present disclosure, synthesizing a virtual audience map based on the preset original three-dimensional virtual image of a virtual object and the synthesis template includes:

[0021] According to the number of grids possessed by the synthesis template, the number of images of the virtual object required to be pulled is calculated, and the original three-dimensional virtual image corresponding to the number of images is pulled from a preset resource library;

[0022] The original three-dimensional virtual image is translated to obtain a role code corresponding to the original three-dimensional virtual image, and the role code is sorted to obtain an array code;

[0023] The array code is synthesized into the synthesis template to obtain the virtual audience map.

[0024] In an exemplary embodiment of the present disclosure, the display method of the live room further comprises:

[0025] According to the size of the grid possessed by the synthesis template, the size of the original three-dimensional virtual image is adjusted to obtain a target three-dimensional virtual image.

[0026] In an exemplary embodiment of the present disclosure, synthesizing the virtual audience map and the position of the audience seat in the target live room to obtain a synthesis video stream comprises:

[0027] According to the correspondence between each seat included in the audience seat and each grid included in the virtual audience map, each seat included in the audience seat is assigned a grid;

[0028] Obtain the code possessed by the grid assigned to each seat included in the audience seat, and obtain the original three-dimensional virtual image or the target three-dimensional virtual image possessed by the virtual object corresponding to the code;

[0029] Synthesize the original three-dimensional virtual image or the target three-dimensional virtual image possessed by each seat included in the audience seat and the position of the seat to obtain the synthesis video stream.

[0030] In an exemplary embodiment of the present disclosure, after rendering the virtual audience map to the position of the audience seat in the target live room, the display method of the live room further comprises:

[0031] In response to a live room entry request sent by a terminal device when detecting that a live viewing user enters a target live room, obtaining a user identifier included in the live room entry request, and obtaining user attribute information according to the user identifier;

[0032] determine a user category of the live viewing user according to the user attribute information, and allocate a user position for the live viewing user in the audience seat according to the user category and an entering order of the live viewing user into the target live room;

[0033] synthesize the user position and an original three-dimensional virtual image or a target three-dimensional virtual image of the virtual object corresponding to the user position to obtain a cloud audience video stream corresponding to the live viewing user;

[0034] push the cloud audience video stream to a terminal device where the live viewing user is located, so that the live viewing user performs live interaction with the target live room through the virtual object.

[0035] In an exemplary embodiment of the present disclosure, the method for displaying the live room further comprises:

[0036] obtaining live content and a live scene that the target live room is currently live streaming, and obtaining a first special effect animation corresponding to the live content and the live scene according to the live content and the live scene;

[0037] synthesizing original three-dimensional virtual images or target three-dimensional virtual images of all virtual objects included in the target live room and the first special effect animation to obtain a first special effect video stream, and pushing the first special effect video stream to a terminal device.

[0038] In an exemplary embodiment of the present disclosure, the user category includes a member user category and / or a normal user category, and the audience seat includes a member audience seat and / or a normal audience seat.

[0039] According to the user category and the entering order of the live viewing user into the target live room, the live viewing user is allocated a user position in the audience seat, including:

[0040] when the user category of the live viewing user is the member user category, the live viewing user is allocated a user position in the member audience seat in a random allocation manner according to the entering order of the live viewing user into the target live room;

[0041] when the user category of the live viewing user is the normal user category, the live viewing user is allocated a user position in the normal audience seat in a sequential allocation manner according to the entering order of the live viewing user into the target live room.

[0042] In an exemplary embodiment of the present disclosure, the method for displaying the live room further comprises:

[0043] In response to a special effect generation request sent by the terminal device in response to the touch event of the live viewing user acting on the virtual object being detected, an identifier of a special effect included in the special effect generation request is acquired;

[0044] A second special effect animation corresponding to the identifier of the special effect is acquired; the second special effect animation includes at least one of an original emotion animation, a custom emotion animation, a placard effect animation, a text slogan effect animation, and a team declaration effect animation;

[0045] The second special effect animation and an original three-dimensional virtual image or a target three-dimensional virtual image of the virtual object are synthesized to obtain a second special effect video stream, and the second special effect video stream is pushed to the terminal device where the live viewing user is located.

[0046] In an exemplary embodiment of the present disclosure, the display method of the live room further includes:

[0047] A plurality of original grid resources having different numbers of rows, different numbers of columns, and different position intervals are created, and a display effect of the original grid resources is tested;

[0048] After determining that the display effect test of the original grid resources is completed, the original grid resources after the test is completed are used as a synthesis template, and the synthesis template is stored in the preset map database.

[0049] In an exemplary embodiment of the present disclosure, the display effect includes a light display effect and / or a display effect of a placement position of an original three-dimensional virtual image or a target three-dimensional virtual image of a virtual object in the original grid resource.

[0050] According to an aspect of the present disclosure, a display device of a live room is provided, including:

[0051] A live stage data processing module is configured to collect live stage data of a target live room and process the live stage data to obtain an arrangement rule of audience seats included in the live stage data;

[0052] A synthesis template matching module is configured to match, based on the arrangement rule, a synthesis template of a virtual audience map corresponding to the audience seats for the target live room from a preset map database;

[0053] A virtual audience map synthesis module is configured to synthesize a virtual audience map based on an original three-dimensional virtual image of a preset virtual object and the synthesis template;

[0054] The synthetic video stream pushing module is configured to synthesize the virtual audience sticker and a position where the audience seat in the target live room is located to obtain a synthetic video stream when it is detected that the target live room starts live streaming, and push the synthetic video stream to a terminal device to display a live room picture in the synthetic video stream on the terminal device.

[0055] According to an aspect of the present disclosure, a computer readable storage medium is provided, which stores a computer program. The computer program is executed by a processor to implement the live room display method according to any one of the preceding aspects.

[0056] According to an aspect of the present disclosure, an electronic device is provided, which comprises:

[0057] a processor; and

[0058] a memory configured to store executable instructions of the processor;

[0059] The processor is configured to execute the live room display method according to any one of the preceding aspects by executing the executable instructions.

[0060] The live room display method provided by the embodiments of the present disclosure, on one hand, collects live stage data of a target live room, processes the live stage data, and obtains arrangement rules of audience seats included in the live stage data; then, based on the arrangement rules, a synthetic template of a virtual audience sticker corresponding to the audience seats is matched from a preset sticker database for the target live room; then, based on a preset original three-dimensional virtual image of a virtual object and the synthetic template, the virtual audience sticker is synthesized; finally, when it is detected that the target live room starts live streaming, the virtual audience sticker is rendered to a position where the audience seats in the target live room are located, so that empty positions on the audience seats can be filled by the virtual audience sticker, and the problem of empty audience seats is avoided; on the other hand, since the synthetic template corresponds to the audience seats, and the virtual audience sticker includes the synthetic template and the original three-dimensional virtual image, when the virtual audience sticker is rendered to the audience seats, the audience seats can be assigned the original three-dimensional virtual image, and the virtual audience can be presented without using video recording or pictures, thus solving the problem in the prior art that the audience cannot be presented in three dimensions by using a camera on site, and thus the display effect of the live room is poor.

[0061] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0062] The accompanying drawings, which are incorporated herein and form a part of the specification, illustrate embodiments consistent with the present disclosure and, together with the description, further serve to explain the principles of the disclosure. It is to be understood that the drawings are designed solely for purposes of illustration to be used in conjunction with the description insofar as specifically described herein. Changes in design or structure can be made in the embodiments described herein, without departing from the scope of the present disclosure.

[0063] Figure 1 A flowchart schematically showing a display method of a live room according to an example embodiment of the present disclosure.

[0064] Figure 2 A block diagram schematically showing a display system of a live room according to an example embodiment of the present disclosure.

[0065] Figure 3 An example diagram schematically showing a synthesis template according to an example embodiment of the present disclosure.

[0066] Figure 4 An example diagram schematically showing a raw three-dimensional virtual figure according to an example embodiment of the present disclosure.

[0067] Figure 5 An example diagram schematically showing a filled array encoded square data diagram according to an example embodiment of the present disclosure.

[0068] Figure 6 An example diagram schematically showing a data map according to an example embodiment of the present disclosure.

[0069] Figure 7 An example diagram schematically showing a data map of different row and column numbers and a raw three-dimensional virtual figure of a virtual object corresponding thereto according to an example embodiment of the present disclosure.

[0070] Figure 8 An example diagram schematically showing another data map of different row and column numbers and a raw three-dimensional virtual figure of a virtual object corresponding thereto according to an example embodiment of the present disclosure.

[0071] Figure 9 An example diagram schematically showing live stage data according to an example embodiment of the present disclosure.

[0072] Figure 10 An example diagram schematically showing position and size information of a stage, and coordinate positions and size information of seats included in a seat of an audience according to an example embodiment of the present disclosure.

[0073] Figure 11 An example diagram schematically showing a virtual audience map according to an example embodiment of the present disclosure.

[0074] Figure 12An example diagram schematically showing another virtual audience map according to example embodiments of the present disclosure.

[0075] Figure 13 An example diagram schematically showing a picture rendered on a display interface of a terminal device according to example embodiments of the present disclosure.

[0076] Figure 14 A flowchart schematically showing a method for a live streaming interaction between a live streaming user and a target live streaming room through a virtual object according to example embodiments of the present disclosure.

[0077] Figure 15 An example diagram schematically showing a special effect animation in a live streaming process according to example embodiments of the present disclosure.

[0078] Figure 16 An example diagram schematically showing another special effect animation in a live streaming process according to example embodiments of the present disclosure.

[0079] Figure 17 A block diagram schematically showing a display device of a live streaming room according to example embodiments of the present disclosure.

[0080] Figure 18 An electronic device for implementing the display method of the live streaming room according to example embodiments of the present disclosure. DETAILED DESCRIPTION

[0081] Example implementations will now be described more fully with reference to the accompanying drawings. Example implementations can be implemented in any

[0082] In addition, the accompanying drawings are only schematic and are non- necessarily drawn to scale. The same reference numerals in different drawings denote the same or similar functionality throughout the various figures. In some embodiments, some or all of the functional entities shown in the figures can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0083] In the live broadcast of the current large-scale e-sports events and various large-scale entertainment parties, a large venue is needed to accommodate the audience, so it costs more manpower and economic cost; at the same time, in some special period, it is also impossible to organize the audience to watch on site; however, if the audience watches on site, it will cause the problem of seat vacancy and awkward atmosphere on the stage; at the same time, in the live broadcast of e-sports events, if the offline audience is to be coordinated with the game link, it will cost more manpower and the specific implementation process is more complicated.

[0084] In order to solve the above problems, the current technology for solving the problem of no audience on site can only present virtual audience through video recording or picture, and it is impossible to show three-dimensional audience through camera on site, and it is impossible to meet the best effect of live broadcast.

[0085] Therefore, in the example embodiment, a live room display method is provided first, which can run on a server, a server cluster or a cloud server of Unreal Engine (UE), of course, those skilled in the art can also run the method of the present disclosure on other platforms according to the needs, which is not specially limited in the example embodiment. Specifically, referring to Figure 1 The live room display method can include the following steps:

[0086] Step S110. Collecting live stage data of a target live room and processing the live stage data to obtain arrangement rules of audience seats included in the live stage data;

[0087] Step S120. Based on the arrangement rules, matching a synthetic template of virtual audience map corresponding to the audience seats for the target live room from a preset map database;

[0088] Step S130. Synthesizing virtual audience map based on a preset original three-dimensional virtual image of virtual object and the synthetic template;

[0089] Step S140. When it is detected that the target live room starts live streaming, the virtual audience sticker and the position where the audience seat in the target live room is located are synthesized to obtain a synthesized video stream, and the synthesized video stream is pushed to a terminal device to display the live room picture in the synthesized video stream on the terminal device.

[0090] In the display method of the live room, on the one hand, the live stage data of the target live room is collected, and the arrangement rule of the audience seat included in the live stage data is obtained by processing the live stage data. Then, based on the arrangement rule, a synthesis template of the virtual audience sticker corresponding to the audience seat is matched for the target live room from a preset sticker database. Then, based on the preset original three-dimensional virtual image of the virtual object and the synthesis template, the virtual audience sticker is synthesized. Finally, when it is detected that the target live room starts live streaming, the virtual audience sticker is rendered to the position where the audience seat in the target live room is located, so that the empty position on the audience seat can be filled by the virtual audience sticker, avoiding the problem of empty audience seat. On the other hand, since the synthesis template corresponds to the audience seat, and the virtual audience sticker includes the synthesis template and the original three-dimensional virtual image, when the virtual audience sticker is rendered to the audience seat, the original three-dimensional virtual image can be allocated to the audience seat, without the need to present the virtual audience through video recording or picture, solving the problem that in the prior art, the three-dimensional audience cannot be displayed on site through the camera, and thus the display effect of the live room is poor.

[0091] In the following, the display method of the live room of the example embodiment of the present disclosure will be explained and described in detail with reference to the accompanying drawings.

[0092] Firstly, the invention purpose of the example embodiment of the present disclosure is explained and described. Specifically, the display method of the live room described in the example embodiment of the present disclosure can generate a three-dimensional virtual audience with a user identification on a real stage through a data program by using an AR(Augmented Reality, augmented reality) cloud audience system technology, and the ID(Identification, user identification) of the cloud audience and the platform user(live streaming watching user) can be presented on the real stage. The cloud audience can replace the real audience on site, improve the overall atmosphere on site, and trigger the action and matrix arrangement effect of the corresponding character according to the game data, and push the synthesized picture, realize the "user->client->UE engine->render->generate" process of the AR special effect in live streaming, and thus enhance the interactive effect in the live streaming process.

[0093] Secondly, the display system of the live room involved in the example embodiment of the present disclosure is explained and described. Specifically, referring to Figure 2As shown, the display system of the live room can include a terminal device 210 where the live client is located and an Unreal Engine 220; wherein the terminal device can be in communication connection with the Unreal Engine through wired network or wireless network. Further, the terminal device can be used to receive user instructions input by a live viewing user, and display a live picture; and the Unreal Engine can be used to implement the live room display method described in the example embodiments of the present disclosure.

[0094] Further, in the Unreal Engine, a UE background server and a cloud audience system are provided, wherein the UE background server can be used to generate a synthesis template, and the cloud audience system is used to synthesize a video stream and push the video stream. The UE background server can generate the synthesis template through a specific image processing framework, which can be Niagara, or other frameworks, and the present example does not make special limitations.

[0095] In the following, the specific generation process of the synthesis template described in the example embodiments of the present disclosure is explained and described in conjunction with the drawings. Specifically, in order to realize the entry of a large number of three-dimensional characters into the event virtual or the placement of characters to the real offline stage by AR, the UE generates each game player by using GPU (Graphics Processing Unit, graphics processor) instance, which can limit the additional drawing calls generated by each generated character; at the same time, by using GPU generation, the draw call will only be based on the maximum value of the type of the character, because GPU instance generation is adopted, the entire data interaction mode needs to be changed.

[0096] The specific generation process of CPU (Central Processing Unit, central processor) generation and GPU generation and its disadvantages are described as follows: first, the CPU generation game player flow: setting the game character, game name, team data of the player cloud audience→ setting the animation state of the player cloud audience→ setting the position of the player cloud audience in the virtual scene; its disadvantage is that each game player needs to set the data based on CPU, and a large number of audiences will produce a large performance overhead, and the bottleneck is limited by CPU and GPU; GPU instance player flow: collecting all the cloud audience data at once, and writing the data into a render target→ through the defined render target data sampling algorithm, let all the cloud audiences sample the data on the render target based on their own index, the render target can store 0-255 values, which correspond to let the cloud audience set its own character attributes and the like based on these values; its disadvantage is that taking the maximum of 19 character types as an example, the maximum CPU drawing call overhead is limited to 19 times, and the bottleneck is placed on the GPU.

[0097] Based on this, the example embodiments of the present disclosure provide a method for generating a synthetic template based on Niagara. Specifically, the method for generating the synthetic template can be implemented in the following way: first, create a plurality of original grid resources with different numbers of rows, different numbers of columns, and different position intervals, and test the display effect of the original grid resources; second, after determining that the display effect test of the original grid resources is complete, take the original grid resources after the test as a synthetic template, and store the synthetic template in the preset map database; wherein the display effect includes the display effect of the placement position of the light display effect and / or the original three-dimensional virtual image or target three-dimensional virtual image of the virtual object in the original grid resource.

[0098] That is, in the specific process of generating a synthetic template, it can be implemented in the following way: Initialize Niagara Parm → Create Array Data → Set RT Size → Draw 2D TexData. Among them, taking the 4*4 square of the pulled data as an example, when the square is pulled, the size ratio of the square can be set, and then the image interval to be drawn can be divided according to the calculated ratio to draw the square data graph of the square, that is, the synthetic template, wherein the generated synthetic template can be specifically referred to as shown in Figure 3 It should be noted here that when setting the size ratio of the Render Target, it can be set in equal proportion (for example, the rows and columns are scaled in the same proportion), or it can be scaled in a custom proportion, or the scaling proportion can be set for rows and columns respectively, etc. This example does not make special restrictions on this.

[0099] Further, after obtaining the square data graph, the original three-dimensional virtual image of the virtual object also needs to be filled into the square data graph, and then the data map is obtained. Among them, in the process of filling the original three-dimensional virtual image, first, the original three-dimensional virtual image of the virtual object needs to be obtained; wherein the obtained original three-dimensional virtual image can be specifically referred to as shown in Figure 4The original three-dimensional virtual image of the virtual object is then role-translated to generate a corresponding role code, and then an array code based on the order is generated according to the role code; for example, the array code obtained may include: 1, 0, 2, 3, 6, 5, 1, 2, 0, 1, 3, 1, 5, 2, 0, 3; further, when the data code is obtained, the array code can be filled into the square data graph according to the order; the square data graph after filling the array code can be referred to in FIG. 5; finally, the data map storing the original three-dimensional virtual image is drawn according to the role code in each square graph, and the data map can be referred to in FIG. 5. Figure 6

[0100] It should be noted here that after obtaining the synthesis template, the synthesis template can be tested by some test data. Specifically, during the test, a test map corresponding to the test data and the synthesis template can be generated, and the test map can be previewed, so as to test the light display effect of the test map and the display effect of the original three-dimensional virtual image or the target three-dimensional virtual image of the virtual object in the original grid resource; if the test is passed, the synthesis template is put into the above-mentioned preset map database, so as to be called later. In the preset map database, a plurality of synthesis templates with the same number of rows and columns but different intervals, a plurality of synthesis templates with the same number of rows but different number of columns, a plurality of synthesis templates with the same number of columns but different number of rows can be included; in the specific application process, when matching the synthesis template, factors such as interval, number of rows and number of columns can be considered, so as to match the synthesis template of the actual scene of the live broadcast room; the example graph of the data map with different number of rows and columns and the original three-dimensional virtual image of the virtual object corresponding thereto can be referred to in FIGS. 1-4. Figure 7 Figure 8

[0101] In an example embodiment, referring to FIG. 5, the number of rows and columns of the data map shown is 5*5, wherein the black square shown in the data map represents the array code bit 0 of the original three-dimensional virtual image corresponding to this position, and the array code at other positions can be other codes except zero, which is not specially limited in this example; at the same time, the virtual object corresponding to the data map shown in FIG. 5 can obtain the original three-dimensional virtual image corresponding to the coding array according to the coding array of the data map, and place it in the corresponding position in sequence, so as to obtain the example graph of the original three-dimensional virtual image of the virtual object shown on the right side of FIG. 5. Figure 7 Figure 7 Figure 7 Figure 8 ​​​​​​As shown, the actual data texture has 10*10 rows and columns. The specific generation process is similar to that in Figure 7, and will not be described in detail here.

[0102] It should be added that, through the above method, it is possible to realize the virtual audience of 3D character models generated by data programmatically. By writing the data to the RENDER TARGET, the NIAGARA technology uses the index to sample the RT data and change the character arrangement, thereby saving GPU overhead. At the same time, it is also possible to present the assets of 3D virtual character audiences, which are calculated in units of tens of thousands, on the live broadcast screen, while meeting the performance and network requirements of live broadcast (there should be no frame drops or stuttering during the live broadcast), without being limited by CPU and GPU.

[0103] The following will combine Figures 2-8 right Figure 1 The method for displaying the live streaming room shown is explained and illustrated. Specifically:

[0104] In step S110, the live stage data of the target live broadcast room is collected and processed to obtain the arrangement rules of the audience seats included in the live stage data.

[0105] In this example embodiment, firstly, live stage data of the target live streaming room is collected; wherein, during the process of collecting live stage data, live stage data can be collected through cameras at the live streaming location; wherein, the live stage data can specifically be as follows: Figure 9 As shown; in Figure 9 The stage data shown may include the stage and audience seating. The stage data captured by the camera may include the stage's location and dimensions, as well as the coordinates and size of the seats in the audience seating area (for details on the stage's location and dimensions, and the coordinates and size of the seats in the audience seating area, please refer to...). Figure 10 (as shown); of course, other information can also be included, but this example does not impose any special restrictions on this; secondly, after the on-site stage data is collected, the on-site stage data can be processed; the part that processes the on-site stage data can be executed by the processing module inside the camera; after the camera finishes processing the on-site stage data, it can transmit the obtained on-site information and other data to the UE backend server through the network. After the UE backend server receives the on-site information data, it can determine the arrangement rules based on the on-site information data.

[0106] Further, in some scenarios (e.g., a live room with a small area or without a confrontation between two parties), the audience seats include audience seats in the same preset area; in other scenarios (e.g., a confrontation between different fan groups), the audience seats can also include audience seats in multiple different preset areas. Therefore, the arrangement rule of the audience seats included in the live stage data can be obtained by processing the live stage data in the following two ways:

[0107] One way is that when the audience seats are in the same preset area, the arrangement rule of the audience seats included in the live stage data can be obtained by processing the live stage data in the following way: image recognition is performed on the live stage data to obtain the number of rows and columns of the audience seats included in the live stage data, and the position coordinate points of each seat in the target live room; the horizontal position interval between two horizontally adjacent seats and the vertical position interval between two vertically adjacent seats are calculated according to the position coordinate points of each seat in the target live room; and the arrangement rule of the audience seats included in the live stage data is obtained according to the number of rows and columns of the audience seats, and the horizontal position interval and the vertical position interval.

[0108] Another way is that when the audience seats are in multiple different preset areas, the arrangement rule of the audience seats included in the live stage data can be obtained by processing the live stage data in the following way: image recognition is performed on the live stage data to obtain the number of areas of the audience seats included in the live stage data, and the number of rows and columns of the audience seats included in each area, and the position coordinate points of each seat in the target live room; the horizontal position interval between two horizontally adjacent seats and the vertical position interval between two vertically adjacent seats in the area are calculated according to the position coordinate points of each seat in the target live room; the sub-arrangement rule of the audience seats in each area is obtained according to the number of rows and columns of the audience seats included in each area, and the horizontal position interval and the vertical position interval; and the arrangement rule of the audience seats included in the live stage data is obtained according to the sub-arrangement rule of the audience seats in each area.

[0109] That is, in specific applications, corresponding arrangement rules can be generated according to the actual needs of the live room, and then the corresponding synthesis template can be matched based on the arrangement rules. Through this method, the visual perception and accuracy of the live room can be improved, thereby achieving the purpose of improving the viewing experience of the live viewing users.

[0110] In step S120, based on the arrangement rule, a synthesis template of a virtual audience sticker corresponding to the audience seat of the target live room is matched from a preset sticker database.

[0111] Specifically, in the specific application process, the UE background server can read the synthesis template corresponding to the position arrangement of the cloud audience (arrangement rule) according to the AR special effect demand (i.e. arrangement rule) of the current live room; in the process of matching the synthesis template, the number of rows, the number of columns, the horizontal position interval and the vertical position interval included in the arrangement rule can be matched, and different regions included in the arrangement rule can also be matched. Through this method, the matching degree and the fitting degree of the synthesis template and the audience seat of the current live room can be further improved, and the visual experience can be further improved.

[0112] In step S130, based on the preset original three-dimensional virtual image of the virtual object and the synthesis template, a virtual audience sticker is synthesized.

[0113] In the example embodiment, first, the number of images of the virtual object required to be pulled is calculated according to the number of grids possessed by the synthesis template, and the original three-dimensional virtual image corresponding to the number of images is pulled from the preset resource library; second, the original three-dimensional virtual image is translated to obtain a role code corresponding to the original three-dimensional virtual image, and the role code is sorted to obtain an array code; further, the array code is synthesized into the synthesis template to obtain the virtual audience sticker. At the same time, before the original three-dimensional virtual image is synthesized into the synthesis template, the display method of the live room can further include: adjusting the size of the original three-dimensional virtual image according to the size of the grid possessed by the synthesis template to obtain a target three-dimensional virtual image. Further, after obtaining the target three-dimensional virtual image, the obtained target three-dimensional virtual image can be synthesized into the synthesis template to obtain the virtual audience sticker; wherein the obtained virtual audience sticker can refer to Figure 11 and Figure 12 .

[0114] At this point, the generation steps of the virtual audience sticker have been completed, and the live waiting link can be entered. It needs to be supplemented here that in the process of generating the virtual audience sticker, the role code of the virtual object is recorded in the grid of the synthesis template, that is, in the actual application process, the obtained virtual audience sticker is not the sticker with the target three-dimensional virtual image as shown in Figure 11 or 12, but as shown in Figure 6The black-and-white grid pattern is shown; when a synthetic video stream needs to be generated, the target three-dimensional virtual image or the original three-dimensional virtual image of the corresponding virtual object can be directly obtained by indexing the position (row and column position) of the black-and-white grid pattern and the character code included in the grid. By this method, the consumption of GPU can be reduced, the synthesis efficiency of the synthetic video stream can be improved, and the user's viewing experience can be improved.

[0115] In step S140, when it is detected that the target live room starts live streaming, the virtual audience sticker and the position of the audience seat in the target live room are synthesized to obtain a synthetic video stream, and the synthetic video stream is pushed to the terminal device to display the live room picture in the synthetic video stream on the terminal device.

[0116] In the example embodiment, when it is detected that the target live room starts live streaming, first, the virtual audience sticker and the position of the audience seat in the target live room are synthesized to obtain a synthetic video stream. Specifically, this can be achieved in the following way: first, according to the correspondence between each seat included in the audience seat and each grid included in the virtual audience sticker, each seat included in the audience seat is assigned a grid; second, the code possessed by the grid to which each seat included in the audience seat is assigned is obtained, and the original three-dimensional virtual image or the target three-dimensional virtual image possessed by the virtual object corresponding to the code is obtained; then, the original three-dimensional virtual image or the target three-dimensional virtual image possessed by each seat included in the audience seat and the position of the seat are synthesized to obtain the synthetic video stream. When the synthetic video stream is obtained, the synthetic video stream can be pushed to the terminal device, and the synthetic video stream can be rendered to the display interface of the terminal device. The picture rendered on the display interface of the terminal device can be specifically referred to as shown in Figure 13 .

[0117] At this point, all the processes of video streaming have been completed, and the user can access the live room for live streaming. Of course, during live streaming, corresponding interactions can also be performed. Specifically, referring to FIG. 14, the display method of the live room can further include the following steps:

[0118] Step S1410, in response to the live room entering request sent by the terminal device when it is detected that the live streaming user enters the target live room, obtaining the user identifier included in the live room entering request, and obtaining the user attribute information according to the user identifier;

[0119] Step S1420, determining a user category of the live viewing user according to the user attribute information, and assigning a user position for the live viewing user in the audience seat according to the user category and an entering order of the live viewing user into the target live room;

[0120] Step S1430, synthesizing the user position and an original three-dimensional virtual image or a target three-dimensional virtual image of the virtual object corresponding to the user position to obtain a cloud audience video stream corresponding to the live viewing user;

[0121] Step S1440, pushing the cloud audience video stream to a terminal device where the live viewing user is located, so as to enable the live viewing user to live interact with the target live room through the virtual object.

[0122] In an example embodiment, the user category includes a member user category and / or a common user category; the audience seat includes a member audience seat and / or a common audience seat; and wherein assigning a user position for the live viewing user in the audience seat according to the user category and an entering order of the live viewing user into the target live room includes: when the user category of the live viewing user is the member user category, assigning a user position for the live viewing user from the member audience seat in a random assignment manner according to the entering order of the live viewing user into the target live room; and when the user category of the live viewing user is the common user category, assigning a user position for the live viewing user from the common audience seat in a sequential assignment manner according to the entering order of the live viewing user into the target live room.

[0123] That is, in a specific application process, the user attribute information corresponding to the user can be obtained according to the user identifier of the live viewing user entering the live room, and then the category to which the live viewing user belongs is judged according to the user attribute information, such as VIP (Very Important Person) user and ordinary user; among the VIP users, there are also diamond VIP, platinum VIP or silver VIP, etc., and among the ordinary users, there are also temporary users or long-term users, etc., which are not specially limited in this example; at the same time, when assigning seats to live viewing users, the above rules can be followed or self-set rules can be set, which are not specially limited in this example; at the same time, if the user is not satisfied with the target three-dimensional virtual image or the original three-dimensional virtual image of the virtual object assigned, a request for corresponding replacement can also be sent, etc., which are not specially limited in this example; at the same time, when assigning seats and target three-dimensional virtual images or original three-dimensional virtual images to the live viewing user, the video stream can be pushed to the terminal device where the live viewing user is located; through this method, the live viewing user can interact with the live room through the target three-dimensional virtual image or the original three-dimensional virtual image; that is, the live audience communicates with the anchor or others on the stage offline, realizing immersive program or match watching.

[0124] In an example embodiment, the display method of the live room can further include: obtaining live content and a live scene being live broadcasted by the target live room, and obtaining a first special effect animation corresponding to the live content and the live scene according to the live content and the live scene; synthesizing the original three-dimensional virtual image or the target three-dimensional virtual image of all virtual objects included in the target live room and the first special effect animation to obtain a first special effect video stream, and pushing the first special effect video stream to a terminal device. That is, in a specific application process, in order to realize the corresponding special effect, the original three-dimensional virtual image or the target three-dimensional virtual image of all virtual objects in the live room can be collectively configured with corresponding animations (such as fans holding placards to cheer, victory cheers, jumping, various emotions of frustration), so as to achieve program effects. Among them, the special effect animation can be specifically referred to as shown in Figure 15

[0125] ​In an example embodiment, the display method of the live room can further include: in response to the terminal device sending a special effect generation request in response to the touch event of the live watching user acting on the virtual object being detected, obtaining a special effect identifier included in the special effect generation request; obtaining a second special effect animation corresponding to the special effect identifier; wherein the second special effect animation includes at least one of an original emotion animation, a custom emotion animation, a placard effect animation, a text slogan effect animation, and a team declaration effect animation; and synthesizing the second special effect animation and the original three-dimensional virtual image or the target three-dimensional virtual image of the virtual object to obtain a second special effect video stream, and pushing the second special effect video stream to the terminal device where the live watching user is located. That is, in a specific application process, the live watching user can also express his / her emotions or send text by touching the original three-dimensional virtual image or the target three-dimensional virtual image of the virtual object corresponding to the user identifier, and thus the interaction is realized; wherein, taking fan support as an example, the obtained effect diagram can be referred to as shown in Figure 16

[0126] It can be undoubtedly known that the display method of the live room described in the example embodiments of the present disclosure has at least the following advantages:

[0127] On the one hand, the display method of the live room described in the example embodiments of the present disclosure can complete the demand instruction issued by the user in real time through data analysis and rendering synthesis, that is, the live broadcast can be realized to realize the AR cloud audience effect, thereby enhancing the interactive experience of the live broadcast audience; at the same time, through the cloud audience system and the analysis process, the cloud audience system can receive the demand instruction issued by the user in real time after the live broadcast starts, and can receive the demand sent by multiple people at the same time;

[0128] On the other hand, through UE data analysis and rendering synthesis, the demand instruction issued by the user is completed in real time, the UE engine accepts data to be read by the cloud audience system, and the player's barrage, ID, and other data are read in real time to be generated in the virtual live room, that is, the AR cloud audience can realize the effect change of multiple forms; and through the AR special effect video stream generation and push stream process, the UE background server can generate the above-mentioned special effect change video stream and send it to the user client, thereby realizing the interaction between the user and the live broadcast content.

[0129] ​In still another aspect, the problem that the prior art technology can only present virtual audiences through video recording or pictures when there is no audience for live broadcast, and it is impossible to show stereoscopic audiences through cameras and to realize real-time interaction between special effects roles and the needs sent by the online audience during the competition, and the best effect of live broadcast cannot be met, can be solved. The display method of the live broadcast room disclosed in the example embodiments of the present disclosure does not need artificial control, and can trigger multiple special effects at the same time, and can carry out special effects triggered by countless audiences on the stage at the same time, as long as the cloud audience system is in a running state, thereby greatly increasing the effectiveness of interaction.

[0130] Further, the problem that the prior art technology cannot realize the problem that the effect of live broadcast of large-scale galas and e-sports competitions needs to be very lively, and the cooperation of links needs to be improved to enhance the atmosphere of live broadcast, which cannot be realized, is avoided. The display method of the live broadcast room disclosed in the example embodiments of the present disclosure has great advantages through the AR cloud audience system, and can complete online and offline live broadcast AR special effect linkage play. The biggest advantage is that the cloud audience system is developed based on the live broadcast platform combined with AR technology, so that VIP users and / or ordinary users have their own roles and IDs to interact online. The system can realize that the online VIP users send instructions in the live broadcast room to trigger the offline AR cool special effects, so as to quickly pull the offline VIP users and the atmosphere of the online live broadcast room, and is suitable for triggering functions such as team PK during the competition, so as to improve the viewing experience, participation experience and interaction experience of online users. In addition, the AR cloud audience system is combined with the offline stage special effects to produce a shocking visual effect.

[0131] Further, in the display method of the live broadcast room disclosed in the example embodiments of the present disclosure, after receiving the online audience data, the data is sent to the UE background server. After receiving the trigger of the online audience, the UE background server binds and synthesizes a large number of virtual crowds according to a preset special effect template, and the virtual crowds can be automatically arranged in position, matched in size, and moved through data programming. At the same time, the UE background server can generate each frame of the synthesized picture into a video stream branch in real time, and send it to the user side live broadcast platform background server. The user side live broadcast platform background server sends the video stream to the corresponding user client, and completes the display of the AR cloud audience effect.

[0132] The example embodiments of the present disclosure also provide a display device of a live broadcast room. Specifically, referring to FIG. 17, the display device of the live broadcast room can include a live stage data processing module 1710, a synthesis template matching module 1720, a virtual audience map synthesis module 1730, and a synthesized video stream pushing module 1740. Wherein:

[0133] The live stage data processing module 1710 can be configured to collect live stage data of a target live broadcast room, and process the live stage data to obtain arrangement rules of audience seats included in the live stage data.

[0134] The synthesis template matching module 1720 can be configured to match, based on the arrangement rules, a synthesis template of a virtual audience sticker corresponding to the audience seats for the target live broadcast room from a preset sticker database.

[0135] The virtual audience sticker synthesis module 1730 can be configured to synthesize a virtual audience sticker based on a preset original three-dimensional virtual image of a virtual object and the synthesis template.

[0136] The synthesized video stream pushing module 1740 can be configured to synthesize the virtual audience sticker and a position of the audience seats in the target live broadcast room to obtain a synthesized video stream when it is detected that the target live broadcast room starts live streaming, and push the synthesized video stream to a terminal device to display a live broadcast room picture in the synthesized video stream on the terminal device.

[0137] In an example embodiment of the present disclosure, the audience seats include audience seats located in a same preset area, or audience seats located in multiple different preset areas.

[0138] In an example embodiment of the present disclosure, when the audience seats are audience seats located in a same preset area, processing the live stage data to obtain the arrangement rules of the audience seats included in the live stage data includes:

[0139] performing image recognition on the live stage data to obtain a number of rows, a number of columns of the audience seats included in the live stage data, and a position coordinate point of each seat in the target live broadcast room;

[0140] calculating a horizontal position interval between two horizontally adjacent seats and a vertical position interval between two vertically adjacent seats according to the position coordinate point of each seat in the target live broadcast room;

[0141] obtaining the arrangement rules of the audience seats included in the live stage data according to the number of rows, the number of columns, the horizontal position interval, and the vertical position interval of the audience seats.

[0142] In an example embodiment of the present disclosure, when the audience seats are audience seats located in multiple different preset areas, processing the live stage data to obtain the arrangement rules of the audience seats included in the live stage data includes:

[0143] perform image recognition on the live stage data to obtain a number of areas of the audience seats included in the live stage data, and a number of rows and columns of the audience seats included in each area, and a position coordinate point of each seat of the audience seats included in the area in the target live room;

[0144] calculate a horizontal position interval between two horizontally adjacent seats and a vertical position interval between two vertically adjacent seats in the area according to the position coordinate point of each seat of the area in the target live room;

[0145] obtain a sub-permutation rule of the audience seats of each area according to the number of rows and columns of the audience seats of each area and the horizontal position interval and the vertical position interval, and obtain a permutation rule of the audience seats included in the live stage data according to the sub-permutation rule of the audience seats of each area.

[0146] In an example embodiment of the present disclosure, synthesizing a virtual audience map based on a preset original three-dimensional virtual image of a virtual object and the synthesis template includes:

[0147] calculating a number of images of the virtual object required to be pulled according to the number of meshes of the synthesis template, and pulling original three-dimensional virtual images corresponding to the number of images from a preset resource library;

[0148] translating the original three-dimensional virtual images to obtain a role code corresponding to the original three-dimensional virtual images, and sorting the role code to obtain an array code;

[0149] synthesizing the array code into the synthesis template to obtain the virtual audience map.

[0150] In an example embodiment of the present disclosure, the display device of the live room further includes:

[0151] The three-dimensional virtual image adjustment module can be used to adjust the size of the original three-dimensional virtual image according to the size of the mesh of the synthesis template to obtain a target three-dimensional virtual image.

[0152] In an example embodiment of the present disclosure, synthesizing a synthesis video stream by synthesizing the virtual audience map and a position of the audience seats in the target live room includes:

[0153] allocating a mesh to each seat of the audience seats according to a corresponding relationship between each seat of the audience seats and each mesh of the virtual audience map;

[0154] acquire a code possessed by a grid to which each seat included in the audience seat is assigned, and acquire an original three-dimensional virtual image or a target three-dimensional virtual image possessed by a virtual object corresponding to the code;

[0155] synthesize the original three-dimensional virtual image or the target three-dimensional virtual image possessed by each seat included in the audience seat and a position of the seat, to obtain the synthesized video stream.

[0156] In an example embodiment of the present disclosure, the display device of the live room further includes:

[0157] The user attribute information acquisition module can be configured to, in response to a live room entry request sent by the terminal device when detecting that the live viewing user enters the target live room, acquire a user identifier included in the live room entry request, and acquire user attribute information according to the user identifier;

[0158] The user position allocation module can be configured to determine a user category of the live viewing user according to the user attribute information, and allocate a user position for the live viewing user in the audience seat according to the user category and an entry order of the live viewing user entering the target live room.

[0159] The cloud audience video stream pushing module can be configured to synthesize the user position and an original three-dimensional virtual image or a target three-dimensional virtual image of a virtual object corresponding to the user position, to obtain a cloud audience video stream corresponding to the live viewing user, and push the cloud audience video stream to a terminal device where the live viewing user is located, so that the live viewing user performs live interaction with the target live room through the virtual object.

[0160] In an example embodiment of the present disclosure, the display device of the live room further includes:

[0161] The special effect animation acquisition module can be configured to acquire live content and a live scene of the target live room being live broadcasted, and acquire a first special effect animation corresponding to the live content and the live scene according to the live content and the live scene.

[0162] The first special effect video stream pushing module can be configured to synthesize the original three-dimensional virtual image or the target three-dimensional virtual image of all virtual objects included in the target live room and the first special effect animation, to obtain a first special effect video stream, and push the first special effect video stream to the terminal device.

[0163] In an example embodiment of the present disclosure, the user category includes a member user category and / or a general user category; and the audience seat includes a member audience seat and / or a general audience seat.

[0164] wherein, according to the user category and the entering order of the live viewing user into the target live room, a user position is assigned to the live viewing user in the audience seat, comprising:

[0165] when the user category of the live viewing user is the member user category, a user position is assigned to the live viewing user from the member audience seat in a random assignment manner according to the entering order of the live viewing user into the target live room;

[0166] when the user category of the live viewing user is the ordinary user category, a user position is assigned to the live viewing user from the ordinary audience seat in a sequential assignment manner according to the entering order of the live viewing user into the target live room.

[0167] In an exemplary embodiment of the present disclosure, the display device of the live room further comprises:

[0168] The special effect identifier acquisition module can be configured to acquire a special effect identifier included in a special effect generation request sent by the terminal device in response to a touch event of the live viewing user acting on the virtual object being detected;

[0169] The second special effect animation acquisition module can be configured to acquire a second special effect animation corresponding to the special effect identifier; wherein the second special effect animation comprises at least one of an original emotion animation, a custom emotion animation, a lantern effect animation, a text slogan effect animation, and a team declaration effect animation;

[0170] The second special effect video stream pushing module can be configured to synthesize the second special effect animation and an original three-dimensional virtual image or a target three-dimensional virtual image of the virtual object to obtain a second special effect video stream, and push the second special effect video stream to the terminal device where the live viewing user is located.

[0171] In an exemplary embodiment of the present disclosure, the display device of the live room further comprises:

[0172] The original grid resource testing module can be configured to create a plurality of original grid resources with different numbers of rows, different numbers of columns, and different position intervals, and test the display effect of the original grid resources;

[0173] The synthesis template storage module can be configured to, after determining that the display effect testing of the original grid resources is completed, store the original grid resources after the testing as a synthesis template, and store the synthesis template into the preset texture database.

[0174] In an example embodiment of the present disclosure, the display effect includes a light display effect and / or a display effect of a placement position of an original three-dimensional virtual image or a target three-dimensional virtual image of a virtual object in the original mesh resource.

[0175] The specific details of the modules in the display device of the live room have been described in detail in the display method of the corresponding live room, and therefore will not be described here.

[0176] It should be noted that, although several modules or units of the device for action execution are mentioned in the foregoing detailed description, such division is not mandatory. Indeed, according to embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into several modules or units embodied.

[0177] In addition, although the steps of the method in the present disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired result. Additionally or alternatively, some steps can be omitted, several steps can be combined into one step, and / or one step can be divided into several steps, etc.

[0178] In an example embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided.

[0179] Those skilled in the art can understand that various aspects of the present disclosure can be implemented as a system, a method or a program product. Therefore, various aspects of the present disclosure can be embodied as a complete hardware embodiment, a complete software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects, which can be collectively referred to as "circuitry", "module" or "system" here.

[0180] The electronic device 1800 according to this embodiment of the present disclosure will be described below with reference to Figure 18 Figure 18 The display electronic device 1800 is merely an example and should not impose any limitation on the function and use range of the embodiments of the present disclosure.

[0181] As shown in Figure 18 The electronic device 1800 is in the form of a general computing device. The components of the electronic device 1800 can include, but are not limited to, the at least one processing unit 1810 described above, the at least one storage unit 1820 described above, a bus 1830 connecting different system components (including the storage unit 1820 and the processing unit 1810), and a display unit 1840.​

[0182] The storage unit stores program codes which can be executed by the processing unit 1810, so that the processing unit 1810 performs the steps according to various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of the present specification. For example, the processing unit 1810 can perform the steps shown in FIG. 11 as follows. Figure 1 Step S110: collecting live stage data of a target live room and processing the live stage data to obtain arrangement rules of audience seats included in the live stage data; Step S120: based on the arrangement rules, matching a synthesis template of a virtual audience sticker corresponding to the audience seats for the target live room from a preset sticker database; Step S130: synthesizing a virtual audience sticker based on a preset original three-dimensional virtual image of a virtual object and the synthesis template; Step S140: when it is detected that the target live room starts live streaming, synthesizing the virtual audience sticker and a position where the audience seats in the target live room are located to obtain a synthesis video stream, and pushing the synthesis video stream to a terminal device to display a live room picture in the synthesis video stream on the terminal device.

[0183] The storage unit 1820 can include a readable medium in the form of a volatile storage unit, such as a random access memory (RAM) 18201 and / or a cache memory unit 18202, and can further include a read-only memory (ROM) 18203.

[0184] The storage unit 1820 can further include program / utilities 18204 having a set of (at least one) program modules 18205, such as an operating system, one or more application programs, other program modules, and program data, and each of these examples, or some combination thereof, can include implementation of a network environment.

[0185] The bus 1830 can represent one or more of several types of bus structures, including a storage unit bus or storage unit controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of a variety of bus structures.

[0186] The electronic device 1800 can also communicate with one or more external devices 1900 such as a keyboard, a pointing device, a Bluetooth device, etc.; and one or more devices that enable a user to interact with the electronic device 1800; and / or one or more devices (e.g. routers, modems, etc.) that enable the electronic device 1800 to communicate with one or more other computing devices. Such communication can be enabled by an input / output (I / O) interface 1850. Further, the electronic device 1800 can communicate with one or more networks, such as a local area network (LAN), a wide area network (WAN), and / or the public network, e.g. the Internet, through a network adapter 1860. As depicted, the network adapter 1860 is in communication with the other modules of the electronic device 1800 through the bus 1830. As will be appreciated by persons skilled in the art, a computer can also include other modules that can or can not be shared with other computing devices. For example, as illustrated in Figure 18, the electronic device 1800 also includes an interface bus 1870 for permitting data exchange between the above-described bus 1830, the I / O interface 1850, and the network adapter 1860.

[0187] From the above description of the embodiments, those skilled in the art will readily appreciate that the example embodiments described herein can be implemented by software and / or by hardware. Accordingly, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product. The software product can be stored in a non-volatile storage medium (e.g., a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or a network, and includes a number of instructions for causing a computing device (e.g., a personal computer, a server, a terminal device, or a network device) to perform the methods according to the embodiments of the present disclosure.

[0188] In the example embodiments of the present disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the above-described methods of the present disclosure is stored. In some possible embodiments, various aspects of the present disclosure can also be implemented in the form of a program product, which includes program codes for causing a terminal device to perform the steps described in the above “Example Methods” section according to various example embodiments of the present disclosure when the program product is run on the terminal device.

[0189] The program product for implementing the above-described methods according to the embodiments of the present disclosure can take the form of a portable compact disc read-only memory (CD-ROM) and include program codes, and can be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited to this, and in this document, a readable storage medium can be any tangible medium that contains or stores a program, which can be used by or in conjunction with an instruction execution system, device, or apparatus.

[0190] The program product can employ any combination of one or more computer-readable media. The computer-readable media can be a computer-readable storage medium or a computer-readable signal medium. The computer-readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0191] The computer-readable signal medium can include a computer-readable storage medium that is communicated, propagated, or transported, for example, over a communication link, a wireless link, or a hard-wired link. The computer-readable signal medium can also be a computer-readable storage medium that is embodied into a computer-readable storage medium or used to manufacture a computer-readable storage medium. The computer-readable storage medium can be any appropriate medium (including the one or more computer-readable media described above) that participates in providing instructions to an instruction execution system, apparatus, or device such that the instructions, which properly render the instruction-execution system, apparatus, or device into a

[0192] The program code embodied on the computer-readable media can be transmitted using any appropriate medium, including but not limited to wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0193] The program code can be executed by one or more programmable processors, which can be individual or grouped processors, to perform the methods disclosed herein. The program code can execute entirely on the user's computing device, partly on the user's computing device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider.

[0194] In addition, the above-described flowcharts are merely illustrative examples of the processes included in the methods of the example embodiments of the present disclosure, and are not intended to limit the purposes. It is easily understood that the processes shown in the above-described flowcharts do not indicate or limit the time sequence of the processes. In addition, it is easily understood that the processes can be executed synchronously or asynchronously, for example, in a plurality of modules.

[0195] Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the features of the disclosure as set forth above. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

Claims

1. A live room display method, characterized in that, The method comprises the following steps: Collecting live stage data of a target live broadcast room and processing the live stage data to obtain arrangement rules of audience seats included in the live stage data; Based on the arrangement rules, a synthesis template of a virtual audience sticker corresponding to the audience seats is matched for the target live broadcast room from a preset sticker database; wherein the synthesis template is obtained by creating a plurality of original grid resources with different numbers of rows, different numbers of columns and different position intervals, and testing the display effect of the original grid resources; after determining that the display effect test of the original grid resources is completed, the original grid resources after the test are taken as the synthesis template; Based on a preset original three-dimensional virtual image of a virtual object and the synthesis template, a virtual audience sticker is synthesized; wherein the virtual audience sticker is synthesized by the following method: according to the number of grids possessed by the synthesis template, the number of images of virtual objects required to be pulled is calculated, and the original three-dimensional virtual image corresponding to the number of images is pulled from a preset resource library; the original three-dimensional virtual image is translated to obtain a role code corresponding to the original three-dimensional virtual image, and the role code is sorted to obtain an array code; the array code is synthesized into the synthesis template to obtain the virtual audience sticker; When it is detected that the target live broadcast room starts live broadcast, each seat included in the audience seats and each grid included in the virtual audience sticker are assigned a grid according to the corresponding relationship between them; the code possessed by the grid to which each seat included in the audience seats is assigned is obtained, and the original three-dimensional virtual image or target three-dimensional virtual image possessed by the virtual object corresponding to the code is obtained; the original three-dimensional virtual image or target three-dimensional virtual image possessed by each seat included in the audience seats and the position of the seat are synthesized to obtain a synthesis video stream, and the synthesis video stream is pushed to a terminal device to display the live broadcast room picture in the synthesis video stream on the terminal device.

2. The live room display method of claim 1, wherein, The audience seats include audience seats located in the same preset area or audience seats located in a plurality of different preset areas.

3. The live room display method of claim 2, wherein, When the audience seats are audience seats located in the same preset area, processing the live stage data to obtain the arrangement rules of the audience seats included in the live stage data comprises: Performing image recognition on the live stage data to obtain the number of rows, the number of columns of the audience seats included in the live stage data, and the position coordinate points of each seat in the target live broadcast room; According to the position coordinate points of each seat in the target live broadcast room, the horizontal position interval between two horizontally adjacent seats and the vertical position interval between two vertically adjacent seats are calculated; According to the number of rows, the number of columns, the horizontal position interval and the vertical position interval of the audience seats, the arrangement rules of the audience seats included in the live stage data are obtained.

4. The live room display method of claim 2, wherein, When the audience seats are audience seats located in different preset areas, the live stage data is processed to obtain the arrangement rule of the audience seats included in the live stage data, including: The live stage data is subjected to image recognition to obtain the number of areas of the audience seats included in the live stage data, and the number of rows and columns of the audience seats included in each area, and the position coordinate points of each seat included in the audience seats in the target live room; According to the position coordinate points of each seat included in each area in the target live room, the horizontal position interval between two horizontally adjacent seats in the area and the vertical position interval between two vertically adjacent seats are calculated; According to the number of rows and columns of the audience seats included in each area and the horizontal position interval and the vertical position interval, the sub-arrangement rule of the audience seats in each area is obtained, and the arrangement rule of the audience seats included in the live stage data is obtained according to the sub-arrangement rule of the audience seats in each area.

5. The live room display method of claim 1, wherein, The display method of the live room further includes: According to the size of the grid of the synthesis template, the size of the original three-dimensional virtual image is adjusted to obtain a target three-dimensional virtual image.

6. The live room display method of claim 1, wherein, After rendering the virtual audience map to the position of the audience seat in the target live room, the display method of the live room further includes: In response to a live room entry request sent by a terminal device when detecting that a live viewing user enters a target live room, a user identifier included in the live room entry request is obtained, and user attribute information is obtained according to the user identifier; According to the user attribute information, the user category of the live viewing user is determined, and the user position of the live viewing user in the audience seat is allocated according to the user category and the entering order of the live viewing user into the target live room; The user position and the original three-dimensional virtual image or the target three-dimensional virtual image of the virtual object corresponding to the user position are synthesized to obtain a cloud audience video stream corresponding to the live viewing user; The cloud audience video stream is pushed to the terminal device where the live viewing user is located, so that the live viewing user performs live interaction with the target live room through the virtual object.

7. The live room display method of claim 6, wherein, The display method of the live room further includes: Obtaining live content and a live scene being live broadcasted in the target live room, and obtaining a first special effect animation corresponding to the live content and the live scene according to the live content and the live scene; The original three-dimensional virtual image or the target three-dimensional virtual image of all virtual objects included in the target live room and the first special effect animation are synthesized to obtain a first special effect video stream, and the first special effect video stream is pushed to a terminal device.

8. The live room display method of claim 6, wherein, The user category includes a member user category and / or a general user category; the audience seat includes a member audience seat and / or a general audience seat; The user position is allocated for the live watching user in the audience seat according to the user category and the entering order of the live watching user into the target live room, and includes: When the user category of the live watching user is the member user category, the user position is allocated for the live watching user from the member audience seat in a random allocation manner according to the entering order of the live watching user into the target live room; When the user category of the live watching user is the ordinary user category, the user position is allocated for the live watching user from the ordinary audience seat in a sequential allocation manner according to the entering order of the live watching user into the target live room.

9. The live room display method of claim 7, wherein, The display method of the live room further includes: In response to a special effect generation request sent by the terminal device in detecting a touch event of the live watching user acting on the virtual object, a special effect identifier included in the special effect generation request is acquired; A second special effect animation corresponding to the special effect identifier is acquired; wherein the second special effect animation includes at least one of an original emotion animation, a custom emotion animation, a placard effect animation, a text slogan effect animation, and a team declaration effect animation; The second special effect animation and the original three-dimensional virtual image or the target three-dimensional virtual image of the virtual object are synthesized to obtain a second special effect video stream, and the second special effect video stream is pushed to the terminal device where the live watching user is located.

10. The live room display method according to any one of claims 1-9, characterized in that, The display method of the live room further includes: The synthesis template is stored in the preset map database.

11. The live room display method of claim 1, wherein, The display effect includes a light display effect and / or a display effect of a placement position of the original three-dimensional virtual image or the target three-dimensional virtual image of the virtual object in the original grid resource.

12. A live room display device, characterized in that, It includes: A live stage data processing module is configured to collect live stage data of a target live room and process the live stage data to obtain an arrangement rule of an audience seat included in the live stage data; A synthesis template matching module is configured to match a synthesis template of a virtual audience map corresponding to the audience seat for the target live room from a preset map database based on the arrangement rule; wherein the synthesis template is obtained by creating a plurality of original grid resources with different numbers of rows, different numbers of columns, and different position intervals, and testing display effects of the original grid resources; after determining that the display effect testing of the original grid resources is completed, the original grid resources after testing are taken as the synthesis template; The virtual audience sticker synthesis module is configured to synthesize a virtual audience sticker based on a preset original three-dimensional virtual image of a virtual object and the synthesis template. The virtual audience sticker is synthesized by calculating the number of images of the virtual object to be pulled according to the number of grids of the synthesis template, and pulling the original three-dimensional virtual image corresponding to the number of images from a preset resource library. The original three-dimensional virtual image is translated to obtain a role code corresponding to the original three-dimensional virtual image, and the role code is sorted to obtain an array code. The array code is synthesized into the synthesis template to obtain the virtual audience sticker. The synthesized video stream pushing module is configured to, when detecting that the target live broadcast room starts live broadcast, allocate a grid to each seat in the audience seat based on a correspondence between each seat in the audience seat and each grid in the virtual audience sticker. The synthesized video stream pushing module is further configured to obtain a code possessed by the grid allocated to each seat in the audience seat, and obtain an original three-dimensional virtual image or a target three-dimensional virtual image possessed by a virtual object corresponding to the code. The synthesized video stream pushing module is further configured to synthesize the original three-dimensional virtual image or the target three-dimensional virtual image possessed by each seat in the audience seat and a position of the seat to obtain a synthesized video stream, and push the synthesized video stream to a terminal device to display a live broadcast room picture in the synthesized video stream on the terminal device.

13. A computer readable storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the live broadcast room display method of any one of claims 1-11.

14. An electronic device, comprising: The computer program is executed by the processor to implement the live broadcast room display method of any one of claims 1-11. The computer program is executed by the processor to implement the live broadcast room display method of any one of claims 1-11. The computer program is executed by the processor to implement the live broadcast room display method of any one of claims 1-11. The computer program is executed by the processor to implement the live broadcast room display method of any one of claims 1-11. The computer program is executed by the processor to implement the live broadcast room display method of any one of claims 1-11. The computer program is executed by the processor to implement the live broadcast room display method of any one of claims 1-11. The computer program is executed by the processor to implement the live broadcast room display method of any one of claims 1-11. The computer program is executed by the processor to implement the live broadcast room display method of any one of claims 1-11. The computer program is executed by the processor to implement the live broadcast room display method of any one of claims 1-11. The computer program is executed by the processor to implement the live broadcast room display method of any one of claims 1-11. The computer program is executed by the processor to implement the live broadcast room display method of any one of claims 1-11. The computer program is executed by the processor to implement the live broadcast room display method of any one of claims 1-11. The computer program is executed by the processor to implement the live broadcast room display method of any one of claims 1-11. The computer program is executed by the processor to implement the live broadcast room display method of any one of claims 1-11. The computer program is executed by the processor to implement the live broadcast room display method of any one of claims 1-11. The computer program is executed by the processor to implement the live broadcast room display method of any one of claims 1-11. The computer program is executed by the processor to implement the live broadcast room display method of any one of claims 1-11. The computer program is executed by the processor to implement the live broadcast room display method of any one of claims 1-11. The computer program is executed by the processor to implement the live broadcast room display method of any one of claims 1-11. The computer program is executed by the processor to implement the live broadcast room display method of any one of claims 1-11. The computer program is executed by the processor to implement the live broadcast room display method of any one of claims 1-11. The computer program is executed by the processor to implement the live broadcast room display method of any one of claims 1-11. The computer program is executed by the processor to implement the live broadcast room display method of any one of claims 1-11. The computer program is executed by the processor to implement the live broadcast room display method of any one of claims 1-11. The computer program is executed by the processor to implement the live broadcast room display method of any one of claims 1-11. The computer program

Images