Live streaming editing method and device

Abstract

The embodiment of the present application discloses a method and device for editing a live stream. A specific implementation of the method includes: obtaining a camera live stream and a demonstration file selected by a user, wherein the camera live stream includes live texture information, parsing the demonstration file, obtaining a demonstration picture set, and presenting the demonstration picture set to the user, in response to receiving a user's selection operation for a demonstration picture in the demonstration picture set, generating demonstration texture information of the selected demonstration picture, and obtaining an edited live stream based on the demonstration texture information and / or the live texture information. This implementation can parse the demonstration file into a demonstration picture for the user to select, and after the user determines the demonstration picture, use the camera live stream to send the demonstration picture, while providing support for the user's presentation process, without adding additional transmission code streams.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a method and device for editing a live stream. Background Art

[0002] Live streaming has become an indispensable part of people's lives. The threshold for live streaming has been lowered, and you only need a mobile smartphone to achieve live streaming. When the anchor is talking about a logically complex topic or idea through a mobile device and the camera is turned on, in order to achieve better results, it is usually necessary to use a demonstration file to explain it.

[0003] Currently, in order to show the content of a presentation file, the anchor can usually only show it to the audience by taking a screenshot of the interface, or by sending the presentation file to a designated terminal in advance. Summary of the invention

[0004] The embodiments of the present application provide a method and device for editing a live stream.

[0005] In a first aspect, an embodiment of the present application provides a method for editing a live stream, comprising: obtaining a camera live stream and a demonstration file selected by a user, wherein the camera live stream includes live texture information; parsing the demonstration file to obtain a demonstration picture set, and presenting the demonstration picture set to the user; in response to receiving a user selection operation on a demonstration picture in the demonstration picture set, generating demonstration texture information of the selected demonstration picture; and obtaining an edited live stream based on the demonstration texture information and / or the live texture information.

[0006] In some embodiments, parsing a demonstration file to obtain a demonstration picture set and presenting the demonstration picture set to a user includes: obtaining operating system information of a current device and determining a parsing method for the demonstration file based on the operating system information; parsing the demonstration file using the parsing method to obtain a demonstration picture set and presenting the demonstration picture set to the user.

[0007] In some embodiments, the operating system information of the current device is obtained, and a parsing method for the demonstration file is determined based on the operating system information, including: in response to determining that the operating system information is an Android system, determining that the parsing method is an HSLF module parsing method; and using the parsing method to parse the demonstration file to obtain a demonstration picture set, and presenting the demonstration picture set to the user, including: calling an interface of the HSLF parsing module in the poi library to parse the demonstration file into multiple demonstration pictures.

[0008] In some embodiments, the operating system information of the current device is obtained, and a parsing method for the presentation file is determined based on the operating system information, including: in response to determining that the operating system information is an IOS system, determining that the parsing method is a webview control parsing method; and using the parsing method to parse the presentation file to obtain a presentation picture set, and presenting the presentation picture set to the user, including: using a webview control to load the presentation file, calling a JavaScript content parsing interface in the webview control, and parsing page information of the presentation file; obtaining a parsing result of the page information, calling a drawing module interface in the webview control, and drawing the parsing result into a UIImage drawing component; and using the UIImage drawing component to draw multiple presentation file pictures.

[0009] In some embodiments, obtaining a set of presentation pictures includes: continuously adding serial number tags according to the generation order of the presentation pictures, and determining the presentation picture set based on the presentation pictures continuously having the serial number tags.

[0010] In some embodiments, the steps of obtaining a demonstration image selected by a selection operation and generating demonstration texture information of the selected demonstration image include: obtaining the selected demonstration image; setting the vertex coordinates and texture coordinates of the selected demonstration image, determining the size information of the selected demonstration image, and determining a transformation matrix based on the size information; based on the determination result of the transformation matrix, calling a texture generation module interface of an OpenGL ES component, and generating the demonstration texture information based on predetermined texture type information.

[0011] In some embodiments, live texture information in a camera live stream is obtained, and a live stream is generated based on image texture information and / or the live texture information to obtain an edited live stream, including: in response to receiving a fused live stream signal, incorporating the live texture information at a preset position in the demonstration texture information; and using the fused texture information to generate a live stream to obtain an edited live stream.

[0012] In some embodiments, live texture information in the camera live stream is obtained, and a live stream is generated based on the image texture information and / or the live texture information to obtain an edited live stream, including: in response to receiving a separate live stream signal, using the demonstration texture information alone to generate a live stream to obtain an edited live stream.

[0013] In a second aspect, an embodiment of the present application provides a live stream editing device, comprising: a live stream and demonstration file acquisition unit, configured to acquire a camera live stream and a demonstration file selected by a user; a target image determination unit, configured to parse the demonstration file, obtain a demonstration image set, and present the demonstration image set to the user; a texture information generation unit, configured to generate demonstration texture information of the selected demonstration image in response to receiving a user's selection operation on a demonstration image in the demonstration image set; a live stream editing unit, configured to obtain an edited live stream based on the demonstration texture information and / or the live texture information.

[0014] In some embodiments, the target image determination unit includes a demonstration file parsing subunit, which is configured to obtain operating system information of the current device and determine a parsing method for the demonstration file based on the operating system information; parse the demonstration file using the parsing method to obtain a demonstration image set; and a picture display subunit, which is configured to present the demonstration image set to the user.

[0015] In some embodiments, the parsing subunit file parsing unit is further configured to determine that the operating system information is an Android system, determine that the parsing method is an hslf module parsing method; call the interface of the hslf parsing module in the poi library to parse the demonstration file into multiple demonstration pictures.

[0016] In some embodiments, the parsing sub-unit file parsing unit is further configured to, in response to determining that the operating system information is an IOS system, determine that the parsing method is a webview control parsing method; use the webview control to load the presentation file, call the JavaScript content parsing interface in the webview control, and parse the page information of the presentation file; obtain the parsing result of the page information, call the drawing module interface in the webview control, and draw the parsing result into a UIImage drawing component; and use the UIImage drawing component to draw multiple presentation file pictures.

[0017] In some embodiments, the target picture determination unit includes a sequence marking subunit configured to continuously add serial number marks according to the generation order of the presentation pictures, and determine a presentation picture set based on the presentation pictures continuously marked with serial numbers.

[0018] In some embodiments, the texture information generation unit is further configured to obtain a selected demonstration image; set the vertex coordinates and texture coordinates of the selected demonstration image, determine the size information of the selected demonstration image, and determine the transformation matrix based on the size information; based on the determination result of the transformation matrix, call the texture generation module interface of the OpenGL ES component, and generate the demonstration texture information based on predetermined texture type information.

[0019] In some embodiments, the live stream editing unit is further configured to, in response to receiving a fused live stream signal, incorporate the live texture information into a preset position in the demonstration texture information; and generate a live stream using the fused texture information to obtain an edited live stream.

[0020] In some embodiments, the live stream editing unit is further configured to generate a live stream using the demonstration texture information alone in response to receiving a separate live stream signal, thereby obtaining an edited live stream.

[0021] In a third aspect, an embodiment of the present application provides a computer device, comprising: one or more processors; a storage device on which one or more programs are stored; when the one or more programs are executed by one or more processors, the one or more processors implement the method described in any implementation method in the first aspect.

[0022] In a fourth aspect, an embodiment of the present application provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.

[0023] The live stream editing method and device provided in the embodiment of the present application obtains the live stream of the camera and the demonstration file selected by the user, parses the demonstration file, obtains a demonstration picture set, and presents the demonstration picture set to the user, and in response to receiving the user's selection operation of the demonstration picture in the demonstration picture set, obtains the edited live stream based on the demonstration picture selected by the selection operation and / or the live texture information in the live stream of the camera. This implementation method can parse the demonstration file into a demonstration picture for the user to select, and after the user determines the demonstration picture, the demonstration picture is sent using the live stream of the camera, while providing support for the user's presentation process without adding additional transmission code streams. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0025] Figure 1 is an exemplary system architecture to which some embodiments of the present application may be applied;

[0026] Figure 2 is a flowchart of a first embodiment of a live stream editing method according to the present application;

[0027] Figure 3 It is a flowchart of an implementation method for parsing a demonstration picture in a live stream editing method according to the present application;

[0028] Figure 4It is a flowchart of another implementation method for parsing a demonstration picture in the live stream editing method according to the present application;

[0029] Figure 5 is a flowchart of an implementation method for generating presentation texture information according to the live stream editing method of the present application;

[0030] Figure 6 is a flowchart of a second embodiment of a live stream editing method according to the present application;

[0031] Figure 7 This is a rendering of an application scenario of the live stream editing method according to the present application;

[0032] Figure 8 It is a structural diagram of a computer system of a computer device suitable for implementing some embodiments of the present application. DETAILED DESCRIPTION

[0033] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It should also be noted that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.

[0034] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0035] Figure 1 An exemplary system architecture 100 is shown to which an embodiment of the live stream editing method of the present application can be applied.

[0036] like Figure 1 As shown, the system architecture 100 may include devices 101, 102, 103, 104 and a network 105. The network 105 is used to provide a medium for communication links between the devices 101, 102, 103, 104. The network 105 may include various connection types, such as wired, wireless target communication links or optical fiber cables.

[0037] Devices 101, 102, 103, and 104 can be hardware devices or software that support network connection to provide various network services. When the device is hardware, it can be various electronic devices, including but not limited to smart phones, tablet computers, laptops, desktop computers, servers, etc. At this time, as a hardware device, it can be implemented as a distributed device group consisting of multiple devices, or it can be implemented as a single device. When the device is software, it can be installed in the electronic devices listed above. At this time, as software, it can be implemented as multiple software or software modules for providing distributed services, for example, or it can be implemented as a single software or software module. No specific limitation is made here.

[0038] In practice, a device can provide corresponding network services by installing corresponding client applications or server applications. After the client application is installed, the device can be embodied as a client in network communication. Correspondingly, after the server application is installed, it can be embodied as a server in network communication.

[0039] As an example, in Figure 1 In the embodiment, devices 101, 102, and 103 are embodied as clients on the live broadcast side, and device 104 is embodied as a server. Specifically, devices 101, 102, and 103 may be clients with live broadcast applications installed, and device 104 may be a server of the live broadcast application.

[0040] It should be noted that the live streaming editing method provided in the embodiment of the present application is usually executed by the devices 101, 102, and 103 embodied as the user end of the live streaming side, so that the editing of the live streaming can be completed directly after the live streaming and the presentation file selected by the user are obtained, thereby avoiding the waste of resources caused by multiple transmissions. Of course, the live streaming editing method provided in the embodiment of the present application is also executed by the device 104 embodied as the server end.

[0041] It should be understood that Figure 1 The number of networks and devices in the figure is only for illustration. Any number of networks and devices may be provided according to implementation requirements.

[0042] Continue to refer Figure 2 , which shows a process 200 of a first embodiment of a live stream editing method according to the present application. The live stream editing method may include the following steps:

[0043] Step 201, obtaining the camera live stream and the demonstration file selected by the user.

[0044] In this embodiment, the execution subject of the live stream editing method (eg Figure 1The devices 101, 102, and 103 shown in the figure can obtain the content of the live stream captured by the camera locally, and obtain the demonstration file selected by the user from the local or other non-local storage devices. The demonstration file can meet all the special storage needs in the actual application scenario. These files can be stored in other devices in a dispersed manner. The files in the storage device can be the original or the backup, which is not specifically limited here.

[0045] Among them, the above-mentioned execution entity is usually the device used by the live content provider user (for example, 101, 102, 103). These users perform live broadcasts through the above-mentioned execution entity, that is, generate a live stream containing live content and send it to the outside. Therefore, when these users have the need to use a presentation file for a lecture, the presentation file can be loaded on the first terminal. The presentation file can usually be a collection of pictures in jpeg (Joint Photographic Experts Group, jpg for short), png (Portable Network Graphics, png for short) format, ppt (Microsoft Office PowerPoint, ppt for short) file and other types of files.

[0046] In practice, the presentation file is usually a ppt file. In order to better give a lecture to the audience, the live content provider loads the presentation file locally, so that the executor of the live stream editing method can load the presentation file and generate a live stream based on the loading result and the live video content of the live content provider, and send it to the server for storage, or transmit it to other audience users.

[0047] It should be understood that the main purpose of the present application is to edit the live stream to obtain the edited live stream, but after obtaining the edited live stream, the server (e.g. 104) can record and save the content therein after receiving the live stream, and generate corresponding files, so that other devices can subsequently obtain these saved files after sending a request to the server and play the content therein.

[0048] It should be understood that in order to present the user's image information in the camera live stream, the camera live stream includes live texture information.

[0049] Step 202: parse the demonstration file to obtain a demonstration picture set, and present the demonstration picture set to the user.

[0050] In this embodiment, after the demonstration file is obtained based on the above step 201, the demonstration file is parsed to obtain a demonstration picture set. The parsing method can be selected according to the system operation information of the above execution subject. For example, when the above execution subject is in the Android system, the parsing tool in the Android system is called to implement the parsing of the demonstration file.

[0051] After parsing the content in the demonstration file into a plurality of pictures, these pictures can be saved to generate a set of demonstration pictures, and the set of demonstration pictures can be presented to the user so that the user can select the demonstration pictures to be used and transmitted according to the set of demonstration pictures.

[0052] It should be understood that other parsing methods can also be selected according to different purposes. For example, when you only want to obtain part of the content in the demonstration file, you can pre-set the image capture area, and then continuously obtain multiple demonstration pictures based on the image capture area during the process of reading the demonstration file, and generate a corresponding demonstration picture set.

[0053] In some embodiments, parsing a demonstration file to obtain a demonstration picture set and presenting the demonstration picture set to a user includes: obtaining operating system information of a current device and determining a parsing method for the demonstration file based on the operating system information; parsing the demonstration file using the parsing method to obtain a demonstration picture set and presenting the demonstration picture set to the user.

[0054] Specifically, a suitable parsing method can be selected based on the current device, that is, the operating system information of the above-mentioned execution subject. The operating system of the above-mentioned execution subject can be an Android system, an IOS system, a Windows operating system, and other operating systems. For example, when the operating system of the above-mentioned execution subject is an Android system, the jxl.jar parsing method, the extractors-0.4.jar parsing method, or the hslf module parsing method can be used to parse the demonstration file, so as to obtain the parsing tool in a targeted manner to parse the demonstration file and improve the parsing efficiency.

[0055] In some embodiments, the operating system information of the current device is obtained, and a parsing method for the demonstration file is determined based on the operating system information, including: in response to determining that the operating system information is an Android system, determining that the parsing method is an HSLF module parsing method; and using the parsing method to parse the demonstration file to obtain a demonstration picture set, and presenting the demonstration picture set to the user, including: calling an HSLF parsing module interface in a poi library to parse the demonstration file into multiple demonstration pictures.

[0056] Specifically, when it is determined that the operating system of the first terminal is the Android system, the HSLF module parsing method can be used to parse the presentation file. The HSLF parsing method refers to using the HSLF module in the POI library to implement a module for parsing files in the Microsoft Office format. The POI library is an open source function library of the Apache Software Foundation that provides an interface to Java programs, i.e., programs supported by the Android system, to implement a library that has the functions of reading and writing Microsoft Office format files. The presentation file can be parsed quickly and accurately by directly calling the HSLF module.

[0057] In some embodiments, the steps of parsing a presentation file using the hslf parsing method are as follows: Figure 3 As shown in the process 300, it includes:

[0058] Step 301, obtaining a demonstration file selected by a user.

[0059] Step 302: call the hslf parsing module interface in the poi library.

[0060] Step 303, determining whether the demonstration file can be parsed by the hslf module.

[0061] Specifically, because the program code in the hslf module usually parses files in Microsoft Office format, when the file format is not Microsoft Office format, or when the Microsoft Office format is damaged or cannot be opened normally, the hslf module cannot be used to complete the parsing work.

[0062] Step 304: in response to determining that the presentation file can be parsed by the hslf module, the file is parsed.

[0063] Specifically, when the demonstration file can be parsed by the hslf module, the parsing work is completed and multiple demonstration pictures are output.

[0064] Step 305: In response to determining that the demonstration file cannot be parsed by the hslf module, a corresponding error flag is output.

[0065] Specifically, when it is determined that the presentation file cannot be parsed by the hslf module, the reason why it cannot be parsed is determined, and a corresponding error mark is output. For example, when the format is not Microsoft Office format, a format error mark is output. When the file is damaged and cannot be opened, a corresponding damage mark is generated.

[0066] By using the hslf parsing method to parse the demonstration file when the operating system information is the Android system, the demonstration file can be quickly and accurately parsed into multiple demonstration images when the demonstration file format supports the hslf parsing method, so that the demonstration images can be retrieved later, and the corresponding identifier can be generated when the parsing fails and fed back to the user so that the user can decide on other parsing strategies later, thereby improving the parsing efficiency of the demonstration file.

[0067] In some embodiments, the operating system information of the current device is obtained, and a parsing method for the presentation file is determined based on the operating system information, including: in response to determining that the operating system information is an IOS system, determining that the parsing method is a webview control parsing method; and using the parsing method to parse the presentation file to obtain a presentation picture set, and presenting the presentation picture set to the user, including: using a webview control to load the presentation file, calling a JavaScript content parsing interface in the webview control, and parsing the page information of the presentation file; obtaining the parsing result of the page information, calling a drawing module interface in the webview control, and drawing the parsing result into a UIImage drawing component; and using the UIImage drawing component to draw multiple presentation file pictures.

[0068] Specifically, webview is a loading control that can be called based on the IOS system, namely, "network view", which can load and display web pages and can be regarded as a browser. It is mainly used to display the content after the network request, that is, to display the content of the network address request in it. Through this control, the content of the demonstration file can be parsed, and the parsing result can be used by other applications in the IOS system. Therefore, the webview control parsing method is used to parse when the operating system information of the first terminal is the IOS system. Under the premise of ensuring the parsing efficiency, it is guaranteed that the obtained parsing can be used by the first terminal device.

[0069] In some embodiments, the specific steps of parsing the presentation file using the webview control parsing method are as follows: Figure 4 As shown in the process 400, it includes:

[0070] Step 401, using the webview control to load the demonstration file, and determining whether the demonstration file can be parsed by the webview control parsing method.

[0071] Specifically, because the webview control is also not compatible with all formats of presentation files, it is also necessary to determine whether the presentation file can be loaded by the webview control.

[0072] Step 402: In response to determining that the presentation file cannot be parsed by the webview module, a corresponding error indicator is output.

[0073] Specifically, when it is determined that the presentation file cannot be parsed by the webview module, a corresponding identifier can be generated based on different failure reasons, so as to adjust the strategy according to the identifier.

[0074] Step 403, in response to determining that the presentation file can be parsed by the webview control, calling the JavaScript content parsing interface in the webview control to parse the file and obtain the parsing result of the page information.

[0075] Specifically, after determining that the presentation file can be parsed by the webview control parsing method, the JavaScript content parsing interface in the webview control is called to parse the size information and page quantity information of the individual pages in the presentation file, wherein the size information includes the length, width, and height data of the page.

[0076] Step 404, calling the RenderInContext interface of the layer layer in the webview control, and sending the parsing result to the UIImage drawing component.

[0077] Specifically, since the IOS system has its own UIImage drawing component, and the image information generated by the UIImage drawing component can be well supported by the IOS system, after obtaining the necessary parameters for drawing using the UIImage drawing component, the UIImage drawing component is used to draw the image to obtain multiple demonstration file images.

[0078] Step 405, using the UIImage drawing component to draw multiple presentation file images.

[0079] By using the webview control parsing method to parse the demonstration file when the operating system information is the IOS system, the demonstration file can be quickly and accurately parsed into multiple demonstration pictures when the demonstration file format supports the webview control parsing method, so that the demonstration pictures can be retrieved later. In addition, when the parsing fails, the corresponding identifier can be generated and fed back to the user so that the user can decide on other parsing strategies later, thereby improving the parsing efficiency of the demonstration file.

[0080] Step 203 , in response to receiving a user's selection operation on a demonstration picture in the demonstration picture set, generating demonstration texture information of the selected demonstration picture.

[0081] In this embodiment, after receiving the user's selection operation on a demonstration picture in the demonstration picture set, that is, when the user selects the demonstration picture he wants to use from the demonstration picture set, the selected picture is obtained, and the demonstration texture information of the selected demonstration picture is generated to facilitate the subsequent editing of the camera live stream according to the demonstration texture information.

[0082] In some embodiments, the step of generating demonstration texture information of a selected demonstration picture includes: obtaining the selected demonstration picture; setting the vertex coordinates and texture coordinates of the selected demonstration picture, and determining the size information of the selected demonstration picture; determining a transformation matrix based on the size information; based on the determination result of the transformation matrix, calling a texture generation module interface of an OpenGL ES component, and generating the demonstration texture information based on predetermined texture type information.

[0083] Specifically, Figure 5 As shown in the process 500 in FIG. 1 , it includes:

[0084] Step 501, obtain the selected demonstration picture.

[0085] Step 502, setting the vertex coordinates and texture coordinates of the selected demonstration image, determining the size information of the selected demonstration image, and determining the transformation matrix based on the size information.

[0086] Specifically, a coordinate system is constructed based on the content of the selected demonstration image, and the vertex coordinates and texture coordinates of the selected demonstration image are obtained through the coordinate system to determine the size information of the selected demonstration image, that is, to determine the width and height of the selected demonstration image, and determine the transformation matrix of the width and height of the selected demonstration image according to the size information, and determine the final change matrix based on the transformation matrix of the width and height through a matrix multiplication algorithm, such as the Matrix.multiplyMM matrix.

[0087] Step 503: Generate selected presentation texture information based on OpenGL ES based on the result determined by the change matrix.

[0088] Specifically, based on the determination result of the transformation matrix, the texture generation module interface of the OpenGL ES (Open Graphics Library for Embedded Systems) tool, such as the texImage2D interface, is called to install the content corresponding to the determination result on a predetermined texture type, such as the GL_TEXTURE_2D type. Among them, OpenGL ES is an open library that is directly integrated or manually installed in the mobile terminal system.

[0089] In this implementation, the texture information of the selected demonstration image can be determined by determining a transformation matrix to improve the quality of the generated texture information, and to facilitate the use of the texture information in subsequent processes to generate a live stream and transmit the demonstration image.

[0090] Step 204: obtaining an edited live stream based on the demonstration texture information and / or the live texture information.

[0091] In this embodiment, after obtaining the demonstration texture information, the editing method of the camera live stream is determined according to the user's needs, that is, according to the user's choice, it is determined whether to use the demonstration texture information alone to generate the edited live stream, to use the live texture information alone to generate the edited live stream, or to combine the demonstration texture information and the live texture information to generate the edited live stream. When the edited live stream is generated by combining the demonstration texture information and the live texture information, the live texture information can be added corresponding to the position of the preset area in the demonstration texture information. Similarly, the demonstration texture information can also be added corresponding to the position of the preset area in the live texture information to form a synthesized texture information, and the contents of the demonstration texture information and the live texture information are displayed at the same time.

[0092] Among them, a rendering tool based on texture information can be used to generate a live stream. For example, the glDrawElements tool can be called. glDrawElements is a primitive drawing function of the Open Graphics Library (OPENGL for short). It can obtain data rendering primitives from an array, and then render based on the presentation texture information and / or the live texture information to generate a corresponding live stream.

[0093] The live streaming editing method provided in the embodiment of the present application, after obtaining the camera live streaming and the demonstration file selected by the user, wherein the camera live streaming includes live texture information, parses the demonstration file, obtains a demonstration picture set, and presents the demonstration picture set to the user, in response to receiving the user's selection operation of the demonstration picture in the demonstration picture set, generates the demonstration texture information of the selected demonstration picture, and obtains the edited live streaming based on the demonstration texture information and / or the live texture information. This implementation method can parse the demonstration file into a demonstration picture for the user to select, and after the user determines the demonstration picture, the demonstration picture is sent using the camera live streaming, while providing support for the user's presentation process, without adding additional transmission code streams.

[0094] In some embodiments, after obtaining the set of presentation pictures, serial number marks are added continuously according to the generation order of the presentation pictures, and the presentation picture set is determined based on the presentation pictures continuously marked with serial numbers.

[0095] Specifically, after obtaining multiple demonstration pictures, the demonstration pictures are numbered according to the generation order of the demonstration pictures using, for example, Arabic numerals or consecutive English letters, to facilitate the correspondence between the demonstration pictures and the contents in the demonstration file during subsequent use, thereby facilitating user retrieval and use.

[0096] Continue to refer Figure 6 , which shows a process 600 of a second embodiment of a live stream editing method according to the present application. The process includes the following steps:

[0097] Step 601, obtaining the camera live stream and the demonstration file selected by the user.

[0098] Step 602: parse the demonstration file to obtain a demonstration picture set, and present the demonstration picture set to the user.

[0099] Step 603 , in response to receiving a user's selection operation on a demonstration picture in the demonstration picture set, generating demonstration texture information of the selected demonstration picture.

[0100] Step 604, in response to receiving the fused live streaming signal, the live texture information is incorporated into the presentation texture information at a preset position.

[0101] Specifically, in response to the combined live streaming signal, the demonstration texture information and the live texture information are obtained, and the live texture information is added at a predetermined position in the demonstration texture information to obtain the synthesized texture information.

[0102] The camera live stream is then edited based on the synthesized texture information to obtain an edited live stream, so that other users can watch the selected demonstration image and the content of the camera live stream at the same time after receiving the edited live stream, thereby realizing the simultaneous display of demonstration files and camera live streams through a single live stream, saving transmission resources.

[0103] Step 605: Generate a live stream using the fused texture information to obtain an edited live stream.

[0104] Step 606: In response to receiving a separate live stream signal, generate a live stream using the demonstration texture information alone to obtain an edited live stream.

[0105] Specifically, in response to receiving a separate live stream signal, a live stream signal is generated based solely on the demonstration texture information, and the live stream signal is sent to replace the existing live stream signal sent from the first terminal to the second terminal. When the user wants to present the demonstration file separately, another video live stream editing method that meets the user's needs is provided.

[0106] In the live streaming editing method provided in this embodiment, steps 601-603 are similar to the above Figure 2Steps 201-203 in the illustrated embodiment are similar, and the same contents are not repeated. Through the live streaming editing method described in this embodiment, after the user determines the demonstration picture, the demonstration picture is sent using the camera live streaming, which provides support for the user's presentation process without adding additional transmission streams.

[0107] For ease of understanding, an application scenario of the live streaming editing method is provided below. In this application scenario, both smart mobile terminals D1 and D2 can be installed with a live streaming application. Server S is the background server of the live streaming application. User U1 is the anchor-side application using D1, and user U2 is the audience-side application using D2. The execution subject of the live streaming editing method selects the smart mobile terminal D1 used by U1.

[0108] Specifically, user U1 pre-selects the presentation file P1 in ppt format to be loaded. After D1 receives the presentation file P1, it selects the HSLF module parsing method based on its own Android system, and then obtains the interface information of the corresponding hslf module in the poi library to call the hslf module, parses the presentation file P1 into multiple pictures, and marks the serial numbers T1.01, T1.02, T1.03...T1.09 in the order of generation, saves them continuously in the local storage directory, generates a presentation file T, and presents it to user U1.

[0109] After receiving the set of pictures in the presentation file, U1 selects the required T1.02 therefrom. After acquiring the picture T1.02, the executing subject D1 of the live streaming editing method calls the texImage2D interface of OpenGL ES to install the picture content on a GL_TEXTURE_2D type texture.

[0110] According to its own needs, U1 chooses to integrate the content in the ppt into the live content and play them together, so it sends a merge live stream signal instruction to D1. After D1 responds to the merge live stream signal instruction, it obtains the video texture information in the live stream currently sent by D1 to server S, and adds the video texture information to the upper right corner of the presentation texture information to generate merged texture information. Then, based on the merged texture information, glDrawElements is called to complete the rendering, generate the final live stream and send it to server S. Finally, the effect presented by the audience based on the live stream is as follows: Figure 7 shown.

[0111] It should be understood that in this process, in the case of direct transmission between user U1 and user U2, the live stream used by the above-mentioned execution entity D1 to obtain video texture information is sent from D1 to D2, and similarly, the above-mentioned execution entity D1 directly sends the final live stream to D2.

[0112] Reference below Figure 8 , which shows a computer device (eg, Figure 1 Schematic diagram of the structure of a computer system 800 of the devices 101, 102, 103, 104 shown. Figure 8 The computer device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0113] like Figure 8 As shown, the computer system 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage part 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the system 800 are also stored. The CPU 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0114] The following components are connected to the I / O interface 805: an input section 806 including a keyboard, a mouse, etc.; an output section 805 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, a modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed, so that a computer program read therefrom is installed into the storage section 808 as needed.

[0115] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or installed from the removable medium 811. When the computer program is executed by the central processing unit (CPU) 801, the above-mentioned functions defined in the method of the present application are executed.

[0116] It should be noted that the computer-readable medium of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0117] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or electronic device. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0118] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0119] The units involved in the embodiments described in the present application may be implemented by software or hardware. The units described may also be set in a processor, for example, it may be described as: a processor includes a file acquisition unit, a file parsing unit, and a file sending unit. The names of these units do not constitute a limitation on the units themselves in this case. For example, the file sending unit may also be described as "sending a request to obtain a demonstration file to the first terminal, and obtaining a demonstration file loaded by a user using the first terminal".

[0120] As another aspect, the present application also provides a computer-readable medium, which may be included in the computer device described in the above embodiment; or it may exist independently without being assembled into the computer device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by the computer device, the computer device: obtains the camera live stream and the demonstration file selected by the user, wherein the camera live stream includes live texture information, parses the demonstration file, obtains a demonstration picture set, and presents the demonstration picture set to the user, in response to receiving the user's selection operation on the demonstration picture in the demonstration picture set, generates the demonstration texture information of the selected demonstration picture, and obtains the edited live stream based on the demonstration texture information and / or the live texture information.

[0121] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

Claims

1. A method for editing a live stream, comprising: Obtaining a live stream of a camera and a demonstration file selected by a user; wherein the live stream of the camera includes live texture information; Parsing the demonstration file to obtain a demonstration picture set includes: based on a preset image capture area, continuously acquiring a plurality of demonstration pictures based on the image capture area in the process of reading the demonstration file, and generating a corresponding demonstration picture set; and presenting the demonstration picture set to the user; In response to receiving a user's selection operation on a demonstration picture in the demonstration picture set, generating demonstration texture information of the selected demonstration picture, the step of generating the demonstration texture information of the selected demonstration picture includes: acquiring the selected demonstration picture; setting the vertex coordinates and texture coordinates of the selected demonstration picture, determining the size information of the selected demonstration picture, and determining a transformation matrix based on the size information; based on the determination result of the transformation matrix, calling the texture generation module interface of the OpenGL ES component, and generating the demonstration texture information based on predetermined texture type information; Based on the demonstration texture information and / or the live broadcast texture information, an edited live broadcast stream is obtained, including: in response to receiving a fused live broadcast stream signal, incorporating the live broadcast texture information at a preset position in the demonstration texture information; using the fused texture information to generate a live broadcast stream to obtain an edited live broadcast stream.

2. The method according to claim 1, wherein: The parsing of the demonstration file to obtain a demonstration picture set, and presenting the demonstration picture set to the user may also include: Acquire operating system information of the current device, and determine a parsing method for the demonstration file according to the operating system information; The demonstration file is parsed using the parsing method to obtain a demonstration picture set, and the demonstration picture set is presented to the user.

3. The method according to claim 2, wherein: The obtaining of operating system information of the current device and determining a parsing method for the presentation file according to the operating system information includes: In response to determining that the operating system information is an Android system, determining that the parsing method is an hslf module parsing method; and The adopting the parsing method to parse the demonstration file to obtain a demonstration picture set, and presenting the demonstration picture set to the user includes: The interface of the hslf parsing module in the poi library is called to parse the demonstration file into multiple demonstration pictures.

4. The method according to claim 2, wherein: The obtaining of operating system information of the current device and determining a parsing method for the presentation file according to the operating system information includes: In response to determining that the operating system information is an IOS system, determining that the parsing method is a webview control parsing method; and The adopting the parsing method to parse the demonstration file to obtain a demonstration picture set, and presenting the demonstration picture set to the user includes: The demonstration file is loaded by using a webview control, and a JavaScript content parsing interface in the webview control is called to parse the page information of the demonstration file; Obtain the parsing result of the page information, call the drawing module interface in the webview control, and draw the parsing result into the UIImage drawing component; Use the UIImage drawing component to draw multiple demonstration file images.

5. The method according to claim 3 or 4, wherein: The obtaining of the demonstration picture set includes: Serial number marks are added continuously according to the generation order of the demonstration pictures, and a demonstration picture set is determined based on the demonstration pictures continuously marked with serial number marks.

6. The method according to claim 1, wherein: The step of obtaining the edited live stream based on the demonstration texture information and / or the live texture information includes: In response to receiving a separate live stream signal, the live stream is generated using the demonstration texture information alone to obtain an edited live stream.

7. A computer device comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images