A live document processing method and apparatus

Abstract

Embodiments of the present application disclose a live document processing method and apparatus; Embodiments of the present application can display a live document editing page, wherein the live document editing page includes a content editing area, and the content editing area includes at least one content element of the live document; In response to an operation on the content element, an editing control corresponding to the content element is displayed, and based on an editing operation on the editing control, an edited live document is displayed during the live broadcast. The edited live document is a live document after replacing the content element with a target replacement content element, and the target replacement content element is determined based on additional metadata and the content element, and the additional metadata is determined based on the editing operation. This solution can improve the processing efficiency of live documents and improve the processing method of live documents, enabling the live broadcast to be carried out efficiently.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular, to a method and apparatus for processing live documents. Background Art

[0002] As a form of online social interaction, webcasting absorbs and continues the advantages of the Internet. By combining live documents, it can conduct on-site online broadcasts in a video conferencing manner. Webcasts can publish content such as product demonstrations, relevant meetings, background introductions, solution evaluations, interviews, and online trainings onto the Internet in real time. Leveraging the intuitive, fast, well-performing, content-rich, highly interactive, geographically unrestricted, and audience-segmentable characteristics of the Internet, the live broadcast effect of activities can be enhanced.

[0003] In the process of researching and practicing related technologies, the inventors of this application found that various types of live documents are widely used in the application scenarios of webcasts. When performing editing operations such as element replacement on the content elements in a live document, the current processing methods are inefficient and the approaches need to be improved. For example, currently, it is necessary to delete the content elements to be replaced and then add new content elements to achieve element replacement, which not only incurs a high cost but also destroys the original document content in the original live document. Therefore, the current processing efficiency and methods for live documents need to be improved. Summary of the Invention

[0004] Embodiments of this application provide a method and apparatus for processing live documents, which can improve the processing efficiency of live documents and improve the processing methods of live documents, enabling webcasts to be carried out efficiently.

[0005] Embodiments of this application provide a method for processing live documents, including:

[0006] Display a live document editing page, where the live document editing page includes a content editing area, and the content editing area includes at least one content element of the live document;

[0007] In response to an operation on the content element, display an editing control corresponding to the content element, and based on the editing operation on the editing control, display the edited live document during the live broadcast. The edited live document is a live document in which the content element is replaced with a target replacement content element, and the target replacement content element is determined based on additional metadata and the content element, and the additional metadata is determined based on the editing operation.

[0008] Correspondingly, embodiments of this application further provide an apparatus for processing live documents, including:

[0009] A page display unit for displaying a live document editing page, where the live document editing page includes a content editing area, and the content editing area includes at least one content element of the live document;

[0010] A document display unit for, in response to an operation on the content element, displaying an editing control corresponding to the content element, and based on an editing operation on the editing control, displaying an edited live document during the live broadcast, where the edited live document is a live document after replacing the content element with a target replacement content element, and the target replacement content element is determined based on additional metadata and the content element, and the additional metadata is determined based on the editing operation.

[0011] In one embodiment, the document display unit includes:

[0012] A list display subunit for, in response to an operation on the content element, displaying a replacement control list on the live document editing page, where the replacement control list includes at least one element replacement control;

[0013] A metadata generation subunit for generating additional metadata describing the target replacement effect based on an element replacement operation on the element replacement control;

[0014] An attachment subunit for attaching the additional metadata to the content element to obtain an edited live document;

[0015] A document display subunit for displaying the edited live document during the live broadcast.

[0016] In one embodiment, the list display subunit is used to:

[0017] In response to an editing operation on the content element, display an editing control list on the live document editing page, where the editing control list includes a replacement trigger control; in response to a replacement trigger operation on the replacement trigger control, display a replacement control list on the live document editing page, where the replacement control list includes at least one element replacement control corresponding to a data acquisition module for acquiring display data in the target replacement content element.

[0018] In one embodiment, the live document editing page includes an editing control menu, and the editing control menu includes at least one editing control; the list display subunit is used to:

[0019] In response to an editing operation on the content element, a replacement trigger control is displayed in the editing control menu; in response to a replacement trigger operation on the replacement trigger control, a list of replacement controls is displayed on the live document editing page, where the list of replacement controls includes at least one element replacement control corresponding to an element display mode.

[0020] In one embodiment, the metadata generation subunit is configured to:

[0021] Based on an element replacement operation on the element replacement control, determine private description data describing the target replacement content element, where the private description data includes the type attribute and content information of the target replacement content element; generate additional metadata describing the target replacement effect according to the content element, the type attribute, and the content information.

[0022] In one embodiment, the list of replacement controls includes at least one element replacement control corresponding to a data acquisition module, and the data acquisition module is used to acquire real-time data displayed in the target replacement content element; the metadata generation subunit is specifically configured to:

[0023] In response to an element replacement operation on the element replacement control, determine the type attribute of the target replacement content element; display a device selection window for determining a data acquisition device on the live document editing page, where the device selection window includes at least one candidate data acquisition device; determine the target data acquisition device selected by the user from the device selection window; in response to a device selection completion operation on the device selection window, determine the content information corresponding to the target replacement content element, where the content information includes the acquisition path information of the real-time data.

[0024] In one embodiment, the list of replacement controls includes at least one element replacement control corresponding to a data acquisition module, and the data acquisition module is used to acquire offline data displayed in the target replacement content element; the metadata generation subunit is specifically configured to:

[0025] In response to an element replacement operation on the element replacement control, determine the type attribute of the target replacement content element; display an offline file selection window for acquiring the offline data on the live document editing page; determine the offline file selected by the user through the offline file selection window, where the offline file includes the offline data; in response to a file selection completion operation on the offline file selection window, determine the content information corresponding to the target replacement content element, where the content information includes the acquisition path information of the offline file.

[0026] In one embodiment, the replacement control list includes element replacement controls corresponding to at least one element display mode; the metadata generation subunit is specifically configured to:

[0027] In response to an element replacement operation on the element replacement control, determine the type attribute of the target replacement content element; based on the element display mode corresponding to the target replacement content element, determine the content information corresponding to the target replacement content element, where the content information includes the display effect information corresponding to the target replacement content element in the element display mode.

[0028] In one embodiment, the metadata generation subunit is specifically configured to:

[0029] According to the content element, obtain a metadata generation template for information to be injected, where the metadata generation template includes at least one identification node; perform type marking on the identification node according to the type attribute to obtain a marked metadata generation template; inject the content information into the marked metadata generation template to generate additional metadata describing the target replacement effect.

[0030] In one embodiment, the document display subunit is configured to:

[0031] Obtain the edited live document, where the edited live document includes content elements, and the content elements are attached with additional metadata describing the target replacement effect; based on the identification nodes in the additional metadata, parse the additional metadata to determine the type attribute and content information of the target replacement content element; according to the type attribute and the content information, generate the target replacement content element of the content element; replace the content element with the target replacement content element to obtain a replaced live document, so that the live client can display the replaced live document during the live broadcast.

[0032] In one embodiment, the document display subunit is specifically configured to:

[0033] According to the type attribute, determine the element type to which the target replacement content element belongs; based on the content information, determine the display data corresponding to the target replacement content element of the element type; fill the display data into the target replacement content element of the element type to generate the target replacement content element of the content element.

[0034] In one embodiment, the page display unit includes:

[0035] The first preparation display subunit is configured to display a live preparation page of the live client, where the live preparation page includes a document import trigger control for the live document;

[0036] A first page display subunit, configured to display a live document editing page based on an import trigger operation for an import trigger control of the document import.

[0037] In one embodiment, the page display unit includes:

[0038] A live display subunit, configured to display a live page during a live broadcast of a live client, where the live page includes a content display area of a live document and a document editing control for the live document;

[0039] A second page display subunit, configured to display a live document editing page in response to a document editing operation for the document editing control.

[0040] In one embodiment, the page display unit includes:

[0041] A second preparation display subunit, configured to display a live preparation page of a live client, where the live preparation page includes document link information of at least one live document;

[0042] A third page display subunit, configured to display a live document editing page in response to a selection operation for the document link information.

[0043] Correspondingly, an embodiment of the present application further provides a storage medium, on which a computer program is stored, where when the computer program is executed by a processor, the steps of the live document processing method shown in the embodiment of the present application are implemented.

[0044] Correspondingly, an embodiment of the present application further provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where when the processor executes the computer program, the steps of the live document processing method shown in the embodiment of the present application are implemented.

[0045] An embodiment of the present application can display a live document editing page, where the live document editing page includes a content editing area, and the content editing area includes at least one content element of a live document; in response to an operation on the content element, an editing control corresponding to the content element is displayed, and based on an editing operation on the editing control, an edited live document is displayed during a live broadcast, where the edited live document is a live document obtained by replacing the content element with a target replacement content element, and the target replacement content element is determined based on additional metadata and the content element, and the additional metadata is determined based on the editing operation.

[0046] This solution can efficiently replace the content elements in the live document with target replacement content elements. Specifically, this solution can determine additional metadata describing the target replacement effect based on the editing operations on the content elements, and further determine the target replacement content elements based on this additional metadata and the content elements to be replaced, thereby achieving element replacement in the live document. In this way, by determining the additional metadata based on the editing operations on the content elements, the additional metadata can become the information carrier describing the target replacement effect desired by the user, that is, the user can present the target replacement effect they want through the generated additional metadata only by performing simple editing operations on the content elements, such as selection operations. Further, this solution can achieve element replacement by attaching the generated additional metadata to the content elements to be replaced and display the replacement effect during the live broadcast. In this way, element replacement can be achieved without damaging the original document content in the live document. Therefore, this solution improves the processing efficiency of the live document and improves the processing method of the live document, enabling the live broadcast to be carried out efficiently. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those skilled in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0048] Figure 1a is a schematic diagram of the scenario of the live document processing method provided by the embodiment of the present application;

[0049] Figure 1b is a flowchart of the live document processing method provided by the embodiment of the present application;

[0050] Figure 2a is an interaction diagram of the live document processing method provided by the embodiment of the present application;

[0051] Figure 2b is another interaction diagram of the live document processing method provided by the embodiment of the present application;

[0052] Figure 2c is another interaction diagram of the live document processing method provided by the embodiment of the present application;

[0053] Figure 2d is another interaction diagram of the live document processing method provided by the embodiment of the present application;

[0054] Figure 2e is another interaction diagram of the live document processing method provided by the embodiment of the present application;

[0055] Figure 2f It is another interaction schematic diagram of the live document processing method provided by the embodiments of the present application;

[0056] Figure 2g It is another interaction schematic diagram of the live document processing method provided by the embodiments of the present application;

[0057] Figure 2h It is another interaction schematic diagram of the live document processing method provided by the embodiments of the present application;

[0058] Figure 2i It is a metadata schematic diagram of the live document processing method provided by the embodiments of the present application;

[0059] Figure 2j It is another metadata schematic diagram of the live document processing method provided by the embodiments of the present application;

[0060] Figure 2k It is another metadata schematic diagram of the live document processing method provided by the embodiments of the present application;

[0061] Figure 2l It is a schematic diagram of the screen principle of the live document processing method provided by the embodiments of the present application;

[0062] Figure 2m It is a metadata parsing schematic diagram of the live document processing method provided by the embodiments of the present application;

[0063] Figure 3 It is another process schematic diagram of the live document processing method provided by the embodiments of the present application;

[0064] Figure 4a It is a structural schematic diagram of the live document processing device provided by the embodiments of the present application;

[0065] Figure 4b It is another structural schematic diagram of the live document processing device provided by the embodiments of the present application;

[0066] Figure 4c It is another structural schematic diagram of the live document processing device provided by the embodiments of the present application;

[0067] Figure 4d It is another structural schematic diagram of the live document processing device provided by the embodiments of the present application;

[0068] Figure 4e It is another structural schematic diagram of the live document processing device provided by the embodiments of the present application;

[0069] Figure 5 It is a structural schematic diagram of the computer device provided by the embodiments of the present application. Detailed implementation manners

[0070] Next, the technical solutions in the embodiments of the present application will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present application.

[0071] The embodiments of the present application provide a live document processing method and apparatus. Specifically, the embodiments of the present application provide a live document processing apparatus applicable to a computer device. Among them, the computer device may be a server or a terminal, etc. Specifically, the server may be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto. The terminal and the server may be directly or indirectly connected through wired or wireless communication methods, and the present application does not limit this.

[0072] The embodiments of the present application will take the case where the live document processing method is executed by the terminal as an example to introduce the live document processing method.

[0073] Among them, a live client may be run on the terminal, and the user may edit the live document required for live broadcast through the live client. For example, the content elements in the live document may be replaced with target replacement content elements.

[0074] Reference Figure 1a , the terminal 10 may display a live document editing page on the live client. Among them, the live document editing page may include a content editing area. The live document to be edited may be displayed in the content editing area so that the content editing area may include the content elements of the live document.

[0075] Further, the terminal 10 can determine private description data for describing the target replacement content element based on the user's operations on the content element, such as click operations, long - press operations, double - click operations, swipe operations, and combinations of a series of operations, etc. The private description data can include the type attribute information and content information of the target replacement content element, so that the terminal can generate additional metadata for describing the target replacement effect according to the content element to be replaced, the type attributes and content information of the target replacement content element. Moreover, the terminal 10 can edit the live document by attaching the generated additional metadata to the content element to be replaced, thereby obtaining an edited live document.

[0076] After completing the editing of the live document, the live client can then start the live broadcast. And during the live broadcast, the live client can parse the edited live document to replace the content element with the target replacement content element to present the display effect of the replaced live document.

[0077] In another embodiment, taking the live document processing method as jointly executed by the terminal and the server and applying the live document processing method to the scenario of online education as an example, the live document processing method will be introduced.

[0078] Reference Figure 1a , online education can include a live client on the teacher side and a live client on the student side. In practical applications, both the live client on the teacher side and the live client on the student side can be clients running in the terminal 10.

[0079] The teacher can edit the live document required for teaching on the live document editing page of the live client on the teacher side through the aforementioned live document processing method. For example, the teacher can prepare the courseware document required for teaching before class. Before starting the classroom live broadcast through the live client on the teacher side, the teacher can also edit the edited courseware document on the live document editing page. For example, the teacher can set the replacement of the content element therein to the target replacement content element. For instance, the teacher can replace the picture element therein with the teacher's camera image; or, the teacher can add dynamic effects to the text element therein, etc., to obtain an edited live document.

[0080] Further, the teacher can start the classroom live broadcast through the live client on the teacher side. And during the classroom live broadcast, the live client on the teacher side can parse the edited live document so that in the displayed live document, the content element is replaced by the target replacement content element.

[0081] Accordingly, the server 20 can obtain the video picture data on the teacher-side live client and send the video picture data to the student-side live client, so that the students can see the edited courseware document on the student-side live client. Among them, in the edited courseware document displayed on the student-side live client, the content element is replaced by the target replacement content element.

[0082] The following will be described in detail respectively. It should be noted that the description order of the following embodiments does not limit the preferred order of the embodiments.

[0083] A live document processing method provided by an embodiment of the present application can be executed by a server or a terminal, or can be jointly executed by a server and a terminal; in the embodiment of the present application, the live document processing method is taken as an example of being executed by a terminal for illustration. Specifically, the terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto. The terminal and the server can be directly or indirectly connected through wired or wireless communication methods, and the present application does not limit this. In the embodiment of the present application, the method can be executed by a live document processing device integrated in the terminal, such as Figure 1b As described, the specific process of the live document processing method can be as follows:

[0084] 101. Display a live document editing page, where the live document editing page includes a content editing area, and the content editing area includes at least one content element of the live document.

[0085] Among them, live broadcast is a technology that collects the data of the live broadcast party through certain devices and performs a series of processing on the live broadcast data, such as compressing the video through video encoding into a viewable and transmissible video stream, and outputting the processed data to the viewing user terminal. There can be various types of live broadcasts. For example, live broadcasts can include news live broadcasts, shopping live broadcasts, game live broadcasts, classroom live broadcasts, etc.

[0086] The application of video technology in life is very extensive. For example, it can be used in the field of online education. Among them, online education, also known as distance teaching and distance education, refers to a teaching mode that uses Internet technology for teaching. It breaks through the boundaries of time and space and is different from the traditional teaching mode that requires students to sit in the classroom in the school building.

[0087] The live client can be an application program running in the terminal for providing live broadcast services. Accordingly, when the live broadcast technology is applied to the online education scenario, the live client can be an application program for providing online education services.

[0088] Among them, the live document can be the document required for the live broadcast. For example, when the live broadcast technology is applied to the office scenario, the live document can be the document required during the office meeting, such as, the requirements analysis report, the bidding presentation file, the department summary file, and so on. Another example, when the live broadcast technology is applied to online education, the live document can be the document required during the online education, such as, the presentation file for introducing new courses, the presentation file for test paper analysis, and so on.

[0089] The format of the live document can be various. For example, it can be the Portable Document Format (PDF), text format, image format, video format, and so on.

[0090] Since the live document can include document content, accordingly, the live document can include several content elements. For example, when the live document is a PDF file, the text content, image content or other multimedia content, link content, background content, etc. included therein can all be the content elements of the live document.

[0091] Among them, the live document editing page can be the page required for editing the live document. The live document editing page can include a content editing area, which can be used to edit the content elements in the live document.

[0092] It should be noted that in the following embodiments, the live broadcast technology can be applied to the field of online education, and taking the live document as a PDF file as an example, the live document processing method will be introduced. In practical applications, the live broadcast technology can be applied to other scenarios, and the format of the live document can also be various, and this application does not limit this.

[0093] In one embodiment, referring to Figure 2a , the live document editing page shown in Figure 2a can be displayed on the live client. In this page, the content editing area shown in 201 can be included. And although not marked, the thumbnail list of the live document and several editing controls can also be included in this page, such as, the undo control, the redo control, the font adjustment control, the font size adjustment control, the bold control, the italic control, the underline control, the strikethrough control, the font color control, the graffiti control, the alignment control, the bullet control, the numbering control, the class trigger control, the add page control, and so on. The manifestation forms of the controls in this embodiment can be in the forms of icons, input boxes, and buttons, etc.

[0094] Referring to Figure 2a, the content display area shown in 201 can be used to display the content of the live document to be edited shown in 202. And in this live document content, there can be several content elements included. For example, there can be an image content element and a text content element included. For instance, the image content element shown in 203 and the text content element shown in 204; and so on.

[0095] There can be multiple ways to trigger the display of the live document editing page. For example, before the class during the lesson preparation stage, the teacher can replace the content elements of the live document required for live teaching. Therefore, correspondingly, the live document editing page can be triggered to be displayed during the lesson preparation stage. Also, during the teaching process, the teacher can also replace the content elements of the live document. Therefore, correspondingly, the live document editing page can also be triggered to be displayed during the teaching process; and so on.

[0096] The following introduces the ways to trigger the display of the live document editing page.

[0097] Since during the lesson preparation stage, the teacher may have a need to edit the live document, specifically, the step of "displaying the live document editing page" can include:

[0098] Display the live preparation page of the live client, where this live preparation page includes a document import trigger control for the live document;

[0099] In response to an import trigger operation on this document import trigger control, display the live document editing page.

[0100] Among them, the live preparation page can be used for the user to prepare for the live document before starting the live broadcast. For example, the live preparation page can be used for the teacher to check or edit the courseware file used for teaching before starting the live classroom.

[0101] In an embodiment, the live preparation page shown in Figure 2b can be displayed on the live client. There is text information of "During lesson preparation" at the top of this page, which can be used to prompt the teacher that before starting the course live broadcast, the live document can be checked or edited on this interface.

[0102] It should be noted that the live preparation page and the live document editing page can be displayed in the live client at the same time, specifically including: the live preparation page and the live document editing page are displayed in split screens; the live document editing page is displayed in a floating layer above the live preparation page. In this way, the live document editing page has transparency and will not completely block the live preparation interface. Of course, the live preparation page and the live document editing page can also be not displayed at the same time. For example, when an import trigger operation on the document import trigger control is received, switch from the live preparation interface to present the live document editing page.

[0103] Among them, the document import trigger control can be used for users to import live documents. The manifestation forms of the control in this embodiment can be forms such as icons, input boxes, and buttons. For example, in Figure 2b the shown live broadcast preparation page, there may be a document import trigger control shown as 205.

[0104] Among them, the import trigger operation can be used to trigger the import of live documents into the live client. The manifestation forms of the import trigger operation in this embodiment can be specific touch operations, such as click operations, long - press operations, double - click operations, and swipe operations, etc., and can also be triggered by voice; optionally, the import trigger operation can also be a combination of a series of operations, and this embodiment has no limitation on this.

[0105] In one embodiment, the teacher can click on Figure 2b the document import trigger control shown as 205 to select the courseware file required for the course live broadcast, and import the courseware file into Figure 2a the shown live document editing page. Specifically, the step "in response to the import trigger operation for the document import trigger control, display the live document editing page" can include:

[0106] In response to the import trigger operation for the document import trigger control, display a document selection window for the live document on the live broadcast preparation page;

[0107] In response to the document selection completion operation for the document selection window, display the live document editing page of the live client.

[0108] Among them, the document selection window can be a window for users to select live documents. For example, users can select the documents stored in the terminal through this window, or users can select the documents stored in the network or server through this window, etc.

[0109] The document selection window and the live broadcast preparation page can be displayed in the live client at the same time, specifically including: the document selection window and the live broadcast preparation page are displayed in split - screen; the document selection window is displayed above the live broadcast preparation page in the form of a floating layer. In this way, the document selection window has transparency and will not completely block the live broadcast preparation page. Of course, the document selection window and the live broadcast preparation page can also be not displayed at the same time. For example, when receiving the import trigger operation for the document import trigger control, switch from the live broadcast preparation page to present the document selection window.

[0110] Among them, the document selection completion operation can be used to complete the selection of the live document. The manifestation of the document selection completion operation in this embodiment can be a specific touch operation, such as a click operation, a long press operation, a double click operation, and a swipe operation, etc., or it can be triggered by voice; optionally, the file selection completion operation can also be a combination of a series of operations, and this embodiment has no limitation on this.

[0111] In one embodiment, when the teacher is in Figure 2b the live broadcast preparation page shown, after clicking the document import trigger control shown in 205, a document selection window for the live document can be displayed above the live broadcast preparation page. The teacher can select the live document through this document selection window. When the live document to be imported is determined, the teacher can click the "OK" control in the document selection window to import the selected live document into the live document editing page. Correspondingly, the live client can display a live document editing page as shown in Figure 2a and, in this page, the document content of the live document selected by the teacher can be displayed.

[0112] Regarding the teacher's need to edit the live document during the lesson preparation stage, in addition to providing a document import trigger control in the live broadcast preparation page, the document link information of the live document can also be provided in the live broadcast preparation page, so that the teacher can use this document link information to import the corresponding live document into the live document editing page. Specifically, the step of "displaying the live document editing page" can include:

[0113] Display the live broadcast preparation page of the live client, where this live broadcast preparation page includes the document link information of at least one live document;

[0114] In response to the selection operation for this document link information, display the live document editing page.

[0115] Among them, the document link information of the live document can be link information that can be used to direct to this live document. For example, this live document can be quickly opened through this document link information.

[0116] The manifestation of the document link information can be various. For example, it can be in the form of the control shown in Figure 2b 206; another example is that the document link information can be in the form of an icon, an input box, a QR code, a hyperlink, etc.

[0117] Among them, the selection operation can be used to select document link information. The manifestation form of the selection operation in this embodiment can be a specific touch operation, such as a click operation, a long - press operation, a double - click operation, and a swipe operation, etc., or it can also be triggered by voice; optionally, the selection operation can also be a combination of a series of operations, and this embodiment has no limitation on this.

[0118] In one embodiment, it can be displayed on the live client Figure 2b the live preparation page shown in the figure. This page may include the document link information of the live document named "Elementary English Introduction" shown in 206. In Figure 2b it, this document link information is presented in the form of a control. The teacher can select the live document named "Elementary English Introduction" by clicking Figure 2b the control shown in 206 in it, and import this live document into Figure 2a the live document editing page shown in the figure. And the live client can correspondingly display the live document editing page as shown in Figure 2a the figure, and in this page, the live document named "Elementary English Introduction" is displayed.

[0119] In addition to being able to import the corresponding live document into the live document editing page for the teacher's need to edit the live document during the lesson preparation stage, it can also trigger the display of the live document editing page of the live client for the teacher's need to edit the live document during the class. Specifically, the step of "displaying the live document editing page" can include:

[0120] Display the live page of the live client during the live broadcast. Among them, this live page includes a content display area of the live document and a document editing control for this live document;

[0121] In response to the document editing operation on this document editing control, display the live document editing page.

[0122] Among them, the live page can be the page displayed after the live client starts the live broadcast. This page can display the live content of the live broadcast. For example, it can display the document content of the live document.

[0123] The live page can include a content display area of the live document, and this content display area can be used to display the document content of this live document. In addition, the live page can also include a document editing control for the live document, and this document editing control can be used to trigger the display of the live document editing page for this live document during the live broadcast. The manifestation form of the control in this embodiment can be in the form of an icon, an input box, a button, etc.

[0124] Among them, the document editing operation can be used to trigger the display of the live document editing page for the live document. The manifestation form of the document editing operation in this embodiment can be a specific touch operation, such as a click operation, a long - press operation, a double - click operation, and a swipe operation, etc., and can also be triggered by voice; optionally, the document editing operation can also be a combination of a series of operations, and this embodiment has no limitation on this.

[0125] In one embodiment, after the teacher starts a classroom live broadcast through the live broadcast client, a live broadcast page as shown in Figure 2c can be displayed on the live broadcast client. Among them, the live broadcast page includes a content display area of the courseware document as shown in 207. In this content display area, the document content of the courseware document is displayed. In addition, the live broadcast page may also include a document editing control as shown in 208. During the classroom live broadcast, the teacher can click on the document editing control as shown in 208 to edit the courseware document. Correspondingly, the live broadcast client can display a live document editing page for the courseware document as shown in Figure 2a shown.

[0126] In practical applications, the position of the document editing control can be adjusted according to requirements. In addition, the live broadcast page and the live document editing page can be displayed simultaneously in the live broadcast client, specifically including: the live broadcast page and the live document editing page are displayed in split - screen mode; the live document editing page is displayed as a floating layer above the live broadcast page. Of course, the live broadcast page and the live document editing page can also be not displayed simultaneously. For example, when a document editing operation is received, the live broadcast page is switched to the live document editing page.

[0127] In addition, it should be noted that the live document editing page displayed on the live broadcast client in the classroom can be synchronously displayed on the live broadcast clients of students, so that the live document editing page or the element replacement setting process implemented through the live document editing page is visible to students; it can also be not synchronously displayed on the live broadcast clients of students, so that the live document editing page or the element replacement setting process implemented through the live document editing page is not visible to students. The specific settings can be adjusted based on business requirements, and this application does not limit this.

[0128] 102. In response to an operation on a content element, display an editing control corresponding to the content element. Based on the editing operation on the editing control, display an edited live document during the live broadcast. The edited live document is a live document after replacing the content element with a target replacement content element, and the target replacement content element is determined based on additional metadata and the content element, and the additional metadata is determined based on the editing operation.

[0129] The presentation forms of the editing controls in this embodiment can be in the forms of icons, input boxes, buttons, etc.

[0130] Among them, the presentation forms of the operations on the content elements or the editing operations on the editing controls in this embodiment can be specific touch operations, such as click operations, long - press operations, double - click operations, and swipe operations, etc., and can also be triggered by voice; optionally, it can also be a combination of a series of operations, and this embodiment has no restrictions on this.

[0131] When performing a replacement operation on a content element in a live document, the element used to replace the content element can be called the target replacement content element. The target replacement content element can be of the same type as the content element to be replaced, or can be of a different type from the content element to be replaced. For example, the target replacement content element can be a text content element, an image content element, a link content element, a background content element, etc.; in addition, the target replacement content element can also be other types of multimedia content elements. For example, it can be a content element including a camera screen, a content element including a video screen, etc.

[0132] Among them, metadata, also known as intermediate data and relay data layer, is data that describes data, or is structured data used to provide information about a certain resource. Metadata is data that describes objects such as information resources or data, and its purpose of use is: to identify resources; to evaluate resources; to trace the changes of resources during use; to achieve simple and efficient management of a large amount of networked data; to achieve effective discovery, search, integrated organization of information resources, and effective management of used resources.

[0133] In this embodiment, the additional metadata can be metadata generated based on the editing operation on the editing control and describing the target replacement effect.

[0134] The following will introduce the step of "responding to the operation on the content element, displaying the editing control corresponding to the content element, and based on the editing operation on the editing control, displaying the edited live document during the live broadcast". Specifically, this step can include:

[0135] Responding to the operation on the content element, displaying a list of replacement controls on the live document editing page, where the list of replacement controls includes at least one element replacement control;

[0136] Generating additional metadata describing the target replacement effect based on the element replacement operation on the element replacement control;

[0137] Attaching the additional metadata to the content element to obtain the edited live document;

[0138] Displaying the edited live document during the live broadcast.

[0139] Among them, the replacement control list is a list including at least one element replacement control, and the element replacement control can be used to trigger a series of element replacement operations for content elements.

[0140] The manifestation forms of the controls in this embodiment can be forms such as icons, input boxes, and buttons.

[0141] Since the manifestation forms of the operations on content elements in this embodiment can be specific touch operations, such as click operations, long-press operations, double-click operations, and swipe operations, etc., and can also be triggered by voice; optionally, the operations on content elements can also be a combination of a series of operations. Therefore, based on the operations on content elements, there can be multiple ways to display the replacement control list on the live document editing page, which will be introduced below.

[0142] In one embodiment, the target replacement content element can be a multimedia content element. For example, it can be a content element including a camera screen, a content element including a video screen, etc. And, based on the combination of the operations of the user on the content element to be replaced, the user can finally be directed to the element replacement control. Therefore, specifically, the step "in response to an operation on a content element, display a replacement control list on the live document editing page, where the replacement control list includes at least one element replacement control" can include:

[0143] In response to an editing operation on a content element, display an editing control list on the live document editing page, where the editing control list includes a replacement trigger control;

[0144] In response to a replacement trigger operation on the replacement trigger control, display a replacement control list on the live document editing page, where the replacement control list includes at least one element replacement control corresponding to a data acquisition module, and the data acquisition module is used to acquire the display data in the target replacement content element.

[0145] The manifestation forms of the controls in this embodiment can be forms such as icons, input boxes, and buttons.

[0146] Among them, the editing control list can be a list including several editing controls. For example, the editing control list can include a replacement trigger control, and the replacement trigger control can be used to trigger the display of optional element replacement controls. For example, by triggering this control, a replacement control list including element replacement controls can be displayed, so that the element replacement controls can also be displayed.

[0147] Among them, the data acquisition module can be a module required to acquire the display data in the target replacement content element. For example, the data acquisition module can include a camera device required to acquire camera screen data. Correspondingly, the display data in the target replacement content element can be the acquired camera screen data. Another example is that it includes relevant devices required to obtain video screen data on a terminal, network, or server. Correspondingly, the display data in the target replacement content element can be the obtained video screen data, and so on.

[0148] In one embodiment, a live broadcast client can display a live broadcast document editing page as shown in Figure 2a which includes an image content element shown in 203. Referring to Figure 2d , a teacher can select the image content element by clicking. Correspondingly, the live broadcast client can display the state of the selected image content element as shown in 209 in Figure 2d . And after selecting the image content element, an edit control list shown in 210 can be displayed on the live broadcast document editing page. Among them, the edit control list includes a replacement trigger control shown in 211.

[0149] Among them, the replacement trigger control shown in 211 can be used to trigger the display of optional element replacement controls. In addition, when the teacher places the mouse on the replacement trigger control 211, the live broadcast client can correspondingly display a prompt text of "enhanced element" below the control to prompt that the image content element can be enhanced by operating on this control, such as replacing the image content element with a content element including a camera screen, or replacing it with a content element including a video screen, and so on.

[0150] Furthermore, after the teacher clicks the replacement trigger control shown in 211, the live broadcast client can display a replacement control list on the live broadcast document editing page as shown in Figure 2e . Among them, the replacement control list can include element replacement controls corresponding to at least one data acquisition module. For example, referring to Figure 2e , it includes an element replacement control shown in 212, which can be used to replace the image content element with a camera screen; it also includes an element replacement control shown in 213, which can be used to replace the image content element with a video screen.

[0151] It should be noted that the manifestation forms of the edit operation or replacement trigger operation in this embodiment can be specific touch operations, such as click operation, long press operation, double click operation, and slide operation, etc., and can also be triggered by voice; optionally, the edit operation or replacement trigger operation can also be a combination of a series of operations, and this embodiment has no limitation on this.

[0152] In another embodiment, the target replacement content element and the content element to be replaced may belong to the same element type but have different content information. For example, the content information in the content element to be replaced may be static content information, while the content information in the target replacement content element may be dynamic content information obtained by adding a dynamic display effect to the static content information, and so on. And when the live document editing page may include an editing control menu and the editing control menu may include at least one editing control, based on the combination of operations of the user on the content element to be replaced, the user may ultimately be directed to the element replacement control. Specifically, the step "in response to an operation on the content element, display a list of replacement controls on the live document editing page, where the list of replacement controls includes at least one element replacement control" may include:

[0153] In response to an operation on the content element, display a replacement trigger control in the editing control menu;

[0154] In response to a replacement trigger operation on the replacement trigger control, display a list of replacement controls on the live document editing page, where the list of replacement controls includes at least one element replacement control corresponding to the element display mode.

[0155] The presentation form of the controls in this embodiment may be in the form of icons, input boxes, buttons, etc.

[0156] Among them, the editing control menu may be a menu including several editing controls. For example, the editing control menu may include a replacement trigger control, and the replacement trigger control may be used to trigger the display of optional element replacement controls. For example, by triggering this control, a list of replacement controls including at least one element replacement control corresponding to the element display mode may be displayed, so that the element replacement control can also be displayed.

[0157] Among them, the element display mode is the mode followed when displaying the content element in the live document. For example, the element display mode may be the mode followed when the content element appears in the live page; another example is that the element display mode may be the mode followed when the content element leaves the live page; another example is that the element display mode may be the mode followed when emphasizing the display of the content element in the live page; and so on.

[0158] In one embodiment, a live document editing page as shown in Figure 2a may be displayed on the live client, which includes the image content element shown in 203. Referring to Figure 2d , the teacher can select the image content element by clicking in the live document editing page. Among them, the live document editing page may include an editing control menu shown in 214, and in this editing control menu, there may be included at least one editing control. Accordingly, the live client may be as shown inFigure 2d As shown in 209, the state after the image content element is selected is shown, and after the image content element is selected, the replacement trigger control shown in 215 can be triggered to be displayed in the edit control menu 214.

[0159] Among them, the replacement trigger control shown in 215 can be used to trigger the display of the corresponding element replacement control in at least one element display mode. Further, after the teacher clicks the replacement trigger control shown in 215, the live client can display the replacement control list 216 on the live document editing page as Figure 2f shown, where the replacement control list can include the corresponding element replacement controls in at least one element display mode.

[0160] For example, referring to Figure 2f , the replacement control list shown in 215 includes multiple element replacement controls corresponding to three element display modes. Specifically, it includes "entrance animation", that is, the display mode followed when the content element appears on the live page, includes "exit animation", that is, the display mode followed when the content element leaves the live page, and also includes "emphasis animation", that is, the display mode followed when emphasizing the display of the content element on the live page. Each display mode can include several corresponding element replacement controls.

[0161] It should be noted that the manifestation form of the editing operation or replacement trigger operation in this embodiment can be a specific touch operation, such as a click operation, a long press operation, a double click operation, and a slide operation, etc., and can also be triggered by voice; optionally, the editing operation or replacement trigger operation can also be a combination of a series of operations, and this embodiment has no limitation on this.

[0162] After introducing the step "In response to an operation on a content element, display a replacement control list on the live document editing page, where the replacement control list includes at least one element replacement control", the following can introduce the step "Generate additional metadata describing the target replacement effect based on the element replacement operation on the element replacement control". Specifically, this step can include:

[0163] Based on the element replacement operation on the element replacement control, determine the private description data describing the target replacement content element, where the private description data includes the type attribute and content information of the target replacement content element;

[0164] Generate additional metadata describing the target replacement effect according to the content element, type attribute, and content information.

[0165] The manifestation form of the element replacement control in this embodiment can be in the form of an icon, an input box, a button, etc.

[0166] Among them, the manifestation form of the element replacement operation in this embodiment can be a specific touch operation, such as a click operation, a long - press operation, a double - click operation, and a slide operation, etc., and can also be triggered by voice; optionally, it can also be a combination of a series of operations, and this embodiment has no limitation on this.

[0167] Among them, the private description data can be related data describing the process of replacing content elements. Specifically, it can be private data describing how to replace the content elements to be replaced with target replacement content elements. Private data is relative to public data and is protected, non - public, or conditionally shared data. For example, in practical applications, an editor can be designed, and this editor can generate private description data to replace the content elements in a PDF document to obtain an edited PDF document. And a reader can be designed, and this reader can read and parse the private description data to display the PDF document after replacing the content elements. In this process, the private description data can only be generated by this editor and can only be read and parsed by this reader, so this private description data is private, protected, and non - public. Another example is that conditional sharing can be set for this private description data, and developers or applications can obtain or parse this private description data after meeting the set conditions to implement the content element replacement operation.

[0168] The format of the private description data can be set according to business requirements. For example, in this embodiment, when replacing the content elements to be replaced with target replacement content elements, the target replacement content elements can be determined by determining the type attribute of the target replacement content element, that is, what type of content element the target replacement content element is, and determining the content information presented in the target replacement content element. Therefore, in this embodiment, the private description data can include the type attribute and content information of the target replacement content element.

[0169] Among them, the type attribute of the target replacement content element can be used to describe the element type to which the target replacement content element belongs. For example, the type attribute of the target replacement content element can be a text content element, an image content element, a link content element, a background content element, etc.; in addition, it can also be other types of multimedia content elements. For example, it can be a content element including a camera screen, a content element including a video screen, etc.

[0170] Among them, the content information of the target replacement content element can be used to describe the display content presented by the target replacement content element whose element type is determined. For example, when the target replacement content element is a text content element, the content information of the target replacement content element can be the text information presented by the text content element; for another example, when the target replacement content element is an image content element, the content information of the target replacement content element can be the image information presented by the image content element.

[0171] For another example, when the target replacement content element is a content element including a camera screen, the content information of the target replacement content element can be the camera screen presented by the target replacement content element; for another example, when the target replacement content element is a content element including a video screen, the content information of the target replacement content element can be the video screen presented by the target replacement content element.

[0172] For another example, when the content element to be replaced and the target replacement content element belong to the same element type, the content information of the target replacement content element can be the content information after setting a dynamic effect on the content information in the content element to be replaced; and so on.

[0173] Since the element replacement control has various forms of expression. For example, it can be Figure 2e the element replacement control for implementing "replacement with a camera screen" shown in 212 in Figure 2e the element replacement control for implementing "replacement with a video screen" shown in 213 in Figure 2f the element replacement control in different element display modes shown in list 216 in , etc. Therefore, based on the element replacement operation for the element replacement control, there can be multiple ways to determine the type attribute and content information of the target replacement content element, which will be introduced below.

[0174] In one embodiment, the replacement control list may include element replacement controls corresponding to at least one data acquisition module. The data acquisition module can be used to acquire real-time data displayed in the target replacement content element. Specifically, the step of "based on the element replacement operation for the element replacement control, determining the private description data describing the target replacement content element, where the private description data includes the type attribute and content information of the target replacement content element" may include:

[0175] In response to the element replacement operation for the element replacement control, determining the type attribute of the target replacement content element;

[0176] Displaying a device selection window for determining a data acquisition device on the live document editing page, where the device selection window includes at least one candidate data acquisition device;

[0177] Determine the target data acquisition device selected by the user from the device selection window of the device;

[0178] In response to the device selection completion operation for the device selection window, determine the content information corresponding to the target replacement content element, where the content information includes the acquisition path information of the real-time data.

[0179] In this embodiment, the manifestation form of the control can be in the form of an icon, an input box, a button, etc.

[0180] Among them, the device selection window can be a window for the user to select a data acquisition device. Therefore, the device selection window can include at least one candidate data acquisition device. For example, it can include at least one optional camera device. Optionally, the device selection window can also include other function controls. For example, refer to Figure 2g , the device selection window can also include a function control for flipping the camera image by 180°, a function control for mirroring the camera preview, a function control for only previewing; and so on.

[0181] Among them, the data acquisition device can be a device required for data acquisition. For example, it can be a camera device, a recorder device, and so on.

[0182] Among them, the real-time data can be data collected in real time by the data acquisition device. For example, the real-time data can be the camera image data collected by the camera device; or, the real-time data can be the recording data collected by the recorder device; and so on.

[0183] Among them, the acquisition path information of the real-time data can be the relevant path information required to acquire the real-time data. The manifestation form of the acquisition path information of the real-time data can be various. For example, it can be the identifier of the data acquisition device, such as the device account (Identity Document, ID) or the device name, etc.; or, it can be the data storage path corresponding to the data acquisition device; and so on.

[0184] In one embodiment, a replacement control list including the element replacement control 212 and the element replacement control 213 as shown in Figure 2e can be displayed on the live document editing page, where the element replacement control 212 can correspond to the real-time data acquisition module, and the real-time data acquisition module can be used to collect the real-time data displayed in the target replacement content element. For example, the picture data collected by the camera.

[0185] Further, the teacher can click Figure 2eReplace the element replacement control 212 shown in the figure, determine to replace the image content element to be replaced with the camera screen, and the live client can accordingly respond to the teacher's element replacement operation on the element replacement control 212, determine that the target replacement content element is the content element including the camera screen, that is, determine the type attribute of the target replacement content element.

[0186] Furthermore, the live client can display a device selection window for determining the data acquisition device on the live document editing page. For example, the device selection window can be as shown in Figure 2g 217 in the figure, and, as shown in Figure 2g 218 in the figure, the device selection window includes at least one candidate data acquisition device: "Camera Device A", "Camera Device B", etc.

[0187] The teacher can select from the candidate data acquisition devices through the device selection window, and the live client can accordingly determine the target data acquisition device selected by the teacher from the device selection window. For example, referring to Figure 2g , the teacher can select "Camera Device A" as the target data acquisition device. After determining the target data acquisition device, the real-time data collected by the target data acquisition device can be determined as the content information corresponding to the target replacement content element. For example, the content information can include the acquisition path information of the real-time data, such as the device account (Identity Document, ID) or device name of the data acquisition device, or the data storage path corresponding to the data acquisition device, etc.

[0188] It should be noted that the manifestation forms of the element replacement operation or the device selection completion operation in this embodiment can be specific touch operations, such as click operation, long press operation, double click operation, and slide operation, etc., and can also be triggered by voice; optionally, the element replacement operation or the device selection completion operation can also be a combination of a series of operations, and this embodiment has no limitation on this.

[0189] In another embodiment, the replacement control list can include at least one element replacement control corresponding to a data acquisition module. The data acquisition module can be used to acquire the offline data displayed in the target replacement content element. Specifically, the step "Based on the element replacement operation for the element replacement control, determine the private description data describing the target replacement content element, where the private description data includes the type attribute and content information of the target replacement content element" can include:

[0190] Respond to the element replacement operation for the element replacement control and determine the type attribute of the target replacement content element;

[0191] Display an offline file selection window for obtaining offline data on the live document editing page;

[0192] Determine the offline file selected by the user through the offline file selection window, where the offline file includes offline data;

[0193] In response to the completion of the file selection operation for the offline file selection window, determine the content information corresponding to the target replacement content element, where the content information includes the acquisition path information of the offline file.

[0194] The presentation form of the control in this embodiment can be in the form of an icon, an input box, a button, etc.

[0195] Among them, the offline file selection window can be a window required for the user to select an offline file. For example, the offline file selection window can be as Figure 2h shown.

[0196] Among them, the offline data is relative to the real-time data and can be existing data resources that do not need to be collected in real time by a data collection device, but can be obtained through query, search, or request. For example, the offline data can be data resources such as video files, images, audio, and text stored on the terminal; another example is that the offline data can be video files, images, audio, text, etc. obtained through the network or requested from the server; and so on.

[0197] Among them, the acquisition path information of the offline data can be the relevant path information required to obtain the offline data. The presentation form of the acquisition path information of the offline data can be various. For example, it can be the storage path or name of the offline data on the terminal; another example is the identifier required when requesting the offline data from the network or the server, such as the website (Uniform Resource Locator, URL), the offline data name, the offline data ID, and so on.

[0198] In one embodiment, a replacement control list including the element replacement control 212 and the element replacement control 213 as shown in Figure 2e can be displayed on the live document editing page, where the element replacement control 213 can correspond to the offline data collection module, and the offline data collection module can be used to collect the offline data displayed in the target replacement content element, such as a video file stored on the terminal.

[0199] Further, the teacher can click on the element replacement control 213 as shown in Figure 2e to determine to replace the image content element to be replaced with a video picture. The live client can then respond to the teacher's element replacement operation for the element replacement control 213 and determine that the target replacement content element is a content element including a video picture, that is, determine the type attribute of the target replacement content element.

[0200] Furthermore, the live broadcast client can display an offline file selection window required for obtaining offline data on the live broadcast document editing page. For example, the offline file selection window can be as shown in Figure 2h . In one embodiment, as shown in Figure 2h , the offline file selection window can include at least one candidate offline file: "Video A", "Video B".

[0201] The teacher can select the offline file required for element replacement through the offline file selection window. Among them, the selected offline file can include the required offline data. For example, referring to Figure 2h , the teacher can select "Video A" from the offline file selection window, and the live broadcast client can also correspondingly determine the offline file selected by the teacher from the offline file selection window.

[0202] After the teacher selects an offline file from the offline file selection window, the live broadcast client can respond to the file selection completion operation for the offline file selection window, such as the trigger operation for the "Open" control in Figure 2h , and determine the offline data included in the offline file selected by the teacher as the content information corresponding to the target replacement content element. In one embodiment, the content information can include the storage path or name of the offline data on the terminal. For example, it can include the storage path or name of "Video A" on the terminal.

[0203] It should be noted that the manifestation forms of the element replacement operation or the device selection completion operation in this embodiment can be specific touch operations, such as click operation, long press operation, double click operation, and swipe operation, etc., and can also be triggered by voice; optionally, the element replacement operation or the device selection completion operation can also be a combination of a series of operations, and this embodiment has no limitation on this.

[0204] In another embodiment, the replacement control list can include at least one element replacement control corresponding to an element display mode. Specifically, the step "Based on the element replacement operation for the element replacement control, determine the private description data describing the target replacement content element, where the private description data includes the type attribute and content information of the target replacement content element" can include:

[0205] Respond to the element replacement operation for the element replacement control and determine the type attribute of the target replacement content element;

[0206] Based on the element display mode corresponding to the target replacement content element, determine the content information corresponding to the target replacement content element, where the content information includes the display effect information corresponding to the target replacement content element in this element display mode.

[0207] The manifestation forms of the controls in this embodiment can be forms such as icons, input boxes, and buttons.

[0208] Among them, the element display mode is the mode followed when displaying content elements in the live document. For example, the element display mode can be the mode followed when the content element appears on the live page; for another example, the element display mode can be the mode followed when the content element leaves the live page; for another example, the element display mode can be the mode followed when emphasizing the display of the content element on the live page; and so on.

[0209] Therefore, different display effects can be set for content elements in different element display modes. Since what determines the content element can include the element type and content information, after determining the element display mode of the content element, the content information with the display effect corresponding to this element display mode can be determined, that is, the content information can include the display effect information corresponding to the content element in this element display mode.

[0210] Among them, the display effect information can be the relevant information describing the display effect of the content element in the corresponding element display mode. The manifestation forms of the display effect information can be various. For example, the display effect can be mapped to a specific display effect through data such as the name and duration of the display effect. Therefore, the name and duration of the display effect and other data can be used as the display effect data; for another example, the response event of the display effect can also be set to accurately display the content element during the display of the live document. Therefore, the name, duration, and response event of the display effect can be used as the display effect data; and so on.

[0211] For example, an element display mode can be set as follows: when the content element appears on the live page, it appears with a "fade-in" effect, and this effect lasts for 0.3 seconds. Then the corresponding display effect information can include "fade-in" and 0.3 seconds; for another example, an element display mode can be set as follows: when the content element leaves the live page, it leaves with a "wipe" effect, and this effect lasts for 0.4 seconds. Then the corresponding display effect information can include "wipe" and 0.4 seconds; for another example, an element display mode can be set as follows: after the content element appears on the live page, in response to the user's emphasis trigger operation on this content element, the content element is emphasized and displayed again with a "magnify" effect. Then the corresponding display effect information can include "magnify" and the emphasis trigger operation; and so on.

[0212] In one embodiment, it can be displayed in the live document editing page as Figure 2fThe replacement control list shown in 216, where the replacement control list 216 includes at least one element replacement control corresponding to the element display mode. The teacher can click on the element replacement control to trigger an enhanced display effect for the content element to be replaced, such as a dynamic display effect. Correspondingly, the live client can, in response to the teacher's element replacement operation on the element replacement control, determine that the target replacement content element is of the same type as the content element to be replaced.

[0213] For example, after the teacher selects the image content element to be replaced and chooses to add a "fade-in" display effect when the image appears, and this effect lasts for 0.3 seconds during display. Therefore, the live client can correspondingly determine the image content element with the "fade-in" effect added as the target replacement content element, where the target replacement content element can include, in addition to the content information of the image to be displayed, the corresponding display effect information, such as "fade-in" and 0.3 seconds.

[0214] It should be noted that the manifestation form of the element replacement operation in this embodiment can be a specific touch operation, such as a click operation, a long-press operation, a double-click operation, and a swipe operation, etc., or it can be triggered by voice; optionally, the element replacement operation can also be a combination of a series of operations, and this embodiment has no restrictions on this.

[0215] After determining the type attribute of the target replacement content element and the corresponding content information, the private description data of the target replacement content element can be correspondingly determined.

[0216] After introducing the step "Based on the element replacement operation on the element replacement control, determine the private description data describing the target replacement content element, where the private description data includes the type attribute and the content information of the target replacement content element", the following can introduce the step "Generate additional metadata describing the target replacement effect according to the content element, the type attribute, and the content information". Specifically, this step can include:

[0217] According to the content element, obtain the metadata generation template for the information to be injected, where the metadata generation template includes at least one identification node;

[0218] According to the type attribute, perform type marking on the identification node to obtain the marked metadata generation template;

[0219] Inject the content information into the marked metadata generation template to generate additional metadata describing the target replacement effect.

[0220] Among them, the metadata generation template can be a template required to generate metadata describing the target replacement effect. The metadata generation template can include preset modules and nodes, and metadata can be generated by injecting the required data into the metadata generation template.

[0221] Among them, the identification node in the metadata generation template can be a node that plays an identification role in the metadata generation template. For example, it can be a markup label in the metadata generation template.

[0222] The manifestation form of the identification node can be various. For example, it can be a keyword enclosed by angle brackets, such as <pdfmetadata>In one embodiment, identification nodes may appear in pairs, for example, <pdfmetadata>and< / pdfmetadata> , and the first identification node in the identification node pair can be a start identification node, and the second identification node can be an end identification node; in another embodiment, the identification node can also be presented separately.

[0223] Regardless of whether the identification nodes appear in pairs or individually, you can assign values to them, thereby marking the identification nodes. For example, when assigning values to identification nodes that appear in pairs, the assigned value can be between the two identification nodes, for example, <exementtype>VideoStream< / exementtype> ; Or, it can be in the start mark node, for example,<File filePath="C: / Media / 视频讲解.mp4”fileName="视频讲解.mp4”> For another example, when assigning a value to a tag that is presented separately, the value can be in the identification node.

[0224] In this embodiment, after determining the type attribute of the target replacement content element, the identification node in the metadata generation template can be type-marked so that the marked metadata generation template includes the type attribute information of the target replacement content element. In this way, the subsequent live broadcast client can accurately replace the content element with the target replacement content element based on the generated additional metadata.

[0225] For example, in the metadata generation template, you can assign the type attribute of the target replacement content element to the identifier node pair. <exementtype>< / exementtype> In this way, the identification node can be marked as a type. For example, when the target replacement content element is a content element including a camera image, or a content element including a video image, the following marking can be performed: <exementtype>VideoStream< / exementtype> For another example, when the target content element is a content element with dynamic display effects, the following markup can be used: <exementtype>Animate< / exementtype> ;etc.

[0226] After the identification node is type-marked according to the type attribute of the target replacement content element and the marked metadata generation template is obtained, the content information can be injected into the marked metadata generation template to generate additional metadata describing the target replacement effect.

[0227] Since the content information of the target replacement content element can be used to describe the display content presented by the target replacement content element of the determined element type, there are many ways to inject the content information into the metadata generation template after the markup, which are described below with examples.

[0228] In one embodiment, the format of the live document can be PDF, and the image content element in the live document can be replaced with a content element including a camera feed, that is, the element type of the target replacement content element can be a video stream element. Refer to Figure 2i , the identification node pair <exementtype>< / exementtype> can be assigned to VideoStream to identify that the target replacement content element is a video stream element, and the content information of the video stream element can be described in detail through the identification node pair <videostreamsrc>< / videostreamsrc> . Refer to Figure 2i , in the metadata generation template, the identification node pair <videostreamsrc>< / videostreamsrc> nests the identification node pair <camera>< / camera> , and by assigning the identification node <camera>< / camera> to the ID (deviceID) and name (deviceName) of the camera device that captures the camera feed, the content information of the target replacement content element is injected into the marked metadata generation template, thereby obtaining additional metadata describing the target replacement effect as shown in b2i.

[0229] In another embodiment, the format of the live document can be PDF, and the text content element in the live document can be replaced with a content element including a video frame, that is, the element type of the target replacement content element can be a video stream element. Refer to Figure 2j , the identification node pair <exementtype>< / exementtype> can be assigned to VideoStream to identify that the target replacement content element is a video stream element, and the content information of the video stream element can be described in detail through the identification node pair <videostreamsrc>< / videostreamsrc> . Refer to Figure 2j , in the metadata generation template, the identification node pair <videostreamsrc>< / videostreamsrc> nests <file>Identify nodes, and by identifying nodes <file>It is assigned to the storage path (filePath) and name (fileName) of the video file, which realizes injecting the content information of the target replacement content element into the tagged metadata generation template, thereby obtaining additional metadata describing the target replacement effect as shown in b2i.

[0230] In another embodiment, the format of the live document can be PDF, and the image content element in the live document can be replaced with an image content element after adding a dynamic display effect. For example, the dynamic display effect can be: fade in and appear for 0.3 seconds, and this effect is displayed when the page appears. Then, when the live client displays the PDF file, the initial image content element can be replaced with the image content element with the added dynamic display effect. That is, the element type of the target replacement content element can be a content element with a dynamic display effect. Refer to Figure 2k , the identification node pair <exementtype>< / exementtype> can be assigned to Animate to identify that the target replacement content element is a content element with a dynamic display effect, and the content information of the target replacement content element can be described in detail through the identification node pair <animatetype>< / animatetype> . Refer to Figure 2k , in the metadata generation template, the identification node pair <animatetype>< / animatetype> nests the identification node pair <in>< / in> , and by assigning the identification node pair <in>< / in> to the type (action), duration (time), and response event (start) of the dynamic display effect, it realizes injecting the content information of the target replacement content element into the tagged metadata generation template, thereby obtaining additional metadata describing the target replacement effect as shown in b2i.

[0231] When generating additional metadata describing the target replacement effect, the additional metadata describing the target replacement effect can be attached to the content element to be replaced, thereby realizing editing of the live document to obtain the edited live document.

[0232] For example, the live document can be a PDF format file. Since the support for element source data streams in PDF files has been added since PDF version 1.4, PDF can be extended, and an Extensible Metadata Platform (XMP) can be used to add extensible metadata based on the Extensible Markup Language (XML) standard like in other file formats. This allows metadata to be attached to any string stream in the document, such as special camera picture elements can be realized regarding adding an entrance animation.

[0233] In one embodiment, with reference to Figure 2l , 219 may be an image content element in a PDF file. Through the live document processing method described in this application, the image content element can be replaced with a content element including a camera screen, so that after replacement, the content element shown by 219 is extended to an enhanced object in the PDF.

[0234] In addition, between the content element corresponding to the extended enhanced object and the ordinary content element, any overlay or layout is possible. For example, with reference to Figure 2l , 219 is the extended video stream content element, and 220, 221, and 222 are all ordinary content elements in the PDF file, that is, the content elements that have not been extended. It can be seen that between the ordinary content element and the video stream content element, any overlay or layout is possible.

[0235] After attaching additional metadata to the content element to obtain the edited live document, the edited live document can be displayed during the live broadcast. Specifically, the step of "displaying the edited live document during the live broadcast" may include:

[0236] Obtain the edited live document, where the edited live document includes a content element, and the content element is attached with additional metadata describing the target replacement effect;

[0237] Based on the identification node in the additional metadata, parse the additional metadata to determine the type attribute and content information of the target replacement content element;

[0238] Generate the target replacement content element of the content element according to the type attribute and content information;

[0239] Replace the content element with the target replacement content element to obtain the replaced live document, so that the live client can display the replaced live document during the live broadcast.

[0240] In one embodiment, a teacher can process a live document in PDF format through the live document processing method described in this application in the live client. For example, the teacher can replace content element A in the live document with video stream content element B. Specifically, the live client can generate additional metadata describing the target replacement effect and attach the additional metadata to the content element to be replaced to obtain the edited live document.

[0241] Further, when the live client wants to display the edited live document, for example, when a teacher wants to use the replaced PDF file to start a live class, the terminal can read and parse the additional metadata in the edited live document, and use the video source described by the additional metadata for rendering, so as to display the edited live document on the live client. That is, it is realized that in the PDF file displayed by the teacher on the live client, the displayed content element is the video stream content element B, rather than the content element A, that is, the content element A is replaced by the video stream content element B.

[0242] In an embodiment, the content element in the live document can be replaced with a video stream content element. For example, the content element including the camera screen or the content element including the video screen. Then, the additional metadata format parsing process can refer to Figure 2m :

[0243] After the terminal obtains the edited live document, it can read the additional metadata from the edited live document and parse the additional metadata. Specifically, first, it can determine the value of ExementType by reading the identification node pair <pdfmetadata>< / pdfmetadata> nested in the <exementtype>< / exementtype> , so as to determine the type attribute of the target replacement content element. Referring to Figure 2m , it can be known that the value of ExementType read is VideoStream, that is, it is determined that the type of the target replacement content element is a video stream content element.

[0244] Further, it can determine the content information of the target replacement content element by reading the sub-identification node nested in the identification node pair <videostreamsrc>< / videostreamsrc> . Specifically, referring to Figure 2m , if the sub-identification node is <file>< / file> , that is, when the target replacement content element is a content element including a video screen, the filePath, that is, the storage path of the video file, can be obtained by reading the value of the sub-identification node <file>< / file> ; if the sub-identification node is <camera>< / camera> , that is, when the target replacement content element is a content element including a camera screen, the deviceID, that is, the camera device required to capture the camera screen, can be obtained by reading the value of the sub-identification node <camera>< / camera> .

[0245] After determining the type attribute and content information of the target replacement content element, the target replacement content element of the content element can be generated according to the type attribute and content information. Specifically, the step "generating the target replacement content element of the content element according to the type attribute and content information" can include:

[0246] Determine the element type to which the target replacement content element belongs according to the type attribute;

[0247] Determine the display data corresponding to the target replacement content element of this element type based on the content information;

[0248] Fill the display data into the target replacement content element of this element type to generate the target replacement content element of the content element.

[0249] In one embodiment, refer to Figure 2m , when the content element to be replaced is an image content element and the target replacement content element is a content element including a video frame, the video file to be displayed can be obtained by opening the file specified by filePath, and the terminal can render the video frame in the video file to the target replacement content area to implement the replacement of the content element.

[0250] In another embodiment, refer to Figure 2m , when the content element to be replaced is an image content element and the target replacement content element is a content element including a camera frame, the camera frame to be displayed can be obtained by opening the device specified by deviceID, and the terminal can render the camera frame to the target replacement content area to implement the replacement of the content element.

[0251] As can be seen from the above, this embodiment can display a live document editing page, where the live document editing page includes a content editing area, and the content editing area includes at least one content element of the live document; in response to an operation on the content element, an editing control corresponding to the content element is displayed, and based on the editing operation on the editing control, the edited live document is displayed during the live broadcast. The edited live document is the live document after replacing the content element with a target replacement content element, and the target replacement content element is determined based on additional metadata and the content element, and the additional metadata is determined based on the editing operation.

[0252] This solution can efficiently replace the content elements in a live document with target replacement content elements. Specifically, based on the editing operations on the content elements, this solution can determine additional metadata that describes the target replacement effect. Further, based on this additional metadata and the content elements to be replaced, it can determine the target replacement content elements, thus achieving element replacement in the live document. In this way, by determining the additional metadata based on the editing operations on the content elements, the additional metadata can become the information carrier that describes the target replacement effect desired by the user. That is, the user can present the target replacement effect they want through the generated additional metadata by simply performing editing operations on the content elements, such as selection operations. Further, this solution can achieve element replacement by attaching the generated additional metadata to the content elements to be replaced and display the replacement effect during the live broadcast. In this way, element replacement can be achieved without damaging the original document content in the live document. Therefore, this solution improves the processing efficiency of the live document and improves the processing method of the live document, enabling the live broadcast to be carried out efficiently.

[0253] Moreover, this solution provides multiple ways to trigger the editing of the live document. For example, the live document can be edited before the live broadcast starts or during the live broadcast. This not only provides multiple functions for the user but also greatly improves the processing efficiency of the live document, facilitating more efficient live broadcasts.

[0254] In addition, taking the live document as a PDF format file as an example, for an already generated PDF, it is possible to replace the content elements therein. For example, they can be replaced with content elements including camera images or video images. In this way, this solution greatly expands the functions of the PDF file and also brings corresponding beneficial effects in applications. For example, when this solution is applied to the online education scenario, the content elements in the PDF courseware can be replaced with content elements including camera images. In this way, during the classroom live broadcast, students can see the teacher's head, body language, and expressions during teaching through the camera image in the PDF file. Further, compared with the traditional online education application scenario, if both the courseware content and the teacher's camera image need to be shown to the students, the server needs to send two video images to the student client. However, this solution replaces the content elements in the PDF file with video stream content elements and nests the video stream content elements in the PDF file. Therefore, the server only needs to send one video image to the student client. In this way, this solution can avoid wasting computing resources and improve the utilization rate of computing resources.

[0255] According to the method described in the above embodiments, the following will give further detailed examples.

[0256] In this embodiment, the live document processing device integrated in the server and the terminal will be taken as an example for illustration. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto. The terminal and the server can be directly or indirectly connected through wired or wireless communication methods, and this application does not make any restrictions here.

[0257] As Figure 3 shown, a live document processing method has the following specific process:

[0258] 301. The terminal displays the live document editing page of the first live client. Among them, the live document editing page includes a content editing area, and the content editing area includes content elements of the live document.

[0259] In this embodiment, the first live client can be the live client of the anchor in the live promotion application scenario, and the live document can be a promotion document in PDF format.

[0260] In actual applications, there can be various situations for triggering the display of the live document editing page of the first live client. For example, it can be during the live preparation stage before the anchor starts the live promotion; another example is that when the anchor needs to edit the promotion document during the live broadcast.

[0261] 302. The terminal determines private description data that describes the target replacement content element based on the editing operation on the content element. Among them, the private description data includes the type attribute and content information of the target replacement content element.

[0262] Among them, the anchor's editing operations on the content element can include various types. For example, it can include simple operations such as inserting and editing pictures, texts, shapes, etc.; another example is that an existing content element can be replaced with a content element of an enhanced picture type so that the existing content element can be replaced with a content element including a camera screen or a video screen; another example is adding a dynamic display effect to the existing content element, where the dynamic display effect can include an entrance animation, such as fading in, sliding in, etc.; or an exit animation, such as fading out, sliding out, etc.; or an emphasis animation, such as flashing, magnifying, etc.

[0263] Further, the terminal may determine private description data for describing a target replacement content element based on the host's editing operation on the content element, where the private description data may include the type attribute and content information of the target replacement content element.

[0264] 303. The terminal generates additional metadata for describing the target replacement effect according to the content element, type attribute, and content information.

[0265] In an embodiment, the terminal may obtain a metadata generation template for information to be injected according to the content element to be replaced, where the metadata generation template may include at least one identification node. Further, the terminal may perform type marking on the identification node according to the type attribute, and obtain the marked metadata generation template. The terminal may inject the content information into the marked metadata generation template to generate additional metadata for describing the target replacement effect.

[0266] 304. The terminal attaches the additional metadata to the content element to obtain an edited live document.

[0267] Since the support for the source data stream by elements in PDF files has been added since the PDF 1.4 version, PDF can be extended, and an Extensible Metadata Platform (XMP) can be used to add extensible metadata based on the Extensible Markup Language (XML) standard as in other file formats. This allows metadata to be attached to any string stream in the document, such as for adding an entrance animation, and special camera picture elements can be realized.

[0268] Therefore, in this embodiment, the live document can be edited by attaching additional metadata for describing the target replacement effect to the content element to be replaced, so as to obtain an edited live document.

[0269] 305. The terminal displays the live broadcast picture on the first live client and sends the live broadcast picture to the server, where the live broadcast picture includes the edited live document.

[0270] In an embodiment, the terminal where the host live client is located may, after obtaining the edited live document, read and parse the additional metadata for describing the target replacement effect from the edited live document. And, based on the identification node in the additional metadata, the metadata may be parsed to determine the type attribute and content information of the target replacement content element. Further, the terminal may generate a target replacement content element for the content element to be replaced according to the type attribute and the content information.

[0271] During the live broadcast, when the terminal needs to display the edited live document on the meeting content page of the host's live broadcast client, it can use the video source described by the additional metadata for rendering, that is, replace the content element to be replaced with the corresponding target replacement content element.

[0272] Meanwhile, the host's live broadcast client can send the live video on it to the server, so that the server can send the live video to the second live broadcast client, that is, the viewer's live broadcast client, to achieve live broadcast.

[0273] It should be noted that in practical applications, an editor can be designed to complete steps 302 and 303, and a reader can be designed to complete step 304 and the part of "displaying the live video on the first live broadcast client" in step 305.

[0274] 306. The server sends the live video to the second live broadcast client, so that the second live broadcast client can display the live video.

[0275] In this embodiment, the second live broadcast client can be the viewer's live broadcast client. And since the content elements in the live document have been replaced, the live video received by the viewer's live broadcast client presents the target replacement content elements, that is, the replaced effect.

[0276] As can be seen from the above, the embodiments of the present application can efficiently replace the content elements in the live document with target replacement content elements. Specifically, the solution can determine the additional metadata describing the target replacement effect based on the editing operation on the content element, and further determine the target replacement content element based on the additional metadata and the content element to be replaced, so as to achieve element replacement for the live document. In this way, by determining the additional metadata based on the editing operation on the content element, the additional metadata can become the information carrier describing the target replacement effect desired by the user. That is, the user can present the target replacement effect he wants through the generated additional metadata only by performing simple editing operations on the content element, such as selection operations. Further, the solution can achieve element replacement by attaching the generated additional metadata to the content element to be replaced and display the replacement effect during the live broadcast. In this way, element replacement can be achieved without damaging the original document content in the live document. Therefore, the embodiments of the present application improve the processing efficiency of the live document and improve the processing method of the live document, enabling the live broadcast to be carried out efficiently.

[0277] Moreover, taking the scenario of live promotion where the live document is a PDF file as an example, this solution can not only send only one video stream to the second live client, improving the utilization rate of computing resources, but also eliminate the need to display two rectangular pages on a rectangular screen, improving the layout of the second live client and increasing the utilization rate of the screen space.

[0278] In addition, by designing an editor and a reader for live documents, this solution can not only add content elements including camera images or video streams, as well as content elements with dynamic display effects, to live documents such as PDF files, but also restore the display effects set by the editor through the reader, enabling more efficient live operations.

[0279] To better implement the above method, correspondingly, an embodiment of the present application further provides a live document processing device, which can be integrated into a terminal. The terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto.

[0280] For example, as Figure 4a shown, the live document processing device may include a page display unit 401 and a document display unit 402, as follows:

[0281] The page display unit 401 is configured to display a live document editing page, where the live document editing page includes a content editing area, and the content editing area includes at least one content element of the live document;

[0282] The document display unit 402 is configured to, in response to an operation on the content element, display an editing control corresponding to the content element, and based on an editing operation on the editing control, display an edited live document during live broadcast. The edited live document is a live document in which the content element is replaced with a target replacement content element, and the target replacement content element is determined based on additional metadata and the content element, and the additional metadata is determined based on the editing operation.

[0283] In one embodiment, referring to Figure 4b , the document display unit 402 may include:

[0284] The list display subunit 4021 may be configured to, in response to an operation on the content element, display a replacement control list on the live document editing page, where the replacement control list includes at least one element replacement control;

[0285] The metadata generation subunit 4022 can be used to generate additional metadata describing the target replacement effect based on the element replacement operation for the element replacement control;

[0286] The attachment subunit 4023 can be used to attach the additional metadata to the content element to obtain an edited live document;

[0287] The document display subunit 4024 can be used to display the edited live document during the live broadcast.

[0288] In one embodiment, the list display subunit 4021 can be used to:

[0289] In response to an editing operation on the content element, display an edit control list on the live document editing page, where the edit control list includes a replacement trigger control; in response to a replacement trigger operation on the replacement trigger control, display a replacement control list on the live document editing page, where the replacement control list includes element replacement controls corresponding to at least one data acquisition module for acquiring display data in the target replacement content element.

[0290] In one embodiment, the live document editing page includes an edit control menu, and the edit control menu includes at least one edit control; the list display subunit 4021 can be used to:

[0291] In response to an editing operation on the content element, display a replacement trigger control in the edit control menu; in response to a replacement trigger operation on the replacement trigger control, display a replacement control list on the live document editing page, where the replacement control list includes element replacement controls corresponding to at least one element display mode.

[0292] In one embodiment, the metadata generation subunit 4022 can be used to:

[0293] Based on the element replacement operation for the element replacement control, determine private description data describing the target replacement content element, where the private description data includes the type attribute and content information of the target replacement content element; generate additional metadata describing the target replacement effect according to the content element, the type attribute, and the content information.

[0294] In one embodiment, the replacement control list includes element replacement controls corresponding to at least one data acquisition module for acquiring real-time data displayed in the target replacement content element; the metadata generation subunit 4022 can be specifically used to:

[0295] In response to an element replacement operation on the element replacement control for the element, determine the type attribute of the target replacement content element; display a device selection window for determining the required data collection device on the live document editing page, where the device selection window includes at least one candidate data collection device; determine the target data collection device selected by the user from the device selection window; in response to a device selection completion operation on the device selection window, determine the content information corresponding to the target replacement content element, where the content information includes the acquisition path information of the real-time data.

[0296] In one embodiment, the replacement control list includes element replacement controls corresponding to at least one data collection module, and the data collection module is used to collect the offline data displayed in the target replacement content element; the metadata generation subunit 4022 may specifically be used for:

[0297] In response to an element replacement operation on the element replacement control, determine the type attribute of the target replacement content element; display an offline file selection window for obtaining the offline data on the live document editing page; determine the offline file selected by the user through the offline file selection window, where the offline file includes the offline data; in response to a file selection completion operation on the offline file selection window, determine the content information corresponding to the target replacement content element, where the content information includes the acquisition path information of the offline file.

[0298] In one embodiment, the replacement control list includes element replacement controls corresponding to at least one element display mode; the metadata generation subunit 4022 may specifically be used for:

[0299] In response to an element replacement operation on the element replacement control, determine the type attribute of the target replacement content element; based on the element display mode corresponding to the target replacement content element, determine the content information corresponding to the target replacement content element, where the content information includes the display effect information corresponding to the target replacement content element in the element display mode.

[0300] In one embodiment, the metadata generation subunit 4022 may specifically be used for:

[0301] According to the content element, obtain a metadata generation template for the information to be injected, where the metadata generation template includes at least one identification node; perform type marking on the identification node according to the type attribute to obtain a marked metadata generation template; inject the content information into the marked metadata generation template to generate additional metadata describing the target replacement effect.

[0302] In one embodiment, the document display subunit 4024 can be used for:

[0303] Obtain the edited live document, where the edited live document includes content elements, and the content elements are attached with additional metadata describing the target replacement effect; based on the identification nodes in the additional metadata, parse the additional metadata to determine the type attribute and content information of the target replacement content element; generate the target replacement content element of the content element according to the type attribute and the content information; replace the content element with the target replacement content element to obtain the replaced live document, so that the live client can display the replaced live document during the live broadcast.

[0304] In one embodiment, the document display subunit 4024 can specifically be used for:

[0305] Determine the element type to which the target replacement content element belongs according to the type attribute; determine the display data corresponding to the target replacement content element of the element type based on the content information; fill the display data into the target replacement content element of the element type to generate the target replacement content element of the content element.

[0306] In one embodiment, referring to Figure 4c , the page display unit 401 may include:

[0307] The first preparation display subunit 4011 can be used to display the live preparation page of the live client, where the live preparation page includes a document import trigger control for the live document;

[0308] The first page display subunit 4012 can be used to display the live document editing page based on the import trigger operation for the document import trigger control.

[0309] In one embodiment, referring to Figure 4d , the page display unit 401 may include:

[0310] The live display subunit 4013 can be used to display the live page of the live client during the live broadcast, where the live page includes a content display area of the live document and a document editing control for the live document;

[0311] The second page display subunit 4014 can be used to display the live document editing page in response to the document editing operation for the document editing control.

[0312] In one embodiment, referring to Figure 4e , the page display unit 401 may include:

[0313] The second preparation display subunit 4015 can be used to display the live preparation page of the live client, where the live preparation page includes document link information of at least one live document;

[0314] The third page display subunit 4016 can be used to display the live document editing page in response to a selection operation on the document link information.

[0315] In specific implementation, each of the above units can be implemented as an independent entity, or can be combined arbitrarily and implemented as the same or several entities. For the specific implementation of each of the above units, reference can be made to the foregoing method embodiments and will not be elaborated here.

[0316] As can be seen from the above, in the live document processing device of this embodiment, the page display unit 401 displays the live document editing page, where the live document editing page includes a content editing area, and the content editing area includes at least one content element of the live document; the document display unit 402 responds to an operation on the content element to display an editing control corresponding to the content element, and based on an editing operation on the editing control, displays the edited live document during the live broadcast. The edited live document is a live document obtained by replacing the content element with a target replacement content element, and the target replacement content element is determined based on additional metadata and the content element, and the additional metadata is determined based on the editing operation.

[0317] This solution can efficiently replace the content element in the live document with the target replacement content element. Specifically, this solution can determine additional metadata describing the target replacement effect based on the editing operation on the content element, and further determine the target replacement content element based on the additional metadata and the content element to be replaced, so as to achieve element replacement of the live document. In this way, by determining the additional metadata based on the editing operation on the content element, the additional metadata can become an information carrier describing the target replacement effect desired by the user. That is, the user can present the target replacement effect he wants through the generated additional metadata only by performing a simple editing operation on the content element, such as a selection operation. Further, this solution can achieve element replacement by attaching the generated additional metadata to the content element to be replaced and display the replacement effect during the live broadcast. In this way, element replacement can be achieved without damaging the original document content in the live document. Therefore, this solution improves the processing efficiency of the live document and improves the processing method of the live document, enabling the live broadcast to be carried out efficiently.

[0318] In addition, an embodiment of the present application further provides a computer device, which may be a device such as a server or a terminal. The server may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It may also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal may be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto. As Figure 5 shown, it shows a schematic structural diagram of the computer device involved in the embodiment of the present application. Specifically:

[0319] The computer device may include a memory 501 having one or more computer-readable storage media, an input unit 502, a display unit 503, an audio circuit 504, a wireless fidelity (WiFi) module 505, a processor 506 having one or more processing cores, and a power supply 507 and other components. Those skilled in the art can understand that Figure 5 the computer device structure shown in does not constitute a limitation on the computer device, and may include more or fewer components than shown, or combine certain components, or different component arrangements. Among them:

[0320] The memory 501 can be used to store software programs and modules. The processor 506 executes various functional applications and data processing by running the software programs and modules stored in the memory 501. The memory 501 may mainly include a program storage area and a data storage area. Among them, the program storage area may store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device (such as audio data, phone book, etc.), etc. In addition, the memory 501 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices. Correspondingly, the memory 501 may further include a memory controller to provide access to the memory 501 by the processor 506 and the input unit 502.

[0321] The input unit 502 can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control. Specifically, in a specific embodiment, the input unit 502 may include a touch-sensitive surface and other input devices. The touch-sensitive surface, also known as a touch display screen or a touchpad, can collect touch operations of the user on or near it (such as operations of the user using a finger, a stylus or any suitable object or accessory on or near the touch-sensitive surface), and drive the corresponding connection device according to a pre-set program. Optionally, the touch-sensitive surface may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the touch position of the user, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it to the processor 506, and can receive and execute the commands sent by the processor 506. In addition, various types such as resistive, capacitive, infrared, and surface acoustic wave can be used to implement the touch-sensitive surface. In addition to the touch-sensitive surface, the input unit 502 may further include other input devices. Specifically, the other input devices may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, etc.

[0322] The display unit 503 can be used to display information input by the user or information provided to the user and various graphical user interfaces of the computer device, and these graphical user interfaces can be composed of graphics, text, icons, videos and any combination thereof. The display unit 503 may include a display panel. Optionally, the display panel may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. Further, the touch-sensitive surface may cover the display panel. When the touch-sensitive surface detects a touch operation on or near it, it is transmitted to the processor 506 to determine the type of touch event, and then the processor 506 provides a corresponding visual output on the display panel according to the type of touch event. Although in Figure 5 it, the touch-sensitive surface and the display panel are implemented as two independent components to achieve input and input functions, but in some embodiments, the touch-sensitive surface and the display panel can be integrated to achieve input and output functions.

[0323] The audio circuit 504, the speaker, and the microphone can provide an audio interface between the user and the computer device. The audio circuit 504 can convert the received audio data into an electrical signal and transmit it to the speaker, which converts it into a sound signal for output. On the other hand, the microphone converts the collected sound signal into an electrical signal, which is received by the audio circuit 504, converted into audio data, and then the audio data is output to the processor 506 for processing. After that, it is sent through the RF circuit 501 to, for example, another computer device, or the audio data is output to the memory 501 for further processing. The audio circuit 504 may also include an earphone jack to provide communication between the peripheral earphone and the computer device.

[0324] WiFi belongs to short - range wireless transmission technology. Through the WiFi module 505, the computer device can help users send and receive emails, browse the web, and access streaming media, etc. It provides users with wireless broadband Internet access. Although Figure 5 the WiFi module 505 is shown, it can be understood that it does not belong to an essential component of the computer device and can be omitted entirely within the scope of not changing the essence of the invention according to needs.

[0325] The processor 506 is the control center of the computer device. It connects all parts of the entire device using various interfaces and circuits. By running or executing software programs and / or modules stored in the memory 501, and by calling the data stored in the memory 501, it executes various functions of the computer device and processes data, thereby performing an overall detection of the device. Optionally, the processor 506 may include one or more processing cores; preferably, the processor 506 may integrate an application processor and a modem processor. Among them, the application processor mainly processes the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communication. It can be understood that the above - mentioned modem processor may not be integrated into the processor 506 either.

[0326] The computer device also includes a power source 507 (such as a battery) that powers each component. Preferably, the power source can be logically connected to the processor 506 through a power management system, thereby realizing functions such as management of charging, discharging, and power consumption management through the power management system. The power source 507 may also include any components such as one or more DC or AC power sources, a recharge system, a power - failure detection circuit, a power converter or inverter, and a power - status indicator.

[0327] Although not shown, the computer device may further include a camera, a Bluetooth module, etc., which will not be elaborated here. Specifically, in this embodiment, the processor 506 in the computer device will load the executable files corresponding to the processes of one or more application programs into the memory 501 according to the following instructions, and the processor 506 will run the application programs stored in the memory 501 to implement various functions as follows:

[0328] Display a live document editing page, where the live document editing page includes a content editing area, and the content editing area includes at least one content element of the live document; in response to an operation on the content element, display an editing control corresponding to the content element, and based on the editing operation on the editing control, display the edited live document during the live broadcast, where the edited live document is the live document after replacing the content element with a target replacement content element, and the target replacement content element is determined based on additional metadata and the content element, and the additional metadata is determined based on the editing operation.

[0329] For the specific implementation of each of the above operations, reference may be made to the previous embodiments, which will not be elaborated here.

[0330] As can be seen from the above, the computer device of this embodiment can efficiently replace the content elements in the live document with target replacement content elements. Specifically, this solution can determine additional metadata describing the target replacement effect based on the editing operation on the content element, and further determine the target replacement content element based on the additional metadata and the content element to be replaced, so as to achieve element replacement in the live document. In this way, by determining the additional metadata based on the editing operation on the content element, the additional metadata can become an information carrier describing the target replacement effect desired by the user, that is, the user can present the target replacement effect he wants through the generated additional metadata only by performing simple editing operations on the content element, such as selection operations. Further, this solution can achieve element replacement by attaching the generated additional metadata to the content element to be replaced and display the replacement effect during the live broadcast. In this way, element replacement can be achieved without damaging the original document content in the live document. Therefore, the computer device of this embodiment improves the processing efficiency of the live document and improves the processing method of the live document, enabling the live broadcast to be carried out efficiently.

[0331] Those of ordinary skill in the art can understand that all or part of the steps in the above various methods can be completed by instructions or by controlling related hardware through instructions. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0332] To this end, an embodiment of the present application provides a storage medium storing multiple instructions that can be loaded by a processor to execute the steps in any of the live document processing methods provided by the embodiments of the present application. For example, the instructions may perform the following steps:

[0333] Display a live document editing page, where the live document editing page includes a content editing area, and the content editing area includes at least one content element of the live document; in response to an operation on the content element, display an editing control corresponding to the content element, and based on the editing operation on the editing control, display the edited live document during the live broadcast, where the edited live document is a live document obtained by replacing the content element with a target replacement content element, and the target replacement content element is determined based on additional metadata and the content element, and the additional metadata is determined based on the editing operation.

[0334] For the specific implementation of each of the above operations, reference may be made to the previous embodiments, which will not be elaborated here.

[0335] Among them, the storage medium may include: read-only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), a magnetic disk, an optical disc, etc.

[0336] Since the instructions stored in the storage medium can execute the steps in any of the live document processing methods provided by the embodiments of the present application, the beneficial effects achievable by any of the live document processing methods provided by the embodiments of the present application can be realized. For details, reference may be made to the previous embodiments, which will not be elaborated here.

[0337] According to one aspect of the present application, a computer program product or a computer program is provided. The computer program product or the computer program includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to execute the methods provided in various optional implementation manners of the above live document processing aspect.

[0338] The above has introduced in detail a live document processing method and apparatus provided by the embodiments of the present application. Specific examples are used in this article to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those skilled in the art, according to the idea of the present application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present application.< / file> < / file> < / pdfmetadata>

Claims

1. A live document processing method, characterized in that, Including: Display a live document editing page, wherein the live document editing page includes a content editing area, and the content editing area includes at least one content element of the live document; In response to an operation on the content element, display an editing control corresponding to the content element, and based on the editing operation on the editing control, display the edited live document during the live broadcast, including: In response to an operation on the content element, display a replacement control list on the live document editing page, wherein the replacement control list includes at least one element replacement control; Based on the element replacement operation on the element replacement control, generate additional metadata describing the target replacement effect; Attach the additional metadata to the content element to obtain the edited live document; Display the edited live document during the live broadcast; The edited live document is the live document after replacing the content element with a target replacement content element, and the target replacement content element is determined based on the additional metadata and the content element, and the additional metadata is metadata describing the target replacement effect determined based on the editing operation.

2. The live document processing method according to claim 1, wherein In response to an operation on the content element, display a replacement control list on the live document editing page, wherein the replacement control list includes at least one element replacement control, including: In response to an editing operation on the content element, display an editing control list on the live document editing page, wherein the editing control list includes a replacement trigger control; In response to a replacement trigger operation on the replacement trigger control, display a replacement control list on the live document editing page, wherein the replacement control list includes at least one element replacement control corresponding to a data acquisition module for acquiring display data in the target replacement content element.

3. The live document processing method according to claim 1, wherein The live document editing page includes an editing control menu, and the editing control menu includes at least one editing control; In response to an operation on the content element, display a replacement control list on the live document editing page, wherein the replacement control list includes at least one element replacement control, including: In response to an editing operation on the content element, display a replacement trigger control in the editing control menu; In response to a replacement trigger operation on the replacement trigger control, display a replacement control list on the live document editing page, wherein the replacement control list includes at least one element replacement control corresponding to the element display mode.

4. The live document processing method according to claim 1, wherein Based on the element replacement operation on the element replacement control, generate additional metadata describing the target replacement effect, including: Based on the element replacement operation on the element replacement control, determine private description data describing the target replacement content element, wherein the private description data includes the type attribute and content information of the target replacement content element; Generate additional metadata describing the target replacement effect according to the content element, the type attribute, and the content information.

5. The live document processing method according to claim 4, wherein The replacement control list includes at least one element replacement control corresponding to a data acquisition module, and the data acquisition module is used to acquire real-time data displayed in the target replacement content element; Based on the element replacement operation for the element replacement control, determine the private description data describing the target replacement content element, where the private description data includes the type attribute and content information of the target replacement content element, including: In response to the element replacement operation for the element replacement control, determine the type attribute of the target replacement content element; On the live document editing page, display a device selection window for determining the data acquisition device, where the device selection window includes at least one candidate data acquisition device; Determine the target data acquisition device selected by the user from the device selection window; In response to the device selection completion operation for the device selection window, determine the content information corresponding to the target replacement content element, where the content information includes the acquisition path information of the real-time data.

6. The live document processing method according to claim 4, wherein The replacement control list includes at least one element replacement control corresponding to a data acquisition module, and the data acquisition module is used to acquire offline data displayed in the target replacement content element; Based on the element replacement operation for the element replacement control, determine the private description data describing the target replacement content element, where the private description data includes the type attribute and content information of the target replacement content element, including: In response to the element replacement operation for the element replacement control, determine the type attribute of the target replacement content element; On the live document editing page, display an offline file selection window for acquiring the offline data; Determine the offline file selected by the user through the offline file selection window, where the offline file includes the offline data; In response to the file selection completion operation for the offline file selection window, determine the content information corresponding to the target replacement content element, where the content information includes the acquisition path information of the offline file.

7. The live document processing method according to claim 4, wherein The replacement control list includes at least one element replacement control corresponding to an element display mode; Based on the element replacement operation for the element replacement control, determine the private description data describing the target replacement content element, where the private description data includes the type attribute and content information of the target replacement content element, including: In response to the element replacement operation for the element replacement control, determine the type attribute of the target replacement content element; Based on the element display mode corresponding to the target replacement content element, determine the content information corresponding to the target replacement content element, where the content information includes the display effect information corresponding to the target replacement content element in the element display mode.

8. The live document processing method according to claim 4, wherein Generate additional metadata describing the target replacement effect according to the content element, the type attribute, and the content information, including: According to the content element, obtain a metadata generation template for information to be injected, where the metadata generation template includes at least one identification node; Type - mark the identification nodes according to the type attributes to obtain a marked metadata generation template; Inject the content information into the marked metadata generation template to generate additional metadata describing the target replacement effect.

9. The live document processing method according to claim 1, wherein During the live broadcast, display the edited live document, including: Obtain the edited live document, where the edited live document includes content elements, and the content elements are attached with additional metadata describing the target replacement effect; Based on the identification nodes in the additional metadata, parse the additional metadata to determine the type attributes and content information of the target replacement content elements; Generate target replacement content elements for the content elements according to the type attributes and the content information; Replace the content elements with the target replacement content elements to obtain a replaced live document, so that the live client can display the replaced live document during the live broadcast.

10. The live document processing method according to claim 9, wherein, Generate target replacement content elements for the content elements according to the type attributes and the content information, including: Determine the element type to which the target replacement content elements belong according to the type attributes; Based on the content information, determine the display data corresponding to the target replacement content elements of the element type; Fill the display data into the target replacement content elements of the element type to generate the target replacement content elements for the content elements.

11. The live document processing method according to claim 1, wherein, Display a live document editing page, including: Display a live preparation page of the live client, where the live preparation page includes a document import trigger control for the live document; Based on an import trigger operation for the document import trigger control, display a live document editing page.

12. The live document processing method according to claim 1, wherein Display a live document editing page, including: Display a live page of the live client during the live broadcast, where the live page includes a content display area of the live document and a document editing control for the live document; In response to a document editing operation for the document editing control, display a live document editing page.

13. The live document processing method according to claim 1, wherein Display a live document editing page, including: Display a live preparation page of the live client, where the live preparation page includes document link information of at least one live document; In response to a selection operation for the document link information, display a live document editing page.

14. A live document processing device, characterized in that, Include: A page display unit for displaying a live document editing page, where the live document editing page includes a content editing area, and the content editing area includes at least one content element of the live document; A document display unit, configured to display an editing control corresponding to the content element in response to an operation on the content element, and display the edited live document during the live broadcast based on an editing operation on the editing control, including: in response to an operation on the content element, displaying a replacement control list on the live document editing page, where the replacement control list includes at least one element replacement control; generating additional metadata describing a target replacement effect based on an element replacement operation on the element replacement control; attaching the additional metadata to the content element to obtain an edited live document; displaying the edited live document during the live broadcast; the edited live document is a live document after replacing the content element with a target replacement content element, the target replacement content element is determined based on the additional metadata and the content element, and the additional metadata is metadata describing the target replacement effect determined based on the editing operation.

15. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the live document processing method according to any one of claims 1-13.

16. A computer device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the live document processing method according to any one of claims 1-13.

17. A computer program product, characterized in that, A computer program product includes computer instructions stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to execute the steps of the live document processing method according to any one of claims 1-13.

Images