File insertion method and device for online collaboration drawing board and storage medium

Abstract

The invention provides a file insertion method and device for an online collaboration drawing board and a storage medium. The method comprises the following steps: in response to an insertion request for inserting a target portable document into a target online collaboration drawing board document, performing content analysis on first document content in the target portable document to obtain document content elements of various data types; determining a plurality of target document content elements in the document content elements based on an association relationship between the document content elements and second document content of the target online collaboration drawing board document; generating a plurality of online collaboration drawing board display elements according to the plurality of target document content elements and the data type corresponding to each target document content element; and displaying the plurality of online collaborative drawing board display elements in the target online collaborative drawing board document. According to the method, the checking efficiency of the inserted file in the online collaboration drawing board can be improved, and the collaboration efficiency of the online collaboration drawing board is further improved.

Description

Technical Field

[0001] This disclosure relates to the field of online documents, and in particular to a method, apparatus, and storage medium for inserting files into an online collaborative drawing board. Background Technology

[0002] Online collaborative whiteboards are virtual whiteboards that can be edited online by multiple users. They can receive and display data in multiple modalities, which can greatly enrich the dimensions of collaborative content display and improve collaboration efficiency.

[0003] In related technologies, when a file is inserted into an online collaboration board, a file link (text link or thumbnail link) is displayed on the board interface. Other users of the online collaboration board can click the file link displayed on the board to jump to the file application to view the file. However, this method results in low efficiency in viewing the content of the inserted file, thereby reducing the collaboration efficiency of the online collaboration board. Summary of the Invention

[0004] This disclosure provides a file insertion method, apparatus, and storage medium for an online collaborative drawing board, which can improve the collaboration efficiency of the online collaborative drawing board.

[0005] According to one aspect of this disclosure, a method for inserting files into an online collaborative drawing board is provided, comprising:

[0006] In response to an insertion request to insert a target portable document into a target online collaborative whiteboard document, the content of the first document in the target portable document is parsed to obtain document content elements of various data types;

[0007] Based on the association between the document content elements and the second document content of the target online collaborative drawing board document, multiple target document content elements are determined in the document content elements;

[0008] Multiple online collaborative whiteboard display elements are generated based on the multiple target document content elements and the data type corresponding to each target document content element;

[0009] The multiple online collaboration whiteboard display elements are displayed in the target online collaboration whiteboard document.

[0010] According to one aspect of this disclosure, a file insertion device for an online collaborative drawing board is provided, the device comprising:

[0011] The parsing unit is used to respond to an insertion request to insert a target portable document into a target online collaborative whiteboard document, and to parse the content of the first document in the target portable document to obtain document content elements of various data types.

[0012] The determining unit is configured to determine multiple target document content elements in the document content elements based on the association relationship between the document content elements and the second document content of the target online collaborative drawing board document;

[0013] The generation unit is used to generate multiple online collaborative whiteboard display elements based on the multiple target document content elements and the data type corresponding to each target document content element;

[0014] The display unit is used to display the multiple online collaborative whiteboard elements in the target online collaborative whiteboard document.

[0015] Optionally, in some embodiments, the determining unit includes:

[0016] A calculation subunit is used to calculate the content similarity between each document content element and the second document content of the target online collaborative drawing board document, and to obtain the first confidence level corresponding to each document content element;

[0017] The first determining subunit is used to determine the second confidence level corresponding to each document content element based on the data type of each document content element.

[0018] The second determining subunit is used to calculate the target confidence level corresponding to each document content element based on the first confidence level and the second confidence level, and to determine multiple target document content elements among the document content elements based on the target confidence level.

[0019] Optionally, in some embodiments, the computing subunit includes:

[0020] The generation module is used to perform content statistics on the document content elements in the target online collaborative whiteboard document and generate summary text based on the statistical content.

[0021] The calculation module is used to extract content text from each document content element, calculate the text similarity between each content text and the summary text, and obtain the first confidence level corresponding to each document content element.

[0022] Optionally, in some embodiments, the first determining subunit includes:

[0023] The statistics module is used to perform element type statistics on the online collaborative whiteboard elements in the target online collaborative whiteboard document, and obtain the statistical quantity corresponding to different element types;

[0024] The first determining module is used to determine the second confidence level corresponding to each document content element based on the statistical quantity.

[0025] Optionally, in some embodiments, the display unit includes:

[0026] The first acquisition subunit is used to acquire viewport information when the online collaborative whiteboard document is displayed in the first client.

[0027] The third determining subunit is used to determine the target image element from multiple candidate image elements in the online collaborative drawing display elements based on the viewport information. The multiple candidate image elements are multiple image elements with different resolutions generated based on the target document image in the target document content element.

[0028] A sending subunit is configured to send the target image element to the first client, so that the target image element is displayed in the display area of ​​the target online collaborative whiteboard document in the first client.

[0029] Optionally, in some embodiments, the third determining subunit includes:

[0030] The second determining module is used to determine the parameter range of the viewport parameters based on the viewport information;

[0031] The third determining module is used to determine the target image element from multiple candidate image elements according to the parameter range, wherein the multiple candidate image elements are multiple image elements with different resolutions generated based on the target document image in the target document content element.

[0032] Optionally, in some embodiments, the display unit includes:

[0033] The first display subunit is used to display an insertion element selection interface in the display area of ​​the target online collaborative canvas document in the second client that initiated the insertion request. The insertion element selection interface includes multiple candidate tags corresponding to the multiple online collaborative canvas elements.

[0034] The fourth determining subunit is used to determine multiple target online collaborative canvas display elements corresponding to the target candidate tag in response to the selection operation of the target candidate tag among the multiple candidate tags;

[0035] The display subunit is used to display the multiple target online collaborative canvas display elements in the target online collaborative canvas document.

[0036] Optionally, the file insertion device for the online collaborative drawing board provided in this application further includes:

[0037] The second display subunit is used to display the viewing permission setting interface of the target candidate tag in response to the triggering operation of the target candidate tag;

[0038] The fifth determining subunit is used to receive the permission setting information input in the viewing permission setting interface, and determine the viewing permission of each target online collaborative canvas display element according to the permission setting information.

[0039] Optionally, in some embodiments, the parsing unit includes:

[0040] An extraction subunit is used to extract image content contained in the first document content to obtain image-type document content elements; and

[0041] The first conversion subunit is used to extract the table content contained in the first document content and convert the table content into a table-type document content element; and

[0042] The second conversion subunit is used to extract the text content contained in the first document content and convert the text content into document content elements of editable text type.

[0043] Optionally, in some embodiments, the second conversion subunit includes:

[0044] The splitting module is used to extract the text content contained in the first document content and split the text content into multiple paragraph texts;

[0045] The conversion module is used to convert the multiple paragraph texts into multiple document content elements of an editable document type.

[0046] Optionally, in some embodiments, the generation unit includes:

[0047] The second acquisition subunit is used to acquire the online collaborative canvas element template corresponding to each data type;

[0048] A generation subunit is used to generate multiple online collaboration board display elements based on the multiple target document content elements and the online collaboration board element template corresponding to each target document content element.

[0049] According to one aspect of this disclosure, a storage medium is provided that stores a computer program, which, when executed by a processor, implements the file insertion method for an online collaborative drawing board as described above.

[0050] According to one aspect of this disclosure, a computer program product is provided, comprising a computer program that is read and executed by a processor of a computer device, causing the computer device to perform the file insertion method for an online collaborative drawing board as described above.

[0051] The file insertion method for online collaborative whiteboards provided in this disclosure specifically includes, in response to an insertion request to insert a target portable document into a target online collaborative whiteboard document, parsing the content of a first document in the target portable document to obtain document content elements of various data types; determining multiple target document content elements among the document content elements based on the association relationship between the document content elements and the second document content of the target online collaborative whiteboard document; generating multiple online collaborative whiteboard display elements according to the multiple target document content elements and the data type corresponding to each target document content element; and displaying the multiple online collaborative whiteboard display elements in the target online collaborative whiteboard document.

[0052] In this embodiment, when a user of the online collaboration whiteboard inserts a target portable document into the target online collaboration whiteboard document, the document content of the target portable document can be automatically parsed to convert it into document content elements of various data types. Then, based on the relationship between the document content elements and the content of the target online collaboration whiteboard document, important target document content elements can be identified from among the various data types of document content elements. These target document content elements are then converted into online collaboration whiteboard display elements and displayed in the target online collaboration whiteboard document. Thus, the file insertion method for the online collaboration whiteboard provided in this application can not only automatically identify important content from portable documents but also directly display this content as collaboration whiteboard elements in the collaboration whiteboard. This avoids the problem of low collaboration efficiency caused by users having to switch to other document applications when viewing these file contents, thereby improving the collaboration efficiency of the online collaboration whiteboard.

[0053] Other features and advantages of this disclosure will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the disclosure. The objectives and other advantages of this disclosure may be realized and obtained by means of the structures particularly pointed out in the description, claims and drawings. Attached Figure Description

[0054] The accompanying drawings are provided to further understand the technical solutions of this disclosure and constitute a part of the specification. They are used together with the embodiments of this disclosure to explain the technical solutions of this disclosure and do not constitute a limitation on the technical solutions of this disclosure.

[0055] Figure 1 This is a system architecture diagram of the file insertion method for an online collaborative whiteboard according to embodiments of the present disclosure;

[0056] Figure 2 yes Figure 1 Software structure diagram of the online collaborative drawing board;

[0057] Figure 3 This is a flowchart illustrating the file insertion method for the online collaborative drawing board disclosed herein.

[0058] Figure 4 A schematic diagram illustrating a scenario for the file insertion method in the online collaborative whiteboard provided in this disclosure;

[0059] Figure 5 Another scenario illustration of file insertion for the online collaborative canvas provided in this application;

[0060] Figure 6 This is another scenario illustration of the file insertion method for the online collaborative canvas provided in this disclosure;

[0061] Figure 7 This is another scenario illustration of the file insertion method for the online collaborative canvas provided in this disclosure;

[0062] Figure 8 This is another scenario illustration of the file insertion method for the online collaborative canvas provided in this disclosure;

[0063] Figure 9 Another flowchart illustrating the file insertion method for the online collaborative canvas provided in this disclosure;

[0064] Figure 10 This is yet another flowchart illustrating the file insertion method for the online collaborative drawing board provided in this disclosure;

[0065] Figure 11 A schematic diagram of the file insertion device for the online collaborative drawing board provided in this embodiment of the disclosure;

[0066] Figure 12 Based on embodiments of this disclosure Figure 3 The terminal structure diagram of the file insertion method in the online collaborative canvas is shown.

[0067] Figure 13 Based on embodiments of this disclosure Figure 3 The server architecture diagram shows the file insertion method for the online collaborative whiteboard. Detailed Implementation

[0068] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of this disclosure.

[0069] Before providing a further detailed description of the embodiments of this disclosure, the terms and concepts used in these embodiments are explained, and they are subject to the following interpretations:

[0070] Portable Document Format (PDF): A file format designed to present documents across different computer systems and devices while maintaining their formatting.

[0071] Cloud Object Storage (COS): A distributed storage service used to store and manage large amounts of unstructured data, such as images, videos, and documents.

[0072] Online collaborative whiteboards are whiteboard documents that allow multiple people to collaborate and edit simultaneously online. They feature unlimited space and support for various content formats. Online collaborative whiteboards can be widely used in various fields such as online meetings, online education, and collaborative design and office work to improve work efficiency in these fields.

[0073] When using online collaboration whiteboards for online meetings, one user may need to upload a PDF document to the whiteboard. This can be done by dragging and dropping the PDF document into the whiteboard interface or by inserting it through the whiteboard's file insertion interface. In related technologies, after a PDF file is inserted, text links, thumbnail links, or card links to the PDF document are displayed on the whiteboard's interface. Other users can then click on these links to view the document's content in a PDF reader. However, this method requires switching from the whiteboard application to a PDF reader, resulting in low file viewing efficiency. Furthermore, this method only allows viewing the entire inserted file; users still need to locate the specific content they need within the file, significantly reducing the overall collaboration efficiency of the online collaboration whiteboard.

[0074] To address the issue that uploaded PDF files to online collaboration boards are only displayed as file links, causing users of other online collaboration boards to have to switch from the online collaboration board application to a PDF reader to view the file and search for specific content within the entire file, resulting in low collaboration efficiency, this application provides a file insertion method for online collaboration boards. This method aims to improve the efficiency of file viewing for other users of online collaboration boards after file insertion, thereby enhancing the overall collaboration efficiency of online collaboration boards.

[0075] System architecture and scenario description of the embodiments disclosed herein

[0076] Figure 1This is a system architecture diagram applied to the file insertion method of the online collaborative whiteboard according to embodiments of the present disclosure. It includes a terminal 140, an Internet 130, a gateway 120, a server 110, etc.

[0077] Terminal 140 includes various forms of devices with display screens, such as desktop computers, laptops, PDAs (personal digital assistants), mobile phones, vehicle terminals, home theater terminals, dedicated terminals, intelligent voice interaction devices, smart home appliances, or aircraft. Furthermore, it can be a single device or a collection of multiple devices. Terminal 140 can communicate with the Internet 130 via wired or wireless means to exchange data. In this embodiment, there can be multiple terminals 140, and each terminal 140 can be equipped with an online collaborative whiteboard application.

[0078] Server 110 refers to a computer system capable of providing certain services to terminal 140. Compared to ordinary terminal 140, server 110 has higher requirements in terms of stability, security, and performance. Server 110 can be a single high-performance computer in a network platform, a cluster of multiple high-performance computers, a portion of a single high-performance computer (e.g., a virtual machine), or a combination of portions of multiple high-performance computers (e.g., virtual machines). In this embodiment, server 110 can specifically be an application server for an online collaborative drawing board, used to process operation data uploaded by clients of the online collaborative drawing board application loaded on multiple terminals 140 and determine the content displayed in the online collaborative drawing board. Server 110 can also be connected to a storage device, which can be the aforementioned distributed storage service, storing data such as the user registration information, permission information, and collaborative drawing board element templates of the online collaborative drawing board application.

[0079] Gateway 120, also known as an internetwork connector or protocol converter, is a computer system or device that acts as a translator, enabling network interconnection at the transport layer. It translates between two systems using different communication protocols, data formats, languages, or even completely different architectures. Gateways can also provide filtering and security functions. Messages sent from terminal 140 to server 110 are forwarded to the corresponding server 110 via gateway 120. Messages sent from server 110 to terminal 140 are also forwarded to the corresponding terminal 140 via gateway 120. In this embodiment, terminal 140 sends operation data received from the online collaborative whiteboard client to server 110 via gateway 120. Server 110 then sends a display screen generated based on the operation data sent by multiple clients to each terminal 140 for display via gateway 120.

[0080] For example, the software structure diagram of an online collaborative whiteboard is as follows: Figure 2As shown, the backend architecture of the online collaborative whiteboard can be divided into a business layer, an access layer, a collaboration layer, an engine layer, and a storage layer. The business layer provides a user interface and receives user editing operations, such as creating, editing, inserting (files), and collaborating. The access layer provides communication interfaces between the online collaborative whiteboard backend and external systems, supporting various communication protocols, such as Remote Procedure Call (RPC), Hypertext Transfer Protocol (HTTP), WebSocket (a protocol for full-duplex communication over a single Transmission Control Protocol connection), and Common Gateway Interface (CGI). The collaboration layer converts various data received from the business and access layers into a unified format before forwarding it to the engine layer. The engine layer is the core of the online collaborative whiteboard's functionality, including process management, basic capability modules, and core capability modules. Processes include gateways and document management; basic capabilities include collaborative editing and operation execution; and core capabilities include file insertion. The storage layer is used to store various types of data from the engine layer, providing a variety of storage services such as local caching, remote dictionary service (Redis), and distributed storage.

[0081] The file insertion method for the online collaborative whiteboard provided in this disclosure can be specifically applied to server 110. Specifically, when the file insertion method for the online collaborative whiteboard provided in this disclosure is applied to server 110, server 110 receives an insertion request sent by terminal 140 to insert a target portable document into a target online collaborative whiteboard document. Then, server 110 obtains the target portable document and parses its content to obtain document content elements of various data types. Then, based on the association between the document content elements and the second document content of the target online collaborative whiteboard document, server 110 determines multiple target document content elements within the document content elements. Further, server 110 generates multiple online collaborative whiteboard display elements according to the multiple target document content elements and the data type corresponding to each target document content element. Then, server 110 updates the multiple online collaborative whiteboard display elements in the target online collaborative whiteboard document to display the multiple online collaborative whiteboard display elements in the target online collaborative whiteboard document.

[0082] The embodiments disclosed herein can be applied in various scenarios, such as online meeting scenarios or online education scenarios.

[0083] When this embodiment of the disclosure is applied to an online meeting scenario, the meeting initiator can create a target online collaborative whiteboard document and then send a link to that document to other members participating in the online meeting. Other participants can access the collaborative editing page of the target online collaborative whiteboard document by clicking the link and authorizing login to the online collaborative whiteboard application. At this time, both the meeting initiator and other participants can edit the target online collaborative whiteboard document to express their meeting views. Editing operations can involve inserting templates, mind maps, flowcharts, tables, or editing text, or inserting files. Specifically, when a participant needs to insert a PDF file, they can drag and drop the PDF file into the target online collaborative whiteboard document interface on their corresponding client. The online collaborative whiteboard backend server can then obtain the PDF file, parse its content, and categorize the content according to data type (e.g., images, tables, and text) to obtain multiple document content elements. Then, the online collaboration whiteboard backend server further analyzes the correlation between these document content elements of various data types and the content of the target online collaboration whiteboard document, removing document content elements with weak correlation and retaining multiple target document content elements. Further, the online collaboration whiteboard backend server can convert each target document content element into a corresponding content element for the online collaboration whiteboard based on its data type, resulting in multiple online collaboration whiteboard display elements. Finally, the online collaboration whiteboard backend server can display these multiple online collaboration whiteboard display elements in the target online collaboration whiteboard document.

[0084] It should be understood that the above description only illustrates some application scenarios of this disclosure. The business scenarios to which this disclosure can be applied may include, but are not limited to, the specific scenarios mentioned above.

[0085] General Description of Embodiments in this Disclosure

[0086] This disclosure provides a method for inserting files into an online collaborative drawing board. For example... Figure 3 The diagram shown is a flowchart illustrating a file insertion method for an online collaborative drawing board provided in this disclosure. This method can be applied to a file insertion device for an online collaborative drawing board, which can be integrated into the backend server of the online collaborative drawing board. The file insertion method for this online collaborative drawing board specifically includes:

[0087] Step 310: In response to the insertion request to insert the target portable document into the target online collaborative whiteboard document, perform content parsing on the first document content in the target portable document to obtain document content elements of various data types.

[0088] To address the inefficient viewing of portable documents after insertion into online collaborative whiteboards in related technologies, this disclosure provides a method for inserting files into online collaborative whiteboards. Specifically, this method can be applied after users of an online collaborative whiteboard insert a portable document into it. Specifically, as an application, an online collaborative whiteboard can create multiple online collaborative whiteboard documents, each usable for different collaborative scenarios. These different scenarios can be due to different whiteboard purposes (online meetings, online collaborative drawing) or different participating collaborative objects. The file insertion method for online collaborative whiteboards provided in this application can be applied to target online collaborative whiteboard documents corresponding to any scenario.

[0089] In a target online collaboration whiteboard document, collaborators can be either the initiator (creator) of the document or other participants (invited parties). This means that anyone inserting a portable file into the target document can be either the initiator or another participant. When any collaborating party inserts a portable document (e.g., the target portable document) into the document interface of the target online collaboration whiteboard document displayed in their corresponding online collaboration whiteboard client, the backend server of the online collaboration whiteboard receives the request from the collaborating party to insert the target portable document. The backend server then retrieves the target portable document and parses its content (referred to as the first document content here to distinguish it from the document content of the target online collaboration whiteboard document below), obtaining document content elements of various data types.

[0090] The backend server of the online collaborative drawing board parses the content of the target portable document. This can be done either by the backend server itself or by calling other functional services, such as a dedicated file parsing service, to parse the initial content of the target portable document. Specifically, the backend server can call the application programming interface (API) (e.g., PDF.js) of the dedicated file parsing service to parse the content of the target portable document.

[0091] In some embodiments, the first document content of the target portable document is parsed, specifically by paginating the target portable document to obtain multiple document pages. Then, each document page is converted into an image, resulting in multiple document page images. In this embodiment, each document page image can be further converted into elements of an online collaborative drawing board, specifically image elements. Specifically, multiple document page images can first be stored in their original form in the storage corresponding to the backend server of the aforementioned online collaborative drawing board. This storage can specifically be a COS (Coding Storage System). Then, image elements of the online collaborative drawing board can be generated based on the original document page images stored in the COS.

[0092] In some embodiments, the content of a first document in the target portable document is parsed to obtain document content elements of various data types, including:

[0093] Extract the image content contained in the first document to obtain image-type document content elements; and

[0094] Extract the table content contained in the first document and convert it into a table-type document content element; and

[0095] Extract the text content contained in the first document and convert it into an editable text document content element.

[0096] In this embodiment of the disclosure, content parsing of the first document content in the target portable document can specifically involve separating document content elements of different data types within the first document content. Specifically, this can involve separating images from the first document content to obtain image-type document content elements, separating table content from the first document content to obtain table-type document content elements, and separating text content from the first document content to obtain text-type document content elements.

[0097] Specifically, the image content contained in the first document can be extracted first to obtain image-type document content elements. This process can involve using a document scanning tool to scan the content of the target portable document and determining the location of the images within the document based on the scan information. Then, the image content can be cropped from the target portable document based on the determined image locations to obtain image-type document content elements.

[0098] Simultaneously, the scanned information can also determine the location of table content and text content within the target portable document. Then, the scanned table content is converted into table-type document content elements. It's understood that the table content in the target portable document is readable but not editable, while the converted table-type document content elements can be either editable in themselves (e.g., adding or deleting rows or columns) or editable in their content (i.e., the text or numbers within the table can be edited). Similarly, the scanned text content can be converted into editable text-type document content elements. Specifically, optical character recognition (OCR) technology can be used to identify the characters corresponding to the text content locations in the target portable document, and then these characters are arranged according to the original layout.

[0099] In some embodiments, extracting text content contained in the first document content and converting the text content into document content elements of an editable text type includes:

[0100] Extract the text content contained in the first document and split the text content into multiple paragraphs;

[0101] Convert multiple paragraph texts into multiple document content elements of an editable document type.

[0102] For each image in the target portable document, a corresponding image-type document content element can be generated; for each table in the target portable document, a corresponding table-type document content element can also be generated; however, for the text content in the target portable document, since the text content is large and appears continuously without obvious segmentation, it is not suitable to include all the text in the target portable document into a single editable document-type document content element.

[0103] Therefore, in this embodiment, the text content in the first document content of the target portable document can be split, specifically by splitting it into paragraphs, thus dividing the text content in the first document content into multiple paragraph texts. Then, for each paragraph text, it is converted into multiple document content elements of an editable document type.

[0104] In some embodiments, the text content in the first document content of the target portable document is split, or it can be split by page to obtain multiple page texts. Then, for each page text, it is converted into multiple document content elements of an editable document type.

[0105] In other embodiments, the text content in the first document can be split according to the content structure, wherein the content structure can be distinguished by subheadings. Multiple subheadings divide the entire text content into multiple thematic areas, each thematic area includes multiple paragraphs, and the multiple paragraphs may occupy less than one page of space or more than one page of space.

[0106] Step 320: Based on the association between document content elements and the second document content of the target online collaborative whiteboard document, identify multiple target document content elements in the document content elements.

[0107] The backend server of the online collaboration whiteboard parses the first document content of the target portable document to obtain document content elements of various data types. It then further identifies several important target document content elements based on the relationship between these content elements and the second document content of the target online collaboration whiteboard document. Specifically, these important target document content elements are those with a similar relationship to the second document content of the target online collaboration whiteboard document. This avoids displaying unimportant document content on the online collaboration whiteboard, which could interfere with the retrieval of useful information, thereby improving the collaboration efficiency of the online collaboration whiteboard.

[0108] In some embodiments, based on the association between document content elements and the second document content of the target online collaborative whiteboard document, multiple target document content elements are determined from the document content elements, including:

[0109] Calculate the content similarity between each document content element and the second document content of the target online collaborative whiteboard document to obtain the first confidence score corresponding to each document content element;

[0110] Determine the second confidence level for each document content element based on its data type.

[0111] Calculate the target confidence level for each document content element based on the first confidence level and the second confidence level, and determine multiple target document content elements among the document content elements based on the target confidence level.

[0112] In this embodiment of the disclosure, the association between document content elements and the second document content of the target online collaborative whiteboard document can be determined based on the content similarity between the document content elements and the second document content, as well as the data type of each document content element.

[0113] Specifically, the content similarity between each document content element and the second document content of the target online collaborative whiteboard document can be calculated first, and this content similarity can be used as the first confidence level for each document content element. Then, the second confidence level for each document content element can be determined based on its data type. The data type of the document content element and the second confidence level can be determined according to a preset mapping relationship. Generally, in portable document type (PDF) files, images and tables convey more important information, so image-type document content elements in PDF files can be set to the first confidence level, for example, 0.8; then, table-type document content elements can be set to the second confidence level, for example, 0.6; further, text-type document content elements can be set to the third confidence level, for example, 0.4.

[0114] After determining the first and second confidence levels for each document content element, a target confidence level can be calculated for each document content element based on the first and second confidence levels. Then, multiple target document content elements are determined from the document content elements based on the target confidence levels.

[0115] The target confidence score for each document content element is calculated based on a first confidence score and a second confidence score. Specifically, the weight coefficients of the first and second confidence scores for each document content element are first obtained. Then, the first and second confidence scores are weighted based on their respective weight coefficients to obtain the target confidence score for the document content element. The weight coefficients for the first and second confidence scores can be the same or different for different document content elements. For example, for document content element A, the weight for the first confidence score is 0.6, and the weight for the second confidence score is 0.4; for document content element B, the weight for the first confidence score can be 0.4, and the weight for the second confidence score can be 0.6. The specific weight can be determined based on the actual nature of the different document content elements. For example, if a document content element is text content, but its content represents the core content of the entire document and has a high similarity to the content of a second document, then the weight of its first confidence score can be higher than the weight of its second confidence score.

[0116] After determining the target confidence level for each document content element, the relationship between the target confidence level and a preset confidence threshold can be used to determine which document content elements should be retained and displayed, and which are unimportant document content elements that should be removed. Specifically, a preset confidence threshold can be obtained first. This threshold can be set by the user of the online collaborative whiteboard that inserts the target portable document, by the backend administrator of the online collaborative whiteboard application, or automatically determined by the online collaborative whiteboard application based on long-term system statistics. Then, document content elements with a target confidence level greater than this confidence threshold can be identified as target document content elements.

[0117] In some embodiments, the content similarity between each document content element and the second document content of the target online collaborative whiteboard document is calculated to obtain a first confidence score corresponding to each document content element, including:

[0118] Perform content statistics on document content elements in the target online collaborative whiteboard document, and generate summary text based on the statistical content;

[0119] Extract the content text from each document content element and calculate the text similarity between each content text and the summary text to obtain the first confidence score for each document content element.

[0120] In this embodiment, the content similarity between each document content element and the second document content of the target online collaborative whiteboard document is calculated. Specifically, content statistics can be performed on the document content elements in the target online collaborative whiteboard document, and a summary text can be generated based on the statistical content. Specifically, generating the summary text of the target online collaborative whiteboard document can be implemented using a summary text generation model. This model can be a large language model that supports multimodal input. The model categorizes the currently edited content in the target online collaborative whiteboard document into multiple modalities, then simultaneously inputs the edited content from these multiple modalities into the large language model, prompting the model to generate a summary text from the input content, resulting in the summary text output by the large language model.

[0121] In some embodiments, considering that classifying and summarizing the currently edited elements in the target online collaborative whiteboard document would result in the loss of the positional relationships between the elements, the target online collaborative whiteboard document can be directly input into a large language model (which is trained to have document analysis capabilities), and the large language model can be prompted to generate a summary text for the target online collaborative whiteboard document, thus obtaining the summary text output by the large language model.

[0122] In some embodiments, when creating an online collaborative canvas, the creator may set a canvas name (topic name) for the newly created online collaborative canvas. The canvas name may also include the use case of the canvas (such as online meetings, collaborative drawing, or online education). Furthermore, the canvas name and the target online collaborative canvas document can be simultaneously input into a large language model to generate summary text, so as to generate more accurate summary text.

[0123] Then, for each document content element, its corresponding content text can be extracted. When the document content element is a text-based element, its text data can be directly used as the content text. When the document content element is a table or image-based element, it can be converted into its corresponding content text, such as the descriptive text corresponding to the table or image.

[0124] After extracting the content text from each document content element, we can further calculate the text similarity between the content text corresponding to each document content element and the summary text corresponding to the target online collaborative whiteboard document, and use this text similarity as the first confidence score for each document content element.

[0125] In some embodiments, when document content elements are image or table types, calculating their similarity to the summary text can be done without converting these document content elements into content text. Semantic features of table or image type document content elements can be directly extracted, along with textual features from the summary text. Then, the semantic features are dimensionalized to match the text feature dimensions. Finally, the feature similarity between the text features and the transformed features is calculated to obtain the first confidence score for each document content element.

[0126] In some embodiments, determining a second confidence level for each document content element based on its data type includes:

[0127] Perform element type statistics on the online collaboration board elements in the target online collaboration board document to obtain the statistical quantity corresponding to different element types;

[0128] The second confidence level for each document content element is determined based on statistical data.

[0129] When determining the second confidence level of a document content element based on its data type, pre-setting the confidence level of image types to be greater than that of table types, and the confidence level of table types to be greater than that of text types, may be too arbitrary and lead to inaccurate determination of the second confidence level.

[0130] This disclosure provides a method for determining the second confidence level of document content elements of different data types based on the usage tendency of element types in a target online collaborative whiteboard document. Specifically, element type statistics can be performed on the online collaborative whiteboard elements in the target online collaborative whiteboard document to obtain the statistical quantity corresponding to different element types. Then, the second confidence level corresponding to document content elements of different data types can be determined based on the statistical quantity of element types of different online collaborative whiteboard elements.

[0131] The file insertion method for online collaborative canvases provided in this disclosure, when needing to insert a PDF document into the online collaborative canvas, no longer mechanically splits the PDF document into multiple document pages and then converts each document page into corresponding image elements for display in the online collaborative canvas. Instead, it can intelligently identify important content in the inserted file based on the content already drawn in the current online collaborative canvas document. This important content is then converted into elements in the online collaborative canvas document and displayed therein. Thus, other users of the online collaborative canvas can view the content of the PDF document without switching to other applications, and it avoids the problem of low content viewing efficiency caused by searching for useful information among numerous pieces of content. In the scenario of multi-person online collaboration using online collaborative canvases, improving content viewing efficiency is extremely important, because the delay of one user can amplify the impact on everyone, thereby greatly affecting collaboration efficiency. Therefore, the file insertion method for online collaborative canvases provided in this application can significantly improve the collaboration efficiency of online collaborative canvases.

[0132] Step 330: Generate multiple online collaborative canvas display elements based on multiple target document content elements and the data type corresponding to each target document content element.

[0133] Once several key target document content elements have been identified based on the association between document content elements and the second document content of the target online collaboration board document, the content corresponding to these target document content elements can be further displayed in the target online collaboration board document. This allows participants in the target online collaboration board document to view this content on their respective client's display interface. Therefore, it is necessary to first convert these target document content elements into display elements for the online collaboration board. Different data types of target document content elements are displayed using different types of display elements on the online collaboration board; thus, multiple online collaboration board display elements can be generated based on the target document content elements and their corresponding data types.

[0134] In some embodiments, multiple online collaborative whiteboard display elements are generated based on multiple target document content elements and the data type corresponding to each target document content element, including:

[0135] Retrieve the online collaborative canvas element template corresponding to each data type;

[0136] Multiple online collaboration whiteboard display elements are generated based on multiple target document content elements and the corresponding online collaboration whiteboard element templates for each target document content element.

[0137] Online collaborative canvases are typically used by multiple users to collaboratively create content simultaneously. Each user can insert PDF documents into the canvas. However, document elements of the same data type in different PDF documents may be created using different parameters. For example, text formatting (KaiTi, SongTi, etc.), size (size 4, size 5, etc.), line spacing, and color may differ; image size, resolution, and color may also vary; table frame size and alignment may differ. Therefore, if the online collaborative canvas display elements are generated according to the original format of the PDF document, when multiple users insert PDF documents into the canvas, the formatting of elements within the PDF documents will be inconsistent, thus affecting the user experience of the online collaborative canvas.

[0138] Therefore, in this embodiment of the disclosure, when generating the corresponding online collaboration board display element for each target document content element, the online collaboration board element template corresponding to each data type can be obtained first. Then, multiple online collaboration board display elements can be generated based on multiple target document content elements and the online collaboration board element template corresponding to each target document content element. In this way, for document content elements of the same data type from different inserted PDF documents, a unified format template can be used to generate the corresponding online collaboration board display elements, thereby ensuring the consistency of the format of the display elements in the online collaboration board and improving the user experience of the online collaboration board.

[0139] Step 340: Display multiple online collaboration board elements in the target online collaboration board document.

[0140] After generating online collaboration canvas display elements corresponding to multiple target document content elements, these elements can be rendered into the target online collaboration canvas document, ensuring their proper display. By default, these elements can be displayed in blank areas of the target online collaboration canvas document, with multiple elements displayed without overlapping.

[0141] In some embodiments, online collaboration whiteboard display elements include image elements, and displaying multiple online collaboration whiteboard display elements in a target online collaboration whiteboard document includes:

[0142] Obtain viewport information when the online collaboration whiteboard document is displayed in the first client;

[0143] The target image element is determined from multiple candidate image elements in the online collaborative drawing display elements based on viewport information. The multiple candidate image elements are multiple image elements with different resolutions generated based on the target document image in the target document content element.

[0144] The target image element is sent to the first client so that the target image element is displayed in the display area of ​​the target online collaborative whiteboard document in the first client.

[0145] In this embodiment, when the determined target document content element is an image-type document content element, multiple candidate image elements can be generated based on an original image stored in COS. These multiple candidate image elements can specifically be multiple image elements of the same image at different resolutions. Then, for multiple clients corresponding to the target online collaboration canvas, when displaying the image element corresponding to the image content in the target online collaboration canvas, the clearest target image element can be selected from the multiple candidate image elements based on the current viewport information of the client (referred to here as the first client), and the target image element can be sent to the corresponding client for display.

[0146] Here, viewport information refers to the viewport information when the target online collaboration canvas is displayed in the first client. This viewport information can be adjusted by the user of the online collaboration canvas by scrolling the mouse wheel or using keyboard shortcuts. Specifically, viewport information can be similar to the display ratio of a document; for example, a normal display is at 100% scale, which can be zoomed in to 200% or zoomed out to 50%. In related technologies, elements displayed in the online collaboration canvas can be set to change accordingly with the viewport. This requires re-rendering the elements displayed in the online collaboration canvas, which may result in poor rendering effects and unclear image elements. In this embodiment, by pre-rendering each image content at different resolutions, multiple clear candidate image elements suitable for different viewport information are obtained. When displaying image elements in the target online collaboration canvas document, the viewport information of the target online collaboration canvas document in the client to be displayed can be obtained first, and then the most suitable target image element can be selected from the multiple candidate image elements for display, thereby ensuring the clarity of the image elements displayed in the client. When the viewport information in the client changes, it can also detect in real time whether the viewport information jumps to another scope. If it is determined to switch to another scope, the target image element can be re-determined from the candidate image elements and displayed in the target online collaborative whiteboard document.

[0147] For example, three candidate image elements, a, b, and c, can be pre-rendered, corresponding to viewport sizes of 200% and above, 50% to 200%, and below 50%, respectively. Thus, when the viewport size is 100%, image element b is displayed as the target image element in the target online collaboration canvas document. When the user increases the viewport size to 220%, image element a is selected as the target image element and displayed in the target online collaboration canvas document.

[0148] In some embodiments, a target image element is determined from multiple candidate image elements in the online collaborative drawing display elements based on viewport information. These multiple candidate image elements are multiple image elements with different resolutions generated from the target document image in the target document content element, including:

[0149] The parameter range of viewport parameters is determined based on viewport information;

[0150] The target image element is determined from multiple candidate image elements based on the parameter range. The multiple candidate image elements are multiple image elements with different resolutions generated from the target document image in the target document content elements.

[0151] In some embodiments, displaying the plurality of online collaboration whiteboard display elements in the target online collaboration whiteboard document includes:

[0152] In the second client that initiated the insertion request, the display area of ​​the target online collaborative canvas document displays an insertion element selection interface, which includes multiple candidate tags corresponding to the multiple online collaborative canvas elements.

[0153] In response to the selection operation of the target candidate label among the plurality of candidate labels, a plurality of target online collaborative canvas display elements corresponding to the target candidate label are determined;

[0154] The multiple target online collaboration whiteboard display elements are displayed in the target online collaboration whiteboard document.

[0155] In this embodiment, the backend server of the online collaboration whiteboard analyzes the uploaded PDF document's content and, in conjunction with the document content of the target online collaboration whiteboard document, determines multiple target document content elements. Furthermore, it generates multiple online collaboration whiteboard display elements based on these target document content elements. This allows for the extraction and conversion of content from the inserted PDF document that is strongly correlated with the target online collaboration whiteboard document into display elements, thereby improving the efficiency of document viewing for users of the online collaboration whiteboard.

[0156] However, the target document content elements selected by the backend server of the online collaboration whiteboard may not be entirely accurate, and some unimportant document content elements may still exist, affecting the efficiency of content viewing. Therefore, in this embodiment, after generating multiple online collaboration whiteboard display elements, these online collaboration whiteboard display elements can be further sent to a second client that initiates an insertion request to insert the target portable document. This allows the second client that initiated the insertion request to display an insertion element selection interface in the display area of ​​the target online collaboration whiteboard document, where multiple candidate tags corresponding to multiple online collaboration whiteboard display elements can be displayed. Then, in response to the selection operation of the target candidate tag among the multiple candidate tags, multiple target online collaboration whiteboard display elements corresponding to the target candidate tag are determined, and further, these multiple target online collaboration whiteboard display elements can be displayed in the target online collaboration whiteboard document.

[0157] like Figure 4The diagram illustrates a scenario of the file insertion method for the online collaborative canvas provided in this disclosure. As shown, in the second client, the target online collaborative canvas 400 displays an insertion element selection interface 410. The insertion element selection interface 410 displays multiple candidate tags 411 corresponding to multiple online collaborative canvas display elements. Users of the online collaborative canvas corresponding to the second client can select one or more of the candidate tags 411 as target online collaborative canvas display elements. After receiving the selected target online collaborative canvas display elements, the backend server of the online collaborative canvas renders these target online collaborative canvas display elements into the target online collaborative canvas document, so that these target collaborative canvas display elements are displayed in the target online collaborative canvas document on other clients.

[0158] In some embodiments, the file insertion method for the online collaborative drawing board provided in this disclosure may further include:

[0159] In response to a triggered action on the target candidate tag, the viewing permission settings interface for the target candidate tag is displayed;

[0160] Receive permission settings information entered in the permission settings interface, and determine the viewing permissions for each target's online collaboration whiteboard display elements based on the permission settings information.

[0161] In this embodiment of the disclosure, when selecting elements in the insert element selection interface, the viewing permissions of the selected target collaboration board display elements can also be set. Specifically, the viewing permission setting interface for the target candidate tag can be displayed in the insert element selection interface by triggering an operation on the selected target candidate tag (e.g., right-clicking the mouse or long-pressing the control). Then, users of the online collaboration board can enter permission setting information in the viewing permission setting interface, thus determining the viewing permissions of each target online collaboration board display element based on the permission setting information received in the viewing permission setting interface.

[0162] like Figure 5 The figure shows another scenario diagram illustrating file insertion in the online collaboration whiteboard provided in this application. As shown in the figure, in the target online collaboration whiteboard 400, a viewing permission setting interface 500 corresponding to the selected target candidate label can also be displayed. In the viewing permission setting interface 500, users can be selected to have viewing permissions for the target online collaboration whiteboard display elements corresponding to the target candidate label.

[0163] After setting viewing permissions for the selected target online collaboration board display element, the online collaboration board's backend server then renders the target online collaboration board display element into the target online collaboration board document in a certain way. For example, for a display element with viewing permissions set, an unknown template can be displayed; users without access permissions cannot view it, while users with access permissions can view it.

[0164] like Figure 6 The diagram illustrates another scenario of the file insertion method for the online collaborative canvas provided in this disclosure. As shown, when access permissions are set for a certain image element, an image template 610 for that image element will be displayed in the target online collaborative canvas 400. When the user of the online collaborative canvas corresponding to a client does not have access permissions to the image, a prompt label "Sorry, you do not have access permissions to this image!" will be displayed near the image template 610.

[0165] like Figure 7 The diagram illustrates another scenario of the file insertion method for the online collaborative canvas provided in this disclosure. As shown, when a user of the online collaborative canvas corresponding to a client has access to an image, another prompt label, "Hello, you have access to this image," will be displayed near the image template. Additionally, a viewing control 710 can be displayed; when the user triggers the viewing control 710, the display can be switched to […]. Figure 8 The scene diagram shown is illustrated below. Figure 8 The screen can also display a close control 810, and in response to a trigger operation on the close control 810, it can also switch to display a template image.

[0166] The file insertion method for online collaborative whiteboards provided in this disclosure specifically includes, in response to an insertion request to insert a target portable document into a target online collaborative whiteboard document, parsing the content of a first document in the target portable document to obtain document content elements of various data types; determining multiple target document content elements among the document content elements based on the association relationship between the document content elements and the second document content of the target online collaborative whiteboard document; generating multiple online collaborative whiteboard display elements according to the multiple target document content elements and the data type corresponding to each target document content element; and displaying the multiple online collaborative whiteboard display elements in the target online collaborative whiteboard document.

[0167] In this embodiment, when a user of the online collaboration whiteboard inserts a target portable document into the target online collaboration whiteboard document, the document content of the target portable document can be automatically parsed to convert it into document content elements of various data types. Then, based on the relationship between the document content elements and the content of the target online collaboration whiteboard document, important target document content elements can be identified from among the various data types of document content elements. These target document content elements are then converted into online collaboration whiteboard display elements and displayed in the target online collaboration whiteboard document. Thus, the file insertion method for the online collaboration whiteboard provided in this application can not only automatically identify important content from portable documents but also directly display this content as collaboration whiteboard elements in the collaboration whiteboard. This avoids the problem of low collaboration efficiency caused by users having to switch to other document applications when viewing these file contents, thereby improving the collaboration efficiency of the online collaboration whiteboard.

[0168] This disclosure provides a detailed description of embodiments in conjunction with specific application scenarios.

[0169] like Figure 9 The diagram shown is another flowchart illustrating the file insertion method for the online collaborative drawing board provided in this disclosure. This embodiment will provide a detailed description of the file insertion method for the online collaborative drawing board in conjunction with the executing entities of each step. The method specifically includes the following steps:

[0170] Step 901: The first client displays the online collaboration whiteboard interface and receives the inserted PDF document in the online collaboration whiteboard interface. The first client sends a request to the online collaboration whiteboard server to insert the PDF.

[0171] When a user inserts a PDF document into the online collaboration canvas, if the PDF file is a local document, they can simply drag and drop it onto the canvas to insert it. If the PDF file is an online document, the user can access the file insertion page through the online collaboration canvas's file insertion interface, and then enter the document link on the file insertion page to insert the PDF document into the online collaboration canvas.

[0172] In related technologies, after a PDF document is inserted into an online collaboration whiteboard, text links, thumbnail links, or card links to the PDF document can be displayed on the whiteboard. Other users of the online collaboration whiteboard can click on these links to view the PDF document in an online document application or a PDF reader application. This process requires switching to other applications, resulting in low efficiency for users of the online collaboration whiteboard in viewing PDF documents.

[0173] To address the issue of low viewing efficiency caused by requiring users to navigate to other applications via links when viewing PDF documents inserted into online collaborative canvases, the inventors of this application first provide a method for inserting files into online collaborative canvases. When inserting a PDF document into an online collaborative canvas using this method, the PDF document can be converted into images page by page. Then, canvas elements corresponding to the online collaborative canvas are generated based on these images, and these elements are displayed in the corresponding online collaborative canvas.

[0174] like Figure 10 The diagram shown is yet another flowchart illustrating the file insertion method for the online collaborative drawing board provided in this disclosure. The method specifically includes the following steps:

[0175] Step 1001: The user uploads a PDF document to the online collaborative whiteboard.

[0176] The specific process of this step has been described in detail in the previous examples and will not be repeated here.

[0177] Step 1002: The online collaborative whiteboard server calls the PDF processing library to convert each page of the PDF document into an image.

[0178] After receiving the uploaded PDF file, the backend service of the online collaborative whiteboard can call a PDF processing library (such as PDF.js or Poppler) to convert each page of the PDF document into an image. During this process, parameters can be adjusted to control the image resolution and quality, minimizing image size while maintaining clarity.

[0179] Step 1003: Store the image in COS and receive the storage address returned by COS.

[0180] After converting each page of the PDF document into an image, the resulting images can be further stored in COS. Specifically, the COS SDK (Software Development Kit) interface can be called to upload the generated local image files to COS and receive the storage address returned by COS. During the upload process, access permissions can be set to ensure that only authorized users can access these images.

[0181] Step 1004: Store the storage address and other settings attributes into the data model of the canvas image element.

[0182] After obtaining the image's storage address, the backend service can insert the image address as an attribute of the canvas image element into the canvas's content. Besides the storage address, the canvas image element's attributes can also include the image element's type, position, size, and rotation angle. These attributes can be written together into the canvas image element's data model.

[0183] Step 1005: Display image elements on the canvas based on the information in the data model.

[0184] When a user browses an online collaborative whiteboard, the whiteboard application can generate image elements at corresponding locations based on information from the data model, load the image from COS, and render and display the image element on the whiteboard. Specifically, the image can be displayed using the HTML `img` element or the Canvas API, and the display effect can be controlled using CSS or JavaScript.

[0185] This method converts each page of a PDF document into an image, and then converts the images into canvas elements for display on an online collaborative canvas. As a result, when users view PDF documents, they can directly view the canvas image elements in the online collaborative canvas without having to switch to other applications, thereby improving the efficiency of viewing PDF document content.

[0186] However, while this method can avoid switching to other applications to view PDF files, in some cases, the PDF files contain a lot of content. Converting each page of the PDF file into image elements of the online collaboration board and displaying them in the online collaboration board will not only take up a lot of space in the online collaboration board, but also make it difficult to find useful information among a large number of images due to the excessive content displayed. As a result, the viewing efficiency of PDF documents inserted into the online collaboration board is still low.

[0187] To address the aforementioned issues, this disclosure provides a method for inserting files into an online collaborative drawing board, which is described in detail below.

[0188] First, the first client (the PDF file upload client) displays the online collaboration board interface on its display device and receives the inserted PDF document within that interface. After receiving the inserted PDF document, the first client sends a request to the online collaboration board server to insert the PDF document.

[0189] Step 902: The online collaborative whiteboard server calls the PDF analysis service to analyze the PDF document and obtain document elements of multiple data types.

[0190] After receiving a request to insert a PDF document, the online collaborative whiteboard server can invoke a PDF analysis service to analyze the PDF document. Specifically, in this embodiment, the PDF analysis service can identify document elements of text, table, image, and other data types within the PDF document content. That is, this method no longer simply splits the PDF document into pages and converts the pages into images; instead, it extracts the content of the PDF document according to data type, obtaining document elements of various data types. For each data type, multiple document elements can also correspond to it.

[0191] Step 903: The online collaboration whiteboard server calls the summary text generation service to summarize the current content of the online collaboration whiteboard and obtain the summary text.

[0192] In this embodiment of the disclosure, while the online collaboration whiteboard server calls the PDF analysis service to analyze the PDF document, it can also call the summary text generation service to summarize the current content of the online collaboration whiteboard and obtain summary text. Specifically, the current content of the online collaboration whiteboard can be the content composed of online collaboration whiteboard elements in the online collaboration whiteboard document that received the uploaded PDF file.

[0193] The specific method for generating summary text of the current content of the online collaborative whiteboard has been described in detail in the foregoing embodiments and will not be repeated here.

[0194] Step 904: The online collaborative whiteboard server extracts text features from the summary text to obtain text features, and extracts element features from document elements of multiple data types to obtain multiple element features.

[0195] After summarizing the current content of the online collaboration whiteboard and generating a summary text, you can use the summary text to filter multiple document elements obtained from PDF parsing, and select document elements that are highly relevant to the content of the online collaboration whiteboard.

[0196] Specifically, in this embodiment of the disclosure, the online collaborative whiteboard server can extract text features from the summary text to obtain text features, and extract element features from document elements of multiple data types to obtain multiple element features. Specifically, extracting text features from the summary text can be implemented using a text feature extraction model (e.g., the BERT model), and extracting element features from document elements of multiple data types can be implemented using a dedicated feature extraction model corresponding to each data type, in order to extract more accurate element features.

[0197] Specifically, text features can be text vectors, and element features can also be represented in vector form.

[0198] In some embodiments, before using the text feature extraction model and the element feature extraction model described above for feature extraction, multiple models can be trained using training sample data. During training, the text feature extraction model and each element feature extraction model can be jointly constructed to form an overall correlation evaluation model. Then, manually labeled sample data is used to train this correlation evaluation model end-to-end, so that the parameters of multiple models are mutually constrained and adjusted, thereby improving the accuracy of the extracted features.

[0199] Step 905: The online collaborative whiteboard server determines the target document element among multiple document elements based on the similarity between text features and element features.

[0200] After extracting the text features of the abstract text and the element features of each document element, the online collaborative whiteboard server can further calculate the similarity between the text features and the element features of each document element. Specifically, the text vectors corresponding to the text features and the element vectors corresponding to the element features can first be dimensionalized to ensure that their dimensions are consistent. Then, the cosine similarity between the dimensionalized text vectors and the dimensionalized element vectors of each document element is calculated, thereby obtaining the relevance between each document element and the content of the online collaborative whiteboard.

[0201] Then, based on the relevance between the document element and the content of the online collaboration whiteboard, the target document element is determined from among multiple document elements. Specifically, document elements with a relevance to the content of the online collaboration whiteboard greater than a preset threshold can be identified as target document elements.

[0202] Step 906: The online collaborative whiteboard server generates element tags based on the target document elements and sends the element tags to the first client for display.

[0203] There can be one or more target document elements. When there are multiple target document elements, after determining the target document elements, the online collaboration board server can further generate corresponding element tags based on the target document elements, and then send the element tags to the first client for display. This allows the online collaboration board user corresponding to the first client to manually select from the multiple target document elements to determine the elements that should ultimately be displayed on the online collaboration board.

[0204] Step 907: The first client displays the element label in the element selection interface of the online collaboration board interface, receives the selection operation of the element label, and sends the target element label determined by the selection operation to the online collaboration board server.

[0205] After receiving the element tags generated by the online collaboration whiteboard server based on the target document elements, the first client can display the element selection interface on the online collaboration whiteboard interface and display these element tags in the element selection interface.

[0206] Then, the first client can receive the user's selection of an element label and determine the target element label based on the selection. Further, the first client can send the target element label to the online collaborative whiteboard server.

[0207] Step 908: The online collaboration whiteboard server generates online collaboration whiteboard display elements based on the target document elements corresponding to the target element tags.

[0208] After receiving the target element tag, the online collaboration whiteboard server can generate online collaboration whiteboard display elements based on the target document elements corresponding to the target element tag. The specific process of generating these online collaboration whiteboard display elements is similar to the previously described process of converting PDF pagination into images and then converting those images into online collaboration whiteboard elements.

[0209] For example, when the target document element is a text-based document element, the document data can be stored in COS and the storage address returned. Then, the storage address, along with attributes such as display position and text box size, are stored in the data model, thus generating the online collaboration whiteboard display element. When displayed on the client, the page can be rendered based on the data in the data model.

[0210] Step 909: The online collaboration board server adds the online collaboration board display elements to the online collaboration board.

[0211] After generating the online collaboration canvas display elements corresponding to the target document elements, the online collaboration canvas server can add these display elements to the online collaboration canvas. Specifically, the online collaboration canvas display elements can be added to the online collaboration canvas's rendering logic, and the corresponding data model can be called for rendering when needed.

[0212] Step 910: The second client receives operation instructions for the online collaboration whiteboard display elements and sends the operation instructions to the online collaboration whiteboard server.

[0213] Once the elements of the online collaboration canvas are displayed, any client (a second client, which can be the aforementioned first client or other clients) can manipulate these displayed elements, such as scaling and dragging. Specifically, the second client receives operation instructions for the displayed elements and sends these instructions to the online collaboration canvas server. This step involves the application of front-end interaction technologies, such as using JavaScript to detect mouse or touchscreen events to implement element scaling and dragging. Specifically, scaling can be implemented by detecting mouse wheel events or touchscreen gesture events, and the element size is updated in real time by calculating the mouse wheel scroll distance or touchscreen gesture scaling ratio. Similarly, dragging can be implemented by detecting mouse drag events or touchscreen gesture events, and the element's position is updated in real time by calculating the mouse drag distance or touchscreen gesture movement distance. When an element is scaled or dragged, the front-end updates the element's size and position information in real time and transmits this information to the online collaboration canvas server so that it can be synchronized on other users' canvases.

[0214] Providing a right-click menu on elements allows users to quickly access frequently used operations such as copy, paste, and delete. This step involves the application of front-end interaction technologies, such as using JavaScript to detect right-click events and show or hide the right-click menu. Various operation options can be added to the right-click menu, such as copy, paste, and delete, and corresponding event handlers can be added to these options to implement their respective functions. Additionally, a toolbar can be added to the element, providing advanced operations such as rotate and flip. This step also involves the application of front-end interaction technologies, such as using JavaScript to show or hide the toolbar and handle event processing for various operation options.

[0215] Step 911: The online collaboration whiteboard server adjusts the displayed elements of the online collaboration whiteboard based on the operation instructions.

[0216] After receiving the operation instructions sent by the second client, the online collaboration whiteboard server can adjust the display position, display attributes, and other information of the online collaboration whiteboard display elements based on the operation instructions.

[0217] The file insertion method for online collaborative whiteboards provided in this disclosure specifically includes, in response to an insertion request to insert a target portable document into a target online collaborative whiteboard document, parsing the content of a first document in the target portable document to obtain document content elements of various data types; determining multiple target document content elements among the document content elements based on the association relationship between the document content elements and the second document content of the target online collaborative whiteboard document; generating multiple online collaborative whiteboard display elements according to the multiple target document content elements and the data type corresponding to each target document content element; and displaying the multiple online collaborative whiteboard display elements in the target online collaborative whiteboard document.

[0218] In this embodiment, when a user of the online collaboration whiteboard inserts a target portable document into the target online collaboration whiteboard document, the document content of the target portable document can be automatically parsed to convert it into document content elements of various data types. Then, based on the relationship between the document content elements and the content of the target online collaboration whiteboard document, important target document content elements can be identified from among the various data types of document content elements. These target document content elements are then converted into online collaboration whiteboard display elements and displayed in the target online collaboration whiteboard document. Thus, the file insertion method for the online collaboration whiteboard provided in this application can not only automatically identify important content from portable documents but also directly display this content as collaboration whiteboard elements in the collaboration whiteboard. This avoids the problem of low collaboration efficiency caused by users having to switch to other document applications when viewing these file contents, thereby improving the collaboration efficiency of the online collaboration whiteboard.

[0219] Description of apparatus and devices according to embodiments of this disclosure

[0220] It is understood that although the steps in the above flowcharts are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated in this embodiment, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the above flowcharts may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps.

[0221] Figure 11 A schematic diagram of the structure of the file insertion device 1100 for the online collaborative drawing board provided in this embodiment of the disclosure. The file insertion device 1100 includes:

[0222] The parsing unit 1110 is used to respond to an insertion request to insert a target portable document into a target online collaborative whiteboard document, and to parse the content of the first document in the target portable document to obtain document content elements of various data types.

[0223] The determining unit 1120 is used to determine multiple target document content elements in the document content elements based on the association relationship between document content elements and the second document content of the target online collaborative drawing board document;

[0224] The generation unit 1130 is used to generate multiple online collaborative whiteboard display elements based on multiple target document content elements and the data type corresponding to each target document content element;

[0225] Display unit 1140 is used to display multiple online collaboration board display elements in a target online collaboration board document.

[0226] Optionally, in some embodiments, the determining unit includes:

[0227] The calculation subunit is used to calculate the content similarity between each document content element and the second document content of the target online collaborative drawing board document, and to obtain the first confidence score corresponding to each document content element.

[0228] The first determining subunit is used to determine the second confidence level corresponding to each document content element based on the data type of each document content element.

[0229] The second determining subunit is used to calculate the target confidence level corresponding to each document content element based on the first confidence level and the second confidence level, and to determine multiple target document content elements in the document content elements based on the target confidence level.

[0230] Optionally, in some embodiments, the computing subunit includes:

[0231] The generation module is used to perform content statistics on document content elements in the target online collaborative whiteboard document and generate summary text based on the statistical content.

[0232] The calculation module is used to extract content text from each document content element, calculate the text similarity between each content text and the summary text, and obtain the first confidence score corresponding to each document content element.

[0233] Optionally, in some embodiments, the first determining subunit includes:

[0234] The statistics module is used to perform element type statistics on online collaborative whiteboard elements in the target online collaborative whiteboard document and obtain the statistical quantity corresponding to different element types;

[0235] The first determination module is used to determine the second confidence level corresponding to each document content element based on statistical data.

[0236] Optionally, in some embodiments, the display unit includes:

[0237] The first acquisition subunit is used to acquire viewport information when the online collaboration whiteboard document is displayed in the first client.

[0238] The third determining subunit is used to determine the target image element from multiple candidate image elements in the online collaborative drawing display elements based on viewport information. The multiple candidate image elements are multiple image elements with different resolutions generated based on the target document image in the target document content element.

[0239] The sending sub-unit is used to send the target image element to the first client so that the target image element is displayed in the display area of ​​the target online collaborative whiteboard document in the first client.

[0240] Optionally, in some embodiments, the third determining subunit includes:

[0241] The second determining module is used to determine the parameter range of the viewport parameters based on the viewport information;

[0242] The third determination module is used to determine the target image element from multiple candidate image elements based on the parameter range. The multiple candidate image elements are multiple image elements with different resolutions generated based on the target document image in the target document content elements.

[0243] Optionally, in some embodiments, the display unit includes:

[0244] The first display subunit is used to display the insertion element selection interface in the display area of ​​the target online collaborative canvas document in the second client that initiated the insertion request. The insertion element selection interface includes multiple candidate labels corresponding to multiple online collaborative canvas elements.

[0245] The fourth determining subunit is used to determine multiple target online collaborative canvas display elements corresponding to the target candidate label in response to the selection operation of the target candidate label among multiple candidate labels;

[0246] The display sub-unit is used to display multiple target online collaboration board elements in the target online collaboration board document.

[0247] Optionally, the file insertion device for the online collaborative drawing board provided in this application further includes:

[0248] The second display subunit is used to display the viewing permission settings interface of the target candidate tag in response to the trigger operation of the target candidate tag;

[0249] The fifth determining subunit is used to receive the permission setting information entered in the viewing permission setting interface, and determine the viewing permission of each target online collaborative canvas display element based on the permission setting information.

[0250] Optionally, in some embodiments, the parsing unit includes:

[0251] Extraction sub-units are used to extract image content contained in the first document content, obtaining image-type document content elements; and

[0252] The first transformation subunit is used to extract the table content contained in the first document content and convert the table content into a table-type document content element; and

[0253] The second conversion subunit is used to extract the text content contained in the first document content and convert the text content into document content elements of editable text type.

[0254] Optionally, in some embodiments, the second conversion subunit includes:

[0255] The splitting module is used to extract the text content contained in the first document and split the text content into multiple paragraphs.

[0256] The conversion module is used to convert multiple paragraph texts into multiple document content elements of an editable document type.

[0257] Optionally, in some embodiments, the generation unit includes:

[0258] The second acquisition subunit is used to acquire the online collaborative canvas element template corresponding to each data type;

[0259] The generation sub-unit is used to generate multiple online collaboration board display elements based on multiple target document content elements and the online collaboration board element template corresponding to each target document content element.

[0260] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0261] Reference Figure 12 , Figure 12The structural block diagram of a portion of the terminal 140 implementing the file insertion method for the online collaborative whiteboard according to this embodiment of the present disclosure includes: a radio frequency (RF) circuit 1410, a memory 1215, an input unit 1230, a display unit 1240, a sensor 1250, an audio circuit 1260, a wireless fidelity (WiFi) module 1270, a processor 1280, and a power supply 1290, etc. Those skilled in the art will understand that... Figure 12 The terminal 140 structure shown does not constitute a limitation on a mobile phone or computer, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0262] The RF circuit 1210 can be used to receive and transmit signals during information transmission or calls. In particular, it receives downlink information from the base station and processes it with the processor 1280; in addition, it transmits uplink data to the base station.

[0263] The memory 1215 can be used to store software programs and modules. The processor 1280 executes various terminal functions and document editing by running the software programs and modules stored in the memory 1215.

[0264] The input unit 1230 can be used to receive input numeric or character information, and to generate key signal inputs related to the terminal's settings and function control. Specifically, the input unit 1230 may include a touch panel 1231 and other input devices 1232.

[0265] The display unit 1240 can be used to display input or provided information, as well as various menus of the terminal. The display unit 1240 may include a display panel 1241.

[0266] Audio circuitry 1260, speaker 1261, and microphone 1262 provide an audio interface.

[0267] In this embodiment, the processor 1280 included in the terminal 140 can execute the file insertion method of the online collaborative drawing board in the previous embodiment.

[0268] The terminal 140 in this disclosure includes, but is not limited to, mobile phones, computers, smart voice interaction devices, smart home appliances, vehicle terminals, aircraft, etc.

[0269] Figure 13This is a partial structural block diagram of a server 110 for implementing the file insertion method of the online collaborative whiteboard according to embodiments of the present disclosure. The server 110 can vary significantly due to different configurations or performance, and may include one or more central processing units (CPUs) 1322 (e.g., one or more processors) and storage devices 1332, and one or more storage media 1330 (e.g., one or more mass storage devices) for storing application programs 1342 or data 1344. The storage devices 1332 and storage media 1330 can be temporary or persistent storage. The program stored in the storage media 1330 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the server 110. Furthermore, the CPU 1322 may be configured to communicate with the storage media 1030 and execute the series of instruction operations in the storage media 1330 on the server 110.

[0270] Server 110 may also include one or more power supplies 1326, one or more wired or wireless network interfaces 1350, one or more input / output interfaces 1358, and / or one or more operating systems 1341, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0271] The central processing unit 1322 in server 110 can be used to execute the file insertion method of the online collaborative canvas of this disclosure.

[0272] This disclosure also provides a storage medium for storing program code, which is used to execute the file insertion method of the online collaborative drawing board in the foregoing embodiments.

[0273] This disclosure also provides a computer program product comprising a computer program. A processor of a computer device reads and executes the computer program, causing the computer device to perform the file insertion method for the online collaborative drawing board described above.

[0274] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in this disclosure and the foregoing drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “including,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatuses.

[0275] It should be understood that in this disclosure, "at least one item" means one or more, and "more than one" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0276] It should be understood that in the description of the embodiments disclosed herein, "multiple" means two or more, "greater than", "less than", "exceeding" etc. are understood to exclude the number itself, and "above", "below", "within" etc. are understood to include the number itself.

[0277] In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0278] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0279] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0280] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0281] It should also be understood that the various implementation methods provided in this disclosure can be combined arbitrarily to achieve different technical effects.

[0282] The above is a detailed description of the embodiments of this disclosure. However, this disclosure is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this disclosure. All such equivalent modifications or substitutions are included within the scope defined by the claims of this disclosure.

Claims

1. A method for inserting files into an online collaborative drawing board, characterized in that, The method includes: In response to an insertion request to insert a target portable document into a target online collaborative whiteboard document, the content of the first document in the target portable document is parsed to obtain document content elements of various data types; Based on the association between the document content elements and the second document content of the target online collaborative drawing board document, multiple target document content elements are determined in the document content elements; Multiple online collaborative whiteboard display elements are generated based on the multiple target document content elements and the data type corresponding to each target document content element; The multiple online collaboration whiteboard display elements are displayed in the target online collaboration whiteboard document.

2. The method according to claim 1, characterized in that, The method of determining multiple target document content elements in the document content elements based on the association relationship between the document content elements and the second document content of the target online collaborative drawing board document includes: Calculate the content similarity between each document content element and the second document content of the target online collaborative drawing board document to obtain the first confidence level corresponding to each document content element; Based on the data type of each document content element, determine the second confidence level corresponding to each document content element; Calculate the target confidence level for each document content element based on the first confidence level and the second confidence level, and determine multiple target document content elements among the document content elements based on the target confidence level.

3. The method according to claim 2, characterized in that, The step of calculating the content similarity between each document content element and the second document content of the target online collaborative drawing board document to obtain the first confidence score corresponding to each document content element includes: The document content elements in the target online collaborative whiteboard document are statistically analyzed, and a summary text is generated based on the statistical content. Extract the content text from each document content element and calculate the text similarity between each content text and the summary text to obtain the first confidence level corresponding to each document content element.

4. The method according to claim 2, characterized in that, The step of determining the second confidence level corresponding to each document content element based on the data type of each document content element includes: Perform element type statistics on the online collaboration whiteboard elements in the target online collaboration whiteboard document to obtain the statistical quantity corresponding to different element types; The second confidence level is determined for each document content element based on the statistical data.

5. The method according to claim 1, characterized in that, The online collaboration whiteboard display elements include image elements, and displaying the multiple online collaboration whiteboard display elements in the target online collaboration whiteboard document includes: Obtain the viewport information of the online collaborative whiteboard document when it is displayed in the first client; Based on the viewport information, a target image element is determined from multiple candidate image elements in the online collaborative drawing display elements. The multiple candidate image elements are multiple image elements with different resolutions generated based on the target document image in the target document content element. The target image element is sent to the first client so that the target image element is displayed in the display area of ​​the target online collaborative whiteboard document in the first client.

6. The method according to claim 5, characterized in that, The step of determining the target image element from multiple candidate image elements in the online collaborative drawing display elements based on the viewport information, wherein the multiple candidate image elements are multiple image elements with different resolutions generated based on the target document image in the target document content element, including: The parameter range of the viewport parameters is determined based on the viewport information; The target image element is determined from multiple candidate image elements based on the parameter range, wherein the multiple candidate image elements are multiple image elements with different resolutions generated based on the target document image in the target document content element.

7. The method according to claim 1, characterized in that, The step of displaying the multiple online collaboration whiteboard elements in the target online collaboration whiteboard document includes: In the second client that initiated the insertion request, the display area of ​​the target online collaborative canvas document displays an insertion element selection interface, which includes multiple candidate tags corresponding to the multiple online collaborative canvas elements. In response to the selection operation of the target candidate label among the plurality of candidate labels, a plurality of target online collaborative canvas display elements corresponding to the target candidate label are determined; The multiple target online collaboration whiteboard display elements are displayed in the target online collaboration whiteboard document.

8. The method according to claim 7, characterized in that, The method further includes: In response to a trigger operation on the target candidate tag, the viewing permission settings interface for the target candidate tag is displayed; The system receives permission setting information input from the viewing permission setting interface and determines the viewing permission for each target online collaborative canvas display element based on the permission setting information.

9. The method according to claim 1, characterized in that, The content parsing of the first document content in the target portable document yields document content elements of various data types, including: Extract the image content contained in the first document content to obtain image-type document content elements; and Extract the table content contained in the first document content and convert the table content into a table-type document content element; and The text content contained in the first document is extracted and converted into document content elements of editable text type.

10. The method according to claim 9, characterized in that, The step of extracting the text content contained in the first document content and converting the text content into editable text type document content elements includes: The text content contained in the first document is extracted and then split into multiple paragraph texts; Convert the multiple paragraph texts into multiple document content elements of an editable document type.

11. The method according to claim 1, characterized in that, The step of generating multiple online collaborative whiteboard display elements based on the multiple target document content elements and the data type corresponding to each target document content element includes: Retrieve the online collaborative canvas element template corresponding to each data type; Based on the multiple target document content elements and the online collaboration whiteboard element template corresponding to each target document content element, multiple online collaboration whiteboard display elements are generated.

12. A file insertion device for an online collaborative drawing board, characterized in that, The device includes: The parsing unit is used to respond to an insertion request to insert a target portable document into a target online collaborative whiteboard document, and to parse the content of the first document in the target portable document to obtain document content elements of various data types. The determining unit is configured to determine multiple target document content elements in the document content elements based on the association relationship between the document content elements and the second document content of the target online collaborative drawing board document; The generation unit is used to generate multiple online collaborative whiteboard display elements based on the multiple target document content elements and the data type corresponding to each target document content element; The display unit is used to display the multiple online collaborative whiteboard elements in the target online collaborative whiteboard document.

13. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the file insertion method for the online collaborative drawing board according to any one of claims 1 to 11.

14. A storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the file insertion method for the online collaborative drawing board according to any one of claims 1 to 11.

15. A computer program product comprising a computer program that is read and executed by a processor of a computer device, causing the computer device to perform the file insertion method for an online collaborative drawing board according to any one of claims 1 to 11.

Images