Same-screen labeling processing method based on cloud conference, same-screen labeling processing equipment and medium

Abstract

The invention discloses a same-screen annotation processing method based on a cloud conference, a same-screen annotation processing device and a medium, relates to the technical field of cloud conferences, and discloses the same-screen annotation processing method based on the cloud conference. Acquiring a local version of a client and a service version of a server corresponding to the same-screen canvas; comparing version numbers between the local version and the service version; if the version number of the local version is smaller than the version number of the service version, synchronizing a difference label between the service version and the local version; and generating a submission instruction based on the newly-added mark of the synchronized service version, and broadcasting the submission instruction to other clients. Thus, it is ensured that asynchronous clients caused by a poor network are aligned with the latest service state when the local operation is submitted, label coverage or conflict caused by version lag is avoided, meanwhile, it is avoided that bandwidth is increased due to direct transmission of complete data, and the data synchronization efficiency of the canvas on the same screen is improved.

Description

Technical Field

[0001] The present application relates to the field of cloud conference technology, and in particular to a same-screen annotation processing method, same-screen annotation processing device, and medium based on cloud conference. Background Art

[0002] Most whiteboard libraries (on-screen annotation tools) in existing cloud conferencing systems rely on a full-chart synchronization mechanism. Regardless of the scope of the modification, each annotation operation, such as drawing or erasing, requires the complete whiteboard data to be transmitted. For example, modifying a single graphic in a meeting of 10 participants requires the full synchronization of all 10 copies of the data.

[0003] With the popularization of 4K (4000 horizontal pixels) materials and the normalization of ultra-large-scale conferences, this fully quantitative data transmission method in high-concurrency scenarios significantly increases bandwidth pressure, resulting in data loss on the same-screen canvas.

[0004] The above content is only used to assist in understanding the technical solution of this application and does not constitute an admission that the above content is prior art. Summary of the Invention

[0005] The main purpose of this application is to provide a same-screen annotation processing method, same-screen annotation processing device and medium based on cloud conferencing, aiming to solve the technical problem of same-screen canvas data loss in cloud conferencing communication when the network is unstable.

[0006] To achieve the above objectives, the present application proposes a method for same-screen annotation processing based on cloud conferencing, the method comprising:

[0007] If an incremental update instruction of a same-screen canvas is received from the client, obtaining a local version of the client and a service version of the server corresponding to the same-screen canvas;

[0008] Comparing the version numbers between the local version and the service version;

[0009] If the version number of the local version is smaller than the version number of the service version, synchronize the difference annotations between the service version and the local version;

[0010] A commit instruction is generated based on the newly added annotation of the synchronized service version, and the commit instruction is broadcast to other clients.

[0011] In one embodiment, if the version number of the local version is smaller than the version number of the service version, the step of synchronizing the difference annotations between the service version and the local version includes:

[0012] If the version number of the local version is smaller than the version number of the service version, a missing mark of the local version in the service version is determined based on the difference in version numbers;

[0013] Sending the vacancy mark to the local version based on synchronization signaling, wherein the client adds the vacancy mark after receiving the synchronization signaling;

[0014] The incremental update instruction is parsed to obtain the marked steps, and the service version is updated based on the marked steps.

[0015] In one embodiment, after the step of comparing the version numbers of the local version and the service version, the cloud conference-based on-screen annotation processing method further includes:

[0016] If the version numbers of the local version and the service version are the same, parsing the incremental update instruction to obtain the marked steps, and updating the service version based on the marked steps;

[0017] Generate target submission signaling based on the updated service version, and broadcast the target submission instruction to the other clients.

[0018] In one embodiment, if the version numbers of the local version and the service version are the same, the step of updating the service version based on the annotation step of the incremental update instruction includes:

[0019] If the version numbers of the local version and the service version are the same, parsing the annotation step in the incremental update instruction;

[0020] Synchronize the marking steps to the service version, and set the version number of the service version as the sum of the number of the marking steps and the version number of the service version;

[0021] Update the version snapshot corresponding to the service version.

[0022] In one embodiment, after the step of comparing the local version and the service version, the cloud conference-based on-screen annotation processing method further includes:

[0023] If the version number of the local version is greater than the version number of the service version, the local version is updated to the service version.

[0024] In one embodiment, before the step of obtaining the local version of the client and the service version of the server upon receiving the incremental update instruction sent by the client, the cloud conference-based same-screen annotation processing method further includes:

[0025] When a marking creation instruction is detected, it is determined whether the current cloud conference will create a marking conference number;

[0026] If so, obtain the snapshot data of the same-screen canvas; otherwise, create a marked conference number and add the same-screen canvas to the current cloud conference.

[0027] In one embodiment, the same-screen annotation processing method based on cloud conference also includes:

[0028] If the end sharing and exit marking signaling is detected, determining whether to retain the same-screen canvas, and if so, setting the same-screen canvas to a read-only state;

[0029] If not, the window of the same-screen canvas is closed, the snapshot data of the same-screen canvas is cleared, and the annotation whiteboard component is uninstalled.

[0030] In one embodiment, the cloud conference-based on-screen annotation processing method further includes:

[0031] If an exit marking instruction is detected from a target client not in conference sharing, determining whether the marked conference number of the same-screen canvas is created by the target client;

[0032] If so, the window of the same-screen canvas is closed, the snapshot data of the same-screen canvas is cleared, and the annotation whiteboard component is uninstalled; otherwise, the canvas of the same-screen canvas is set to a read-only state.

[0033] In addition, to achieve the above-mentioned purpose, the present application also proposes a same-screen annotation processing device, which includes: a memory, a processor, and a computer program stored on the memory and runnable on the processor, and the computer program is configured to implement the steps of the same-screen annotation processing method based on cloud conferencing as described above.

[0034] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the same-screen annotation processing method based on cloud conferencing as described above are implemented.

[0035] One or more technical solutions proposed in this application have at least the following technical effects:

[0036] After receiving the incremental update instruction from the client, it actively obtains and compares the service version number of the local version of the client with that of the cloud server. When it is identified that the local version lags behind the service version, it automatically synchronizes the difference annotations between the server and the local client to ensure that the client's operations can be notified to the server. At the same time, the client completes the missing annotation content. In this way, the server generates a submission instruction based on the latest service version that has been synchronized and broadcasts it to other clients. At the same time, the incremental update mechanism only transmits the annotation operation steps instead of the full amount of whiteboard data, which greatly reduces the network transmission load and effectively improves the data synchronization efficiency of the same-screen canvas. Therefore, in a high-concurrency meeting scenario, when the network environment fluctuates, the system can avoid annotation coverage, loss or conflict caused by version lag, ensure the real-time consistency and operation continuity of multi-user collaborative annotation, reduce bandwidth pressure, and significantly improve the accuracy and fluency of remote collaboration. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0038] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0039] Figure 1 This is a design diagram of the overall annotation architecture of the same-screen annotation processing method based on cloud conferencing in this application;

[0040] Figure 2 This is a design diagram of the annotation code architecture for the same-screen annotation processing method based on cloud conferencing in this application;

[0041] Figure 3 This is a flowchart of the first embodiment of the same-screen annotation processing method based on cloud conferencing of this application;

[0042] Figure 4 This is a schematic diagram of the processing flow when the local version of the same-screen annotation processing method based on cloud conferencing in this application is smaller than the service version;

[0043] Figure 5 This is a schematic diagram of the processing flow when the local version of the same-screen annotation processing method based on cloud conferencing in this application is equal to the service version;

[0044] Figure 6 This is a schematic diagram of the processing flow when the local version of the same-screen annotation processing method based on cloud conferencing in this application is larger than the service version;

[0045] Figure 7 This is a brief flowchart of the same-screen annotation processing method based on cloud conferencing in this application;

[0046] Figure 8 A diagram showing the process of opening the same-screen annotation for this application;

[0047] Figure 9 A flowchart for the user's exit marking for this application;

[0048] Figure 10 This is a schematic diagram of the device structure of the hardware operating environment involved in the same-screen annotation processing method based on cloud conferencing in an embodiment of the present application.

[0049] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0050] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the present application.

[0051] In order to better understand the technical solution of this application, this application will be described in detail in conjunction with the accompanying drawings and specific implementation methods.

[0052] Most whiteboard libraries (on-screen annotation tools) in existing cloud conferencing systems rely on a full-chart synchronization mechanism. Regardless of the scope of the modification, each annotation operation, such as drawing or erasing, requires the complete whiteboard data to be transmitted. For example, modifying a single graphic in a meeting of 10 participants requires the full synchronization of all 10 copies of the data.

[0053] With the popularization of 4K (4000 horizontal pixels) materials and the normalization of ultra-large-scale conferences, this fully quantitative data transmission method in high-concurrency scenarios significantly increases bandwidth pressure, resulting in data loss on the same-screen canvas.

[0054] Based on this, the main solution of the embodiment of the present application is: if an incremental update instruction of the same-screen canvas sent by the client is received, obtain the local version of the client corresponding to the same-screen canvas and the service version of the server; compare the version numbers between the local version and the service version; if the version number of the local version is less than the version number of the service version, synchronize the difference annotations between the service version and the local version; generate a submission instruction based on the newly added annotations of the synchronized service version, and broadcast the submission instruction to other clients.

[0055] In this way, in high-concurrency meeting scenarios, when the network environment fluctuates, the system can avoid annotation overlap, loss or conflict caused by version lag, ensuring the real-time consistency and operational continuity of multi-user collaborative annotation, reducing bandwidth pressure, and significantly improving the accuracy and fluency of remote collaboration.

[0056] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, mobile phone, etc., or an electronic device capable of implementing the above functions, a same-screen annotation processing device, etc. The following uses the same-screen annotation processing device as an example to illustrate this embodiment and the following embodiments.

[0057] In this embodiment, the same-screen annotation technology is implemented based on the electron (open source framework developed by GitHub) + websocket technology stack, and the carrier is a desktop application. The desktop application nests the same-screen annotation tool web whiteboard sdk tool page to implement the basic function of conference collaborative annotation. In the screen sharing state, start the annotation whiteboard, set the whiteboard background color to transparent, the person sharing the screen is covered on the shared screen, and the viewer is covered on the shared remote screen, and then based on websocket (net set) to implement canvas collaborative signaling in the conference screen sharing scenario.

[0058] The annotation architecture of the same-screen annotation processing method based on cloud conference in this application is as follows Figure 1 As shown, it includes the main window and whiteboard window for the sharer to share the cloud conference, as well as the main window and whiteboard window for the viewer's cloud conference. Among them, real-time signaling communication is realized based on websocket during the annotation process. On the sharing end of the sharer, the window whiteboard transmits data to the main window socket (socket), and the main window receives the downlink message and passes it to the child window. On the viewer's end, the web component is overlaid on the shared screen and communicates with the back end through the main window socket. Among them, floweditorref is the whiteboard window on the screen content. Annotatemanage is the annotation manager, and Windowinstance is the window instance.

[0059] Furthermore, the annotation code architecture is as follows Figure 2 As shown, by drawing a customized UI (User Interface) in the upper-level code of the cloud conference, the user selects a specified cursor such as a brush, arrow, text or laser pen in the UI interface, and the logical business layer calls based on the selected content. The bottom layer will trigger the event API (Application Programming Interface) to expose the upper layer, and the conference upper layer will call the bottom layer capabilities. In this way, the user can annotate the whiteboard based on the independent warehouse component of the flowchart canvas. At the same time, the upper layer can inject role information to implement permission control, and finally trigger a callback through the socket to transmit the actual annotation content.

[0060] Based on this, the embodiment of the present application provides a method for processing same-screen annotation based on cloud conference, referring to Figure 3 , Figure 3 This is a flow chart of the first embodiment of the same-screen annotation processing method based on cloud conferencing of this application.

[0061] In this embodiment, the same-screen annotation processing method based on cloud conference includes steps S10 to S40:

[0062] Step S10: If an incremental update instruction of the same-screen canvas is received from the client, the local version of the client and the service version of the server corresponding to the same-screen canvas are obtained.

[0063] In this embodiment, the same-screen canvas refers to a virtual collaborative canvas shared by all participating clients in a cloud conference, which is used to display annotations (such as lines, graphics, and text) in real time. Its status is uniquely determined by a series of ordered operation steps, namely the annotation steps and the corresponding annotation meeting number (annotateDocId). The incremental update instruction is that the client only sends the relevant data of the user's latest annotation operation, rather than transmitting the complete image of the entire whiteboard canvas or all historical data. For example, the annotation operation is to draw a line or erase a graphic, and the relevant data are the starting point coordinates, end point coordinates, brush properties, etc. corresponding to these operations. Incremental update means that when performing an update operation, only the newly added, modified, or deleted data is processed, and the unchanged part is skipped. Therefore, the data corresponding to the incremental update instruction is the changed data.

[0064] It's understood that each time the content of a canvas changes, the server updates the version number corresponding to that canvas based on the changes, and the client also updates the corresponding version number locally. Typically, the client updates the version number locally after the server updates it, and under stable network bandwidth conditions, the two versions will be the same.

[0065] As an optional implementation method, the server continuously monitors network requests from clients of each conference, and receives a specific instruction data packet sent by a client, such as Client A, through its network communication module in the network interface module of the server. After parsing the data packet, it is determined that it is an incremental update instruction. Then, the local version number corresponding to the current screen whiteboard state claimed by the client Client A that initiated the instruction is read from the instruction, as well as the newly added / modified / deleted content contained in the instruction.

[0066] At the same time, the server queries its own persistent memory for the service version number corresponding to the same-screen canvas in the current cloud conference, such as serviceVersion 15.

[0067] For example, a user draws a line on client A using a red brush from coordinates (100, 200) to (300, 400). Client A generates an incremental update instruction containing the local version number 15 and the steps {type:'drawLine', start:[100,200], end:[300,400], color:'red'} and sends it to the server over the network. The server receives this instruction.

[0068] This embodiment directly reads the version information corresponding to the local and server-side canvases on the same screen, so as to process the version differences under different network conditions based on the version information.

[0069] Step S20: comparing the version numbers of the local version and the service version.

[0070] As an optional implementation, the server performs a comparison operation based on the version control module. The comparison results may be in three situations: the client version number is equal to the server version number, the client version number is less than the server version number, and the client version number is greater than the server version number. For example, the server compares the received local version number 15 of client A with the stored service version number 15 and finds that the version numbers are identical.

[0071] It should be noted that, under normal circumstances, after each operation is completed, the client will send the newly added annotation steps to the server, and then the server will synchronize the annotation steps and update the version number, while saving the snapshot corresponding to the new version number, and then broadcast the newly added content. After receiving the broadcast data, other clients will synchronize the newly added annotation steps to the local and update the local client version number after synchronization, such as version number + 1. Therefore, under normal network conditions, the local version and the service version are the same, but in a high-concurrency environment, when the network is congested, the server broadcast is interrupted, or the client cannot receive the broadcast, causing the local version to lag behind the service version. In extreme cases, the client assumes that the network environment is unobstructed, and after each upload of the incremental update instruction, the local version number is + 1, resulting in a situation where the version number is greater than the server version number, or the local version number and the server version number are slightly different, but the actual difference in annotation content is large.

[0072] It is understandable that each time a service version update is completed, it is necessary to record the version snapshot corresponding to each operation sequence so as to obtain the newly added operation content of each version from the version snapshot.

[0073] Step S30: If the version number of the local version is smaller than the version number of the service version, synchronize the difference annotations between the service version and the local version.

[0074] In this embodiment, difference annotations refer to the differences in the annotation data added, modified, or deleted between the two versions. The synchronization process involves sending the steps in the service version that are missing from the local version to the client. After the client completes the synchronization, it then sends the local brush content back to the server, allowing the server to perform the corresponding brush operations.

[0075] Therefore, as an optional implementation method, in the case of a normal version number update, the vacancy mark of the local version in the service version can be determined based on the version number difference, such as extracting the version incremental data relative to the local version from the operation log pool or snapshot data of the server based on the version number difference. Then, based on the synchronization signaling (hereinafter referred to as sync signaling), the vacancy mark is sent to the local version, so that the client adds the vacancy mark after receiving the sync signaling. The server then parses the incremental update instruction sent by the client, obtains the marking step, and then updates the current service version based on the marking step. Among them, updating the service version includes two processing actions, including adding the marking step to the end of the whiteboard operation history sequence maintained by the server, and increasing the version number of the server, such as adding one. Or add a corresponding brush operation at the position in the service version to complete the marking update of the new content.

[0076] For example, please refer to Figure 4When the client version number is less than the server version number, the client sends an incremental update instruction (commit type) to the server. The incremental update instruction contains client_version=3 and the annotated steps are [step4, step5, step6]. At this time, the server detects that the client's local version (client_version=3) is less than the service version (server_version=5), triggering differential synchronization. Since the version number difference is 5-3=2, it indicates that the client has two actual steps, corresponding to step_server_4 and step_server_5. At this time, the server retrieves the incremental operations step_server_4 and step_server_5 from version 4 to version 5 from the version snapshot, and sends server_version=5, steps=[step_server_4, step_server_5] to the client through sync signaling. The client then performs a rebase action based on step_server_4 and step_server_5 to align the annotated content, and inserts the difference annotation [step_server_4, step_server_5] into the local version, so that the local client version number is changed to client_version=3+2=5. Finally, the client renders the whiteboard canvas based on the reorganized sequence ([step_server_4, step_server_5, step4, step5, step6]), making the client whiteboard state completely consistent with the server. In subsequent data synchronization, the newly added steps [step4, step5, step6, step7...] can be resent to the server normally, so that the server can process these newly added data and broadcast the newly added canvas data to all clients.

[0077] It should be noted that each canvas change is calculated based on the changes in the full data before the change, and a step is broadcasted via signaling. The transmitted steps are divided into three categories: UPDATE (update brush data operation), INSET (add brush data operation), and REMOVE (remove brush data operation).

[0078] As another optional implementation, under the default network environment with smooth version number update logic, when synchronizing the difference annotations between the service version and the local version, after receiving the incremental update instruction, the server sends all the annotation steps to the client. The client reorganizes the sequence in chronological order based on all the received annotation steps, thereby completing the completion of the client's missing annotations and synchronizing the version numbers between the local version and the service version. Subsequently, after the client feedbacks the reorganization instruction, the client generates a new incremental update instruction based on the reorganized sequence, using all the annotation steps sent by the server as new incremental content, except for the annotation steps sent by the server, and sends it to the server, so that the server can process the newly added data under the same version number.

[0079] Step S40: Generate a commit instruction based on the newly added annotation of the synchronized service version, and broadcast the commit instruction to other clients.

[0080] In this embodiment, the commit signaling is a notification message generated by the server. Its core content includes the annotation steps that triggered this broadcast and the new version number generated by the server after applying these steps. It is essentially an incremental update instruction that has been confirmed by the server and assigned a new version number.

[0081] The newly added annotation of the service version is usually the annotation step in the incremental update quality. After the difference annotation between the service version and the local version is synchronized, a commit signal is generated and submitted to other clients. After the service version broadcasts the data, the current version number will be updated. At the same time, after the client receives the broadcast signal, it will synchronously update the brush content and version number.

[0082] This embodiment provides a same-screen annotation processing method based on cloud conferencing, which transmits instructions through incremental updates rather than full whiteboard images or status, greatly reducing the network transmission load. When the client submits a new brush operation, its outdated version is aligned with the current server version, eliminating the risk of annotation overwriting due to network delays or concurrent operations, while avoiding the loss of same-screen canvas data. Ensure that all clients generate new annotations based on the latest global status during incremental updates, fundamentally guarantee the strong consistency of multi-terminal annotation data, and significantly improve the reliability of real-time collaboration in cloud conferencing scenarios.

[0083] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the first embodiment can be referred to the above introduction and will not be repeated hereafter. On this basis, after step S10, the same-screen annotation processing method based on cloud conference further includes steps S50 to S60:

[0084] Step S30: If the version numbers of the local version and the service version are the same, parse the incremental update instruction to obtain the marking steps, and update the service version based on the marking steps.

[0085] In this embodiment, updating the service version involves two processing actions: adding the annotation step to the end of the whiteboard operation history sequence maintained by the server and increasing the server version number by one, thereby marking the status as updated. Optionally, during the process of adding the annotation step to the server, a corresponding brush operation can be added at the location in the service version.

[0086] As an optional implementation method of adding annotation steps to the server, when the version numbers are the same, first read and parse the incremental update instruction to obtain the annotation steps corresponding to the instruction, then synchronize the annotation content corresponding to the annotation steps to the service version, and then set the sum of the number of annotation steps and the version number of the service version as the new version number, and use this to update the version snapshot corresponding to the service version. After completing the update of the service version, it is necessary to save the snapshot information corresponding to the current version so that when other users join the meeting, they can quickly synchronize the whiteboard display content based on the snapshot information. Finally, update the version snapshot corresponding to the service version to complete the update of the service version.

[0087] Optionally, after updating the version snapshot corresponding to the service version, the server will also synchronize the version number to the client so that the client can synchronize the version number.

[0088] For example, when the client version number is equal to the server version number, please refer to Figure 5 , the client version number client_version is 7, and the server version number server_version is also 7. The annotation steps include step = [step8, step9], indicating two different actions. At this time, after the client sends the incremental update quality, including the version number and annotation steps, to the server, if the versions are the same, the server sends a commit signaling and performs a brush operation to add the annotation content to the service version, thereby updating the server version. At the same time, because there are two steps, the original service version number is 7, and the updated version number server_version is now 9. Finally, the client responds to the commit signaling and synchronously updates the client version number, that is, client_version is updated to 9.

[0089] In this embodiment, after a client adjusts the annotation content of the displayed whiteboard, signaling is sent through incremental updates, which effectively reduces the amount of data transmission and facilitates the subsequent server to synchronize the changed content to other clients based on incremental updates, thereby improving the data update efficiency of the same-screen canvas.

[0090] It is understandable that the above numbers are only used to distinguish data and do not have any mathematical meaning.

[0091] Step S60: Generate a target submission signaling based on the updated service version, and broadcast the target submission signaling to the other clients.

[0092] In this embodiment, please continue to refer to Figure 5 After completing the version synchronization update, the server's commit signaling is generated based on the newly added content, namely step 8 and step 9. In addition to sending it back to the client and updating the client version number, the server will also broadcast the instruction to add the annotation step to the local version corresponding to other clients and synchronize the version.

[0093] Optionally, when the version number of the local version is greater than the version number of the service version, it is considered that the client network is abnormal, and the local version is directly updated to the service version. For example, please refer to Figure 6 , the version number of the local version, client_version, is 7, which is greater than the server's server_version = 5. At this time, after the client sends an incremental update instruction to the server, the server returns a reset instruction to reset the local version. In response to the instruction, the local version requests the server for the snapshot corresponding to the current version 5. The server then returns the snapshot and version corresponding to the current version, that is, returns the rich text of the snapshot.

[0094] Further, in order to help understand the implementation process of the same-screen annotation processing method based on cloud conference obtained by combining this embodiment with the above first embodiment, please refer to Figure 7 , Figure 7 A brief flow chart of a same-screen annotation processing method based on cloud conferencing is provided. Specifically, when drawing a whiteboard in the collaborative network, after user A successfully draws the new element on his client, the local version sends the commit signaling of the incremental data to the server. After receiving the signaling, the socket service of the server performs a data comparison. When the received version and the service version are the same, the server version +1 is completed, and the server version update is completed. At the same time, the commit incremental data of the server version is broadcast to all clients (including the clients of users A, B and C). Then, the versions of these clients are compared. If the original versions of the two are the same and the operation is for the user (user A), no processing is required. Otherwise, the drawing operation is performed, and the local version +1 is used. If the service version is lower than the local version, the local version is refreshed and the latest service version snapshot is obtained. When the service version is higher than the local version, data synchronization is performed through sync signaling to supplement the missing version brush data.

[0095] Alternatively, please continue to refer to Figure 7After the socket service receives the commit incremental data, if the received version is smaller than the service version, it synchronizes the version based on the sync signaling, and then performs the same comparison process. If the local version is too high, the socket service returns a reset signaling to refresh the local version and obtain the latest snapshot version. For the server (taking the Java server as an example), a full snapshot needs to be stored after each service version update. If the local version is too high, it is necessary to obtain the latest full snapshot to update the local version.

[0096] This embodiment provides a cloud-based on-screen annotation processing method. When the local version and service version number are the same, it achieves efficient synchronization of multi-terminal whiteboard annotation operations with minimized data transmission, simplified state verification logic, and a deterministic operation synchronization mechanism, effectively ensuring a smooth remote collaboration experience. Based on a broadcast processing method, after receiving the submission signal, other clients directly apply the operation and update the local version number without requesting the complete whiteboard status from the server. This greatly reduces the amount of data and time required for synchronization and avoids data synchronization loss due to network instability.

[0097] Based on the first or second embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those in the above embodiment can be referred to the above introduction, and no further details will be given later. On this basis, before step S10, the same-screen annotation processing method based on cloud conference further includes steps S01 to S03:

[0098] Step S01: When a marking creation instruction is detected, it is determined whether the current cloud conference creates a marked meeting number.

[0099] Step S02: If yes, obtain the snapshot data of the same-screen canvas; otherwise, create a marked conference number and add the same-screen canvas to the current cloud conference.

[0100] In this embodiment, there is only one annotateDocId (the meeting ID) during a sharing process. Exiting the annotation process clears the snapshot data. Therefore, when opening a meeting annotation, it is necessary to record the UId of the person opening the annotation, createAnnotateDocUid.

[0101] Considering that two users may send the signal to start annotation at the same time, the backend should prevent duplicate document creation when creating docId and only process the first instruction. In a meeting, only one annotateDocId is generated for each sharing. When the annotation function is exited, the snapshot is cleared when the sharing ends.

[0102] At the code level, a new data type is added to the participant list for identification: "annotateDocId":'xxxx' / / The docId of the annotated document. If a value is present, the annotation whiteboard feature is enabled; if empty, it is not. The annotation creation command is "createAnnotateDocUid":'xxx', which represents the UI ID of the person who initiated the annotation. This is used to determine when viewer A starts annotation or exits annotation.

[0103] For details, please refer to Figure 8 When user A enables annotation, the server-side socket service responds to the ANNOTATE_CRRETE command and determines whether a docid has been created. If not, it creates the document docid. The socket only processes the first create command and updates the participant list, allowing the sharer to enable annotation with "annotateDocId":'xxxx'. If not, the participant list is updated directly. After updating the participant list, a snapshot of the same-screen canvas is obtained and the content is shared to the canvas. The screen sharer has an initialization window at their corresponding screen location, which is used to update the screen sharer configuration and exclude the whiteboard window ID. The viewer initializes the mask element based on the content of the main conference view.

[0104] Optionally, after creating the annotation function of the cloud conference, when other participants want to edit the annotation content, they need to send a corresponding annotation request to the conference, and after the request is approved, they will have the whiteboard annotation function.

[0105] Based on the first or second embodiment of the present application, in the fourth embodiment of the present application, the same or similar contents as those in the above embodiments can be referred to the above introduction, and no further details will be given later. On this basis, only one meeting number is generated for each sharing in a meeting. Usually, when the user who turns on the annotation function exits the annotation, it means that the sharing ends, and the server clears the snapshot data related to the annotation. Or when the user exits the annotation, the annotation is retained. At this time, the canvas in the meeting becomes read-only and cannot be edited.

[0106] Therefore, if the end of sharing and exit of annotation signaling is detected, it is necessary to determine whether to retain the same-screen canvas. If so, the canvas of the same-screen canvas is set to read-only status. If not, the window of the same-screen canvas is closed, the snapshot data of the same-screen canvas is cleared and the annotation whiteboard component is uninstalled.

[0107] For example, please refer to Figure 9After the sharer ends sharing and exits annotation, confirm whether the sharer needs to keep the annotation. If so, the canvas is set to read-only. If not, close the annotation whiteboard window based on the ANNOTATE_CLOSE instruction. At the same time, clear the snapshot data corresponding to the annotation meeting number based on the ANNOTATE_CLOSE instruction and uninstall the annotation whiteboard component.

[0108] Alternatively, please continue to refer to Figure 9 In another scenario, user A is sharing a conference, and user B turns on the annotation function. During this period, only user B is annotating. At this time, B exits the annotation, and the whiteboard should be cleared. Therefore, after the non-sharer exits the annotation, the exit annotation instruction sent by the target client of the non-conference sharing is detected, and it is determined whether the annotation meeting number of the same-screen canvas is created by the target client; if so, the window of the same-screen canvas is closed, the snapshot data of the same-screen canvas is cleared, and the annotation whiteboard component is uninstalled, otherwise the canvas of the same-screen canvas is set to read-only state. Among them, the process of clearing the snapshot and uninstalling the component is the same, and this application will not elaborate on it.

[0109] This embodiment provides a method for processing on-screen annotations in a cloud conference, introducing an annotation synchronization mechanism. Upon detecting an exit annotation signal, the system first determines whether the on-screen canvas needs to be retained. If so, the canvas is switched to read-only mode. This prevents the destruction of collaborative work due to misoperation. If retention is not necessary, the system closes the canvas window, clears snapshot data, and uninstalls the annotation component, reducing client memory usage and improving data synchronization.

[0110] The present application provides a same-screen annotation processing device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the cloud conference-based same-screen annotation processing method in the above-mentioned first embodiment.

[0111] Reference below Figure 10 , which shows a structural schematic diagram of a same-screen annotation processing device suitable for implementing an embodiment of the present application. Figure 10 The same-screen annotation processing device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0112] like Figure 10As shown, the same-screen annotation processing device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. Various programs and data required for the operation of the same-screen annotation processing device are also stored in the random access memory 1004. The processing device 1001, the read-only memory 1002, and the random access memory 1004 are connected to each other via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the same-screen annotation processing device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a same-screen annotation processing device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have instead.

[0113] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a read-only memory 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are performed.

[0114] The same-screen annotation processing device provided by this application adopts the same-screen annotation processing method based on cloud conferencing in the above-mentioned embodiment, which can solve the technical problem of the loss of same-screen canvas data in cloud conferencing communication when the network is unstable. Compared with the existing technology, the beneficial effects of the same-screen annotation processing device provided by this application are the same as the beneficial effects of the same-screen annotation processing method based on cloud conferencing provided by the above-mentioned embodiment, and the other technical features of the same-screen annotation processing device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0115] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0116] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0117] The present application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the same-screen annotation processing method based on cloud conferencing in the above-mentioned embodiment.

[0118] The computer-readable storage medium provided in this application can be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared or semiconductor systems, systems or devices, or any combination thereof. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory (EPROM, Erasable Programmable ReadOnly Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM, CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium can be transmitted using any appropriate medium, including but not limited to: wires, optical cables, radio frequency (RF, Radio Frequency), etc., or any suitable combination thereof.

[0119] The computer-readable storage medium may be included in the same-screen annotation processing device; or may exist independently without being assembled into the same-screen annotation processing device.

[0120] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the same-screen annotation processing device, the same-screen annotation processing device: upon receiving an incremental update instruction for a same-screen canvas sent by a client, obtains the local version of the client and the service version of the server corresponding to the same-screen canvas;

[0121] Comparing the version numbers between the local version and the service version;

[0122] If the version number of the local version is smaller than the version number of the service version, synchronize the difference annotations between the service version and the local version;

[0123] A commit instruction is generated based on the newly added annotation of the synchronized service version, and the commit instruction is broadcast to other clients.

[0124] The computer program code for performing the operations of the present application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., using an Internet service provider to connect via the Internet).

[0125] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment or part of code, and the module, program segment or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, as well as the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or can be implemented using a combination of dedicated hardware and computer instructions.

[0126] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0127] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., a computer program) for executing the above-mentioned cloud conference-based same-screen annotation processing method, and can solve the technical problem of the loss of same-screen canvas data in cloud conference communications when the network is unstable. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the cloud conference-based same-screen annotation processing method provided in the above-mentioned embodiment, and will not be repeated here.

[0128] The above description is only part of the embodiments of the present application and does not limit the patent scope of the present application. All equivalent structural transformations made by using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for same-screen annotation processing based on cloud conference, characterized in that: The same-screen annotation processing method based on cloud conference includes: If an incremental update instruction of a same-screen canvas is received from the client, obtaining a local version of the client and a service version of the server corresponding to the same-screen canvas; Comparing the version numbers between the local version and the service version; If the version number of the local version is smaller than the version number of the service version, synchronize the difference annotations between the service version and the local version; A commit instruction is generated based on the newly added annotation of the synchronized service version, and the commit instruction is broadcast to other clients.

2. The same-screen annotation processing method based on cloud conference according to claim 1, characterized in that: If the version number of the local version is smaller than the version number of the service version, the step of synchronizing the difference annotations between the service version and the local version includes: If the version number of the local version is smaller than the version number of the service version, a missing mark of the local version in the service version is determined based on the difference in version numbers; Sending the vacancy mark to the local version based on synchronization signaling, wherein the client adds the vacancy mark after receiving the synchronization signaling; The incremental update instruction is parsed to obtain the marked steps, and the service version is updated based on the marked steps.

3. The same-screen annotation processing method based on cloud conference according to claim 1, characterized in that: After the step of comparing the version numbers of the local version and the service version, the cloud conference-based on-screen annotation processing method further includes: If the version numbers of the local version and the service version are the same, parsing the incremental update instruction to obtain the marked steps, and updating the service version based on the marked steps; Generate target submission signaling based on the updated service version, and broadcast the target submission instruction to the other clients.

4. The same-screen annotation processing method based on cloud conference according to claim 3, characterized in that: If the version numbers of the local version and the service version are the same, the step of updating the service version based on the annotation step of the incremental update instruction includes: If the version numbers of the local version and the service version are the same, parsing the annotation step in the incremental update instruction; Synchronize the marking steps to the service version, and set the version number of the service version as the sum of the number of the marking steps and the version number of the service version; Update the version snapshot corresponding to the service version.

5. The same-screen annotation processing method based on cloud conference according to claim 1, characterized in that: After the step of comparing the local version and the service version, the cloud conference-based on-screen annotation processing method further includes: If the version number of the local version is greater than the version number of the service version, the local version is updated to the service version.

6. The same-screen annotation processing method based on cloud conference according to claim 1, characterized in that: Before the step of obtaining the local version of the client and the service version of the server upon receiving the incremental update instruction sent by the client, the same-screen annotation processing method based on cloud conference further includes: When a marking creation instruction is detected, it is determined whether the current cloud conference will create a marking conference number; If so, obtain the snapshot data of the same-screen canvas; otherwise, create a marked conference number and add the same-screen canvas to the current cloud conference.

7. The same-screen annotation processing method based on cloud conference according to claim 1, characterized in that: The same-screen annotation processing method based on cloud conference also includes If the end sharing and exit marking signaling is detected, determining whether to retain the same-screen canvas, and if so, setting the same-screen canvas to a read-only state; If not, the window of the same-screen canvas is closed, the snapshot data of the same-screen canvas is cleared, and the annotation whiteboard component is uninstalled.

8. The same-screen annotation processing method based on cloud conference according to claim 7, characterized in that: The same-screen annotation processing method based on cloud conference also includes: If an exit marking instruction is detected from a target client not in conference sharing, determining whether the marked conference number of the same-screen canvas is created by the target client; If so, the window of the same-screen canvas is closed, the snapshot data of the same-screen canvas is cleared, and the annotation whiteboard component is uninstalled; otherwise, the canvas of the same-screen canvas is set to a read-only state.

9. A same-screen annotation processing device, characterized in that: The same-screen annotation processing device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the same-screen annotation processing method based on cloud conferencing as described in any one of claims 1 to 8.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the same-screen annotation processing method based on cloud conferencing as described in any one of claims 1 to 8 are implemented.

Images