Annotating apparatus and method in remote collaboration

Abstract

The present application relates to the technical field of augmented reality remote collaboration, and particularly relates to a labeling device and method in remote collaboration. The present application provides a labeling device in remote collaboration, comprising a picture freezing module configured to intercept a current video frame from a target video stream and generate a static collaboration canvas based on the current video frame in response to a screen freezing instruction sent by a first collaboration terminal; a labeling synchronization module configured to synchronize the static collaboration canvas to multiple collaboration terminals participating in the same remote collaboration session, and receive labeling operation data sent by any collaboration terminal in the multiple collaboration terminals after the static collaboration canvas is synchronized, and synchronize the labeling operation data to other collaboration terminals in real time, so that the other collaboration terminals synchronously present corresponding labeling content locally. The present application improves the accuracy of remote guidance, supports real-time labeling interaction of multi-terminal synchronization and multi-person collaboration, and realizes an efficient collaboration mode of multi-person labeling while watching and seeing the labeled content immediately.

Description

Technical Field

[0001] This invention relates to the field of augmented reality remote collaboration technology, and more particularly to a labeling device and method for remote collaboration. Background Technology

[0002] Augmented reality (AR)-based remote collaboration systems are widely used in highly specialized scenarios such as industrial equipment maintenance, safety inspections, and on-site training. In these scenarios, the on-site environment is typically in a state of continuous dynamic change, such as assembly line operation, robotic arm movement, or personnel movement. Remote experts need to provide guidance to on-site operators through real-time video streams transmitted by smart devices worn by on-site personnel.

[0003] Currently, mainstream AR remote collaboration systems generally employ a "real-time video stream and screen overlay annotation" approach to provide guidance. This means that remote experts, while watching a real-time video stream, can draw and annotate on the screen using a touchscreen or mouse. These annotations are then overlaid on the video stream and transmitted back to the on-site personnel in real time. However, due to the continuous changes in the live video feed, annotations made by experts at one moment may shift from the actual device alignment point the next moment due to changes in the viewpoint and object position. This phenomenon known as "annotation drift" leads to unclear or even misleading annotations, severely impacting the accuracy and efficiency of collaboration.

[0004] To address this issue, some systems have introduced screenshot annotation functionality. This feature allows users to manually capture the current video frame and send the screenshot to remote experts for offline annotation. After the experts complete the annotation, they send the image back. However, this approach has significant limitations: First, the screenshot and annotation process is offline and asynchronous, meaning the annotation process cannot be viewed in real-time by on-site personnel, lacking interactivity. Second, screenshot annotation typically only supports single-time, single-person annotation, and does not support real-time collaborative editing, undoing, style adjustments, or other interactive operations by multiple collaborators on the same screen. Finally, the annotation results often exist as independent images, making it difficult to associate them with specific maintenance tasks or fault work orders, hindering knowledge accumulation and retrospection.

[0005] Therefore, in a dynamically changing remote collaboration environment, there is an urgent need for a technical solution to overcome problems such as annotation drift, asynchronous collaboration, and irreversible operation in existing technologies. Summary of the Invention

[0006] The following provides a brief overview of one or more aspects to offer a basic understanding of them. This overview is not an exhaustive summary of all conceived aspects, nor is it intended to identify key or decisive elements of all aspects, nor to define the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form to prepare for the more detailed descriptions that follow.

[0007] The purpose of this invention is to provide an annotation device and method for remote collaboration, solving the technical problem of annotation drift caused by continuous changes in the image in existing augmented reality remote collaboration systems.

[0008] To achieve the above objectives, the present invention provides an annotation device for remote collaboration, including a screen freeze module and an annotation synchronization module: The screen freeze module is configured to respond to a screen freeze command sent by the first collaborative terminal, extract the current video frame from the target video stream, and generate a static collaborative canvas based on the current video frame. The annotation synchronization module is configured as follows: Synchronize the static collaborative canvas to multiple collaborative terminals participating in the same remote collaborative session; and After the static collaborative canvas is synchronized, the annotation operation data sent by any one of the multiple collaborative terminals is received, and the annotation operation data is synchronized to other collaborative terminals in real time, so that the other collaborative terminals can synchronously display the corresponding annotation content on the static collaborative canvas that is already displayed locally.

[0009] In some embodiments, the annotation operation data includes one or more of drawing trajectories, graphics, and text.

[0010] In some embodiments, it also includes: The interactive control module is configured to respond to interactive control commands sent by the first collaborative terminal and perform global operations on all annotation content on the static collaborative canvas. The interactive control commands include undo commands, clear commands, save commands, or commands to adjust annotation styles.

[0011] In some embodiments, the instructions for adjusting annotation styles include instructions for adjusting brush color or adjusting brush thickness.

[0012] In some embodiments, it also includes: The annotation storage module is configured to respond to the save command sent by the first collaborative terminal by merging the current static collaborative canvas with all annotation content superimposed on it, generating an annotation result file and storing it.

[0013] In some embodiments, the annotation storage module is further configured to: The configuration is set to associate the annotation result file with a predefined task work order.

[0014] In some embodiments, the plurality of collaborative terminals include one or more of augmented reality smart glasses, mobile terminals, mobile surveillance cameras, and personal computer PC terminals.

[0015] To achieve the above objectives, the present invention provides an annotation method for remote collaboration, comprising: In response to the screen freeze command sent by the first collaborative terminal, the current video frame is extracted from the target video stream, and a static collaborative canvas is generated based on the current video frame. The static collaborative canvas is synchronized to multiple collaborative terminals participating in the same remote collaborative session; After the static collaborative canvas is synchronized, it receives annotation operation data sent by any collaborative terminal. The annotation operation data is synchronized to other collaborative terminals in real time, so that the corresponding annotation content is displayed synchronously on the static collaborative canvas displayed locally on the other collaborative terminals.

[0016] In some embodiments, it also includes: In response to interactive control commands sent by the collaborative terminal, a global operation is performed on all annotation content on the static collaborative canvas. The interactive control commands include undo commands, clear commands, or commands to adjust annotation styles.

[0017] In some embodiments, it also includes: In response to the save command sent by the collaboration terminal, the current static collaboration canvas is merged with all the annotation content currently superimposed on it, generating an annotation result file and storing it.

[0018] In some embodiments, it also includes: The annotation result file is associated with a pre-defined task work order.

[0019] To achieve the above objectives, the present invention also provides a remote collaboration system, comprising: The first collaborative terminal is used to participate in remote collaborative sessions; And, as described above, a labeling device for remote collaboration, the device being communicatively connected to the first collaboration terminal, for executing the labeling method for remote collaboration as described above.

[0020] This invention proposes an annotation device and method for remote collaboration. By introducing a screen freeze mechanism into the dynamic video stream, the real-time image is locked as a static collaborative canvas, which significantly improves the accuracy of remote guidance. Moreover, the annotation device supports real-time annotation interaction with multiple terminals and multiple people, realizing an efficient collaborative mode where multiple people can annotate while watching and the annotations are immediately visible. Attached Figure Description

[0021] The above-described features and advantages of the present invention will be better understood after reading the following detailed description of embodiments of the present disclosure in conjunction with the accompanying drawings. In the drawings, components are not necessarily drawn to scale, and components having similar related characteristics or features may have the same or similar reference numerals.

[0022] Figure 1 A block diagram of an annotation device in remote collaboration according to an embodiment of the present invention is disclosed; Figure 2 A block diagram of a remote collaboration system according to an embodiment of the present invention is disclosed; Figure 3 A schematic diagram of a frozen screen annotation interface according to an embodiment of the present invention is disclosed; Figure 4 This invention discloses an overall architecture diagram of a remote collaboration system according to an embodiment of the present invention; Figure 5 A flowchart of a labeling method in remote collaboration according to an embodiment of the present invention is disclosed. Detailed Implementation

[0023] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Although the description of the present invention will be presented in conjunction with preferred embodiments, this does not mean that the features of the invention are limited to these embodiments. On the contrary, the purpose of describing the invention in conjunction with embodiments is to cover other options or modifications that may be derived based on the claims of the present invention. To provide a deep understanding of the invention, many specific details will be included in the following description, and the invention may also be implemented without using these details. Furthermore, to avoid confusion or obscuring the focus of the invention, some specific details will be omitted in the description.

[0024] Figure 1 An overall block diagram of a labeling device in remote collaboration according to an embodiment of the present invention is disclosed, such as... Figure 1 As shown, an annotation device for remote collaboration includes at least a screen freeze module 101 and an annotation synchronization module 102. The screen freeze module 101 is configured to respond to a screen freeze command sent by the first collaborative terminal, extract the current video frame from the target video stream, and generate a static collaborative canvas based on the current video frame.

[0025] The annotation synchronization module 102 is configured to synchronize the static collaborative canvas to multiple collaborative terminals participating in the same remote collaborative session; and

[0026] After the static collaborative canvas is synchronized, the annotation operation data sent by any one of the multiple collaborative terminals is received, and the annotation operation data is synchronized to other collaborative terminals in real time, so that the other collaborative terminals can synchronously display the corresponding annotation content on the static collaborative canvas that is already displayed locally.

[0027] This invention proposes an annotation device for remote collaboration. By introducing a screen freeze mechanism into the dynamic video stream, the real-time image is locked as a static collaborative canvas. This not only solves the annotation drift problem caused by continuous changes in the image and significantly improves the accuracy of remote guidance, but also supports real-time annotation interaction with multiple terminals and multiple people working together, realizing an efficient collaborative mode where multiple people can view and annotate at the same time and the annotations are immediately visible.

[0028] Figure 2 A block diagram of a remote collaboration system according to an embodiment of the present invention is disclosed, such as... Figure 2 As shown, the remote collaboration system includes: the annotation device and the first collaboration terminal 200 described above. The annotation device is deployed on the collaboration server 100. The first collaboration terminal 200 is used to participate in the remote collaboration session. The collaboration server 100 is communicatively connected to the first collaboration terminal 200 and is used to realize functions such as video stream forwarding, annotation synchronization and storage.

[0029] The first collaborative terminal 200 includes multiple collaborative terminals, which can be on-site terminals 201 (such as AR smart glasses, Android / iOS mobile phones, etc.) and remote expert terminals 202 (such as PCs, tablets, etc.).

[0030] The following is combined with Figure 1 , Figure 2 The annotation device for remote collaboration provided by the present invention will be described in detail with specific embodiments.

[0031] The screen freeze module 101 is used to respond to the screen freeze command sent by the first collaborative terminal 200, extract the current video frame from the target video stream, and generate a static collaborative canvas based on the current video frame.

[0032] The first collaborative terminal 200 is the initiator of the screen freeze action. The screen freeze action command can be initiated by on-site personnel or by remote experts. That is to say, such as Figure 2 As shown, the first collaborative terminal can be a field terminal 201 or a remote expert terminal 202. The field terminal 201 includes, but is not limited to, AR glasses / mobile devices (phones) of field personnel, and the remote expert terminal 202 includes, but is not limited to, PCs / mobile devices.

[0033] Screen freeze is a technology that freezes a webpage using keyboard shortcuts. It can be used to fix the content of a viewed page as a static image and has applications in document editing, AR remote guidance, and smart education screens. In AR remote guidance scenarios, experts can perform screen freeze annotations and provide real-time feedback to the operator.

[0034] The target video stream can be a real-time video stream collected by the on-site terminal 201, or a desktop stream or video stream shared by the remote expert terminal 202.

[0035] The annotation synchronization module 102 is used to synchronize the static collaborative canvas generated by the screen freeze module 101 to multiple collaborative terminals participating in the same remote collaborative session; and

[0036] After the static collaborative canvas is synchronized, the annotation operation data sent by any collaborative terminal is received, and the annotation operation data is synchronized to other collaborative terminals in real time, so that the corresponding annotation content is synchronously displayed on the static collaborative canvas displayed locally by the other collaborative terminals.

[0037] The multiple collaborative terminals include, but are not limited to, one or more of augmented reality smart glasses, mobile terminals, mobile surveillance cameras, and personal computer PC terminals.

[0038] In one embodiment, the annotation operation data includes one or more of drawing trajectories, graphics, and text.

[0039] In this embodiment, the frozen screen supports multi-user synchronous drawing and annotation. That is, after the screen is frozen, each member can draw, doodle, and write text on the frozen screen interface. The collaborative terminal will transmit the image to the collaborative server in real time, including personalized settings such as the drawing position, color, and pen size. After receiving the data, the collaborative server will forward the member data to each member terminal during the call. After receiving the data, other terminals will draw synchronously on the current canvas.

[0040] The annotation synchronization module enables interactive annotation through real-time collaboration across multiple devices, allowing multiple users to simultaneously annotate on the same static collaborative canvas. Annotation operation data from each collaborative terminal is forwarded to all participants in real time via the server, achieving a real-time collaborative experience of "annotating as you look, and seeing what you annotate immediately." It also reduces the reliance on the image description capabilities of on-site personnel, enabling non-professional users to quickly understand complex operation instructions through intuitive visual annotations. This makes the remote guidance process more interactive and timely, effectively reducing communication costs in cross-border and cross-language collaborations.

[0041] During remote collaboration, the annotation synchronization module 102 responds to the screen freeze command from any participant, instantly captures the current video frame to generate a static collaborative canvas and synchronizes it to all collaborator terminals. This decouples the dynamically changing real-time video stream from the annotation operation, ensuring that the annotation content is stably attached to the frozen key screen. This effectively avoids problems such as annotation drift and mis-pointing caused by continuous changes in the on-site environment, equipment movement, or perspective shifts. It provides remote experts with a stable and clear annotation benchmark, significantly improving the accuracy of guidance in dynamic operation scenarios such as industrial maintenance and safety inspection.

[0042] In one embodiment, the annotation device further includes an annotation storage module 103, configured to, in response to a save command sent by the first collaborative terminal, merge the current static collaborative canvas with all annotation content currently superimposed thereon, generate an annotation result file, and store it.

[0043] In this embodiment, the annotation storage module 103 is configured to, in response to a save command sent by the first collaborative terminal, merge the current static collaborative canvas with all annotation content currently superimposed on it, generate an annotation result file (e.g., a PNG format image), and store it on a local device or in the cloud. Furthermore, the annotation storage module 103 is also configured to associate the annotation result file with a predetermined task work order or fault record for subsequent backtracking analysis.

[0044] The annotation device for remote collaboration provided by this invention does not rely on SLAM (Simultaneous Localization and Mapping). Instead, it uses multi-sensor data, such as LiDAR and cameras, to estimate the current position and orientation of the device in real time and build an environmental map. Under the premise of spatial modeling or 3D registration, it achieves highly reliable annotation in a lightweight manner. It is suitable for edge operation scenarios with limited network conditions or terminal computing power, and realizes the functions of image freezing and multi-terminal synchronous annotation in dynamic remote collaboration scenarios.

[0045] In one embodiment, the labeling device further includes: The interactive control module is configured to respond to interactive control commands sent by the first collaborative terminal and perform global operations on all annotation content on the static collaborative canvas. The interactive control commands include undo commands, clear commands, save commands, or commands to adjust annotation styles.

[0046] Instructions for adjusting annotation styles include, but are not limited to, instructions for adjusting brush color or brush thickness.

[0047] In this embodiment, under the frozen screen mode, users of each collaborating terminal can make real-time annotations on the frozen screen.

[0048] Figure 3 A schematic diagram of a frozen screen annotation interface according to an embodiment of the present invention is shown, as follows: Figure 3 As shown, the frozen screen annotation interface is set on the collaborative terminal and may include: a static canvas 301, an annotation toolbar 302, an interactive control area 303, an annotation content display area 304, and an exit frozen screen button 305.

[0049] The following is combined with Figure 2 and Figure 3 To further explain the basic operation process of screen freeze annotation: During a remote collaborative call, users can select one member's video stream to be displayed in full screen. After clicking the freeze button, the system enters freeze mode and captures the current screen (executed by screen freeze module 101).

[0050] The static canvas 301 is the frozen display area. On the static canvas 301, users can select annotation tools, such as brushes, graphic or text tools, through the annotation toolbar 302 on the interface, and adjust the annotation style, such as brush color and thickness.

[0051] The captured image can be synchronously displayed to all other collaborating members (executed by the annotation synchronization module 102), and users can annotate on the image. Specifically, when a user performs drawing operations in the annotation content display area 304, their terminal will send the annotation operation data (including the position coordinates, color, thickness, and other personalized settings of the drawing trajectory) to the collaboration server 100 in real time.

[0052] After receiving the annotation operation data, the annotation synchronization module 102 immediately forwards it to all other collaborating terminals. Upon receiving the data, the other terminals synchronously draw the same annotation content on their local static collaborative canvas, achieving a real-time collaborative effect of "what is annotated is what you see".

[0053] Users can also use advanced interactive functions through the interactive control area 303. For example, clicking the "Undo" button can undo the most recent annotation operation; clicking the "Clear All" button can delete all annotations on the current canvas. These interactive control commands can also be synchronized to all terminals via the collaboration server.

[0054] Users can use the "Save as Image" function to merge the current annotations into the original frozen image and save it locally as a PNG image (annotation storage module 103). If they want to exit the frozen screen mode, they can click the exit frozen screen button 305 to return to the real-time video stream.

[0055] Freeze-screen annotation can be performed seamlessly on a variety of terminal devices, including AR smart glasses, Android / iOS phones, and PCs. On AR glasses, it can be combined with the first-person view to achieve an immersive collaborative experience of "what you see is what you annotate".

[0056] This invention makes the annotation process flexible and reversible through interactive functions such as undoing the previous step, clearing all annotations, and dynamically adjusting pen color and thickness. Users can adjust the annotation style in real time according to different precision requirements, improving operational convenience and user experience. Simultaneously, annotation results can be saved locally and linked to specific task orders or fault records, transforming temporary collaborative behavior into structured knowledge assets. This provides traceable and reusable visual evidence for optimizing enterprise fault handling SOPs and building new employee training systems.

[0057] Figure 4 This invention discloses an overall architecture diagram of screen freeze annotation in remote collaboration according to an embodiment of the present invention, such as... Figure 4 As shown, the overall architecture of the remote collaboration system is divided into three layers: the terminal layer, the platform (collaboration server), and multiple application scenarios. Each layer works together to provide stable and efficient screen freeze annotation functionality for remote collaboration.

[0058] The terminal layer mainly includes two types of devices: field terminals 201 and remote expert terminals 202. Field terminals 201 can be AR glasses, mobile phones, or tablets. AR glasses can be used for first-person view capture and augmented reality display by on-site personnel, while mobile devices can serve as backup on-site terminals or remote expert access points. All collaborative terminals connect to the platform service layer via a wireless network to achieve video stream uploading, annotation command transmission and reception, and image synchronization.

[0059] The platform can be deployed on cloud servers or other private servers, enabling functions such as remote collaborative AR annotation, remote screen freeze annotation, annotation storage, call logs, task management, offline editing, resource management, search services, user management, and access control. Among them, the screen freeze annotation function calls the platform-level screen freeze module 101 (deployed on the server) to capture real-time video frames to generate a static canvas, and achieves real-time collaboration across multiple terminals through the annotation synchronization module 102. Finally, the annotation results are stored in the annotation storage module 103 and associated with the task.

[0060] The annotation storage function is mainly used to persistently store the provided images, videos, annotation results and other data, supporting both local and cloud backups.

[0061] The call log function mainly enables full-domain call management, recording metadata for each remote collaboration session, including participants, time, and associated tasks.

[0062] The task management function covers task execution, task recording, and task publishing, realizing full lifecycle management from task creation, assignment, execution to archiving. The frozen screen annotation results can be linked to specific task work orders.

[0063] The offline editing function allows annotations to be temporarily saved locally when the network is unstable, and automatically synchronized to the server when the network is restored.

[0064] The resource management function enables the management of resources such as enterprise knowledge base, SOP documents, and equipment drawings, which can be quickly accessed during collaboration. SOP (Standard Operating Procedure) refers to a standard operating procedure, which aims to describe the standard operating steps and requirements for a certain event in a unified format, and is used to guide and standardize daily work.

[0065] The search service function enables full-text retrieval of historical annotation records, tasks, and resources.

[0066] User management and access control features enable multi-role login, identity authentication, and role-based operation permission management to ensure data security.

[0067] Based on the above architecture, the system can flexibly support a variety of business scenarios, such as remote collaboration, equipment maintenance, and security inspection.

[0068] This invention also provides a labeling method for remote collaboration, comprising: In response to a screen freeze command sent by the first collaborative terminal, the current video frame is extracted from the target video stream, and a static collaborative canvas is generated based on the current video frame. The static collaborative canvas is synchronized to multiple collaborative terminals participating in the same remote collaborative session; After the static collaborative canvas is synchronized, it receives annotation operation data sent by any collaborative terminal. The annotation operation data is synchronized to other collaborative terminals in real time, so that the corresponding annotation content is displayed synchronously on the static collaborative canvas displayed locally on the other collaborative terminals.

[0069] In one embodiment, the method further includes: In response to interactive control commands sent by the collaborative terminal, a global operation is performed on all annotation content on the static collaborative canvas. The interactive control commands include undo commands, clear commands, or commands to adjust annotation styles.

[0070] In one embodiment, the method further includes: In response to the save command sent by the collaboration terminal, the current static collaboration canvas is merged with all the annotation content currently superimposed on it to generate an annotation result file and store it, and the annotation result file is associated with the predetermined task work order.

[0071] Figure 5 A flowchart of an annotation method in remote collaboration according to an embodiment of the present invention is disclosed below. Figure 4 and Figure 5In detail, the above operation process should be explained. It should be noted that the annotation device in the remote collaboration described above is connected in communication with the first collaboration terminal and is used to execute the annotation method in the remote collaboration.

[0072] During a remote collaborative call, any participant (such as on-site personnel or remote experts) can select a member's real-time video stream as the target screen and click the "Freeze Screen" button on the interface to trigger the freeze screen command.

[0073] In response to the freeze command, the collaborative server's screen freeze module extracts the current video frame from the target video stream and generates a static collaborative canvas based on that frame. This canvas serves as the base map for all subsequent annotation operations.

[0074] The annotation synchronization module will synchronously push the generated static collaborative canvas to all collaborative terminals participating in the remote collaborative session.

[0075] Specifically, in the SDK (Software Development Kit) / plugin, the camera video stream is selected and the focus / freeze operation is triggered to lock the current image, i.e., the current video frame is captured, a static collaborative canvas is generated, and the local marker image library is loaded. The image is then preprocessed, including but not limited to noise reduction, cropping, and other optimizations of the video frames. The frozen frame is then synchronously uploaded to the cloud recognition service and pushed to all other collaborative terminals.

[0076] After the cloud recognition service completes feature comparison with the developer's Marker image library, it returns the Marker metadata and recognition results. The SDK / plugin side completes pose locking through local recognition and tracking before entering the rendering stage. It supports real-time annotation (drawing, text, graphics, etc.) on a static canvas by multiple users, and allows adjustment of annotation style (color, thickness) and interaction (undo, clear). Finally, it overlays 3D AR (Augmented Reality) content for presentation.

[0077] The management platform uses a web management service to configure accounts, database management, images, and recognition strategies. It also saves collaborative annotation results to local devices or links them to task work orders, forming a complete closed loop from terminal interaction and cloud recognition to collaborative annotation and result accumulation.

[0078] This invention provides an annotation device and method for remote collaboration, which can quickly lock key images in dynamic environments, support multi-user synchronous visual annotation, have rich interactive functions and does not rely on complex 3D reconstruction. It can be widely adapted to various devices such as AR smart glasses, mobile terminals, and PCs. Furthermore, it can provide stable, intuitive and savable annotation guidance capabilities while preserving the on-site context, thereby effectively improving the practicality and reliability of remote collaboration in real industrial scenarios.

[0079] It should be understood that the embodiments described above are merely illustrative. The embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or any combination thereof. For hardware implementation, the processor may be implemented within one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, and / or other electronic units designed to perform the functions described herein, or combinations thereof.

[0080] Some aspects of this application can be executed entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The aforementioned hardware or software may be referred to as a "data block," "module," "engine," "unit," "component," or "system." The processor may be one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DAPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, or combinations thereof. Furthermore, aspects of this application may manifest as computer products residing in one or more computer-readable media, including computer-readable program code. For example, computer-readable media may include, but are not limited to, magnetic storage devices (e.g., hard disks, floppy disks, magnetic tapes, etc.), optical discs (e.g., compressed CDs, digital multifunction DVDs, etc.), smart cards, and flash memory devices (e.g., cards, sticks, key drives, etc.).

[0081] A computer-readable medium may contain a propagated data signal containing computer program code, for example, on baseband or as part of a carrier wave. This propagated signal may take various forms, including electromagnetic, optical, and so on, or suitable combinations thereof. A computer-readable medium can be any computer-readable medium other than a computer-readable storage medium, which can be connected to an instruction execution system, apparatus, or device to enable communication, propagation, or transmission of a program for use. The program code located on the computer-readable medium can be propagated through any suitable medium, including radio, cable, fiber optic cable, radio frequency signals, or similar media, or any combination of the above media.

[0082] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0083] Furthermore, the terms "upper," "lower," "left," "right," "top," "bottom," "horizontal," and "vertical" used in the following description should be understood as the orientations shown in the relevant paragraphs and accompanying drawings. These relative terms are for illustrative purposes only and do not imply that the described apparatus must be manufactured or operated in a specific orientation, and therefore should not be construed as limiting the invention.

[0084] It is understood that although terms such as "first," "second," and "third" may be used herein to describe various components, regions, layers, and / or parts, these components, regions, layers, and / or parts should not be limited by these terms, and these terms are only used to distinguish different components, regions, layers, and / or parts. Therefore, the first components, regions, layers, and / or parts discussed below may be referred to as second components, regions, layers, and / or parts without departing from some embodiments of the present invention.

[0085] The foregoing description of this disclosure is intended to enable any person skilled in the art to make or use it. Various modifications to this disclosure will be apparent to those skilled in the art, and the general principles defined herein may be applied to other variations without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not intended to be limited to the examples and designs described herein, but should be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A labeling device for remote collaboration, characterized in that, Includes a freeze-frame module and a synchronized annotation module: The screen freeze module is configured to respond to a screen freeze command sent by the first collaborative terminal, extract the current video frame from the target video stream, and generate a static collaborative canvas based on the current video frame. The annotation synchronization module is configured as follows: Synchronize the static collaborative canvas to multiple collaborative terminals participating in the same remote collaborative session; and After the static collaborative canvas is synchronized, the annotation operation data sent by any one of the multiple collaborative terminals is received, and the annotation operation data is synchronized to other collaborative terminals in real time, so that the other collaborative terminals can synchronously display the corresponding annotation content on the static collaborative canvas that is already displayed locally.

2. The annotation device for remote collaboration according to claim 1, characterized in that, The annotation operation data includes one or more of the following: drawing trajectory, graphics, and text.

3. The annotation device for remote collaboration according to claim 1 or 2, characterized in that, Also includes: The interactive control module is configured to respond to interactive control commands sent by the first collaborative terminal and perform global operations on all annotation content on the static collaborative canvas. The interactive control commands include undo commands, clear commands, save commands, or commands to adjust annotation styles.

4. The annotation device for remote collaboration according to claim 3, characterized in that, The instructions for adjusting annotation styles include instructions for adjusting brush color or brush thickness.

5. The annotation device for remote collaboration according to claim 1, characterized in that, Also includes: The annotation storage module is configured to respond to the save command sent by the first collaborative terminal by merging the current static collaborative canvas with all annotation content superimposed on it, generating an annotation result file and storing it.

6. The annotation device for remote collaboration according to claim 5, characterized in that, The annotation storage module is further configured as follows: The configuration is set to associate the annotation result file with a predefined task work order.

7. The annotation device for remote collaboration according to claim 1, characterized in that, The plurality of collaborative terminals include one or more of augmented reality smart glasses, mobile terminals, mobile surveillance cameras, and personal computer PC terminals.

8. A labeling method in remote collaboration, characterized in that, include: In response to a screen freeze command sent by the first collaborative terminal, the current video frame is extracted from the target video stream, and a static collaborative canvas is generated based on the current video frame. The static collaborative canvas is synchronized to multiple collaborative terminals participating in the same remote collaborative session; After the static collaborative canvas is synchronized, it receives annotation operation data sent by any collaborative terminal. The annotation operation data is synchronized to other collaborative terminals in real time, so that the corresponding annotation content is displayed synchronously on the static collaborative canvas displayed locally on the other collaborative terminals.

9. The annotation method in remote collaboration according to claim 8, characterized in that, Also includes: In response to interactive control commands sent by the collaborative terminal, a global operation is performed on all annotation content on the static collaborative canvas. The interactive control commands include undo commands, clear commands, or commands to adjust annotation styles.

10. The annotation method in remote collaboration according to claim 8, characterized in that, Also includes: In response to the save command sent by the collaboration terminal, the current static collaboration canvas is merged with all the annotation content currently superimposed on it, generating an annotation result file and storing it.

11. The annotation method in remote collaboration according to claim 10, characterized in that, Also includes: The annotation result file is associated with a pre-defined task work order.

12. A remote collaboration system, characterized in that, include: The first collaborative terminal is used to participate in remote collaborative sessions; And, the annotation device in remote collaboration as described in any one of claims 1-7, wherein the device is communicatively connected to the first collaboration terminal and is used to perform the annotation method in remote collaboration as described in any one of claims 8-11.

Images