A method and system for synchronizing whiteboard data for remote consultation

Abstract

The present application relates to the field of medical device technology, and specifically to a method and system for synchronizing whiteboard data for remote consultation, the method comprising: the acquisition end acquires multiple medical images, and sends the acquired multiple medical videos to the guidance end through the server end, and the guidance end displays multiple medical videos on multiple video screens; the guidance end uses the electronic whiteboard on the video screen to acquire the handwriting data marked by the user on the video screen of the guidance end, and sends the handwriting data to the server end; the server end stores the handwriting data in a one-to-one correspondence with the video screen, and sends the handwriting data to the acquisition end and the remaining guidance ends on the same video screen for display. The above scheme ensures the real-time and synchronization of the handwriting data displayed on each end, and users on multiple ends can view the handwriting data marked on a certain end at the same time on each end, and determine whether to render the handwriting data as needed.

Description

Technical Field

[0001] The present application relates to the field of medical device technology, and in particular to a method and system for synchronizing whiteboard data for remote consultation. Background Art

[0002] During remote surgical consultation, the acquisition end will encode various video images of the surgical process, such as multi-channel video images of the laparoscope, microscope, and surgical field camera, as well as voice, and send them to multiple guidance ends.

[0003] Although the video + voice interaction method is very convenient, there is still a problem of unclear description. In order to facilitate the guidance of the surgical process by the guidance end, it is necessary to support any guidance end to mark on any video screen, and mark it on other clients watching the same screen to facilitate understanding.

[0004] In the above scheme, it is crucial that multiple clients synchronously display the annotation content. Therefore, there is an urgent need for a synchronization method of whiteboard data for remote consultation, which can realize the synchronization of whiteboard data in real time. Summary of the invention

[0005] In view of this, an embodiment of the present application provides a method and system for synchronizing whiteboard data of remote consultation, which can synchronize whiteboard data on multiple guidance terminals in real time.

[0006] In a first aspect, the present application provides a method for synchronizing whiteboard data of a remote consultation, the method being applied to a synchronization system for whiteboard data of a remote consultation, the system comprising a service end, a collection end, and at least one guidance end, the method comprising:

[0007] The acquisition end acquires multiple medical images, and sends the acquired multiple medical videos to the guidance end through the service end, and the guidance end displays the multiple medical videos in multiple video screens; each video screen corresponds to one medical video;

[0008] The guidance terminal collects handwriting data marked by the user on the video screen of the guidance terminal through the electronic whiteboard on the video screen, and sends the handwriting data to the service terminal;

[0009] The server stores the handwriting data and the video screen in a one-to-one correspondence, and sends the handwriting data to the collection terminal and other guiding terminals for display on the same video screen.

[0010] According to the above technical means, the electronic whiteboard of the present application can allow multiple terminals to see the handwriting data marked by a certain terminal together, so that the handwriting data of multiple terminals can be displayed synchronously in real time.

[0011] In combination with the first aspect, in one implementation, the acquisition end acquires multiple medical images, encodes the acquired multiple medical videos and sends them to the server end, and the guidance end obtains the multiple medical videos from the server end.

[0012] According to the above technical means, the acquisition end needs to perform an encoding operation after encoding the collected multiple medical videos and sending them to the server end, and the guidance end also needs to perform a corresponding decoding operation when obtaining the multiple medical videos from the server end.

[0013] In combination with the first aspect, in one implementation, when the guiding terminal collects the handwriting data marked by the user through the electronic whiteboard on the video screen, continuous points in the window are captured according to the events on the video screen window, and the captured continuous points are formed into an array to form the handwriting data; the data type of the handwriting data includes the handwriting operation type, the handwriting shape type, the handwriting color, the handwriting thickness and the attribution information;

[0014] The guiding end encodes and compresses the handwriting data and sends it to the service end.

[0015] According to the above-mentioned technical means, because there are multiple video screens in the remote consultation process, the screen layouts of the acquisition end and each guidance end may be different. Therefore, when extracting the handwriting data, the event capture points on the video screen window are used, and those beyond the video screen window are not recorded, so that the handwriting data are all one-to-one on the same video screen during rendering, and there will be no misalignment.

[0016] In combination with the first aspect, in one implementation, the server stores the handwriting data and the video screen in a one-to-one correspondence, and records the sender address, timestamp, and absolute coordinate value of each handwriting data on the corresponding video screen;

[0017] According to the sender address and the absolute coordinate value of the handwriting data on the corresponding video screen, the handwriting data is sent to the same video screen of the collection end and other guiding ends for display.

[0018] According to the above technical means, the collection of handwriting data at this end and its display at other ends are converted based on the absolute coordinate values ​​of the handwriting data on the corresponding video screen, thereby ensuring the consistency of handwriting data rendering.

[0019] In combination with the first aspect, in one implementation, when canceling the current handwriting data of the guiding terminal, the guiding terminal sends a cancel command to the server terminal;

[0020] The server cancels the handwriting data corresponding to the cancellation command on the same video screen of the collection terminal and all the guiding terminals according to the cancellation command.

[0021] In combination with the first aspect, in one implementation, when clearing all handwriting data of the target guiding end, the target guiding end sends a clear command to the service end;

[0022] The service end cancels all handwriting data recorded by the target guiding end on the same video screen of the collection end and all guiding ends according to the clear command.

[0023] In combination with the first aspect, in one implementation, when there are other guidance terminals joining the remote consultation, the service terminal sends all the saved handwriting data to the corresponding video screen of the newly joined guidance terminal.

[0024] According to the above technical means, all the saved handwriting data are sent to the corresponding video screen of the newly added guidance terminal, so that the application scenario of this application is broader and is not limited by the number and access status of the guidance terminals.

[0025] In combination with the first aspect, in one implementation, the acquisition end and each of the guiding ends record each video screen and the handwriting data superimposed on the video screen to produce teaching courseware.

[0026] According to the above technical means, the handwriting data during the remote consultation process is recorded to produce teaching courseware for easy postoperative review and teaching.

[0027] In combination with the first aspect, in one implementation, when the collection end and each guiding end display the handwriting data, the curve portion of the handwriting data is drawn using an N-order Bezier curve.

[0028] According to the above technical means, the curve part of the handwriting data is drawn using an N-order Bezier curve, so that when the handwriting data is displayed, the curve part is smoother and has no jagged feeling.

[0029] In a second aspect, the present application provides a system for synchronizing whiteboard data for remote consultation, the system comprising a service end, a collection end, and at least one guidance end;

[0030] The acquisition end is used to acquire multiple medical images and send the acquired multiple medical videos to the guidance end through the service end;

[0031] The guidance terminal is used to display multiple video screens of the multiple medical videos, collect handwriting data marked by the user on the video screen of the guidance terminal through the electronic whiteboard on the video screen, and send the handwriting data to the service terminal; each video screen corresponds to one medical video;

[0032] The server is used to store the handwriting data and the video screen in a one-to-one correspondence, and send the handwriting data to the collection end and other guiding ends for display on the same video screen.

[0033] In a third aspect, the present application provides a computer device, comprising a processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the above-mentioned method for synchronizing whiteboard data of remote consultation.

[0034] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the storage medium stores at least one instruction, and the at least one instruction is loaded and executed by a processor to implement the above-mentioned method for synchronizing whiteboard data of remote consultation.

[0035] The technical solution provided by this application may have the following beneficial effects:

[0036] In this application, if the user needs to mark notes on the electronic whiteboard on the video screen, the guidance end will collect the handwriting data marked by the user on the video screen of the guidance end through the electronic whiteboard on the video screen, and send the handwriting data to the server end; the server end stores the handwriting data and the video screen in a one-to-one correspondence, and sends the handwriting data to the collection end and the other guidance ends on the same video screen for display, ensuring the real-time and synchronization of the handwriting data displayed on each end, and users of multiple ends can view the handwriting data marked on a certain end at the same time on each end;

[0037] The handwriting data collected by this application is stored on the server side, so other unconnected terminals can join the remote consultation at any time, and can receive complete historical handwriting data from the server side and draw it on the corresponding video screen; in addition, since the handwriting data of this application is drawn on the video screen being rendered, if necessary, a certain terminal can pause the video screen rendering of all terminals to facilitate viewing and annotation, or continue video rendering, and the application is more flexible; that is, if a certain guidance terminal does not pay attention to some screens, it can turn off the screen rendering, or render it when needed;

[0038] Furthermore, the present application captures points based on events on the video screen window to collect handwriting data at each end, and does not record data beyond the video screen window. Moreover, the collection of handwriting data at this end and the display at the other end are converted and rendered based on the absolute coordinate values ​​of the handwriting data on the corresponding video screen, thereby ensuring the consistency of handwriting data rendering. Even if the layout and number of video screens at each end are different, the handwriting data of the present application can also be drawn on any video screen in a targeted manner without causing confusion. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0040] Figure 1 The diagram is a schematic structural diagram of a system for synchronizing whiteboard data for remote consultation according to an exemplary embodiment.

[0041] Figure 2 The present invention is a flowchart of a method for synchronizing whiteboard data of a remote consultation according to an exemplary embodiment.

[0042] Figure 3 The present invention is a flowchart of a method for synchronizing whiteboard data of a remote consultation according to an exemplary embodiment.

[0043] Figure 4 A method for synchronizing whiteboard data for remote consultation according to an embodiment of the present application is shown.

[0044] Figure 5 A structural block diagram of a computer device shown in an exemplary embodiment of the present application is shown. DETAILED DESCRIPTION

[0045] The technical solution of the present application will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0046] Figure 1 is a structural diagram of a synchronization system of whiteboard data for remote consultation according to an exemplary embodiment, the system comprising a service end 120, a collection end 110 and at least one guidance end 130;

[0047] The acquisition terminal 110 is used to acquire multiple medical images and send the acquired multiple medical videos to the guidance terminal 130 through the service terminal 120;

[0048] The guidance terminal 130 is used to display multiple video screens of the multiple medical videos, collect the handwriting data marked by the user on the video screen of the guidance terminal 130 through the electronic whiteboard on the video screen, and send the handwriting data to the service terminal 120; each video screen corresponds to one medical video;

[0049] The server 120 is used to store the handwriting data and the video screen in a one-to-one correspondence, and send the handwriting data to the collection terminal and other guiding terminals 130 for display on the same video screen.

[0050] To summarize, in the present application, if the user needs to mark notes on the electronic whiteboard on the video screen, the guidance end will collect the handwriting data marked by the user on the video screen of the guidance end through the electronic whiteboard on the video screen, and send the handwriting data to the server end; the server end stores the handwriting data in a one-to-one correspondence with the video screen, and sends the handwriting data to the collection end and the other guidance ends for display on the same video screen, thereby ensuring the real-time and synchronization of the handwriting data displayed on each end, and users on multiple ends can view the handwriting data marked on a certain end at the same time on each end.

[0051] Figure 2 is a method flow chart of a method for synchronizing whiteboard data of a remote consultation according to an exemplary embodiment. The method is applied to Figure 1 A synchronization system of whiteboard data for remote consultation is shown. Figure 2 As shown, the method may include the following steps:

[0052] Step S201, the acquisition end acquires multiple medical images, and sends the acquired multiple medical videos to the guidance end through the service end, and the guidance end displays the multiple medical videos in multiple video screens; each video screen corresponds to one medical video.

[0053] In a possible implementation, the acquisition end acquires multiple medical images during the operation, and the multiple medical images are medical images at different angles and directions. The acquisition end encodes the acquired multiple medical images and sends them to the server end in real time, and the server end sends them to each connected guidance end. Each guidance end decodes the received multiple medical images and renders them in real time. During rendering, each medical image corresponds to a video screen display. Therefore, multiple video screens can be displayed in real time on the display screen of each guidance end. The video screen layout of each diagnosis and guidance end can be set according to actual conditions or user preferences.

[0054] Step S202: The guidance terminal collects the handwriting data marked by the user on the video screen of the guidance terminal through the electronic whiteboard on the video screen, and sends the handwriting data to the service terminal.

[0055] In one possible implementation, when a user takes notes on an electronic whiteboard on a video screen, the corresponding guidance terminal collects the handwriting data marked by the user on the video screen of the guidance terminal through the electronic whiteboard on the video screen, encodes and compresses the handwriting data, and sends it to the server, which stores and forwards it.

[0056] Step S203: The server stores the handwriting data and the video screen in a one-to-one correspondence, and sends the handwriting data to the collection terminal and other guiding terminals for display on the same video screen.

[0057] In a possible implementation, when the server stores the handwriting data, it will record the sender address of the handwriting data, the absolute coordinates on the video screen, and the marked timestamp for easy distinction. When the handwriting data is sent to the acquisition end and other guidance ends for display on the same video screen, the acquisition end and other guidance ends will decompress and decode the received handwriting data, and render it on the same video screen according to the sender address and absolute coordinates, so that the handwriting data displayed on each end is synchronized.

[0058] In summary, in the present application, if the user needs to mark notes on the electronic whiteboard on the video screen, the guidance end will collect the handwriting data marked by the user on the video screen of the guidance end through the electronic whiteboard on the video screen, and send the handwriting data to the server end; the server end stores the handwriting data and the video screen in a one-to-one correspondence, and sends the handwriting data to the collection end and the other guidance ends on the same video screen for display, thereby ensuring the real-time and synchronization of the handwriting data displayed on each end, and users of multiple ends can view the handwriting data marked on a certain end at the same time on each end;

[0059] The handwriting data collected by this application is stored on the server side, so other unconnected terminals can join the remote consultation at any time, and can receive complete historical handwriting data from the server side and draw it on the corresponding video screen; in addition, since the handwriting data of this application is drawn on the video screen being rendered, if necessary, a certain terminal can pause the video screen rendering of all terminals to facilitate viewing and annotation, or continue video rendering, and the application is more flexible; that is, if a certain guidance terminal does not pay attention to some screens, it can turn off the screen rendering, or render it when needed;

[0060] Furthermore, the present application captures points based on events on the video screen window to collect handwriting data at each end, and does not record data beyond the video screen window. Moreover, the collection of handwriting data at this end and the display at the other end are converted and rendered based on the absolute coordinate values ​​of the handwriting data on the corresponding video screen, thereby ensuring the consistency of handwriting data rendering. Even if the layout and number of video screens at each end are different, the handwriting data of the present application can also be drawn on any video screen in a targeted manner without causing confusion.

[0061] Figure 3 is a method flow chart of a method for synchronizing whiteboard data of a remote consultation according to an exemplary embodiment. The method is applied to Figure 1 A synchronization system of whiteboard data for remote consultation is shown. Figure 3 As shown, the method may include the following steps:

[0062] Step S301, the acquisition end acquires multiple medical images, and sends the acquired multiple medical videos to the guidance end through the service end, and the guidance end displays the multiple medical videos in multiple video screens; each video screen corresponds to one medical video.

[0063] In a possible implementation, the acquisition end acquires multiple medical images, encodes the acquired multiple medical videos and sends them to the server end, and the guidance end obtains the multiple medical videos from the server end.

[0064] Furthermore, the acquisition end collects medical equipment images or operation videos, encodes the medical images collected by this end and sends them to the server through a tcp / udp socket (socket socket of TCP protocol or socket socket of UDP protocol), and then the guidance end obtains multiple medical images from the server, decodes them and renders them in real time.

[0065] Optionally, data encoding and decoding are performed using custom encoding rules, using a protobuf compiler or json (JavaScript Object Notation).

[0066] Step S302: The guidance terminal collects the handwriting data marked by the user on the video screen of the guidance terminal through the electronic whiteboard on the video screen, and sends the handwriting data to the service terminal.

[0067] In a possible implementation, when the guiding terminal collects the handwriting data marked by the user through the electronic whiteboard on the video screen, it captures continuous points in the window according to the events on the video screen window, and forms an array of the captured continuous points to form the handwriting data;

[0068] The guide end encodes and compresses the handwriting data and sends it to the server end. Since the compressed handwriting data is stored in the server end, a new guide end can be added at any time and can receive the complete historical handwriting data from the server end and draw it on the video screen.

[0069] Furthermore, any guiding terminal can open an electronic whiteboard or guiding interface on any video screen (the electronic whiteboard can allow multiple terminals to see the handwriting data marked by a certain terminal together). The electronic whiteboard is used to mark the video screen. Any guiding terminal draws handwriting on the opened real-time video electronic whiteboard, and after collecting the handwriting data, the handwriting data encoding is compressed and then sent to the server through TCP socket. The server saves the handwriting data of the guiding terminal corresponding to the video and sends it to the collecting terminal and the remaining guiding terminals; the handwriting data of this embodiment is sent to the server after binary compression processing, which reduces the data volume and cache pressure. The server stores the compressed handwriting data and sends it to all other clients (collecting terminal and other guiding terminals) in sequence. Due to the characteristics of the TCP protocol, data will not be lost or disordered. The client will save it in order in an array or linked list, and it will be decompressed when rendering the handwriting data. There is no need to add additional tags and it will not cause disorder.

[0070] Furthermore, when the guiding end pulls the stream of a video screen sent by a certain acquisition end, it first creates a video window locally, then saves the id of the window (for example, window_a), and binds it to the id of this video screen (for example, video_a). When the handwriting data is sent, it will include the id of the video screen (video_a). When rendering the handwriting data, the id of the video window (window_a) is found by searching the id of the video screen (video_a), and then the handwriting data is rendered onto the video screen. That is to say, when the guidance end displays the target video screen of the multiple medical videos (the target video screen is any video screen of the multiple medical videos), it first creates a video window locally on the guidance end, and binds and saves the address of the video window and the address of the target video screen. When the handwriting data collected by the guidance end is sent to the collection end and other guidance ends for display, the handwriting data sent by the guidance end is sent together with the address of the video window and the address of the target video screen. According to the address of the target video screen, the video screen addresses of the collection end and the other guidance ends are matched, and the same video screen is found, and the handwriting data collected at the address of the video window is displayed.

[0071] Furthermore, since there are multiple video screens during the consultation process, the screen layouts of the acquisition end and each guidance end may be different. Therefore, when extracting handwriting data, a mouse, touch screen, motion capture or other possible acquisition devices will be used to capture and record continuous points in a certain video window to form an array. Points outside the video window will not be recorded. When rendering, the handwriting data are all one-to-one on the same video screen and will not be misplaced.

[0072] Step S303: The server stores the handwriting data and the video screen in a one-to-one correspondence, and sends the handwriting data to the collection terminal and other guiding terminals for display on the same video screen.

[0073] In a possible implementation, the server stores the handwriting data and the video screen in a one-to-one correspondence, and records the sender address, timestamp, and absolute coordinate value of each handwriting data on the corresponding video screen;

[0074] According to the sender address and the absolute coordinate value of the handwriting data on the corresponding video screen, the handwriting data is sent to the same video screen of the acquisition end and other guiding ends for display.

[0075] Furthermore, after the collection end and the other guidance ends decode the handwriting data, they obtain the same video corresponding to the handwriting data on the collection end and the other guidance ends according to the sender address, and render the handwriting data on the corresponding same video according to the absolute coordinate value of the handwriting data for display. In other words, the collected handwriting data needs to be converted into the absolute coordinates of the current video screen, and the same applies to rendering, so as to ensure consistency.

[0076] Furthermore, the server will record who sent the handwriting data under the video screen and the timestamp for easy distinction. The handwriting data drawn by the collection end and the guidance end will also be saved locally on the collection end, so as to achieve the purpose of remote medical guidance.

[0077] Step S304: When the current handwriting data of the guiding terminal is to be cancelled, the guiding terminal sends a cancellation command to the server terminal; the server terminal cancels the handwriting data corresponding to the cancellation command on the same video screen of the acquisition terminal and all guiding terminals according to the cancellation command.

[0078] Furthermore, the guidance end can cancel the handwriting data of this end and send the cancellation command to the server end. The server end cancels the current handwriting data recorded by the guidance end and transmits it to the collection end and other guidance ends, and then removes the canceled handwriting data from the corresponding video screen.

[0079] Step S305, when all handwriting data of the target guidance end are cleared, the target guidance end sends a clear command to the server end; the server end cancels all handwriting data recorded by the target guidance end on the same video screen of the acquisition end and all guidance ends according to the clear command.

[0080] Furthermore, the guidance end can also clear all the handwriting data of this end and send a clear command to the server end. The server end cancels all the handwriting data recorded by the guidance end and transmits it to the collection end and other guidance ends, and then clears the handwriting data on the corresponding same video screen.

[0081] In a possible implementation, when there are other guiding terminals joining the remote consultation, the server sends all the saved handwriting data to the corresponding video screen of the newly joined guiding terminal.

[0082] Furthermore, during the consultation and guidance process, more guidance terminals may be added (or a guidance terminal may quit and rejoin due to itself or network problems). When joining the guidance, the handwriting data of the remaining guidance terminals previously saved by the server will also be drawn on the same video screen on the newly added guidance terminal.

[0083] In a possible implementation, the acquisition end and each of the guiding ends record each video screen and the handwriting data superimposed on the video screen to produce teaching courseware.

[0084] Furthermore, this embodiment supports multiple guidance terminals, and the teaching function can also be turned on at the collection terminal to realize whiteboard teaching and support the video pause function. Both the guidance terminal and the collection terminal can record the video screen and the handwriting data superimposed on it, which is convenient for postoperative review and the production of teaching courseware. Since the handwriting data is depicted on the video screen being rendered, if necessary, one end can pause the video screen rendering of all ends to facilitate viewing of the annotated note data, or continue video rendering.

[0085] In a possible implementation manner, when the collection end and each guiding end display the handwriting data, the curve part of the handwriting data is drawn using an N-order Bezier curve.

[0086] Furthermore, when displaying handwriting data, in order to make the curve part smoother and less jagged, the N-order Bezier curve can be set for drawing during rendering. The display of handwriting data supports thickness design, and can also be operated similarly to drawing software. You can directly type or choose various shapes such as rectangles and ellipses to draw. The drawn handwriting data can be any doodle, including not only triangles, straight lines, circles, etc.

[0087] Furthermore, the data type of the handwriting data includes handwriting operation type, handwriting shape type, handwriting color, handwriting thickness and attribution information;

[0088] The following is a detailed description of the data type and data information of handwriting data:

[0089]

[0090]

[0091]

[0092]

[0093] Furthermore, the layout and number of video screens at each end of this embodiment may be different, and the handwriting data can be drawn on any video screen without confusion. The handwriting data rendering screens at the acquisition end and the guidance end are described below through a simple example:

[0094] Assume that there is a collection end and three guidance ends ABC. The collection end pushes four different medical videos. Although each guidance end receives the same four video images, the image layout of the guidance end can be different. For example, Figure 4 The collection end is a 4K screen, the guidance end A is a 1080p screen, and the handwriting is red (such as Figure 4 The guide terminal B is a dual screen, and the drawing handwriting is green (such as red 1, red 2 and red 4). Figure 4 Green 1, Green 2 and Green 3 in the figure), the guide terminal C is a 2K screen, and the drawing handwriting is blue (such as Figure 4 Blue 1 and Blue 3).

[0095] In summary, in the present application, if the user needs to mark notes on the electronic whiteboard on the video screen, the guidance end will collect the handwriting data marked by the user on the video screen of the guidance end through the electronic whiteboard on the video screen, and send the handwriting data to the server end; the server end stores the handwriting data and the video screen in a one-to-one correspondence, and sends the handwriting data to the collection end and the other guidance ends on the same video screen for display, thereby ensuring the real-time and synchronization of the handwriting data displayed on each end, and users of multiple ends can view the handwriting data marked on a certain end at the same time on each end;

[0096] The handwriting data collected by this application is stored on the server side, so other unconnected terminals can join the remote consultation at any time, and can receive complete historical handwriting data from the server side and draw it on the corresponding video screen; in addition, since the handwriting data of this application is drawn on the video screen being rendered, if necessary, a certain terminal can pause the video screen rendering of all terminals to facilitate viewing and annotation, or continue video rendering, and the application is more flexible; that is, if a certain guidance terminal does not pay attention to some screens, it can turn off the screen rendering, or render it when needed;

[0097] Furthermore, the present application captures points based on events on the video screen window to collect handwriting data at each end, and does not record data beyond the video screen window. Moreover, the collection of handwriting data at this end and the display at the other end are converted and rendered based on the absolute coordinate values ​​of the handwriting data on the corresponding video screen, thereby ensuring the consistency of handwriting data rendering. Even if the layout and number of video screens at each end are different, the handwriting data of the present application can also be drawn on any video screen in a targeted manner without causing confusion.

[0098] See also Figure 5 , which is a schematic diagram of a computer device provided according to an exemplary embodiment of the present application, the computer device includes a memory and a processor, the memory is used to store a computer program, and when the computer program is executed by the processor, the above-mentioned method for synchronizing whiteboard data of remote consultation is implemented.

[0099] The processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0100] As a non-transitory computer-readable storage medium, the memory can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions / modules corresponding to the method in the implementation of the present application. The processor executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory, that is, implementing the method in the above method implementation.

[0101] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0102] In an exemplary embodiment, a computer-readable storage medium is also provided, which is used to store at least one computer program, and the at least one computer program is loaded and executed by a processor to implement all or part of the steps in the above method. For example, the computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0103] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the application disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0104] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for synchronizing whiteboard data for remote consultation, characterized in that: The method is applied to a synchronization system of whiteboard data of remote consultation, the system comprising a service end, a collection end and at least one guidance end, and the method comprises: The acquisition end acquires multiple medical images, and sends the acquired multiple medical videos to the guidance end through the service end, and the guidance end displays the multiple medical videos on multiple video screens; each video screen corresponds to one medical video; when the guidance end displays the target video screen in the multiple medical videos, a video window is created locally on the guidance end, and the address of the video window is bound to the address of the target video screen and saved; The guidance terminal collects handwriting data marked by the user on the video screen of the guidance terminal through the electronic whiteboard on the video screen, and sends the handwriting data to the service terminal; The server stores the handwriting data and the video screen in a one-to-one correspondence, and sends the handwriting data to the collection terminal and other guiding terminals for display on the same video screen; The guiding end collects handwriting data marked by the user on the video screen of the guiding end through the electronic whiteboard on the video screen, and sends the handwriting data to the service end, including: When the guiding terminal collects the handwriting data marked by the user through the electronic whiteboard on the video screen, it captures continuous points in the window according to the events on the video screen window, and forms an array of the captured continuous points to form the handwriting data; the data type of the handwriting data includes the handwriting operation type, the handwriting shape type, the handwriting color, the handwriting thickness and the attribution information; The guiding end encodes and compresses the handwriting data and sends it to the service end; The server stores the handwriting data and the video screen in a one-to-one correspondence, and sends the handwriting data to the acquisition terminal and other guiding terminals for display on the same video screen, including: The server stores the handwriting data and the video screen in a one-to-one correspondence, and records the sender address, timestamp and absolute coordinate value of each handwriting data on the corresponding video screen; According to the sender address and the absolute coordinate value of the handwriting data on the corresponding video screen, the handwriting data is sent to the same video screen of the acquisition terminal and other guiding terminals for display; The method further comprises: After the acquisition end and the other guidance ends decode the handwriting data, they obtain the same video screen corresponding to the handwriting data on the acquisition end and the other guidance ends according to the sender address, and render the handwriting data on the corresponding same video screen according to the absolute coordinate value of the handwriting data for display.

2. The method according to claim 1, characterized in that The acquisition end acquires multiple medical images and sends the acquired multiple medical videos to the guidance end through the service end, including: The acquisition end acquires multiple medical images, encodes the acquired multiple medical videos and sends them to the service end, and the guidance end obtains the multiple medical videos from the service end.

3. The method according to claim 1, characterized in that The method further comprises: When canceling the current handwriting data of the guiding terminal, the guiding terminal sends a cancel command to the server terminal; The server cancels the handwriting data corresponding to the cancellation command on the same video screen of the collection terminal and all the guiding terminals according to the cancellation command.

4. The method according to claim 1, characterized in that: The method further comprises: When clearing all handwriting data of the target guiding end, the target guiding end sends a clearing command to the service end; The service end cancels all handwriting data recorded by the target guiding end on the same video screen of the collection end and all guiding ends according to the clear command.

5. The method according to claim 1, characterized in that: The method further comprises: When there are other guiding terminals joining the remote consultation, the server sends all the saved handwriting data to the corresponding video screen of the newly joined guiding terminal.

6. The method according to claim 1, characterized in that The method further comprises: The acquisition end and each of the guiding ends record each video screen and the handwriting data superimposed on the video screen to produce teaching courseware.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: When the collection end and each guiding end display the handwriting data, the curve part of the handwriting data is drawn using an N-order Bezier curve.

8. A synchronization system for whiteboard data of remote consultation, characterized in that: The system includes a service end, a collection end, and at least one guidance end; The acquisition end is used to acquire multiple medical images and send the acquired multiple medical videos to the guidance end through the service end; The guidance terminal is used to display multiple video screens of the multiple medical videos, collect handwriting data marked by the user on the video screen of the guidance terminal through the electronic whiteboard on the video screen, and send the handwriting data to the service terminal; each video screen corresponds to one medical video; when the guidance terminal displays the target video screen in the multiple medical videos, a video window is created locally on the guidance terminal, and the address of the video window is bound to the address of the target video screen and saved; The server is used to store the handwriting data and the video screen in a one-to-one correspondence, and send the handwriting data to the acquisition terminal and other guiding terminals for display on the same video screen; The guiding terminal is also used to collect the handwriting data marked by the user through the electronic whiteboard on the video screen, capture the continuous points in the window according to the events on the video screen window, and form an array of the captured continuous points to form the handwriting data; the data type of the handwriting data includes the handwriting operation type, the handwriting shape type, the handwriting color, the handwriting thickness and the attribution information; The guiding end is also used to encode and compress the handwriting data and send it to the service end; The server is further used to store the handwriting data and the video screen in a one-to-one correspondence, and record the sender address, timestamp and absolute coordinate value of the handwriting data on the corresponding video screen of each handwriting data; according to the sender address and the absolute coordinate value of the handwriting data on the corresponding video screen, the handwriting data is sent to the same video screen of the acquisition terminal and other guiding terminals for display; The acquisition end and the other guidance ends are also used to decode the handwriting data, obtain the same video screen corresponding to the handwriting data on the acquisition end and the other guidance ends according to the sender address, and render the handwriting data on the corresponding same video screen according to the absolute coordinate value of the handwriting data for display.

Images