Method, device, equipment and storage medium for synchronizing electronic whiteboard display content

Abstract

The present invention relates to the field of video data processing technology, and provides a method, device, equipment and storage medium for synchronizing the display content of an electronic whiteboard. The method comprises: in the process of sharing an electronic whiteboard with a second visual networking terminal, converting the display content of the electronic whiteboard into display data; encapsulating the display data into a first visual networking protocol packet; adding a header to the first visual networking protocol packet; and sending the first visual networking protocol packet carrying the header to the second visual networking terminal through a visual networking transparent transmission channel between the second visual networking terminal. This allows the display content of the electronic whiteboard to be stably synchronized between the visual networking terminals that are conducting a video conference or a video call.

Description

Technical Field

[0001] The present invention relates to the technical field of video data processing, and in particular to a method, device, equipment and storage medium for synchronizing display content of an electronic whiteboard. Background Art

[0002] Currently, in electronic whiteboard sharing, the display content of the electronic whiteboard interface is mainly transmitted through the TCP / IP protocol. The data transmission speed is easily affected by the network status, resulting in delays in the display of electronic whiteboard strokes and pictures.

[0003] The visual network is a real-time network that can achieve real-time transmission of high-definition video across the entire network, which is currently unavailable on the Internet, and enable high-definition face-to-face video conferencing. Multiple terminals on the Internet can collaboratively operate the same electronic whiteboard, providing convenience for users to work remotely and conduct distance education. However, using the TCP / IP protocol to synchronize stroke data in the visual network cannot achieve the effect of real-time synchronization of strokes edited on the electronic whiteboard. Summary of the invention

[0004] In view of the above problems, embodiments of the present invention are proposed to provide a method, apparatus, device and storage medium for synchronizing electronic whiteboard display content that overcomes the above problems or at least partially solves the above problems.

[0005] In order to solve the above problem, a first aspect of an embodiment of the present invention discloses a method for synchronizing content displayed on an electronic whiteboard, which is applied to a first visual network terminal. The method includes:

[0006] In the process of sharing the electronic whiteboard with the second visual network terminal, converting the display content of the electronic whiteboard into display data;

[0007] Encapsulating the display data into a first visual networking protocol packet;

[0008] Adding a header to the first visual networking protocol packet;

[0009] The first visual networking protocol packet carrying the packet header is sent to the second visual networking terminal through the visual networking transparent transmission channel between the second visual networking terminal and the second visual networking terminal.

[0010] Optionally, during the process of sharing the electronic whiteboard with the second visual networking terminal, a video conference or video call is simultaneously conducted with the second visual networking terminal; the method further includes:

[0011] The interface of the electronic whiteboard is displayed in the first display area of ​​the visual networking terminal, and the screen of the video conference is displayed in the second display area of ​​the visual networking terminal.

[0012] Optionally, the display content is a display picture, and the method further includes:

[0013] When detecting an editing instruction of the user for the displayed image, editing the displayed image according to the editing instruction;

[0014] Display the edited picture on the electronic whiteboard interface;

[0015] Obtaining editing data according to the final state of the edited image;

[0016] Encapsulating the edited data into a second visual networking protocol packet;

[0017] Adding a header to the second visual networking protocol packet according to the editing data;

[0018] The second visual networking protocol packet carrying the packet header is sent to the second visual networking terminal through the visual networking transparent transmission channel between the second visual networking terminal and the second visual networking terminal.

[0019] Optionally, the display content is a stroke editing record; adding a header to the visual networking protocol packet includes:

[0020] Obtaining the stroke type and timestamp of the stroke editing record;

[0021] A header is added to the visual networking protocol packet with the stroke type and timestamp.

[0022] Optionally, the display content is a display picture; encapsulating the display data into a first visual networking protocol packet includes:

[0023] According to the display picture, the display data is divided into N picture data;

[0024] Mark each image data with a coding identifier;

[0025] Encapsulating the N picture data into N first visual networking protocol packets;

[0026] Adding a header to the first visual networking protocol packet includes:

[0027] Determine the encoding identifier corresponding to the image data encapsulated in each of the first visual networking protocol packets;

[0028] Add a header to each of the first visual Internet protocol packets with the encoding identifier.

[0029] Optionally, the method further comprises:

[0030] Adding the first visual networking protocol packet to a sending queue so that the first visual networking protocol packet is arranged in sequence in the sending queue;

[0031] The first visual networking protocol packets arranged in sequence are sent to the second visual networking terminal in sequence through a transparent transmission channel between the first visual networking terminal and the second visual networking terminal.

[0032] A second aspect of an embodiment of the present invention discloses a method for synchronizing content displayed on an electronic whiteboard, which is applied to a second visual networking terminal. The method includes:

[0033] In the process of sharing the electronic whiteboard with the first visual networking terminal, receiving a first visual networking protocol packet carrying a packet header sent by the first visual networking terminal;

[0034] Extracting display data from the first visual networking protocol packet;

[0035] According to the packet header, restoring the display data to display content;

[0036] The display content is displayed on the electronic whiteboard interface.

[0037] Optionally, the display content is a stroke editing record; extracting the display data in the first visual networking protocol packet includes:

[0038] Extracting stroke data in the first visual networking protocol packet;

[0039] According to the packet header, restoring the display data to display content includes:

[0040] Obtaining the stroke type and timestamp in the packet header;

[0041] Restoring the stroke data to the stroke editing record according to the stroke type;

[0042] The stroke editing records are sequentially displayed on the electronic whiteboard interface according to the timestamp.

[0043] Optionally, the display content is a display picture; extracting the display data in the first visual networking protocol packet includes:

[0044] Extracting the image data in the first visual networking protocol packet;

[0045] According to the packet header, restoring the display data to display content includes:

[0046] Obtaining the coding identifier in the packet header;

[0047] Arrange the N picture data in sequence according to the coding identifier and splice them into the overall picture data;

[0048] The overall picture data is restored to a display picture.

[0049] A third aspect of an embodiment of the present invention discloses a device for synchronizing display content of an electronic whiteboard, which is applied to an operation terminal. The device includes:

[0050] A conversion module, used for converting the display content of the electronic whiteboard into display data during the process of sharing the electronic whiteboard with the second visual network terminal;

[0051] A first encapsulation module, used for encapsulating the display data into a first visual networking protocol packet;

[0052] A first adding module, used for adding a header to the first visual networking protocol packet;

[0053] The first sending module is used to send the first visual networking protocol packet carrying the packet header to the second visual networking terminal through the visual networking transparent transmission channel between the second visual networking terminal and the first visual networking terminal.

[0054] Optionally, the device further comprises:

[0055] The first display module is used to display the interface of the electronic whiteboard in the first display area of ​​the visual networking terminal and to display the screen of the video conference in the second display area of ​​the visual networking terminal.

[0056] Optionally, the device further comprises:

[0057] An editing module, configured to edit the displayed image according to the editing instruction when a user's editing instruction for the displayed image is detected;

[0058] A second display module, used for displaying the edited picture on the electronic whiteboard interface;

[0059] An acquisition module, used for acquiring editing data according to the final state of the edited image;

[0060] A second encapsulation module, used for encapsulating the edited data into a second visual networking protocol packet;

[0061] A second adding module, configured to add a header to the second visual networking protocol packet according to the editing data;

[0062] The second sending module is used to send the second visual networking protocol packet carrying the packet header to the second visual networking terminal through the visual networking transparent transmission channel between the second visual networking terminal and the second visual networking terminal.

[0063] Optionally, the first adding module includes:

[0064] Get a stroke type submodule, used to obtain the stroke type and timestamp of the stroke editing record;

[0065] The first submodule for adding a header adds a header to the visual networking protocol packet with the stroke type and timestamp.

[0066] Optionally, the first packaging module includes:

[0067] A data segmentation submodule, used for segmenting the display data into N picture data according to the display picture;

[0068] A marking submodule is used to mark each image data with a coding identifier;

[0069] Encapsulating the N picture data into N first visual networking protocol packets;

[0070] The first adding module comprises:

[0071] A coding identifier determination module, used to determine the coding identifier corresponding to each of the image data encapsulated in the first visual networking protocol packet;

[0072] The second packet header adding submodule is used to add a packet header to each of the first visual networking protocol packets with the coding identifier.

[0073] Optionally, the device further comprises:

[0074] A cache module, used for adding the first visual networking protocol packet to a sending queue, so that the first visual networking protocol packet is arranged in sequence in the sending queue;

[0075] A sequential sending module is used to send the first visual networking protocol packets arranged in sequence to the second visual networking terminal in sequence through a transparent transmission channel between the second visual networking terminal.

[0076] A fourth aspect of an embodiment of the present invention discloses a device for synchronizing display content of an electronic whiteboard, which is applied to a second visual network terminal, and the device includes:

[0077] A receiving module, used for receiving a first visual networking protocol packet carrying a packet header sent by the first visual networking terminal during the process of sharing an electronic whiteboard with the first visual networking terminal;

[0078] An extraction module, used to extract display data in the first visual networking protocol packet;

[0079] A restoration module, used for restoring the display data to display content according to the packet header;

[0080] The display module is used to display the display content on the electronic whiteboard interface.

[0081] Optionally, the extraction module includes:

[0082] A stroke data extraction submodule, used to extract the stroke data in the first visual networking protocol packet;

[0083] The restoration module comprises:

[0084] A stroke type extraction submodule is used to obtain the stroke type and timestamp in the packet header;

[0085] A first restoration submodule, used for restoring the stroke data to the stroke editing record according to the stroke type;

[0086] The display submodule sequentially displays the stroke editing records on the electronic whiteboard interface according to the timestamp.

[0087] Optionally, the extraction module includes:

[0088] A picture data extraction submodule, used to extract the picture data in the first visual networking protocol packet;

[0089] The restoration module comprises:

[0090] Extracting the coding identifier submodule, used to obtain the coding identifier in the packet header;

[0091] A splicing submodule, used to sequentially arrange the N picture data according to the coding identifier and splice them into the overall picture data;

[0092] The second restoration submodule is used to restore the entire image data into a display image.

[0093] A fifth aspect of an embodiment of the present invention provides a readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps in the method described in the first aspect or the second aspect of the present invention are implemented.

[0094] A sixth aspect of an embodiment of the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the method described in the first aspect or the second aspect of the present invention are implemented.

[0095] The embodiments of the present invention include the following advantages:

[0096] The embodiment of the present invention applies the characteristics of visual networking to enable a first visual networking terminal and a second visual networking terminal to share an electronic whiteboard during a video conference or a video call; the first visual networking terminal converts the display content received by the electronic whiteboard, such as stroke editing records, display pictures, etc., into binary data through a transparent transmission channel, and encapsulates it into a visual networking protocol packet for transmission to the second visual networking terminal, which is not affected by network conditions, thereby ensuring that the second visual networking terminal can synchronously display the display content of the electronic whiteboard of the first visual networking terminal.

[0097] When the displayed content is a stroke editing record, the first visual networking terminal adds a header of the visual networking protocol packet with a timestamp and stroke type, so that the second visual networking terminal can more accurately restore the stroke data to strokes according to the stroke type and display the strokes in order according to the timestamp.

[0098] When the display content is a picture, the first visual networking terminal divides the picture data into N picture data according to the picture attributes, and encapsulates the N picture data into N visual networking protocol packets, and adds a header to the N visual networking protocol packets according to the encoding identifier of the N picture data to ensure that the second visual networking terminal can accurately splice the N picture data into the overall picture data, and then accurately restore the picture data to a picture. BRIEF DESCRIPTION OF THE DRAWINGS

[0099] Figure 1 is a flowchart of steps for synchronizing the display content of an electronic whiteboard according to an embodiment of the present invention;

[0100] Figure 2 is a flowchart of steps for editing and synchronizing the display content of an electronic whiteboard according to an embodiment of the present invention;

[0101] Figure 3 is a flowchart of steps for synchronizing the electronic whiteboard display content in another embodiment of the present invention;

[0102] Figure 4 is a flow chart of a first visual networking terminal transmitting stroke data to a second visual networking terminal according to an embodiment of the present invention;

[0103] Figure 5 is a flow chart of a first visual networking terminal transmitting picture data to a second visual networking terminal according to an embodiment of the present invention;

[0104] Figure 6 is a flow chart of a first visual networking terminal transmitting edited data to a second visual networking terminal according to an embodiment of the present invention;

[0105] Figure 7 is a schematic diagram of the structure of an electronic whiteboard display content synchronization device in an embodiment of the present invention;

[0106] Figure 8 is a structural schematic diagram of an electronic whiteboard display content synchronization device in another embodiment of the present invention;

[0107] Fig. 9 It is a networking schematic diagram of a visual network of the present invention;

[0108] Fig.10 It is a schematic diagram of the hardware structure of a node server of the present invention;

[0109] Fig.11It is a schematic diagram of the hardware structure of an access switch of the present invention;

[0110] Fig.12 It is a hardware structure diagram of an Ethernet protocol conversion gateway of the present invention. DETAILED DESCRIPTION

[0111] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0112] Under the existing technology, multiple terminals mainly transmit data through the TCP / IP protocol to achieve electronic whiteboard sharing. The TCP / IP protocol is easily affected by the network status, resulting in delays in the electronic whiteboard data received by the terminal, resulting in a poor electronic whiteboard sharing experience. In addition, if the Internet is attacked by unsafe attacks, the electronic whiteboard data sent by the terminal will be at risk of leakage; and the terminal display interface cannot display other data (such as audio data, video data, etc.) when displaying the data shared by the electronic whiteboard.

[0113] In view of this, the present invention proposes a method for synchronizing the display content of an electronic whiteboard, which is applied to a first visual networking terminal so that the visual networking terminal can synchronize the display content of the electronic whiteboard.

[0114] The display content of the electronic whiteboard refers to the data displayed on the display interface of the visual network terminal according to the user's operating instructions. The display content may include: the stroke data, picture data, and the enlargement, reduction, deletion, rotation, etc. of the picture displayed by the electronic whiteboard according to the user's instructions.

[0115] The first visual network terminal and the second visual network terminal are Android terminals connected to the visual network.

[0116] The chairman terminal can initiate a video conference with a visual network. Other Android terminals that conduct a video conference with the chairman terminal are participating terminals. During the video conference with the participating terminals, the chairman terminal can initiate electronic whiteboard sharing. After the participating terminal responds to the electronic whiteboard sharing, the chairman terminal can edit the electronic whiteboard (draw strokes, select pictures for display, etc.). The participating terminal displays the chairman terminal's editing operations on the electronic whiteboard. At this time, the chairman terminal can operate the electronic whiteboard and is the first video terminal. The participating terminal only displays the chairman terminal's editing of the electronic whiteboard on the display interface and is the second video terminal.

[0117] Based on the same principle, the calling terminal can initiate a visual network video call with the called terminal. During the video call with the called terminal, the calling terminal can initiate electronic whiteboard sharing. After the called terminal responds to the electronic whiteboard sharing, the calling terminal can edit the electronic whiteboard. At this time, the calling terminal can operate the electronic whiteboard, which is the first visual network terminal; the called terminal only displays the editing of the electronic whiteboard by the calling terminal on the display interface, which is the second visual network terminal.

[0118] After the chairman terminal transfers the operation authority of the electronic whiteboard to the called terminal, the called terminal can operate the electronic whiteboard, and the called terminal can be regarded as the first visual network terminal; it can be understood that in a visual network video conference or a visual network video call, the Android terminal that can operate the shared electronic whiteboard and display the edited content of the electronic whiteboard is the first visual network terminal, and the Android terminal that can only display the edited content of the electronic whiteboard on the display interface is the second visual network terminal.

[0119] The first visual networking terminal and the second visual networking terminal can simultaneously display the video conference screen and the electronic whiteboard interface on the display interface, or simultaneously display the video call screen and the electronic whiteboard interface on the display interface.

[0120] During the process of sharing the electronic whiteboard with the second visual networking terminal, simultaneously conducting a video conference or a video call with the second visual networking terminal; the method further includes:

[0121] The interface of the electronic whiteboard is displayed in the first display area of ​​the visual networking terminal, and the screen of the video conference is displayed in the second display area of ​​the visual networking terminal.

[0122] The first display area refers to an area where a video conference or video call screen is displayed, and the second display area refers to an area where an electronic whiteboard interface is displayed.

[0123] The first display area and the second display area can coexist on the display screen of the Android terminal in any plane combination relationship. For example, the second display area is one-fourth the size of the first display area, or the first display area and the second display area are arranged horizontally in a 1:1 ratio, which is not limited in the embodiment of the present invention.

[0124] Taking editing the electronic whiteboard on the chairperson terminal, displaying the edited electronic whiteboard, and synchronizing the electronic whiteboard display content as an example, synchronizing the electronic whiteboard display content specifically includes the following contents:

[0125] refer to Figure 1 , Figure 1 It is a flow chart of the steps of synchronizing the electronic whiteboard display content according to an embodiment of the present invention.

[0126] Step S101: During the process of sharing the electronic whiteboard with the second Visual Networking Terminal, convert the display content of the electronic whiteboard into display data;

[0127] Step S102: Package the display data into the first Visual Networking Protocol Packet;

[0128] When the user draws a stroke on the electronic whiteboard of the chairman terminal, the stroke editing record of the user is displayed on the electronic whiteboard interface of the chairman terminal. The stroke editing record can be the component strokes of text or curves, straight lines, etc. that form a graphic.

[0129] The chairman terminal then converts the component strokes of text or curves, straight lines, etc. that form a graphic displayed on the electronic whiteboard interface into binary data.

[0130] Suppose the character "田" is displayed on the electronic whiteboard interface. The chairman terminal converts "丨", "", "￣", "丨" into binary data respectively, and packages the binary data into the Visual Networking Protocol Packet.

[0131] Step S103: Add a packet header to the first Visual Networking Protocol Packet;

[0132] The display content is the stroke editing record; adding a packet header to the Visual Networking Protocol Packet includes:

[0133] Obtain the stroke type and timestamp of the stroke editing record;

[0134] Use the stroke type and timestamp to add a packet header to the Visual Networking Protocol Packet.

[0135] The stroke type includes but is not limited to: curves, straight lines, broken lines, quadrilaterals, circles, triangles, etc. In addition to the line information of the stroke, the stroke information included in the stroke instruction can also be: the color, contrast, and thickness of the stroke, etc., as well as stroke types such as horizontal folds, vertical hooks, dots, etc.

[0136] The timestamp refers to the digital sequence generated by the time information when the electronic whiteboard of the chairman terminal receives the stroke.

[0137] The packet header of the Visual Networking Protocol Packet encapsulating the stroke data carries the stroke type and the time information of obtaining the stroke.

[0138] The timestamp of the first stroke editing record drawn by the user through the display screen of the chairman terminal is 01, and the timestamp of the second stroke editing record is 02. After the participating terminal receives the visual networking protocol packet carrying the header, it obtains the stroke type and time information about the strokes in the header. Specifically, the first stroke editing record is obtained as an ellipse with a timestamp of 01, and the second stroke editing record is obtained as a straight line with a timestamp of 02. According to the ellipse, the binary data in the visual networking protocol packet encapsulating the first stroke editing record is restored to the strokes of the ellipse and displayed on the electronic whiteboard interface; then according to the straight line, the binary data in the visual networking protocol packet encapsulating the second stroke editing record is restored to the strokes of the straight line and displayed on the electronic whiteboard interface.

[0139] The display content is a display picture; the display data is encapsulated into a first visual networking protocol packet, including:

[0140] According to the display picture, the display data is divided into N picture data;

[0141] Mark each image data with a coding identifier;

[0142] When the user selects a display image on the electronic whiteboard of the chairman terminal, the chairman terminal displays the target display image selected by the user on the interface of the electronic whiteboard and converts the target display image into binary image data.

[0143] In actual applications, the size of data that can be transmitted through the transparent channel is about 1K, so the number of final picture data is determined according to the size of the picture. For example, if the size of picture B is 520K, then the picture data needs to be divided into 520 picture data.

[0144] For a 520K picture, there is a unique identifier B. Combined with the pixels of the picture, each pixel is sequentially matched to a specific picture data in the N picture data. For the picture data corresponding to the pixel M, the corresponding coding identifier can be marked. The coding identifier can be the total number of picture data of the picture and the encoding of any picture data in the N picture data. The coding identifier of the picture data corresponding to the pixel M can be (M, 520), or the picture can be marked as a B picture, and the coding identifier of the pixel M is (M, B). According to the coding identifier, the position and order of the pixel points corresponding to the picture data in the picture can be determined.

[0145] Adding a header to the first visual networking protocol packet includes:

[0146] Determine the encoding identifier corresponding to the image data encapsulated in each of the first visual networking protocol packets;

[0147] Add a header to each of the first visual Internet protocol packets with the encoding identifier.

[0148] When encapsulating the image data corresponding to the pixel point M, the coding identifier (M, 520) or (M, B) is added to the packet header.

[0149] After receiving the visual networking protocol packet, the participating terminal obtains the coding identifier corresponding to the image data encapsulated in the visual networking protocol packet from the packet header, and according to the coding identifier, arranges the 520 image data of picture B in sequence and splices them into the overall image data. The spliced ​​overall image data is then restored to picture B and displayed on the local electronic whiteboard interface.

[0150] Step S104: sending the first visual networking protocol packet carrying the packet header to the second visual networking terminal through the visual networking transparent transmission channel between the second visual networking terminal and the second visual networking terminal.

[0151] The binary data converted according to the display content of the electronic whiteboard can be transmitted through the transparent channel, and the video data is transmitted through the video channel without affecting each other.

[0152] The user can use the chairman terminal or the calling terminal to zoom in, zoom out, move, rotate, delete, etc. the display image of the electronic whiteboard interface displayed in the first display area.

[0153] refer to Figure 2 , Figure 2 It is a flow chart of the steps of editing and synchronizing the display content of the electronic whiteboard according to an embodiment of the present invention.

[0154] Step S201: when a user's editing instruction for the displayed image is detected, the displayed image is edited according to the editing instruction;

[0155] Step S202: Displaying the edited picture on the electronic whiteboard interface;

[0156] Step S203: obtaining editing data according to the final state of the edited image;

[0157] For example, the coordinates of the four vertices of image B on the electronic whiteboard interface of the chairperson terminal are (1, 0), (2, 0), (1, 1), and (2, 1). The chairperson terminal receives a command from the user to enlarge image B by 2 times. The coordinates of the four vertices of the enlarged image B displayed on the electronic whiteboard interface are (0, -1), (3, -1), (0, 2), and (3, 2). The vertex coordinates of the enlarged image B are used as the editing data of the image and are transmitted to the participating terminals sharing the electronic whiteboard through the transparent transmission channel.

[0158] The final state of the edited image is represented by its final coordinates (0, -1), (3, -1), (0, 2), (3, 2).

[0159] Step S204: Encapsulating the edited data into a second visual networking protocol packet;

[0160] Step S205: adding a header to the second visual networking protocol packet according to the editing data;

[0161] Step S206: Send the second visual networking protocol packet carrying the packet header to the second visual networking terminal through the visual networking transparent transmission channel between the second visual networking terminal and the second visual networking terminal.

[0162] The image editing data may also be encapsulated as a visual networking protocol packet. The header of the visual networking protocol packet may be a coding identifier corresponding to the pixel point of the vertex of image B.

[0163] The first visual networking protocol packet encapsulates data for display content conversion, such as image data or stroke data; the second visual networking protocol packet encapsulates editing data for display content, for example: editing data.

[0164] After the participating terminal of the shared electronic whiteboard receives the second visual network protocol packet, it extracts the vertex coordinates (0, -1), (3, -1), (0, 2), and (3, 2) of the enlarged image B, and operates the image B displayed on the electronic whiteboard interface of the display terminal through the Google Android open source method, and transforms the four vertices (1, 0), (2, 0), (1, 1), and (2, 1) of image B into (0, -1), (3, -1), (0, 2), and (3, 2) respectively, thereby completing the enlargement of image B.

[0165] In another embodiment of the present invention, the chairman terminal can encapsulate the display data into a visual networking protocol packet, and then add the visual networking protocol packet with a packet header to a sending queue, and transmit it to the display terminal through a transparent transmission channel.

[0166] Adding the first visual networking protocol packet to a sending queue so that the first visual networking protocol packet is arranged in sequence in the sending queue;

[0167] The first visual networking protocol packets arranged in sequence are sent to the second visual networking terminal in sequence through a transparent transmission channel between the first visual networking terminal and the second visual networking terminal.

[0168] The sending queue is used to cache the visual networking protocol packets to be sent. If the chairman's terminal obtains a lot of display data, resulting in a large number of visual networking protocol packets encapsulating the display data, directly sending a large number of visual networking protocol packets through the transparent transmission protocol is prone to packet loss. Therefore, the visual networking protocol packets are cached in the sending queue. When there are a large number of visual networking protocol packets, the visual networking protocol packets are sent in sequence to ensure the complete transmission of the display data.

[0169] refer to Figure 3 , Figure 3 It is a flowchart of steps for synchronizing the electronic whiteboard display content in another embodiment of the present invention, and is applied to a second visual network terminal.

[0170] Step S301: in the process of sharing an electronic whiteboard with a first visual networking terminal, receiving a first visual networking protocol packet carrying a packet header sent by the first visual networking terminal;

[0171] Step S302: extracting display data in the first visual networking protocol packet;

[0172] Step S303: restoring the display data to display content according to the packet header;

[0173] Step S304: displaying the display content on the electronic whiteboard interface.

[0174] Taking the participating terminal of the shared electronic whiteboard as an example, the processing of display data by the second visual network terminal is explained.

[0175] When the displayed data is stroke data, the participating terminal can parse the packet header of the video network protocol packet to obtain the stroke type and timestamp.

[0176] And extract the stroke data in the visual network protocol package, and restore the stroke data according to the stroke type. For example, when the stroke type is an ellipse, restore the stroke data to an ellipse.

[0177] The stroke data in multiple visual network protocol packets are displayed in sequence on the interface of the local electronic whiteboard according to their respective timestamps.

[0178] The display content is a stroke editing record; extracting the display data in the first visual networking protocol packet, including:

[0179] Extracting stroke data in the first visual networking protocol packet;

[0180] According to the packet header, restoring the display data to display content includes:

[0181] Obtaining the stroke type and timestamp in the packet header;

[0182] Restoring the stroke data to the stroke editing record according to the stroke type;

[0183] The stroke editing records are sequentially displayed on the electronic whiteboard interface according to the timestamp.

[0184] When the display data is image data, the participating terminal can parse the packet header of the visual network protocol packet to obtain the encoding identifier of the image data.

[0185] The image data extracted from each visual network protocol packet are arranged in sequence according to the coding identifier, spliced ​​into the overall image data, and the overall image data is restored to the image and displayed on the interface of the local electronic whiteboard.

[0186] The display content is a display picture; extracting the display data in the first visual networking protocol packet includes:

[0187] Extracting the image data in the first visual networking protocol packet;

[0188] According to the packet header, restoring the display data to display content includes:

[0189] Obtaining the coding identifier in the packet header;

[0190] Arrange the N picture data in sequence according to the coding identifier and splice them into the overall picture data;

[0191] The overall picture data is restored to a display picture.

[0192] refer to Figure 4 , Figure 4 This is a flow chart of a first visual networking terminal transmitting stroke data to a second visual networking terminal according to an embodiment of the present invention.

[0193] A1: The electronic whiteboard interface of the first visual network terminal receives the stroke instruction and displays the stroke drawing record;

[0194] A2: The first visual networking terminal converts the stroke instruction into binary stroke data, and obtains the stroke type and timestamp according to the stroke type and the time when the electronic whiteboard interface receives the stroke instruction.

[0195] A3: The first visual networking terminal encapsulates the stroke data into a visual networking protocol packet, and then adds a header of the visual networking protocol packet according to the stroke type and timestamp.

[0196] A4: The operating terminal sends the visual network protocol packet with a packet header to the display terminal through the transparent transmission channel of the visual network.

[0197] A5: The second visual networking terminal receives the visual networking protocol packet with a packet header, and extracts the stroke type and timestamp in the packet header.

[0198] A6: The second visual network terminal obtains the stroke data.

[0199] A7: The second visual network terminal will restore the stroke data to the stroke editing record according to the stroke type and timestamp.

[0200] A8: The display terminal uploads the stroke editing record to the display screen.

[0201] refer to Figure 5 , Figure 5 This is a flow chart of a first visual networking terminal transmitting picture data to a second visual networking terminal according to an embodiment of the present invention.

[0202] B1: The first visual networking terminal will convert the displayed image into image data, divide it into N image data, and display it on the local electronic whiteboard interface.

[0203] B2: The first visual networking terminal encapsulates the N picture data into N visual networking protocol packets, and adds the coding identifier to the header of the visual networking protocol packet;

[0204] B3: The first visual networking terminal sends N visual networking protocol packets with packet headers to the second visual networking terminal.

[0205] B4: The second visual networking terminal receives the visual networking protocol packet with a header and extracts N picture data.

[0206] B5: The second visual networking terminal combines the N picture data into overall picture data according to the coding identifiers in the packet headers of the N visual networking protocol packets;

[0207] B6: The entire image data of the second visual network terminal is restored to an image.

[0208] refer to Figure 6 , is a flow chart of a first visual networking terminal transmitting edited data to a second visual networking terminal according to an embodiment of the present invention.

[0209] C1: The first visual network terminal performs operations such as zooming in, zooming out, moving, and rotating the displayed image on the electronic whiteboard interface.

[0210] C2: The first visual networking terminal obtains the coordinate data of the four vertices of the image after it is enlarged, reduced, moved, and rotated.

[0211] C3: The first visual networking terminal encapsulates the coordinate data into a visual networking protocol packet and sends it to the second visual networking terminal.

[0212] C4: The second visual networking terminal receives the visual networking protocol packet with a packet header, extracts data from it, and obtains the coordinate data of the image.

[0213] C5: The second vision network terminal uses the Google Android open source method and the coordinate data of the image to realize the operation of the image on the local electronic whiteboard interface.

[0214] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0215] refer to Figure 7 , Figure 7 It is a schematic diagram of the structure of the electronic whiteboard display content synchronization device in an embodiment of the present invention.

[0216] Applied to the first visual networking terminal, the device comprises:

[0217] 701: a conversion module, configured to convert the display content of the electronic whiteboard into display data during the process of sharing the electronic whiteboard with the second visual network terminal;

[0218] 702: a first encapsulation module, configured to encapsulate the display data into a first visual networking protocol packet;

[0219] 703: a first adding module, configured to add a header to the first visual networking protocol packet;

[0220] 704: A first sending module, configured to send a first visual networking protocol packet carrying a packet header to the second visual networking terminal via a visual networking transparent transmission channel between the first visual networking terminal and the second visual networking terminal.

[0221] exist Figure 7 On the basis of, optionally, the device further includes:

[0222] The first display module is used to display the interface of the electronic whiteboard in the first display area of ​​the visual networking terminal and to display the screen of the video conference in the second display area of ​​the visual networking terminal.

[0223] exist Figure 7 On the basis of, optionally, the device further includes:

[0224] An editing module, configured to edit the displayed image according to the editing instruction when a user's editing instruction for the displayed image is detected;

[0225] A second display module, used for displaying the edited picture on the electronic whiteboard interface;

[0226] An acquisition module, used for acquiring editing data according to the final state of the edited image;

[0227] A second encapsulation module, used for encapsulating the edited data into a second visual networking protocol packet;

[0228] A second adding module, configured to add a header to the second visual networking protocol packet according to the editing data;

[0229] The second sending module is used to send the second visual networking protocol packet carrying the packet header to the second visual networking terminal through the visual networking transparent transmission channel between the second visual networking terminal and the second visual networking terminal.

[0230] exist Figure 7 On the basis of, the first adding module comprises:

[0231] Get a stroke type submodule, used to obtain the stroke type and timestamp of the stroke editing record;

[0232] The first submodule for adding a header adds a header to the visual networking protocol packet with the stroke type and timestamp.

[0233] exist Figure 7 Based on this, the first encapsulation module comprises:

[0234] A data segmentation submodule, used for segmenting the display data into N picture data according to the display picture;

[0235] A marking submodule is used to mark each image data with a coding identifier;

[0236] Encapsulating the N picture data into N first visual networking protocol packets;

[0237] The first adding module comprises:

[0238] A coding identifier determination module, used to determine the coding identifier corresponding to each of the image data encapsulated in the first visual networking protocol packet;

[0239] The second packet header adding submodule is used to add a packet header to each of the first visual networking protocol packets with the coding identifier.

[0240] exist Figure 7 On the basis of

[0241] A cache module, used for adding the first visual networking protocol packet to a sending queue, so that the first visual networking protocol packet is arranged in sequence in the sending queue;

[0242] A sequential sending module is used to send the first visual networking protocol packets arranged in sequence to the second visual networking terminal in sequence through a transparent transmission channel between the second visual networking terminal.

[0243] refer to Figure 8 , Figure 8 It is a structural diagram of an electronic whiteboard display content synchronization device in another embodiment of the present invention.

[0244] Applied to a second visual networking terminal, the device comprises:

[0245] 801: A receiving module, configured to receive a first visual networking protocol packet carrying a packet header sent by a first visual networking terminal during the process of sharing an electronic whiteboard with the first visual networking terminal;

[0246] 802: an extraction module, configured to extract display data from the first visual networking protocol packet;

[0247] 803: a restoration module, configured to restore the display data to display content according to the packet header;

[0248] 804: a display module, used for displaying the display content on an electronic whiteboard interface.

[0249] exist Figure 8 Based on this, the extraction module includes:

[0250] A stroke data extraction submodule, used to extract the stroke data in the first visual networking protocol packet;

[0251] The restoration module comprises:

[0252] A stroke type extraction submodule is used to obtain the stroke type and timestamp in the packet header;

[0253] A first restoration submodule, used for restoring the stroke data to the stroke editing record according to the stroke type;

[0254] The display submodule sequentially displays the stroke editing records on the electronic whiteboard interface according to the timestamp.

[0255] exist Figure 8 Based on this, the extraction module includes:

[0256] A picture data extraction submodule, used to extract the picture data in the first visual networking protocol packet;

[0257] The restoration module comprises:

[0258] Extracting the coding identifier submodule, used to obtain the coding identifier in the packet header;

[0259] A splicing submodule, used to sequentially arrange N picture data according to the coding identifier and splice them into the whole picture data;

[0260] The second restoration submodule is used to restore the entire image data into a display image.

[0261] As for the embodiment of the electronic whiteboard picture synchronization device, since it is basically similar to the embodiment of the electronic whiteboard picture synchronization method, the description is relatively simple, and the relevant parts can refer to the partial description of the embodiment of the electronic whiteboard picture synchronization method.

[0262] An embodiment of the present invention further provides a readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the method of the above embodiment of the present invention are implemented.

[0263] An embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the method of the above embodiment of the present invention are implemented.

[0264] The visual Internet is an important milestone in the development of the Internet. It is a real-time network that can realize real-time transmission of high-definition video and push many Internet applications towards high-definition video and high-definition face-to-face communication.

[0265] The visual network adopts real-time high-definition video exchange technology. It can integrate the required services such as high-definition video conferencing, video monitoring, intelligent monitoring analysis, emergency command, digital radio and television, delayed TV, online teaching, live broadcast, VOD on demand, TV mail, personalized recording (PVR), intranet (self-operated) channels, intelligent video broadcast control, information release and dozens of other video, voice, picture, text, communication, data and other services on a single system platform, and realize high-definition quality video playback through TV or computer.

[0266] In order to enable those skilled in the art to better understand the embodiments of the present invention, the visual network is introduced as follows:

[0267] Some of the technologies used in the visual Internet of Things are as follows:

[0268] Network Technology

[0269] The network technology innovation of visual networking improves the traditional Ethernet to cope with the huge potential video traffic on the network. Different from simple network packet switching or network circuit switching, visual networking technology uses packet switching to meet streaming needs. Visual networking technology has the flexibility, simplicity and low price of packet switching, and the quality and security guarantee of circuit switching, realizing the seamless connection of the whole network switching virtual circuit and data format.

[0270] Switching Technology

[0271] Visual networking uses the two advantages of Ethernet: asynchronism and packet switching. It eliminates Ethernet defects under the premise of full compatibility, and has seamless end-to-end connection across the entire network, directly connects to user terminals, and directly carries IP data packets. User data does not require any format conversion within the entire network. Visual networking is a more advanced form of Ethernet and a real-time switching platform that can achieve large-scale real-time transmission of high-definition video across the entire network, which is currently unattainable on the Internet, and pushes many network video applications towards high definition and unification.

[0272] Server Technology

[0273] The server technology on the visual network and unified video platform is different from the traditional server. Its streaming media transmission is based on connection-oriented. Its data processing capability is independent of traffic and communication time. A single network layer can include signaling and data transmission. For voice and video services, the complexity of streaming media processing on the visual network and unified video platform is much simpler than data processing, and its efficiency is more than 100 times higher than that of traditional servers.

[0274] Storage Technology

[0275] The ultra-high-speed storage technology of the unified video platform adopts the most advanced real-time operating system to adapt to the media content with ultra-large capacity and ultra-large flow, mapping the program information in the server command to the specific hard disk space. The media content no longer passes through the server, but is directly delivered to the user terminal in an instant. The user generally waits for less than 0.2 seconds. The optimized sector distribution greatly reduces the mechanical movement of the hard disk head seeking, and the resource consumption accounts for only 20% of the same level of IP Internet, but it generates 3 times more concurrent traffic than the traditional hard disk array, and the overall efficiency is improved by more than 10 times.

[0276] Network Security Technology

[0277] The structural design of the visual network completely eradicates the network security issues that plague the Internet through separate licensing for each service and complete isolation of equipment and user data. Generally, anti-virus programs and firewalls are not required, and attacks by hackers and viruses are eliminated, providing users with a structurally worry-free and secure network.

[0278] Service Innovation Technology

[0279] The unified video platform integrates services and transmission. Whether it is a single user, a private network user or a network, it is just an automatic connection. User terminals, set-top boxes or PCs are directly connected to the unified video platform to obtain rich and colorful multimedia video services in various forms. The unified video platform uses a "recipe-style" table configuration mode to replace traditional complex application programming. Complex applications can be implemented with very little code, realizing "unlimited" new business innovation.

[0280] The networking of the visual network is as follows:

[0281] Visual networking is a centrally controlled network structure. The network can be a tree network, star network, ring network, etc., but on this basis, there needs to be a centralized control node in the network to control the entire network.

[0282] like Fig. 9 As shown in the figure, the visual network is divided into two parts: access network and metropolitan area network.

[0283] The equipment in the access network can be divided into three categories: node server, access switch, and terminal (including various set-top boxes, encoding boards, storage, etc.). The node server is connected to the access switch, and the access switch can be connected to multiple terminals and can be connected to Ethernet.

[0284] The node server is a node in the access network that performs centralized control functions and can control access switches and terminals. The node server can be directly connected to the access switch or directly connected to the terminal.

[0285] Similarly, the devices in the metropolitan area network can also be divided into three categories: metropolitan area servers, node switches, and node servers. The metropolitan area servers are connected to the node switches, and the node switches can be connected to multiple node servers.

[0286] The node server is a node server of the access network part, that is, the node server belongs to both the access network part and the metropolitan area network part.

[0287] The metropolitan area server is a node with centralized control function in the metropolitan area network, which can control the node switch and node server. The metropolitan area server can be directly connected to the node switch or the node server.

[0288] It can be seen from this that the entire visual network is a hierarchical and centrally controlled network structure, and the network controlled by the node server and the metropolitan area server can be various structures such as tree, star, and ring.

[0289] Figuratively speaking, the access network part can form a unified video platform (the part in the dotted circle), and multiple unified video platforms can form a visual network; each unified video platform can be interconnected through the metropolitan area and wide area visual networks.

[0290] Visual networking equipment classification

[0291] 1.1 The devices in the visual network of the embodiment of the present invention can be mainly divided into three categories: servers, switches (including Ethernet gateways), terminals (including various set-top boxes, encoding boards, storage devices, etc.). The visual network as a whole can be divided into a metropolitan area network (or a national network, a global network, etc.) and an access network.

[0292] 1.2 The equipment in the access network can be mainly divided into three categories: node server, access switch (including Ethernet gateway), terminal (including various set-top boxes, encoding boards, storage, etc.).

[0293] The specific hardware structure of each access network device is as follows:

[0294] Node server:

[0295] like Fig.10 As shown, it mainly includes a network interface module 1001, a switching engine module 1002, a CPU module 1003, and a disk array module 1004;

[0296] Among them, the packets coming from the network interface module 1001, the CPU module 1003, and the disk array module 1004 all enter the switching engine module 1002; the switching engine module 1002 performs an operation of checking the address table 1005 on the incoming packets, thereby obtaining the packet's guidance information; and according to the packet's guidance information, the packet is stored in the queue of the corresponding packet buffer 1006; if the queue of the packet buffer 1006 is close to full, it is discarded; the switching engine module 1002 polls all packet buffer queues, and forwards if the following conditions are met: 1) the port sending buffer is not full; 2) the queue packet counter is greater than zero. The disk array module 1004 mainly implements the control of the hard disk, including the initialization, reading and writing of the hard disk; the CPU module 1003 is mainly responsible for the protocol processing between the access switch and the terminal (not shown in the figure), the configuration of the address table 1005 (including the downstream protocol packet address table, the upstream protocol packet address table, and the data packet address table), and the configuration of the disk array module 1004.

[0297] Access switch: f

[0298] like Fig.11 As shown, it mainly includes a network interface module (a downlink network interface module 1101, an uplink network interface module 1102), a switching engine module 1103 and a CPU module 1104;

[0299] Among them, the packet (uplink data) coming from the downlink network interface module 1101 enters the packet detection module 1105; the packet detection module 1105 detects whether the destination address (DA), source address (SA), data packet type and packet length of the packet meet the requirements. If they meet the requirements, the corresponding stream identifier (stream-id) is allocated and enters the switching engine module 1103, otherwise it is discarded; the packet (downlink data) coming from the uplink network interface module 1102 enters the switching engine module 1103; the data packet coming from the CPU module 1104 enters the switching engine module 1103; the switching engine module 1103 processes the incoming The packet performs an operation of looking up the address table 1106 to obtain the packet's guidance information; if the packet entering the switching engine module 1103 is from the downstream network interface to the upstream network interface, the packet is stored in the queue of the corresponding packet buffer 1107 in combination with the stream identifier (stream-id); if the queue of the packet buffer 1107 is nearly full, it is discarded; if the packet entering the switching engine module 1103 is not from the downstream network interface to the upstream network interface, then according to the packet's guidance information, the data packet is stored in the queue of the corresponding packet buffer 1107; if the queue of the packet buffer 1107 is nearly full, it is discarded.

[0300] The switching engine module 1103 polls all packet buffer queues. In the embodiment of the present invention, there are two situations:

[0301] If the queue is from the downlink network interface to the uplink network interface, it is forwarded if the following conditions are met: 1) the port send buffer is not full; 2) the queue packet counter is greater than zero; 3) a token generated by the bit rate control module is obtained;

[0302] If the queue is not going from the downstream network interface to the upstream network interface, it is forwarded if the following conditions are met: 1) the port send buffer is not full; 2) the queue packet counter is greater than zero.

[0303] The code rate control module 1108 is configured by the CPU module 1104 to generate tokens for all packet buffer queues from the downstream network interface to the upstream network interface within a programmable interval to control the code rate of the upstream forwarding.

[0304] The CPU module 1004 is mainly responsible for protocol processing with the node server, configuration of the address table 1106 , and configuration of the bit rate control module 1108 .

[0305] Ethernet protocol conversion gateway :

[0306] like Fig.12As shown, it mainly includes a network interface module (a downlink network interface module 1201, an uplink network interface module 1202), a switching engine module 1203, a CPU module 1204, a packet detection module 1205, a bit rate control module 1208, an address table 1206, a packet buffer 1207, a MAC adding module 1209, and a MAC deleting module 1210.

[0307] The data packet from the downlink network interface module 1201 enters the packet detection module 905; the packet detection module 1205 detects whether the Ethernet MAC DA, Ethernet MAC SA, Ethernet length or frame type, visual network destination address DA, visual network source address SA, visual network data packet type and packet length of the data packet meet the requirements, and if they meet the requirements, the corresponding stream identifier (stream-id) is assigned; then, the MAC deletion module 1210 subtracts the MAC DA, MACSA, length or frametype (2byte) and enters the corresponding receiving buffer, otherwise it is discarded;

[0308] The downstream network interface module 1201 detects the sending buffer of the port. If there is a packet, it obtains the Ethernet MAC DA of the corresponding terminal based on the visual network destination address DA of the packet, adds the Ethernet MAC DA of the terminal, the MACSA of the Ethernet protocol conversion gateway, and the Ethernet length or frametype, and sends it.

[0309] The functions of other modules in the Ethernet protocol conversion gateway are similar to those of the access switch.

[0310] terminal:

[0311] It mainly includes network interface module, business processing module and CPU module; for example, the set-top box mainly includes network interface module, audio and video encoding and decoding engine module, CPU module; the encoding board mainly includes network interface module, audio and video encoding engine module, CPU module; the memory mainly includes network interface module, CPU module and disk array module.

[0312] 1.3 The equipment in the metropolitan area network can be divided into two categories: node server, node switch, and metropolitan area server. Among them, the node switch mainly includes network interface module, switching engine module and CPU module; the metropolitan area server mainly includes network interface module, switching engine module and CPU module.

[0313] 2. Definition of Visual Network Data Packet

[0314] 2.1 Access Network Data Packet Definition

[0315] The data packet of the access network mainly includes the following parts: destination address (DA), source address (SA), reserved bytes, payload (PDU), and CRC.

[0316] As shown in the following table, the data packet of the access network mainly includes the following parts:

[0317] DA SA Reserved Payload CRC

[0318] in:

[0319] The destination address (DA) consists of 8 bytes. The first byte indicates the type of data packet (such as various protocol packets, multicast data packets, unicast data packets, etc.), with a maximum of 256 possibilities. The second to sixth bytes are the metropolitan area network address, and the seventh and eighth bytes are the access network address.

[0320] The source address (SA) is also composed of 8 bytes and has the same definition as the destination address (DA).

[0321] The reserved bytes consist of 2 bytes;

[0322] The payload part has different lengths depending on the type of datagram. If it is a protocol packet, it is 64 bytes. If it is a single multicast data packet, it is 32+1024=1056 bytes. Of course, it is not limited to the above two types.

[0323] CRC consists of 4 bytes and its calculation method follows the standard Ethernet CRC algorithm.

[0324] 2.2 Definition of Metropolitan Area Network Data Packet

[0325] The topology of the metropolitan area network is a graph. There may be two or more connections between two devices, that is, there may be more than two connections between a node switch and a node server, a node switch and a node switch, and a node switch and a node server. However, the metropolitan area network address of a metropolitan area network device is unique. In order to accurately describe the connection relationship between metropolitan area network devices, a parameter: a label is introduced in the embodiment of the present invention to uniquely describe a metropolitan area network device.

[0326] The definition of labels in this manual is similar to the definition of labels in MPLS (Multi-Protocol Label Switch). Assuming that there are two connections between device A and device B, there are two labels for the data packet from device A to device B, and there are two labels for the data packet from device B to device A. Labels are divided into input labels and output labels. Assuming that the label of the data packet entering device A (input label) is 0x0000, the label of the data packet when leaving device A (output label) may become 0x0001. The access process of the metropolitan area network is a centralized access process, which means that the address allocation and label allocation of the metropolitan area network are dominated by the metropolitan area server, and the node switch and node server are only passively executed. This is different from the label allocation of MPLS, which is the result of mutual negotiation between the switch and the server.

[0327] As shown in the following table, the data packets of the metropolitan area network mainly include the following parts:

[0328] DA SA Reserved Label Payload CRC

[0329] That is, destination address (DA), source address (SA), reserved byte (Reserved), label, payload (PDU), CRC. The label format can refer to the following definition: the label is 32 bits, of which the upper 16 bits are reserved and only the lower 16 bits are used. It is located between the reserved bytes and payload of the data packet.

[0330] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0331] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0332] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0333] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0334] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0335] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0336] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0337] The above is a detailed introduction to the electronic whiteboard display content synchronization method, device, equipment and storage medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for synchronizing content displayed on an electronic whiteboard, characterized in that: Applied to a first visual networking terminal, the first visual networking terminal is used to initiate electronic whiteboard sharing to a second visual networking terminal, and perform an editing operation on the electronic whiteboard, wherein the editing operation is to select a picture for display; The second visual networking terminal is used to display the editing of the electronic whiteboard by the first visual networking terminal on the display interface; The method comprises: In the process of sharing the electronic whiteboard with the second visual network terminal, converting the display content of the electronic whiteboard into display data; Encapsulating the display data into a first visual networking protocol packet; Adding a header to the first visual networking protocol packet; Sending the first visual networking protocol packet carrying the packet header to the second visual networking terminal through the visual networking transparent transmission channel between the second visual networking terminal and the first visual networking protocol packet; The display content is a display picture, and the display data is encapsulated into a first visual networking protocol packet, including: According to the display picture, the display data is divided into N picture data; Mark each image data with a coding identifier; the coding identifier includes the total number of the image data and the encoding of the image data; the coding identifier is used to determine the position and order of the pixel points corresponding to the image data in the displayed image; Encapsulating the N picture data into N first visual networking protocol packets; Adding a header to the first visual networking protocol packet includes: Determine the encoding identifier corresponding to the image data encapsulated in each of the first visual networking protocol packets; Add a header to each of the first visual Internet protocol packets with the encoding identifier.

2. The method according to claim 1, characterized in that During the process of sharing the electronic whiteboard with the second visual networking terminal, simultaneously conducting a video conference or a video call with the second visual networking terminal; the method further includes: The interface of the electronic whiteboard is displayed in the first display area of ​​the visual networking terminal, and the screen of the video conference is displayed in the second display area of ​​the visual networking terminal.

3. The method according to claim 1, characterized in that The display content is a display picture, and the method further includes: When detecting an editing instruction of the user for the displayed image, editing the displayed image according to the editing instruction; Display the edited picture on the electronic whiteboard interface; Obtaining editing data according to the final state of the edited image; Encapsulating the edited data into a second visual networking protocol packet; Adding a header to the second visual networking protocol packet according to the editing data; The second visual networking protocol packet carrying the packet header is sent to the second visual networking terminal through the visual networking transparent transmission channel between the second visual networking terminal and the second visual networking terminal.

4. The method according to claim 1, characterized in that The method further comprises: Adding the first visual networking protocol packet to a sending queue so that the first visual networking protocol packet is arranged in sequence in the sending queue; The first visual networking protocol packets arranged in sequence are sent to the second visual networking terminal in sequence through a transparent transmission channel between the first visual networking terminal and the second visual networking terminal.

5. A method for synchronizing electronic whiteboard display content, characterized in that: Applied to a second visual networking terminal, the second visual networking terminal is used to display the editing of the electronic whiteboard by the first visual networking terminal on a display interface; the first visual networking terminal is used to initiate electronic whiteboard sharing to the second visual networking terminal, and perform an editing operation on the electronic whiteboard, wherein the editing operation is to select a picture for display; the method comprises: In the process of sharing the electronic whiteboard with the first visual networking terminal, receiving a first visual networking protocol packet carrying a packet header sent by the first visual networking terminal; Extracting display data from the first visual networking protocol packet; According to the packet header, restoring the display data to display content; Displaying the display content on the electronic whiteboard interface; The display content is a display picture; extracting the display data in the first visual networking protocol packet includes: Extracting the image data in the first visual networking protocol packet; According to the packet header, restoring the display data to display content includes: Obtaining a coding identifier in the packet header; the coding identifier includes the total number of the image data and the encoding of the image data; the coding identifier is used to determine the position and order of the pixel points corresponding to the image data in the display image; Arrange the N picture data in sequence according to the coding identifier and splice them into the overall picture data; The overall picture data is restored to a display picture.

6. A device for synchronizing the display content of an electronic whiteboard, characterized in that: Applied to a first visual networking terminal, the first visual networking terminal is used to initiate electronic whiteboard sharing to a second visual networking terminal, and perform an editing operation on the electronic whiteboard, wherein the editing operation is to select a picture for display; The second visual networking terminal is used to display the editing of the electronic whiteboard by the first visual networking terminal on the display interface; the device includes: A conversion module, used for converting the display content of the electronic whiteboard into display data during the process of sharing the electronic whiteboard with the second visual network terminal; A first encapsulation module, used for encapsulating the display data into a first visual networking protocol packet; A first adding module, used for adding a header to the first visual networking protocol packet; A first sending module, configured to send a first visual networking protocol packet carrying a packet header to the second visual networking terminal through a visual networking transparent transmission channel between the first visual networking terminal and the second visual networking terminal; The display content is a display picture; the first packaging module includes: A data segmentation submodule, used for segmenting the display data into N picture data according to the display picture; A marking submodule, used to mark each image data with a coding identifier; the coding identifier includes the total number of the image data and the encoding of the image data; the coding identifier is used to determine the position and order of the pixel points corresponding to the image data in the displayed image; Encapsulating the N picture data into N first visual networking protocol packets; The first adding module comprises: A coding identifier determination module, used to determine the coding identifier corresponding to each of the image data encapsulated in the first visual networking protocol packet; The second packet header adding submodule is used to add a packet header to each of the first visual networking protocol packets with the coding identifier.

7. A device for synchronizing the display content of an electronic whiteboard, characterized in that: Applied to a second visual networking terminal, the second visual networking terminal is used to display the editing of the electronic whiteboard by the first visual networking terminal on the display interface; the first visual networking terminal is used to initiate electronic whiteboard sharing to the second visual networking terminal, and perform an editing operation on the electronic whiteboard, wherein the editing operation is to select a picture for display; the device comprises: A receiving module, used for receiving a first visual networking protocol packet carrying a packet header sent by the first visual networking terminal during the process of sharing an electronic whiteboard with the first visual networking terminal; An extraction module, used to extract display data in the first visual networking protocol packet; A restoration module, used for restoring the display data to display content according to the packet header; A display module, used to display the display content on the electronic whiteboard interface; The display content is a display picture; the extraction module includes: A picture data extraction submodule, used to extract the picture data in the first visual networking protocol packet; The restoration module comprises: Extracting the coding identifier submodule, used to obtain the coding identifier in the packet header; the coding identifier includes the total number of the image data and the encoding of the image data; the coding identifier is used to determine the position and order of the pixel points corresponding to the image data in the display image; A splicing submodule, used to sequentially arrange N picture data according to the coding identifier and splice them into the whole picture data; The second restoration submodule is used to restore the entire image data into a display image.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 4 or claim 5 are implemented.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When executed, the processor implements the steps in the method according to any one of claims 1 to 4 or claim 5.

Images