Video conference picture marking and extracting method

Abstract

The invention relates to the technical field of video conferences, in particular to a video conference picture marking and extracting method, which comprises the following steps that: a marking party terminal triggers a conference extracting module to carry out image interception on a live video, acquires image identification information and image data, and sends the image identification information and the image data to a service server; the business server carries out decryption operation on the original image and sends the original image to the MINIO, and the MINIO returns a resource bucket address to the business server; the service server updates and stores the resource bucket address and the image identification information in a service database; the business database sends the theme information to a business server; the business server pushes the theme information to EMQX middleware; the terminal carries out subscription consumption on the EMQX middleware; and the terminal consumes the ProtoBuf flow, and carries out a deserialization operation on the ProtoBuf flow. The problem of low communication efficiency in the traditional video conference process is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of video conferencing, and in particular to a method for marking and annotating video conferencing screens. Background Art

[0002] As a tool for online communication, video conferencing breaks the constraints of space and time through equipment and the Internet. Compared with traditional offline meetings, video conferencing has both visual and real-time features, which greatly improves the efficiency of multi-location and multi-terminal conference communication and reduces the cost of corporate conference communication. Video is conducive to enriching the way people communicate with each other, saving energy and reducing environmental pollution.

[0003] At present, in the live broadcast process, the video conferencing system basically realizes the capture, storage, and playback and sharing of key images through video recording and desktop sharing. Typically, in order to share and playback the key images of the live conference such as action demonstration, the user has to intercept part of the video, manually send the conference message through screenshots, or share the desktop to let all participants see it. For example, at a medical surgery technology exchange meeting, the two parties have poor communication. Using traditional methods to solve the poor communication will result in the following problems:

[0004] 1. Reduce communication efficiency. The instantaneous picture may need to be expressed for the participants to understand, which is difficult for multiple parties to understand;

[0005] 2. Increase the cost burden. Local screen recordings and subsequent material screenshots need to be manually sorted and sent. There is no convenient sharing function, and videos generally consume a lot of disk space. Summary of the invention

[0006] Aiming at the deficiencies of the existing methods, the present invention solves the problem of low communication efficiency in the traditional video conference process.

[0007] The technical solution adopted by the present invention is: a video conference screen marking and summary method comprises the following steps:

[0008] Step 1: The marking party terminal triggers the conference summary module to capture the image of the live video, obtain image identification information and image data, and send the image identification information and image data to the business server;

[0009] As a preferred embodiment of the present invention, the conference summary module is used to share real-time images and record notes in real time.

[0010] As a preferred implementation manner of the present invention, the image identification information includes: conference number, marking party terminal name, marking party terminal IP and label.

[0011] As a preferred implementation of the present invention, the image data includes: an original image and a compressed image.

[0012] Step 2: The business server decrypts the original image and sends it to MINIO. MINIO returns the resource bucket address to the business server. The business server updates and stores the resource bucket address and image identification information in the business database. The business database sends the subject information to the business server.

[0013] As a preferred implementation mode of the present invention, the subject information includes: meeting information and time.

[0014] Step 3: The business server pushes the topic information to the EMQX middleware; the terminal subscribes to and consumes the EMQX middleware;

[0015] As a preferred implementation of the present invention, the EMQX middleware packages the number of messages, compression graph, and resource bucket address in the business database into ProtoBuf for terminal subscription and consumption.

[0016] Step 4: The terminal consumes the ProtoBuf stream and deserializes the ProtoBuf stream; the terminal displays thumbnails in real time and updates the number of messages;

[0017] As a preferred embodiment of the present invention, when the terminal views all thumbnails, it first queries the business database for all thumbnails of the current meeting through the https interface, and then the business database sends the thumbnail corresponding binding information to the terminal; the terminal displays all thumbnails.

[0018] As a preferred implementation of the present invention, the terminal clicks on the resource bucket address corresponding to the thumbnail, and the MINIO storage returns the original image.

[0019] As a preferred embodiment of the present invention, a video conference screen marking and annotating system includes: a memory for storing instructions executable by a processor; and a processor for executing the instructions to implement a video conference screen marking and annotating method.

[0020] As a preferred embodiment of the present invention, a computer readable medium stores a computer program code, and the computer program code implements a video conference screen marking and annotating method when executed by a processor.

[0021] Beneficial effects of the present invention:

[0022] 1. All participants have significantly improved meeting efficiency and reduced communication costs. Meeting minutes can be used as part of the meeting summary and provided to different end users for download, so that the meeting is recorded and the results are significant. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a flow chart of the video conference screen marking and annotating method of the present invention;

[0024] Figure 2 This is the process of viewing thumbnails by a terminal of the present invention;

[0025] Figure 3 This is the process of viewing the original image from a thumbnail of the present invention. DETAILED DESCRIPTION

[0026] The present invention is further described below in conjunction with the accompanying drawings and embodiments. This figure is a simplified schematic diagram, which only illustrates the basic structure of the present invention in a schematic manner, and therefore it only shows the components related to the present invention.

[0027] like Figure 1 As shown, a video conference screen marking and annotating method comprises the following steps:

[0028] Step 1: The marking party terminal triggers the conference summary module to capture the image of the live video, obtain image identification information and image data, and send the image identification information and image data to the business server;

[0029] The meeting summary module is used to share real-time images and record notes in real time, that is, during the meeting, video screen recording, real-time sharing, meeting summary functions, etc. can be performed.

[0030] Image identification information includes: meeting number, terminal name of the marking party, terminal IP of the marking party, and label; wherein, label is to mark several screenshots of a certain meeting; for example, if there are 10 screenshots of a meeting, then the meeting will have 10 labels, which can be sorted according to the time of the label.

[0031] Image data includes: original image and compressed image; original image refers to screenshot; during the transmission process from the marking party terminal to the business server, the original image needs to be encrypted to prevent image leakage; the original image is decrypted on the business server side;

[0032] Data is transmitted between the marking party terminal and the business server through the https port.

[0033] For example: Terminal A (the marking party terminal) and Terminal B (the application party terminal) conduct a remote video conference with the theme of "ERPC Standardized Operation Series Surgical Live Broadcast", and Terminal B broadcasts the surgical process live; at this time, the expert on the Terminal A side is confused about certain processing actions during the operation, so he clicks the meeting summary module on Terminal A several times to take screenshots and / or record notes of the doctor's operation screen on the Terminal B side.

[0034] Terminal A performs operations such as label sorting and image compression on the acquired real-time images; and then uploads the compressed images, original images, labels, current meeting information and other data to the business server via https.

[0035] Step 2: The business server decrypts the original image and sends it to the resource storage component (MINIO). The resource storage component returns the MINIO resource bucket address to the business server; the business server updates and stores the resource bucket address and image identification information in the business database; the business database sends the subject information to the business server;

[0036] The subject information includes: meeting information, time, etc.; the meeting information includes: meeting number.

[0037] That is, each original image has a unique MINIO resource bucket address;

[0038] For example, the business server will first verify the data, that is, check whether the transmitted data is valid; perform byte decryption and MINIO storage on the original image, and update the MINIO resource bucket address and conference information and store them in the business database.

[0039] Step 3: The business server pushes the topic information to the EMQX middleware; the terminal subscribes to and consumes the EMQX middleware;

[0040] EMQX middleware packages the information such as the number of messages, compression graph, resource bucket address, etc. in the business database into different ProtoBufs for consumption by the tagging terminal and the application terminal;

[0041] For example: the business database sends a message to the EMQX middleware, which includes the resource bucket address, thumbnail, label, etc., and is serialized into byte stream data through ProtoBuf.

[0042] Step 4: The terminal consumes the ProtoBuf stream and deserializes the ProtoBuf stream; the terminal displays thumbnails in real time and updates the number of messages;

[0043] The terminal can be a tagging terminal or an application terminal, that is, both the tagging terminal and the application terminal can consume the ProtoBuf stream, and the roles of the tagging terminal and the application terminal can be interchangeable.

[0044] The ProtoBuf stream includes sharing or not sharing. If it is not shared, the application terminal cannot obtain the ProtoBuf stream; the deserialization operation is to obtain the image identification information, original image address and other data corresponding to a certain meeting.

[0045] like Figure 2 When a terminal (marker terminal or application terminal) in a conference needs to view all thumbnails, it first queries the business database of the business server through the https interface for all thumbnail information of the current conference. Then, the business database sends the thumbnail corresponding binding information to the terminal; the terminal displays all thumbnails;

[0046] The binding information includes: tags and terminal information for generating thumbnails.

[0047] like Figure 3 ,The process of a terminal viewing the original image based on the thumbnail includes: clicking the resource bucket address corresponding to the thumbnail, and the MINIO storage returns the original image.

[0048] Constraints:

[0049] Considering the robustness and security of the system, this part of the function is managed as a key permission, and the frequency and number of marking are restricted; in addition, the screenshot of the terminal during marking is affected by the image quality of the video conference itself.

[0050] Based on the above ideal embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. A video conference screen marking and annotating method, characterized in that: The following steps are involved: Step 1: The marking party terminal triggers the conference summary module to capture the image of the live video, obtain image identification information and image data, and send the image identification information and image data to the business server; Step 2: The business server decrypts the original image and sends it to MINIO, which returns the resource bucket address to the business server. The business server updates and stores the resource bucket address and image identification information in the business database; the business database sends the subject information to the business server; Step 3: The business server pushes the topic information to the EMQX middleware; the terminal subscribes to and consumes the EMQX middleware; Step 4: The terminal consumes the ProtoBuf stream and deserializes the ProtoBuf stream; the terminal displays thumbnails in real time and updates the number of messages.

2. The video conference screen marking and annotation method according to claim 1, characterized in that: The conference summary module is used to share real-time images and record notes in real time.

3. The video conference screen marking and annotation method according to claim 1, characterized in that: The image identification information includes: conference number, marking party terminal name, marking party terminal IP and label.

4. The video conference screen marking and annotation method according to claim 1, characterized in that: Image data includes: original image and compressed image.

5. The video conference screen marking and annotation method according to claim 1, characterized in that: The subject information includes: meeting information and time.

6. The video conference screen marking and annotating method according to claim 1, characterized in that: The EMQX middleware packages the number of messages, compressed graphs, and resource bucket addresses in the business database into ProtoBuf for terminal subscription and consumption.

7. The video conference screen marking and annotating method according to claim 1, characterized in that: When the terminal views all thumbnails, it first queries the business database for all thumbnails of the current meeting through the https interface, and then the business database sends the thumbnail corresponding binding information to the terminal; the terminal displays all thumbnails.

8. The video conference screen marking and annotating method according to claim 1, characterized in that: The terminal clicks the resource bucket address corresponding to the thumbnail, and MINIO storage returns the original image.

9. A video conference screen marking and annotation system, characterized in that: include: a memory for storing instructions executable by a processor; A processor, used to execute instructions to implement the video conference screen marking and annotating method as described in any one of claims 1-8.

10. A computer readable medium storing computer program code, characterized in that: When the computer program code is executed by a processor, the computer program code implements the video conference screen marking and annotating method as described in any one of claims 1 to 8.

Images