Conference content display method, conference terminal and conference system

Abstract

The invention relates to a conference content display method, a conference terminal and a conference system, and the method comprises the steps: obtaining conference video information including writing picture content needing to be displayed, and transmitting the conference video information to a video preview region for display. And then identifying according to the video preview area to obtain a writing area, and transmitting the corresponding writing picture content in the writing area to a far end for display. According to the scheme, the writing area can be accurately positioned through further recognition in the video preview area, and the writing picture content corresponding to the writing area is transmitted to the far end for display, so that the display reliability of the writing picture content is improved.

Description

Technical Field

[0001] The present application relates to the field of image transmission technology, and is characterized by a conference content display method, a conference terminal and a conference system. Background Art

[0002] A remote conference refers to a meeting held across regions using modern communication methods. It is not restricted by time or region and can be held at any time, bringing great convenience to people's daily production and life.

[0003] In a teleconference, there is often a need to display the written content on a whiteboard or other writing area to remote users. At this time, a video image including the whiteboard or other writing area is generally acquired through a camera and transmitted to the remote end for display, but it is difficult to accurately locate the written content. Summary of the invention

[0004] Based on this, it is necessary to provide a conference content display method, a conference terminal and a conference system to accurately locate the content of the written screen and display it remotely to improve the display reliability of the written screen content.

[0005] In a first aspect, the present application provides a conference content display method, including: obtaining conference video information and displaying it in a video preview area; identifying a writing area based on the video preview area; transmitting the writing screen content corresponding to the writing area to a remote end for remote display of the writing screen content of the writing area.

[0006] The above solution obtains conference video information including the writing screen content to be displayed, and sends it to the video preview area for display. Then, the writing area is identified based on the video preview area, and the corresponding writing screen content in the writing area is transmitted to the remote end for display. This solution can accurately locate the writing area by further identifying it in the video preview area, and transmit the writing screen content corresponding to the writing area to the remote end for display, thereby improving the display reliability of the writing screen content.

[0007] In a second aspect, the present application provides a conference terminal, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the above-mentioned conference content display methods when executing the computer program.

[0008] In a third aspect, the present application provides a conference system, including an image acquisition device and the above-mentioned conference terminal, wherein the image acquisition device and the conference terminal are communicatively connected. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 This is a schematic diagram of an application scenario of a conference content display method in an embodiment of the present application;

[0010] Figure 2 This is a flowchart of a method for displaying conference content in one embodiment of the present application;

[0011] Figure 3 This is a flowchart of a conference content display method in another embodiment of the present application;

[0012] Figure 4 This is a flowchart of a conference content display method in another embodiment of the present application;

[0013] Figure 5 This is a flowchart of a conference content display method in another embodiment of the present application;

[0014] Figure 6 A schematic diagram of a normal display detection process in an embodiment of the present application;

[0015] Figure 7 This is a flowchart of a conference content display method in another embodiment of the present application;

[0016] Figure 8 This is a flowchart of a conference content display method in another embodiment of the present application;

[0017] Fig. 9 This is a flowchart of a conference content display method in another embodiment of the present application;

[0018] Fig.10 This is a flowchart of a conference content display method in another embodiment of the present application;

[0019] Fig.11 This is a flowchart of a conference content display method in another embodiment of the present application;

[0020] Fig.12 This is a flowchart of a conference content display method in another embodiment of the present application;

[0021] Fig.13 This is a schematic diagram of the structure of a conference content display device in one embodiment of the present application;

[0022] Fig.14 This is a schematic diagram of the internal structure of a conference terminal in one embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0024] The conference content display method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the image acquisition device 101 is communicatively connected with the conference terminal 103, and the image acquisition device 101 has a video acquisition function, and can adjust the direction according to the actual use needs to realize the video acquisition of the corresponding area, obtain the conference video information, and then send the conference video information to the conference terminal 103, and preview and display it in the video preview area corresponding to the conference terminal 103. Optionally, the conference content display method provided in the embodiment of the present application can also be applied to the conference terminal scenario with an image acquisition function, that is, the conference terminal is equipped with an image acquisition device such as a camera, and the conference terminal acquires the conference video information according to the camera configured by itself.

[0025] In the conference scene, the image acquisition device 101 can acquire the conference scene in real time, obtain the corresponding conference video information and send it to the conference terminal 103 for display. By adjusting the image acquisition device 101, the video screen including the writing screen content can be displayed in the video preview area. When the user needs to display the writing screen content to the remote end, he only needs to operate at the conference terminal 103 to identify the screen corresponding to the writing content from the video preview area, that is, the writing screen content, and finally transmit it to the remote end for display.

[0026] The specific type of conference terminal 103 is not limited to a single one, and can be, but not limited to, various personal computers, laptops, smart phones, tablet computers, etc., and can be selected according to actual needs. For example, in one embodiment, the conference terminal 103 includes a personal computer, and the personal computer is installed with a conference-related application. In actual application, by starting the application, the conference video information from the image acquisition device 101 is received and displayed in the video preview area corresponding to the application.

[0027] In one embodiment, Figure 2 As shown, a conference content display method is provided, which is applied to Figure 1 The conference terminal 103 in FIG. 1 is taken as an example to illustrate, including step 202, step 204 and step 206.

[0028] Step 202: Acquire conference video information and display it in the video preview area.

[0029] Specifically, the conference video information includes written screen content that needs to be displayed. The video preview area is an area where the video screen is pre-displayed on the conference terminal for users to preview on the conference terminal. There is no unique way to obtain conference video information. In one embodiment, the conference terminal may collect the information through its own camera. In another embodiment, the conference terminal may communicate with an image acquisition device, and after the image acquisition device starts collecting and obtains the conference video information, it transmits it to the conference terminal. After the conference terminal obtains the conference image information, it can display it in the preview area corresponding to the conference terminal, so that the user can make timely adjustments when the displayed image is inaccurate, to ensure the accuracy of the video image finally transmitted to the remote end.

[0030] It should be noted that the specific type of the image acquisition device is not limited, as long as it is a device capable of capturing video images, it can be used. For example, in a more detailed embodiment, the image acquisition device includes a camera.

[0031] It can be understood that the number of cameras and the number of video preview areas are not unique. In one embodiment, an image acquisition device may be set, and a video preview area may be configured for the conference terminal. After the conference video information is acquired by the image acquisition device, the corresponding video screen is displayed in the video preview area. In other embodiments, an image acquisition device may be set, and the conference terminal may be configured with two or more video preview areas, and the conference video information acquired by the image acquisition device may be displayed in the form of a video screen in two or more video preview areas. In another embodiment, two or more image acquisition devices may be set, and a video preview area may be configured for the conference terminal at the same time, and multiple conference video information may be displayed in the form of a video screen in one video preview area. In another embodiment, two or more image acquisition devices may be set, and a video preview area may be configured for each image acquisition device at the conference terminal, and the video screen corresponding to the conference video information of each image acquisition device may have a corresponding video preview area. The specific setting of the image acquisition device and the configuration of the video preview area may be selected in combination with actual needs.

[0032] There is not only one way to configure the video preview area of ​​a conference terminal. In a more detailed embodiment, the conference terminal is installed with an application related to the display of conference content. After the conference terminal starts the application, a corresponding preview area will be generated on the interface of the conference terminal. After the conference terminal receives the conference video information captured by the image acquisition device, it will be displayed in the preview area in the form of a video screen.

[0033] Step 204: Identify the writing area based on the video preview area.

[0034] Specifically, the writing area is the area where the written content is displayed. The specific type of the writing area is not unique. For example, in a more detailed embodiment, the conference content display method is applied in a conference scene, and the writing area includes a whiteboard area. In the scheme of this embodiment, the whiteboard area refers to the area corresponding to the writing screen. Since the writing operation is generally performed on the whiteboard, it is called the whiteboard area. It should be pointed out that in other embodiments, the writing operation can also be performed on a wall, a writable glass wall, etc. Correspondingly, the portion of the wall or the writable glass wall corresponding to the video preview area can also be called the writing area.

[0035] Step 206, transmitting the writing picture content corresponding to the writing area to a remote end, so as to display the writing picture content of the writing area at the remote end.

[0036] Specifically, the far end is also the remote device. The conference terminal is also connected to the remote device in communication. After the conference terminal obtains the conference video information, it not only displays the video screen corresponding to the conference video information in the video preview area, but also can transmit all the video screens or the video screens of a specific area to the remote device according to actual needs so that the user of the remote device can view it.

[0037] The specific type of the remote device is not unique, it can be a conference terminal device or a server, and when the video screen is transmitted to the remote device, it can be displayed at the remote device or stored at the remote device, and there is no specific limitation. When the remote device is a conference terminal device, the conference terminal device can be the same type as the conference terminal, for example, both are conference terminals.

[0038] The technical solution of the present application is specifically that the conference terminal transmits the writing screen content of a specific area in the video screen of the video preview area to the remote device for display. The conference terminal selects the writing area in the video preview area according to the conference content display requirements, that is, in the video screen displayed in the video preview area, after selecting the written writing screen content, the written writing screen content will be transmitted to the remote end for display. Similarly, taking the writing area as a whiteboard area as an example, through this solution, the writing content displayed in the whiteboard area can be transmitted to the remote end for display.

[0039] The above scheme acquires the conference video information including the writing picture content to be displayed through the image acquisition device, and sends it to the video preview area for display. Then, the writing area is identified based on the video preview area, and the corresponding writing picture content in the writing area is transmitted to the remote end for display. This scheme can accurately locate the writing area by further identifying it in the video preview area, and transmit the writing picture content corresponding to the writing area to the remote end for display, thereby improving the display reliability of the writing picture content.

[0040] In one embodiment, see Figure 3 , step 202 includes step 302 and step 304.

[0041] Step 302: Receive the conference video information collected by the image collection device and check whether the preview function is turned on. Step 304: If the preview function is turned on, display the conference video information in the video preview area.

[0042] Specifically, the preview function is a function that displays the conference video information in advance on its own display interface before transmitting it to the remote end. After receiving the video conference information, the conference terminal will choose whether to enable the preview function according to actual needs. If the preview function is enabled, the conference video information will be displayed in the video preview area. If the preview function is not enabled, the conference video information will be directly transmitted to the remote end for display.

[0043] It should be noted that the conference terminal has not the only way to enable the preview function. In one embodiment, the image acquisition device may recognize relevant gesture signals, generate a preview enable signal and send it to the conference terminal to control the conference terminal to enable the preview function.

[0044] Through the above scheme, when the conference terminal verifies that the preview function is turned on, the conference video information is sent to the video preview area for display, so that the writing screen content can be accurately located based on the preview screen content to improve the display accuracy of the writing screen content.

[0045] In one embodiment, see Figure 4 , after step 202, the method further includes step 402.

[0046] Step 402, checking whether there is a need to display written content.

[0047] If there is a demand for displaying written content, step 204 is executed.

[0048] Specifically, the demand for displaying written content is the demand for transmitting the written image of the writing area in the video screen of the video preview area to the remote end for display. During the startup and operation of the conference terminal, it will detect in real time whether the user has the demand to send the screen of the writing area to the remote end for display. When the conference terminal detects the demand for displaying written content, it will further identify the writing area in the video preview area, obtain the writing area including the writing screen content, and finally transmit the screen content corresponding to the area to the remote end for display.

[0049] It can be understood that in one embodiment, when there is no need to display conference content, the conference terminal can also transmit the entire video screen displayed in the video preview area to the remote device for display and / or storage according to actual conditions, so that the user of the remote device can promptly know the real-time status of the environment in which the conference terminal is located.

[0050] In another embodiment, when there is no need to display the conference content, the conference terminal does not send any video images to the remote end. At this time, the conference video information collected by the image acquisition device can be previewed and displayed in the video preview area of ​​the conference terminal, so that a timely response can be made when there is a subsequent display demand.

[0051] Optionally, in one embodiment, when the preview function is turned on at the conference terminal, it is assumed that there is a need to display written content. In this solution, after the conference video information is transmitted to the video preview area for display, the video preview area is identified to obtain the writing area, and the writing screen content corresponding to the writing area is transmitted to the remote end for display.

[0052] In the above scheme, after obtaining the conference video information and displaying it in the video preview area, the conference terminal will further verify whether there is a need to display written content at this time, and then choose whether to display the written content according to actual needs, further improving the reliability of the display of the written content.

[0053] In one embodiment, checking whether there is a need to display written content includes: checking whether a whiteboard function is turned on; if the whiteboard function is turned on, determining that there is a need to display written content.

[0054] Specifically, the whiteboard function is to display the written content on a whiteboard or other writing carrier in the conference scene. In the solution of this embodiment, when the whiteboard function is turned on in the conference terminal, it is assumed that the user has a need to display the written content at this time. At this time, the writing area will be further determined in the video preview area, so as to realize the display of the written content.

[0055] It can be understood that the way to enable the whiteboard function is not unique. Similar to the way to enable the preview function mentioned above, in one embodiment, the image acquisition device can recognize relevant gesture signals, generate a whiteboard function activation signal and send it to the conference terminal, thereby controlling the conference terminal to enable the preview function.

[0056] It should be pointed out that the conference terminal is not the only one in which to determine whether there is a need to display conference content. In another embodiment, it can also determine whether there is a need to display conference content by detecting whether a display instruction is received. If a display instruction is received, it indicates that there is a need to display conference content at this time.

[0057] In more detail, there is not only one way to send the display instruction. In one embodiment, the user may interact with the conference terminal and send the display instruction to the conference terminal. For example, the user clicks or touches a related button on the conference terminal to send the display instruction to the conference terminal.

[0058] In the above scheme, the conference terminal verifies whether there is a need to display written content at this time by detecting whether the whiteboard function is turned on. This has the advantages of simple verification method and high verification efficiency.

[0059] See also Figure 5 In one embodiment, after step 204 , the method further includes step 502 .

[0060] Step 502: If the writing image content corresponding to the writing area cannot be displayed normally at the remote end, output a writing area abnormality prompt message.

[0061] If the writing image content corresponding to the writing area can be normally displayed at the remote end, step 206 is executed.

[0062] Specifically, in the solution of this embodiment, after the conference terminal selects the writing area in the video preview area according to the conference content display requirements, it will also perform an operation of analyzing whether the writing screen content of the writing area can be normally displayed at the remote end. If the writing screen content cannot be normally displayed at the remote end at this time, there is no need to transmit it to the remote end. Correspondingly, the conference terminal will output a prompt message of abnormality in the selected area to remind the user to adjust the image acquisition device or end the display operation.

[0063] Specifically, the conference terminal outputs the selected area abnormal prompt information in a way that is not unique. In one embodiment, the conference terminal may output the selected area abnormal prompt information in the form of sound, light signal, text, or image to inform the user. For example, in a more detailed embodiment, if it is detected that the writing picture content of the writing area cannot be displayed normally at the remote end, the selected area abnormal prompt information of the text type is output in the video preview area to inform the user. Correspondingly, if it is detected that the writing picture content can be displayed normally at the remote end, the operation of transmitting the writing picture content corresponding to the writing area to the remote end for display will continue at this time, so that the user at the remote end can view the writing picture content of the writing area. Through the above scheme, it can ensure that the writing picture content of the remote end that is finally transmitted is displayed normally, thereby improving the operational reliability of the conference content display method.

[0064] See also Figure 6 In one embodiment, before step 502, the method further includes steps 602, 604 and 606.

[0065] Step 602, obtaining the distance between the image acquisition device and the writing carrier;

[0066] Step 604, if the distance is greater than or equal to the preset distance, it is determined that the writing picture content corresponding to the writing area cannot be normally displayed at the remote end;

[0067] Step 606: If the distance is less than the preset distance, it is determined that the writing picture content corresponding to the writing area can be normally displayed at the remote end.

[0068] Specifically, there is not only one way for the conference terminal to determine whether the writing screen content of the writing area can be displayed remotely. In one embodiment, it can be achieved by detecting the resolution of the screen corresponding to the writing area, or detecting whether the writing screen content of the writing area is missing, etc., without specific limitation.

[0069] In the scheme of this embodiment, the writing area includes a whiteboard area, and the whiteboard area must include a writing carrier for writing, such as a whiteboard, a wall, or a writable glass wall, etc., without specific limitation. In addition to collecting conference video information, the image acquisition device (taking a camera as an example) can also collect the distance between the writing carrier and the camera, and then send it to the conference terminal. The conference terminal compares and analyzes the received distance with the preset distance. If the received distance is greater than or equal to the preset distance, it means that the distance between the camera and the writing carrier is too large at this time, the clarity of the collected video image is limited, and the video picture cannot be displayed normally at the remote end. If the distance is less than the preset distance, it means that the distance between the camera and the writing carrier is appropriate at this time, which is sufficient to ensure that the camera captures a clear video picture. At this time, it is considered that the writing picture content corresponding to the writing area can be displayed normally at the remote end.

[0070] The above scheme analyzes whether the content of the writing picture can be normally displayed at the remote end by detecting the distance between the camera and the writing carrier. The detection method is simple and has high detection efficiency.

[0071] See also Figure 7 In one embodiment, after step 204 , the method further includes step 702 .

[0072] Step 702: store the writing screen content corresponding to the writing area according to the received saving instruction.

[0073] Specifically, the save instruction is an instruction to save the writing screen content of the writing area. After obtaining the writing screen content of the writing area, the conference terminal can also save the writing screen content according to actual needs for subsequent reference, thereby improving the operational reliability of the conference terminal.

[0074] It should be pointed out that there is not only one way to send a save instruction. In one embodiment, a user may interact with a conference terminal and send a save instruction to the conference terminal by touching, pressing, or clicking. After receiving the save instruction, the conference terminal saves the writing screen content in the writing area to the local storage of the conference terminal, or transmits it to the cloud for storage.

[0075] In another embodiment, the image acquisition device may also have an AI (Artificial Intelligence) recognition function. When the user sends a relevant gesture instruction to save, the image acquisition device recognizes it and notifies the conference terminal of this status through the image acquisition device. The conference terminal then saves the written screen content to the local memory of the conference terminal or transmits it to the cloud for storage.

[0076] See also Figure 8 In one embodiment, step 204 includes step 802 and step 804.

[0077] Step 802, identifying a pre-selected area in the video preview area according to a preset selection box;

[0078] Step 804, adjust the pre-selected area according to the adjustment instruction to determine the writing area.

[0079] Specifically, the preset selection box is a preset selection box used to select a pre-selected area from the video preview area. Its shape can be a rectangle, a circle or other polygon, or an irregular shape, which is not specifically limited. In the solution of this embodiment, a preset selection box is set in the video preview area of ​​the conference terminal. When there is a need to display conference content, the preset selection box will be displayed at the preset position of the video preview area, and the writing screen content to be displayed will be pre-selected. After that, the user determines whether the writing screen content selected by the preset selection box meets the display requirements. If it does, the writing screen content selected by the preset selection box is directly used as the writing screen content to be displayed, and transmitted to the remote end for display. If it does not meet the requirements, the conference terminal needs to adjust the preset selection box according to the adjustment instruction, so as to frame the writing screen content to be displayed, and transmit it to the remote end for display.

[0080] The above scheme selects the writing area by setting a preset selection box in the video preview area, that is, determines the writing screen content to be displayed, and has high processing efficiency.

[0081] It can be understood that the conference terminal is not the only way to select the writing area in the video preview area. In another embodiment, the application configured by the conference terminal may have an algorithm recognition function. When a need to display conference content is detected, the conference terminal starts the algorithm recognition function to identify the whiteboard (wall or writable glass wall) in the video preview area, thereby generating one or more pre-selected areas. The user then interacts with the conference terminal, selects one of the pre-selected areas for adjustment, and finally determines the writing area.

[0082] See also Fig. 9 In one embodiment, step 204 includes step 902 , step 904 , and step 906 .

[0083] Step 902: two or more preset areas are identified in the video preview area according to the preset selection box.

[0084] Step 904: identifying a pre-selected area from each preset area according to the selection instruction.

[0085] Step 906, adjusting the pre-selected area according to the adjustment instruction to determine the writing area.

[0086] Specifically, similar to the solution of the above-mentioned embodiment, when there is a need to display conference content, the video preview area can select a pre-selected area according to the preset selection box. The difference is that the number of preset selection boxes in the solution of this embodiment is not unique. Two or more preset selection boxes can be set. Each preset selection box may not overlap with each other, or may partially overlap. It can be set according to actual needs. Correspondingly, after the conference terminal displays each preset selection box in the video preview area, two or more preset areas will be generated. Then, the user selects one of the preset areas as the pre-selected area according to actual needs, and adjusts the pre-selected area (in combination with demand selection, it may not be adjusted in other embodiments), and then the writing area can be obtained.

[0087] It can be understood that, similar to the save instruction in the above embodiment, the selection instruction can also be sent by the user interacting with the conference terminal, or by the image recognition device recognizing the user's gesture signal and transmitting it to the conference terminal, without specific limitation.

[0088] The above scheme sets two or more preset selection boxes in the video preview area, and can select pre-selected areas at different positions in the video preview area. When the user needs to switch the content of the written screen to be displayed, he only needs to switch the preset selection box, select the corresponding pre-selected area and then make adjustments, thereby improving the convenience of switching the conference content display.

[0089] See also Fig.10, in one embodiment, step 204 includes step 1002.

[0090] Step 1002: identifying a writing area from the video preview area according to the selected instruction.

[0091] Specifically, in the solution of this embodiment, when there is a need to display conference content in the conference terminal, the writing area is selected in the video preview area by obtaining a selection instruction, thereby determining the writing screen content that needs to be displayed. Taking the conference terminal as a personal computer as an example, after the video screen is displayed in the video preview area of ​​the conference terminal, the user can send a selection instruction to the conference terminal through the mouse, and use the mouse to select the writing area, so that the writing screen content of the writing area is transmitted to the remote end for display. This solution determines the writing area by receiving the selection instruction, ensures that the selected writing screen content meets the user's needs, and improves the accuracy of conference content display.

[0092] It can be understood that in one embodiment, the preset selection box involved in the above embodiment can be configured by the conference terminal when it leaves the factory. For example, the central area or corner area of ​​the video preview area is used as the pre-selected area, and a preset selection box of a rectangle or other shape is configured for it. In another embodiment, the preset selection box can also be set by the user after leaving the factory. For example, when the conference terminal uses the conference content display related function for the first time, it receives the selection instruction sent by the user and selects the writing area from the video preview area. Afterwards, the conference terminal will store the selection box selected by the selection instruction as the preset selection box to obtain the preset selection box.

[0093] See also Fig.11 In one embodiment, step 204 includes step 112 and step 114 .

[0094] Step 112, identifying the writing area when the writing content was displayed last time from the video preview area, and using it as the pre-selected area for displaying the current writing content.

[0095] Step 114, adjusting the pre-selected area according to the adjustment instruction to determine the writing area.

[0096] Specifically, in the same conference scene, the positions of the conference terminal, camera, and writing carrier are relatively fixed. Therefore, in the solution of this embodiment, the conference terminal directly uses the writing area obtained after identification and adjustment when the writing content was displayed last time as the pre-selected area required for the current writing content display. After that, adjustments are made based on the determined pre-selected area to accurately locate the writing area in the current state, thereby realizing the display of the writing screen content.

[0097] This solution directly uses the writing area when the written content was displayed last time as the pre-selected area, and adjusts the current writing area on this basis, thereby improving the display efficiency to a certain extent.

[0098] See also Fig.12 , in one embodiment, step 206 includes step 122.

[0099] Step 122, performing homography transformation correction on the writing picture content of the writing area and then transmitting it to a remote end for remote display of the writing picture content of the writing area.

[0100] Specifically, homography (transmission transformation) is a concept in projective geometry, also known as projective transformation. It can map a point on a projective plane to the corresponding position on another plane, and map a straight line to a straight line, with the property of line preservation. Unlike epipolar geometry, which maps points to lines, the homography matrix is ​​a point-to-point relationship. Its description equation is: x1=Hx2, where the matrix H is a 3x3 non-singular matrix, and the matrix H multiplied by a non-zero constant still holds true.

[0101]

[0102]

[0103] Assuming that the homography matrix between the two images has been obtained, we can get:

[0104]

[0105] Among them, the left side is the point in image 1, and the right side is the point in image 2. From the above formula, we can get:

[0106]

[0107]

[0108] The equation is multiplied by a coefficient both above and below, and the homography matrix remains unchanged:

[0109]

[0110]

[0111] Can be set to h 33 =1, then:

[0112]

[0113]

[0114] Rearranging the above formula, we can get:

[0115] h 11 x+h 12 y+h 13 -h 31 xx′-h 32 yx′=x′

[0116] h 21 x+h 22 y+h 23 -h 31 xy′-h 32 yy′=y′

[0117] If we get the corresponding N point pairs (feature point matching pairs) in the two pictures, we can get the following linear equations:

[0118]

[0119] It can be seen from the above that a pair of matching points can obtain two constraints. Therefore, in the scheme of this embodiment, the writing picture content of the writing area can be corrected to an image of a specific size by detecting or manually selecting the four corner points of the writing area through the homography matrix for perspective transformation.

[0120] In the above scheme, when transmitting the writing picture content to the remote end, the homography matrix is ​​used for perspective transformation to ensure that the writing picture content displayed by the conference terminal is consistent with the writing picture content of the writing area, thereby improving the display accuracy.

[0121] See also Fig.13 In one embodiment, step 206 includes step 132 and step 134 .

[0122] Step 132, obtaining the remote image display mode.

[0123] Step 134 , transmitting the writing picture content corresponding to the writing area to the remote end according to the remote picture display mode, so as to display the writing picture content of the writing area remotely.

[0124] Specifically, the remote screen display mode is the form of remote screen display, including picture-in-picture mode, all main screen mode and all selected screen mode. The picture-in-picture mode is to overlap the writing screen content of the writing area and the writing screen content of the video preview area on the same display interface; the all main screen mode is that the remote display interface only displays the overall video screen of the video preview area; the all selected screen mode is that the remote display interface only displays the writing screen content corresponding to the selected area. In the scheme of this embodiment, when displaying the conference content, the screen layout can be customized for the transmission screen to make the display of the writing screen content more in line with user needs.

[0125] In one of the embodiments, if there is a need to display conference content, the conference content display method further includes: sending a shutdown instruction to the image acquisition device to turn off the artificial intelligence acquisition function of the image acquisition device.

[0126] Specifically, in the solution of this embodiment, the image acquisition device has an artificial intelligence acquisition function, such as a camera with an AI recognition function. After the conference terminal detects the need to display the conference content, that is, enters the conference content display state, the conference terminal will send a shutdown instruction to the image acquisition device connected to it, and turn off the AI ​​acquisition function of the image acquisition device, thereby preventing the image acquisition device from changing the video screen due to changes in the AI ​​acquisition function, thereby affecting the real-time writing screen content of the writing area.

[0127] In order to facilitate understanding of the technical solution of the present application, the present application is explained below using a conference scenario as an example in combination with a more detailed implementation method.

[0128] The camera collects the conference scene in real time to obtain the conference video information. If the conference terminal opens the preview function according to the gesture command sent by the user, the conference video information is transmitted to the conference terminal (a personal computer with conference-related applications installed), and the video image is displayed in the video preview area of ​​the conference terminal to obtain the video image. At this time, if the camera does not collect the gesture command related to the conference content display requirements, that is, it does not enter the whiteboard mode, the conference terminal chooses to transmit the video image to the remote end for display according to actual needs, or only previews the video image in the video preview area. If the camera collects the gesture command related to the conference content display requirements, it informs the conference terminal, and then the conference terminal selects two or more preset areas in the video preview area according to the preset selection box. The user uses a tool such as a mouse to select a preset area corresponding to the whiteboard (or a preset area closer to the whiteboard, etc.) as the pre-selected area in two or more preset areas, and then uses a tool such as a mouse to adjust the pre-selected area (including but not limited to dragging, shrinking, etc.) to determine the writing area.

[0129] After that, if the camera detects the need to save relevant gesture instructions, it will feed back the demand to the conference terminal, and the conference terminal will store the writing screen content of the writing area in the form of pictures, etc. locally or in the cloud. The conference terminal obtains the distance between the whiteboard and the camera collected by the camera, and compares it with the preset distance. If the distance is greater than or equal to the preset distance, it is considered that the writing screen content corresponding to the writing area (that is, the writing content of the whiteboard) cannot be displayed normally at the remote end. At this time, a prompt message will be output in the video preview area to inform the user. If the distance is less than the preset distance, it is considered that the writing screen content corresponding to the writing area can be displayed normally at the remote end, and the conference terminal will send a shutdown command to the camera to turn off the camera's AI acquisition function.

[0130] In whiteboard mode, the conference terminal obtains the remote screen display mode according to the instructions sent by the user, or uses the default remote screen display mode (such as picture-in-picture mode, with the whiteboard screen as a large window and the overall screen of the video preview area as a small window), and performs homography transformation correction on the writing screen content corresponding to the writing area before displaying it remotely.

[0131] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0132] In one embodiment, a conference terminal is provided, whose internal structure diagram can be as follows: Fig.14 As shown. The conference terminal includes a processor, a memory and a network interface connected via a system bus. The processor of the conference terminal is used to provide computing and control capabilities. The memory of the conference terminal includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the conference terminal is used to store video data. The network interface of the conference terminal is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a content display method is implemented.

[0133] Those skilled in the art will understand that Fig.14 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0134] In one embodiment, a conference terminal is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0135] Acquire conference video information and display it in the video preview area; identify the writing area based on the video preview area; transmit the writing screen content corresponding to the writing area to the remote end for remote display of the writing screen content of the writing area.

[0136] In one embodiment, when the processor executes the computer program, the following steps are also implemented: obtaining conference video information and verifying whether the preview function is enabled; if the preview function is enabled, displaying the conference video information in the video preview area.

[0137] In one embodiment, when the processor executes the computer program, the following steps are also implemented: checking whether there is a need to display written content; if there is a need to display written content, performing an operation of identifying the writing area based on the video preview area.

[0138] In one embodiment, when the processor executes the computer program, the following steps are also implemented: checking whether the whiteboard function is turned on; if the whiteboard function is turned on, determining whether there is a need to display written content.

[0139] In one embodiment, when the processor executes the computer program, the following steps are also implemented: if the writing picture content corresponding to the writing area cannot be displayed normally at the remote end, output a writing area abnormal prompt message; if the writing picture content corresponding to the writing area can be displayed normally at the remote end, execute the step of transmitting the writing picture content corresponding to the writing area to the remote end for remote display of the writing picture content of the writing area.

[0140] In one embodiment, when the processor executes the computer program, the following steps are also implemented: obtaining the distance between the image acquisition device and the writing carrier; if the distance is greater than or equal to a preset distance, determining that the writing picture content corresponding to the writing area cannot be displayed normally at the remote end; if the distance is less than the preset distance, determining that the writing picture content corresponding to the writing area can be displayed normally at the remote end.

[0141] In one embodiment, when the processor executes the computer program, the processor further implements the following steps: according to the received saving instruction, storing the writing picture content corresponding to the writing area.

[0142] In one embodiment, when the processor executes the computer program, the following steps are also implemented: a pre-selected area is identified in the video preview area according to a preset selection box; and the pre-selected area is adjusted according to the adjustment instruction to determine the writing area.

[0143] In one embodiment, when the processor executes the computer program, the following steps are also implemented: two or more preset areas are identified in the video preview area according to the preset selection box; a pre-selected area is identified from each preset area according to the selection instruction; and the pre-selected area is adjusted according to the adjustment instruction to determine the writing area.

[0144] In one embodiment, when the processor executes the computer program, the following steps are further implemented: identifying a writing area from the video preview area according to the selected instruction.

[0145] In one embodiment, when the processor executes the computer program, the following steps are also implemented: the writing area when the writing content was last displayed is identified from the video preview area as the preselected area for the current writing content display; the preselected area is adjusted according to the adjustment instruction to determine the writing area.

[0146] In one embodiment, when the processor executes the computer program, the following steps are also implemented: performing homography transformation correction on the writing picture content of the writing area and then transmitting it to a remote end for remote display of the writing picture content of the writing area.

[0147] In one embodiment, when the processor executes the computer program, the following steps are also implemented: obtaining the remote screen display mode; transmitting the writing screen content corresponding to the writing area to the remote end according to the remote screen display mode, for remote display of the writing screen content of the writing area.

[0148] In one embodiment, when the processor executes the computer program, the processor further implements the following steps: sending a shutdown instruction to the image acquisition device to shut down the artificial intelligence acquisition function of the image acquisition device.

[0149] In one embodiment, please refer to Figure 1 The present application also provides a conference system, which includes an image acquisition device 101 and the above-mentioned conference terminal (ie Figure 1 The terminal 103 shown is specifically a conference terminal), the image acquisition device 101 is in communication connection with the conference terminal, and the conference terminal is used to implement the steps of any of the above-mentioned content display methods. Among them, the specific type of the image acquisition device 101 is not unique. In a more detailed embodiment, the image acquisition device 101 can be a camera. Furthermore, the image acquisition device 101 is a camera with AI acquisition function.

[0150] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0151] Acquire conference video information and display it in the video preview area; identify the writing area based on the video preview area; transmit the writing screen content corresponding to the writing area to the remote end for remote display of the writing screen content of the writing area.

[0152] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0153] Acquire conference video information and display it in the video preview area; identify the writing area based on the video preview area; transmit the writing screen content corresponding to the writing area to the remote end for remote display of the writing screen content of the writing area.

[0154] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0155] The above-mentioned conference terminal, conference system, computer-readable storage medium and computer program product obtain conference video information including the writing screen content to be displayed, and send it to the video preview area for display. Then, the writing area is identified based on the video preview area, and the corresponding writing screen content in the writing area is transmitted to the remote end for display. This solution can accurately locate the writing area by further identifying it in the video preview area, and transmit the writing screen content corresponding to the writing area to the remote end for display, thereby improving the display reliability of the writing screen content.

[0156] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0157] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the present application. It should be noted that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A conference content display method, characterized in that: include: Get the conference video information and display it in the video preview area; According to the video preview area, identifying a writing area; The writing picture content corresponding to the writing area is transmitted to a remote end for remotely displaying the writing picture content of the writing area.

2. The conference content display method according to claim 1, characterized in that: The obtaining of conference video information and displaying it in the video preview area includes: Get the conference video information and verify whether the preview function is enabled; If the preview function is turned on, the conference video information will be displayed in the video preview area.

3. The conference content display method according to claim 1, characterized in that: After obtaining the conference video information and displaying it in the video preview area, the method further includes: Check whether there is a need to display written content; If there is a need to display written content, the step of identifying the writing area based on the video preview area is performed.

4. The conference content display method according to claim 3, characterized in that: The verification of whether there is a need to display written content includes: Verify whether the whiteboard function is enabled; If you turn on the whiteboard function, make sure you have a need to display written content.

5. The conference content display method according to any one of claims 1 to 4, characterized in that: After identifying the writing area according to the video preview area, the method further includes: If the writing picture content corresponding to the writing area cannot be displayed normally at the remote end, output a prompt message indicating that the writing area is abnormal; If the writing picture content corresponding to the writing area can be normally displayed at the remote end, the step of transmitting the writing picture content corresponding to the writing area to the remote end for remotely displaying the writing picture content of the writing area is performed.

6. The conference content display method according to claim 5, characterized in that: If the writing picture content corresponding to the writing area cannot be displayed normally at the remote end, before outputting the writing area abnormality prompt information, the method further includes: Acquiring the distance between the image acquisition device and the writing carrier; If the distance is greater than or equal to the preset distance, it is determined that the writing picture content corresponding to the writing area cannot be normally displayed at the remote end; If the distance is less than the preset distance, it is determined that the writing picture content corresponding to the writing area can be normally displayed at the remote end.

7. The conference content display method according to any one of claims 1 to 4, characterized in that: After identifying the writing area according to the video preview area, the method further includes: According to the received saving instruction, the writing picture content corresponding to the writing area is stored.

8. The conference content display method according to any one of claims 1 to 4, characterized in that: The step of identifying a writing area according to the video preview area includes: Identify a pre-selected area in the video preview area according to a preset selection box; The preselected area is adjusted according to the adjustment instruction to determine the writing area.

9. The conference content display method according to any one of claims 1 to 4, characterized in that: The step of identifying a writing area according to the video preview area includes: Two or more preset areas are identified in the video preview area according to the preset selection box; Identify a pre-selected area from each of the preset areas according to the selection instruction; The preselected area is adjusted according to the adjustment instruction to determine the writing area.

10. The conference content display method according to any one of claims 1 to 4, characterized in that: The step of identifying a writing area according to the video preview area includes: A writing area is identified from the video preview area according to the selected instruction.

11. The conference content display method according to any one of claims 1 to 4, characterized in that: The step of identifying a writing area according to the video preview area includes: Identify the writing area when the writing content was displayed last time from the video preview area, and use it as the pre-selected area for displaying the current writing content; The preselected area is adjusted according to the adjustment instruction to determine the writing area.

12. The conference content display method according to any one of claims 1 to 4, characterized in that: The step of transmitting the writing picture content corresponding to the writing area to a remote end for remotely displaying the writing picture content of the writing area includes: The writing picture content of the writing area is subjected to homographic transformation correction and then transmitted to a remote end for remote display of the writing picture content of the writing area.

13. The conference content display method according to any one of claims 1 to 4, characterized in that: The step of transmitting the writing picture content corresponding to the writing area to a remote end for remotely displaying the writing picture content of the writing area includes: Get the remote screen display mode; The writing picture content corresponding to the writing area is transmitted to the remote end according to the remote picture display mode, so as to be used for remotely displaying the writing picture content of the writing area.

14. The conference content display method according to any one of claims 1 to 4, characterized in that: Also includes: Send a shutdown instruction to the image acquisition device to shut down the artificial intelligence acquisition function of the image acquisition device.

15. A conference terminal, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the conference content presentation method described in any one of claims 1 to 14 are implemented.

16. A conference system, characterized in that: It comprises an image acquisition device and the conference terminal as claimed in claim 15, wherein the image acquisition device is communicatively connected with the conference terminal.

Images