Video processing method and device and storage medium

Abstract

The invention discloses a video processing method and device, and a storage medium, relates to the technical field of video processing, and can improve the experience and reality of participants. The method comprises the steps of obtaining an identifier of a target person and a shared data picture; the shared data picture is a video picture corresponding to the shared data; wherein the target personnel are personnel participating in discussion and / or operating shared data in the participants; the participants participate in the video conference and the data conference by using the conference participating terminals, and the shared data is the data shared by the data conference; overlapping the identifier of the target person in the shared data picture to obtain a first target video; and sending the first target video to the participating terminal.

Description

Technical Field

[0001] The present application relates to the field of video processing technology, and in particular to a video processing method, device and storage medium. Background Art

[0002] Video conferencing is the use of voice and video to achieve remote face-to-face communication, while data conferencing is the use of shared data screens (such as shared whiteboards or shared desktops) to achieve remote information exchange. In many remote conferencing scenarios, video conferencing and data conferencing need to work together synchronously to meet user needs.

[0003] In traditional technology, an auxiliary video is usually added to the video conference to synchronize the data conference. This method displays the shared data screen in multiple conference terminals through the added auxiliary video. The user can see the content written in the shared data screen on the screen of the conference terminal. However, in this case, the shared data screen and the video in the video conference are displayed separately on the terminal screen. When multiple people participate in the discussion or write content in the shared data screen, the other participants cannot know in time which person is currently speaking or writing, which will bring inconvenience to the participants and reduce the experience and sense of reality of the participants. Summary of the invention

[0004] The embodiments of the present application provide a video processing method, device and storage medium, which can enhance the experience and sense of reality of conference participants.

[0005] On the one hand, a video processing method is provided, including: obtaining an identification of a target person and a shared data screen; the shared data screen is a video screen corresponding to the shared data; wherein the target person is a person among the participants who participate in the discussion and / or operate the shared data; the participant is a person who uses a participant terminal to participate in a video conference and a data conference, and the shared data is data shared in the data conference; superimposing the identification of the target person on the shared data screen to obtain a first target video; and sending the first target video to the participant terminal.

[0006] On the other hand, a video processing device is provided, including: an acquisition unit, used to acquire the identification of a target person and a shared data screen; the shared data screen is a video screen corresponding to the shared data; wherein the target person is a person among the participants who participate in the discussion and / or operate the shared data; the participant is a person who uses a participant terminal to participate in a video conference and a data conference, and the shared data is data shared by the data conference; a processing unit, used to superimpose the identification of the target person on the shared data screen to obtain a first target video; and a sending unit, used to send the first target video to the participant terminal.

[0007] On the other hand, a video processing device is provided, comprising: a memory and a processor; the memory and the processor are coupled; the memory is used to store a computer program; and the processor implements the video processing method described in any of the above embodiments when executing the computer program.

[0008] On the other hand, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the video processing method described in any of the above embodiments is implemented.

[0009] On the other hand, a computer program product is provided. The computer program product includes computer program instructions. When the computer program instructions are executed by a processor, the video processing method described in any of the above embodiments is implemented.

[0010] In the embodiment of the present application, the video screen displayed in the conference terminal includes both the identification of the target person and the screen of the shared data. When the target person speaks or writes in the screen of the shared data, the conference terminal will synchronously display the identification of the target person, allowing the conference participants to timely perceive the identity of the person currently speaking or writing, thereby increasing the sense of reality and experience for the participants. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solution in the present application, the drawings required for use in some embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only drawings of some embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0012] Figure 1 A schematic diagram of an implementation environment involved in a video processing method provided in some embodiments of the present application;

[0013] Figure 2 A flowchart of a video processing method provided in some embodiments of the present application;

[0014] Figure 3 A schematic diagram of a terminal interface provided for some embodiments of the present application;

[0015] Figure 4 Another terminal interface schematic diagram provided for some embodiments of the present application;

[0016] Figure 5 Another terminal interface schematic diagram provided for some embodiments of the present application;

[0017] Figure 6 Another terminal interface schematic diagram provided for some embodiments of the present application;

[0018] Figure 7Another terminal interface schematic diagram provided for some embodiments of the present application;

[0019] Figure 8 Another terminal interface schematic diagram provided for some embodiments of the present application;

[0020] Fig. 9 A schematic diagram of the structure of a video processing device provided in some embodiments of the present application;

[0021] Fig.10 A schematic structural diagram of another video processing device provided for some embodiments of the present application. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in this application to clearly and completely describe the technical solutions in this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0023] It should be noted that, in this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0024] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features.

[0025] In the description of this application, unless otherwise specified, " / " means "or", for example, A / B can mean A or B. "And / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, "at least one" means one or more, and "multiple" means two or more.

[0026] In traditional video conferences and data conferences, videos and shared data are displayed separately. Participants cannot see the shared data and the people discussing / operating the shared data on the same interface at the same time. When multiple people participate in the discussion or write content on the shared data screen, the other participants cannot know in time which person is currently speaking or writing, which will cause inconvenience to the participants and reduce their experience and sense of reality.

[0027] Based on this, the embodiment of the present application proposes a video processing method, in which the identification of the target person is superimposed on the shared data screen and sent to the conference participant terminal. It can be understood that the video screen displayed in the conference participant terminal contains both the identification of the target person and the screen of the shared data. When the target person speaks or writes in the screen of the shared data, the conference participant terminal will synchronously display the identification of the target person, which enables the conference participant to timely perceive the identity of the person currently speaking or writing, thereby increasing the sense of reality and experience of the conference participant.

[0028] The implementation of the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0029] Please refer to Figure 1 , which shows an implementation environment involved in a video processing method provided in an embodiment of the present application, the implementation environment includes: a multi-point control unit (MCU) 110, a data conferencing device 120 and a terminal 130.

[0030] MCU110 is a key device in the multi-point video conferencing system. Its function is equivalent to that of a switch. It will synchronously separate the information flows from various conference sites, extract audio, video, data and other information and signaling, and then send the information and signaling of each conference site to the same processing module to complete the corresponding audio mixing or switching, video mixing or switching, data broadcasting and routing selection, timing and conference control and other processes. Finally, it will reassemble the various information required by each conference site and send it to the corresponding terminal system equipment.

[0031] In the embodiment of the present application, MCU110 is used to hold a video conference, encode and decode videos, and coordinate communications between various devices. Specifically, MCU110 can support multiple terminals to conduct video conferences. The terminals participating in the video conference all communicate with MCU110. MCU110 manages video conference resources, negotiates between multiple terminals to determine the audio or video encoder / decoder to be used, and can process media streams.

[0032] In a possible implementation, the MCU 110 includes a decoding module 111 and a data conference processing module 112. The decoding module 111 may be a hardware or software module having video decoding and video processing functions. The data conference processing module 112 may be a hardware or software module for communicating with a data conference device.

[0033] The data conference device 120 is used to hold a data conference and can support multiple terminals to share data. The shared data, for example, shares the content displayed on the desktop or the content displayed on the whiteboard.

[0034] Exemplarily, the data conferencing device 120 may be a computing device such as a server, a terminal, a notebook, a desktop computer, or a handheld computer.

[0035] The terminal 130 is used to participate in a video conference and / or a data conference, and can send and receive information, video or data conference requests, etc., and can encode and decode videos, and directly or indirectly capture videos. In the embodiment of the present application, the terminal 130 participating in a video conference and / or a data conference is called a participating terminal, and the terminal participating in the discussion and / or operating the shared data among the participating terminals is called a target terminal.

[0036] Exemplarily, the terminal 130 may include a mobile phone, a tablet computer, a desktop computer, a laptop computer, a PDA or other devices.

[0037] In the embodiment of the present application, the MCU 110 and the data conference device 120 can be integrated together or independently configured and connected via a network. The embodiment of the present application does not limit how the MCU 110 and the data conference device 120 are specifically configured.

[0038] In the embodiment of the present application, the MCU 110 , the data conferencing device 120 and the terminal 130 may be one or more respectively, and each device may be arranged in different places respectively. The devices are connected through the network. The embodiment of the present application does not limit the number of the above devices.

[0039] In an application scenario, multiple terminals 130 conduct a video conference through MCU 110 and a data conference through data conference equipment 120. Part of the multiple terminals 130 participate in the discussion or operate the shared data screen. After MCU 110 obtains the identification of the target person of the part of the terminals, it superimposes the identification of the target person and the shared data screen and sends it to the multiple terminals 130. The screen displayed by the multiple terminals 130 contains the identification of multiple target persons and the shared data screen at the same time.

[0040] The following is a description of the video processing method provided in the embodiment of the present application:

[0041] The video processing method provided in the embodiment of the present application can be applied to MCU110. The execution subject of the video processing method provided in the embodiment of the present application can also be a video processing device. The video processing device can be an MCU, or a control module in the MCU for executing the video processing method. The method provided in the embodiment of the present application is MCU as an example for explanation below.

[0042] Please refer to Figure 2 , is a flow chart of a video processing method provided in an embodiment of the present application. Figure 2 As shown, the method may include S101-S109.

[0043] S101: The MCU creates a video conference.

[0044] Video conferencing refers to a meeting in which people in multiple locations have face-to-face conversations using communication equipment and the Internet.

[0045] Optionally, the MCU may first receive an operation to create a video conference, and create a video conference in response to the operation.

[0046] Optionally, the MCU may receive a request for creating a video conference sent by any participating terminal, and respond to the request to create a video conference.

[0047] The above-mentioned participating terminals are terminals participating in the video conference.

[0048] S102: The MCU establishes a video conference connection with the participating terminals.

[0049] The MCU establishes a video conference connection with the participating terminal, that is, the participating terminal joins the video conference created by the MCU.

[0050] The following are two ways to establish a video conference connection between the MCU and the participating terminals.

[0051] In a possible implementation, the MCU sends a video conference invitation to the participating terminal based on the identifier of the participating terminal. After the participating terminal responds to the video conference invitation, the MCU establishes a video conference connection with the participating terminal.

[0052] The MCU is preset with the identification of the participating terminal. The identification of the participating terminal preset by the MCU can be directly input on the MCU, can be pre-stored on the MCU, or can be sent by any terminal, which is not limited in the embodiment of the present application. As an example, the identification of the participating terminal can be the identification document (ID) of the participating terminal or the Internet protocol (IP) address of the participating terminal.

[0053] In an example, the participating terminals are two mobile phones, and the identifications of the participating terminals are the mobile phone numbers "18000xxxx01" and "18023xxxx28" of the two mobile phones. After the MCU creates the video conference, it dials the mobile phone numbers "18000xxxx01" and "18023xxxx28" respectively to invite the participating terminals to join the video conference. After the participating terminals accept the invitation sent by the MCU, the MCU completes establishing a video conference connection with the participating terminals.

[0054] In another possible implementation manner, the MCU accepts a request from a participating terminal to join a video conference, so as to establish a video conference connection with the participating terminal.

[0055] In one example, after the MCU creates a video conference, it generates a conference number "AABB0102". Participants using the participating terminals can join the video conference by entering "AABB0102" on the conference joining interface displayed on the participating terminals. Figure 3 As shown, Figure 3 The figure shows an interface diagram of a participating terminal with the theme of "XX conference", which includes a "conference number" input position. After entering the conference number "AABB0102" of the video conference, click "Confirm" to join the video conference.

[0056] S103: The MCU sends a creation request to the data conference device.

[0057] The creation request is used to create a data conference for the participating terminals.

[0058] A data conference is a meeting where people in multiple locations discuss or operate shared data through communication equipment and the Internet.

[0059] Specifically, the process can be implemented through S103a-S103c.

[0060] S103a: The MCU sends a creation request to the data conference device.

[0061] Optionally, the MCU may first receive an operation to create a data conference, and in response to the operation, send a creation request to the data conference device.

[0062] Optionally, the MCU may receive a request for creating a data conference sent by any participating terminal, and in response to the request, send a creation request to the data conference device.

[0063] S103b: The data conference device creates a data conference in response to the creation request sent by the MCU.

[0064] S103c: The data conference device establishes a data conference connection with the participating terminal.

[0065] The data conference device establishes a data conference connection with the participating terminal, indicating that the participating terminal has joined the data conference created by the data conference device.

[0066] The following are three methods for establishing a data conference connection between a data conference device and a participating terminal.

[0067] In a possible implementation, the data conferencing device sends a data conference invitation to the participating terminal based on the identifier of the participating terminal. After the participating terminal responds to the data conference invitation, the data conferencing device establishes a data conference connection with the participating terminal.

[0068] The identifier of the participating terminal may be sent by the MCU to the data conferencing device, or may be sent by any terminal to the data conferencing device. The embodiment of the present application does not limit how the data conferencing device obtains the identifier of the participating terminal.

[0069] In one example, the participating terminals are two mobile phones, and the identification of the participating terminals is the mobile phone numbers "18000xxxx01" and "18023xxxx28". After the data conferencing device creates the data conference, it dials the mobile phone numbers "18000xxxx01" and "18023xxxx28" respectively to invite the participating terminals to join the data conference. After the participating terminals accept the invitation sent by the data conferencing device, the data conferencing device establishes a data conference connection with the participating terminals.

[0070] In another possible implementation, the data conference device accepts a request from a participant terminal to join a data conference, so as to establish a data conference connection with the participant terminal.

[0071] In one example, after the data conference device creates a data conference, it generates a conference number "AABB0101". Participants using the conference terminal can join the data conference by entering "AABB0101" on the conference joining interface displayed on the conference terminal. Figure 4 As shown, Figure 4 The figure shows an interface diagram of a participant terminal with the theme of "XX conference", which includes a "conference number" input position. After entering the conference number "AABB0101" of the data conference, click "Confirm" to join the data conference.

[0072] In another possible implementation, after the data conference device creates the data conference, it first sends a message of the data conference creation to the MCU. After receiving the message of the successful creation of the data conference, the MCU sends a prompt message to the participating terminal that has joined the video conference to prompt the participating terminal to join the data conference.

[0073] In one example, if Figure 5 As shown, Figure 5The figure shows an interface diagram of a participating terminal with the theme of "XX Conference", which includes a prompt message "Please join the data conference" sent by the MCU to the participating terminal that has joined the video conference. If "Join" is selected in the interface, the participating terminal can join the data conference.

[0074] It is understandable that the MCU may first execute S102 to establish a video conference connection with the participating terminal, or first execute S103 to enable the data conference device to create a data conference, or execute S102 and S103 simultaneously. The embodiment of the present application does not limit the order of executing S102 and S103.

[0075] The embodiment of the present application does not limit the order in which the participating terminals join the video conference or the data conference.

[0076] S104 (optional): The data conference device receives a request from a target terminal to participate in a discussion and / or operate shared data, creates a target list, and sends the target list to the MCU.

[0077] The target list is used to record the identifiers of the target terminals.

[0078] The target terminal is the terminal used by the target person among the participating terminals. The target terminal can be one terminal or multiple terminals.

[0079] Target persons are those who participate in discussions and / or operate shared data among the participants; participants are those who participate in video conferences and data conferences using participating terminals.

[0080] Shared data refers to the data shared in a data conference. For example, the content written on a shared whiteboard or the content displayed on a shared desktop (such as a document). The content displayed on the shared desktop can be the content displayed on the shared desktop of any participating terminal.

[0081] In one example, if Figure 6 As shown, Figure 6 A schematic diagram of a conference interface with the theme "XX Conference" in a participant terminal is shown, and the interface includes a video display area for displaying a first target video. After the participant terminal establishes a data conference connection with the data conference device, a function button for applying for "Participate in Discussion" is displayed in the interface of the participant terminal. The participant selects "Participate in Discussion" on the participant terminal interface to join the discussion or explain the shared data. At this time, the participant terminal that joins the "Participate in Discussion" is the target terminal, and the person who uses the target terminal to participate in the discussion or operate the shared data is the target person.

[0082] It can be understood that the target list can be updated in real time. When new target terminals join in the discussion or operate shared data at different times, or when target terminals withdraw from the discussion or operate shared data, the data conferencing equipment can add or delete the identification of the target terminal in the target list in real time and send the updated target list to the MCU.

[0083] S105: The MCU obtains the identification of the target person.

[0084] Optionally, when the identification of the target person is a video including a picture of the target person, the MCU obtaining the identification of the target person includes: the MCU obtaining a second target video.

[0085] The second target video is a video including a target person's picture collected by the target terminal. When there are multiple target terminals, there are correspondingly multiple second target videos.

[0086] The video collected by the target terminal including the target person's screen refers to the video collected by the camera of the target terminal or the camera connected to the target terminal when the target terminal conducts a video conference. The video includes the target person's screen, that is, the video includes the screen of the person participating in the discussion or operating the shared data.

[0087] In a possible implementation manner, a specific implementation manner of S105 may include: S105a-S105b.

[0088] S105a: The MCU sends a request for acquiring the identification of the target person to the target terminal based on the identification of the target terminal in the target list.

[0089] In a possible implementation, if there are multiple IP addresses of target terminals in the target list, the MCU sends requests for obtaining the identification of the target person to the IP addresses of the multiple target terminals respectively.

[0090] S105b: After receiving the request for obtaining the identification of the target person sent by the MCU, the target terminal sends the identification of the target person to the MCU.

[0091] Optionally, when the target person is identified as a video including a picture of the target person, the target terminal sends the collected second target video to the MCU.

[0092] Correspondingly, the MCU receives the second target video sent by the target terminal.

[0093] Since the video conference is conducted in real time, the second target video is a video stream collected by the target terminal in real time.

[0094] Optionally, the identifier of the target person may also be a name, nickname and / or work number, etc., which is not limited in the embodiments of the present application.

[0095] It is understandable that, generally, participants (including target personnel) participate in video conferences or data conferences by logging into the conference software in the participating terminal. When logging into the conference software, participants need to log in with an account name and password. The account name can be the user's name, mobile phone number, ID card information, nickname or work number, etc. The account name can be used as an identifier for the participant.

[0096] It is understandable that the target person identification can be in various forms, and using the target person's picture as the target person identification can more intuitively represent the target person. When the first target video containing the target person's picture is subsequently displayed, the participants can clearly see the target person's actions or expressions, which increases the participants' experience.

[0097] S106: The MCU obtains the shared data screen.

[0098] Sharing data screens includes: sharing a whiteboard or sharing a desktop.

[0099] In a possible implementation, the shared data screen is a shared whiteboard, and the MCU sends a request to obtain the shared whiteboard to the data conferencing device. After receiving the request to obtain the shared whiteboard sent by the MCU, the data conferencing device creates a shared whiteboard and sends the shared whiteboard to the MCU.

[0100] In another possible implementation, the shared data screen is a shared desktop. The participating terminal sends the shared desktop to the data conferencing device, and the data conferencing device sends the shared desktop to the MCU after receiving it.

[0101] S107: The MCU superimposes the identification of the target person on the shared data screen to obtain a first target video.

[0102] When the target person is identified as a video including a picture of the target person, a possible implementation method is proposed below, which may include: S107a-S107e.

[0103] S107a: The MCU extracts the image of the target person from the image frame of the second target video to obtain a third target video.

[0104] Specifically, in the first step, the MCU performs boundary value analysis on the image frame of the second target video to identify the image of the target person in the image frame of the second target video.

[0105] In the second step, the MCU extracts the image of the target person in the image frame of the second target video to obtain a third target video.

[0106] Optionally, when there are multiple second target videos, the MCU extracts images of the target person from the image frames of the multiple second target videos respectively, and obtains multiple third target videos accordingly.

[0107] In this possible method, the MCU performs boundary value analysis on each image frame of the first target video, identifies the target person image in each image frame, extracts the target person image in each image frame, and then combines each image frame containing only the target person image into a third target video.

[0108] This method is only one possible implementation method, and other methods may also be used to extract the image of the target person in the first target video, which is not limited in this application.

[0109] S107b (optional): The MCU adjusts the size of the target person image in the third target video based on the size of the shared data image to obtain a fourth target video.

[0110] Adjusting the size of the target person picture in the third target video includes: reducing or enlarging the size of the target person picture in the third target video.

[0111] This optional step is to prevent the target person's screen size in the third target video from being too large or too small compared to the shared data screen, resulting in a poor superposition effect during subsequent superposition.

[0112] S107c: The MCU determines the target position on the shared data screen.

[0113] The target position is the position where the fourth target video is superimposed on the shared data screen. When there is a fourth target video, there are multiple corresponding target positions.

[0114] The multiple target positions do not overlap each other and can be located on the upper side, lower side, left side or right side of the shared data screen in sequence. The target position can also be set in other ways, based on not blocking the shared data screen, and this application does not limit this.

[0115] S107d: The MCU superimposes the fourth target video on the target position of the shared data screen.

[0116] Optionally, the MCU superimposes multiple fourth target videos on multiple target positions that are respectively superimposed on the shared data screen.

[0117] S107e: When there is occlusion between the target person images in the multiple fourth target videos superimposed on the shared data screen, the MCU performs transparency processing on the target person images in the multiple fourth target videos that are occluded to obtain the first target video.

[0118] Since the target persons in the video captured by the target terminal are moving, there is sometimes occlusion between multiple target persons and sometimes no occlusion. The S107e only performs transparency processing on the target persons when there is occlusion between at least two target persons, or only performs partial transparency processing on the occluded part, and no transparency processing is required when there is no occlusion.

[0119] The purpose of transparent processing is to prevent the key actions of the target person who is blocked from being blocked by other target persons. The purpose of opaque processing when there is no blockage is to allow the participants to see the actions of the target person more clearly. This method of dynamically making the target person's image transparent based on the blockage situation can enhance the experience of the participants.

[0120] In one example, if the shared data screen is a shared whiteboard, such as Figure 7 As shown, Figure 7 A schematic diagram of a screen in the first target video displayed on the target terminal is shown in the figure, and the screen includes target person A, target person B, target person C, target person D and a shared whiteboard. The "Xxxxxx..." included in the shared whiteboard is shared data discussed or written by multiple target persons. Since target person C and target person D block each other in the screen, the screen of target person C and target person D displayed is a transparent screen.

[0121] The above S107e is a possible processing method. In fact, after the MCU superimposes multiple target persons on the shared data screen, when some target persons discuss or operate the shared data, the target person screen is displayed at the target position in the shared data screen, and for target persons who do not discuss or operate the shared data, the target person's name and other identification are displayed at the target position in the shared data screen. This processing method can also reduce the occurrence of occlusion to a certain extent, and make the screen look simpler.

[0122] S108: The MCU sends the first target video to the participating terminals.

[0123] Optionally, when the first target video is superimposed with a picture of a target person, the MCU may also send identifiers such as the name, surname, nickname and / or work number of the target person to the participating terminal.

[0124] It is understandable that the MCU sends the name, nickname and / or work number and other identifiers of the target personnel to the participating terminal, which enables the participating terminal to synchronously display the name, nickname and / or work number and other identifiers of the target personnel on the interface, so that the participants can know who the target personnel are in a timely manner, and know who is currently discussing or operating the shared data, giving the participants a more intuitive experience.

[0125] S109: The participating terminal displays the first target video.

[0126] In one example, if Figure 8 As shown, Figure 8 The interface diagram of the participating terminal participating in the theme "xx conference" is shown, and the first target video and the name of the target person are displayed in the interface. The first target video contains the screen of the shared whiteboard and the target person. The names of the target persons are "Zhang xx", "Li xx", "Wang x" and "Zhao x", and the shared data "Xxxxxx..." is displayed on the shared whiteboard. When any of the target persons discusses or operates the shared data, a "microphone" icon is displayed to the right of the name of the person.

[0127] In the video processing method proposed in the embodiment of the present application, the MCU can superimpose the identification of the target person on the shared data screen and send it to each participating terminal, so that the screen displayed by the participating terminal contains both the identification of the target person and the shared data screen. When the target person speaks or writes in the shared data screen, the participating terminal will synchronously display the identification of the target person, allowing the participating persons to timely perceive the identity of the person currently speaking or writing, thereby increasing the sense of reality and experience of the participating persons.

[0128] It is understandable that, in order to realize the above functions, the video processing device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in conjunction with the algorithm steps of each example described in the embodiments of the present application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0129] The embodiment of the present application can divide the functional modules of the video processing device according to the above method embodiment. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one functional module. The above integrated module can be implemented in the form of hardware or software. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. The following is an example of dividing each functional module corresponding to each function.

[0130] Fig. 9 is a schematic diagram of the structure of a video processing device provided in an embodiment of the present application. The video processing device can execute the video processing method provided in the above method embodiment. Fig. 9As shown, the video processing device 200 includes: an acquisition unit 201, which is used to acquire the identification of the target person and the shared data screen; the shared data screen is the video screen corresponding to the shared data; wherein the target person is a person who participates in the discussion and / or operates the shared data among the participants; the participants are people who use the participating terminals to participate in the video conference and the data conference, and the shared data is the data shared by the data conference; the processing unit 202 is used to superimpose the identification of the target person on the shared data screen to obtain the first target video; the sending unit 203 is used to send the first target video to the participating terminal. For example, in combination with Figure 2 The acquisition unit 201 can be applied to S105 and S106 in the method embodiment, the processing unit 202 can be applied to S107 in the method embodiment, and the sending unit 203 can be applied to S108 in the method embodiment.

[0131] Optionally, the target person is identified as a video including the target person's screen; the acquisition unit 201 is specifically used to acquire a second target video; the second target video is a video including the target person's screen collected by the target terminal; the target terminal is the terminal used by the target person among the participating terminals.

[0132] Optionally, the shared data screen includes: a shared whiteboard or a shared desktop; the shared data includes: content written on the shared whiteboard or content displayed on the shared desktop.

[0133] Optionally, the video processing device 200 further includes a connection unit 204, which is used to establish a video conference connection with the participating terminal before obtaining the identification of the target person and sharing the data screen; the sending unit 203 is also used to initiate a creation request to the data conference device, and the creation request is used to create a data conference for the participating terminal. Figure 2 The connecting unit 204 can be applied to S103 in the method embodiment, and the sending unit 203 can be applied to S104 in the method embodiment.

[0134] Optionally, the video processing device 200 also includes a receiving unit 205, which is used to receive a target list sent by the data conference device after initiating a creation request to the data conference device; the target list is used to record the identifier of the target terminal; the target terminal is the terminal used by the target person in the participating terminal; the acquisition unit 201 is specifically used to receive the shared data screen sent by the data conference device; based on the identifier of the target terminal in the target list, send a request to the target terminal to obtain the identifier of the target person; and receive the identifier of the target person sent by the target terminal. For example, in combination with Figure 2 The receiving unit 205 can be applied to S104 in the method embodiment, and the acquiring unit 201 can be applied to S105a-S105b in the method embodiment.

[0135] Optionally, the processing unit 202 is specifically configured to extract the image of the target person from the image frame of the second target video to obtain a third target video; adjust the size of the target person image in the third target video based on the size of the shared data screen to obtain a fourth target video; determine the target position on the shared data screen; and superimpose the fourth target video on the target position in the shared data screen to obtain the first target video. For example, the processing unit 202 can be applied to S107a-S107e in the method embodiment.

[0136] Optionally, the processing unit 202 is specifically configured to, when there are multiple fourth target videos, determine multiple target positions on the shared data screen; respectively superimpose the multiple fourth target videos on the multiple target positions in the shared data screen; and when there is occlusion between the target person images in the multiple fourth target videos superimposed on the shared data screen, respectively perform transparency processing on the target person images in the multiple fourth target videos that are blocked, so as to obtain the first target video. For example, the processing unit 202 can be applied to S107a-S107e in the method embodiment.

[0137] Optionally, the multiple target locations are located at the top, bottom, left or right side of the shared data screen.

[0138] Optionally, the processing unit 202 is specifically configured to perform boundary value analysis on the image frame of the second target video to identify the image of the target person in the image frame of the second target video; and extract the image of the target person in the image frame of the second target video to obtain the third target video. For example, the processing unit 202 can be applied to S107a in the method embodiment.

[0139] In the case of implementing the functions of the above-mentioned integrated modules in the form of hardware, the embodiment of the present application provides another possible structure of the video processing device involved in the above-mentioned embodiment. Fig.10 As shown, the video processing device 300 includes: a processor 302 and a bus 304. Optionally, the video processing device may further include a memory 301; optionally, the video processing device may further include a communication interface 303.

[0140] The processor 302 may be a processor that implements or executes various exemplary logic blocks, modules, and circuits described in conjunction with the embodiments of the present application. The processor 302 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array, or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the embodiments of the present application. The processor 302 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0141] The communication interface 303 is used to connect with other devices via a communication network, such as Ethernet, wireless access network, wireless local area network (WLAN), etc.

[0142] The memory 301 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program codes in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0143] As a possible implementation, the memory 301 may exist independently of the processor 302, and the memory 301 may be connected to the processor 302 via a bus 304 for storing instructions or program codes. When the processor 302 calls and executes the instructions or program codes stored in the memory 301, the video processing method provided in the embodiment of the present application can be implemented.

[0144] In another possible implementation, the memory 301 may also be integrated with the processor 302 .

[0145] The bus 304 may be an extended industry standard architecture (EISA) bus, etc. The bus 304 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.10 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0146] Some embodiments of the present application provide a computer-readable storage medium (e.g., a non-transitory computer-readable storage medium), which stores computer program instructions. When the computer program instructions are executed on a computer, the computer executes the video processing method described in any of the above embodiments.

[0147] Exemplarily, the above-mentioned computer-readable storage media may include, but are not limited to: magnetic storage devices (e.g., hard disks, floppy disks or magnetic tapes, etc.), optical disks (e.g., compact disks (CD), digital versatile disks (DVD), etc.), smart cards and flash memory devices (e.g., erasable programmable read-only memory (EPROM), cards, sticks or key drives, etc.). The various computer-readable storage media described in this application may represent one or more devices and / or other machine-readable storage media for storing information. The term "machine-readable storage medium" may include, but is not limited to, wireless channels and various other media capable of storing, containing and / or carrying instructions and / or data.

[0148] An embodiment of the present application provides a computer program product including instructions. When the computer program product is run on a computer, the computer is enabled to execute the video processing method described in any one of the above embodiments.

[0149] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A video processing method, characterized in that: The method comprises: Obtaining the identification of the target person and the shared data screen; the shared data screen is the video screen corresponding to the shared data; wherein the target person is a person among the conference participants who participates in the discussion and / or operates the shared data; the conference participant is a person who uses a conference terminal to participate in the video conference and the data conference, and the shared data is the data shared by the data conference; Superimposing the identification of the target person on the shared data screen to obtain a first target video; Send the first target video to the conference participant terminal.

2. The method according to claim 1, characterized in that The identification of the target person is a video including a picture of the target person; and obtaining the identification of the target person includes: Acquire a second target video; the second target video is a video collected by a target terminal and including a picture of the target person; the target terminal is a terminal used by the target person among the participating terminals.

3. The method according to claim 1 or 2, characterized in that: The shared data screen includes: a shared whiteboard or a shared desktop; the shared data includes: content written on the shared whiteboard or content displayed on the shared desktop.

4. The method according to claim 1 or 2, characterized in that: Before obtaining the identification of the target person and the shared data screen, the method further includes: Establishing a video conference connection with the participating terminal; A creation request is initiated to a data conference device, where the creation request is used to create the data conference for the participating terminal.

5. The method according to claim 4, characterized in that After initiating a creation request to the data conference device, the method further includes: receiving a target list sent by the data conference device; the target list is used to record the identifier of the target terminal; the target terminal is a terminal used by the target person among the participating terminals; The obtaining of the target person's identification and the shared data screen includes: Receiving the shared data screen sent by the data conferencing device; Based on the identification of the target terminal in the target list, sending a request to the target terminal to obtain the identification of the target person; Receive the identification of the target person sent by the target terminal.

6. The method according to claim 2, characterized in that The step of superimposing the identification of the target person on the shared data screen to obtain a first target video includes: Extracting the image of the target person from the image frame of the second target video to obtain a third target video; Based on the size of the shared data screen, adjusting the size of the target person screen in the third target video to obtain a fourth target video; determining a target location on the shared data screen; The fourth target video is superimposed on the target position in the shared data screen to obtain the first target video.

7. The method according to claim 6, characterized in that In the case where there are a plurality of the fourth target videos, determining the target position on the shared data screen includes: determining a plurality of target locations on the shared data screen; The step of superimposing the fourth target video on the target position in the shared data screen to obtain the first target video includes: Respectively superimposing a plurality of the fourth target videos on the plurality of target positions in the shared data screen; In the case where there is occlusion between the target person images in the multiple fourth target videos superimposed on the shared data screen, the target person images in the multiple fourth target videos with occlusion are respectively processed for transparency to obtain the first target video.

8. The method according to claim 6 or 7, characterized in that: The plurality of target locations are located at an upper side, a lower side, a left side or a right side of the shared data screen.

9. The method according to claim 6, characterized in that The step of extracting the image of the target person from the image frame of the second target video to obtain a third target video includes: Performing boundary value analysis on the image frame of the second target video to identify the image of the target person in the image frame of the second target video; An image of the target person in the image frame of the second target video is extracted to obtain a third target video.

10. A video processing device, characterized in that: It comprises a memory and a processor; the memory and the processor are coupled; the memory is used to store computer program code, and the computer program code includes computer instructions; wherein, when the processor executes the computer instructions, the video processing device executes the video processing method as described in any one of claims 1-9.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions; wherein, when the computer instructions are executed on a video processing device, the video processing device executes the video processing method according to any one of claims 1 to 9.

Images