Microphone connection method and apparatus

Abstract

The embodiments of the present application relate to the technical field of video microphone connections. Provided are a microphone connection method and apparatus. The method comprises: acquiring at least two video frames, which are obtained by means of respectively performing video shooting by at least two client devices participating in a video microphone connection; respectively extracting target objects in the at least two video frames, so as to acquire at least two target images; and generating a presentation interface of the video microphone connection on the basis of the at least two target images.

Description

A method and device for connecting to a microphone

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on July 24, 2023, with application number 202310914696.8 and invention name “A method and device for connecting to a microphone”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of video conferencing technology, and in particular to a method and device for conferencing. Background Art

[0003] During video calls, live video broadcasts, and other video conferences, the screen of each participating terminal is usually displayed in a fixed area corresponding to each participating terminal and cannot exceed the corresponding fixed area. The screen content outside the display area will be directly cropped, reducing the richness of the content. In addition, the screens of multiple participating terminals will be displayed completely separately, affecting the interactive experience of the conference.

[0004] Summary of the Invention

[0005] In view of this, embodiments of the present application provide a method and apparatus for connecting to a microphone.

[0006] In order to achieve the above objectives, the embodiments of the present application provide the following technical solutions:

[0007] In a first aspect, an embodiment of the present application provides a method for connecting to a microphone, including:

[0008] Obtain at least two video frames captured by at least two client devices participating in the video link;

[0009] respectively extracting the target objects in the at least two video frames to obtain at least two target images;

[0010] A display interface for the video link is generated based on the at least two target images.

[0011] As an optional implementation of the embodiment of the present application, generating the video link display interface based on the at least two target images includes:

[0012] Respectively obtaining position coordinates of the at least two target images in corresponding image frames;

[0013] According to the position coordinates of the at least two target images in the corresponding image frames, the at least two target images are added to a display interface template to generate the display interface of the video link.

[0014] As an optional implementation of the embodiment of the present application, adding the at least two target images to the display interface template according to the position coordinates of the at least two target images in the corresponding image frames includes:

[0015] Respectively obtaining position coordinates of the at least two video frames in the display interface template;

[0016] Acquire the position coordinates of the at least two target images in the display interface template according to the position coordinates of the at least two video frames in the display interface template and the position coordinates of the at least two target images in the corresponding image frames;

[0017] Adding the at least two target images to the display interface template according to the position coordinates of the at least two target images in the display interface template.

[0018] As an optional implementation of the embodiment of the present application, adding the at least two target images to the display interface template according to the position coordinates of the at least two target images in the display interface template includes:

[0019] Obtaining a layer order of the at least two target images;

[0020] The at least two target images are added to the display interface template according to the position coordinates of the at least two target images in the display interface template and the layer order of the at least two target images.

[0021] As an optional implementation of the embodiment of the present application, adding the at least two target images to the display interface template according to the position coordinates of the at least two target images in the display interface template and the layer order of the at least two target images includes:

[0022] According to the position coordinates of the at least two target images in the corresponding image frames, the at least two target images are added to the display interface template in order of layers from the bottom layer to the top layer.

[0023] As an optional implementation of the embodiment of the present application, adding the at least two target images to the display interface template according to the position coordinates of the at least two target images in the display interface template and the layer order of the at least two target images includes:

[0024] fusing the at least two target images to generate a fused image according to the position coordinates of the at least two target images in the display interface template and the layer order of the at least two target images, and obtaining the position coordinates of the fused image in the display interface template;

[0025] The fused image is added to the display interface template according to the position coordinates of the fused image in the display interface template.

[0026] As an optional implementation of the embodiment of the present application, the display interface template includes display areas corresponding to each client device participating in the video link; the method further includes:

[0027] The display area corresponding to each client device is rendered according to the background area of ​​the video frame obtained by shooting the video of each client device participating in the video connection.

[0028] As an optional implementation of the embodiment of the present application, the display interface template includes a fusion display area; the method further includes:

[0029] determining a target video frame from the at least two video frames;

[0030] The fused display area is rendered according to the background area of ​​the target video frame.

[0031] As an optional implementation of the embodiment of the present application, the display interface template includes a fusion display area; the method further includes:

[0032] The fusion display area is rendered according to a preset background image.

[0033] As an optional implementation of the embodiment of the present application, obtaining at least two video frames obtained by respectively shooting videos by at least two client devices participating in the video conferencing includes:

[0034] Video shooting is performed to obtain a first video frame, and at least one second video frame sent by the video link server is received.

[0035] In a second aspect, an embodiment of the present application provides a microphone connection device, including:

[0036] An acquisition unit, configured to acquire at least two video frames captured by at least two client devices participating in a video link;

[0037] a processing unit, configured to extract the target objects in the at least two video frames respectively to obtain at least two target images;

[0038] A generating unit is configured to generate a display interface for the video link based on the at least two target images.

[0039] As an optional implementation of an embodiment of the present application, the generation unit is specifically used to respectively obtain the position coordinates of the at least two target images in the corresponding image frames; according to the position coordinates of the at least two target images in the corresponding image frames, the at least two target images are added to the display interface template to generate the display interface of the video connection.

[0040] As an optional implementation of the embodiment of the present application, the generation unit is specifically used to respectively obtain the position coordinates of the at least two video frames in the display interface template; obtain the position coordinates of the at least two target images in the display interface template according to the position coordinates of the at least two video frames in the display interface template and the position coordinates of the at least two target images in the corresponding image frames; and add the at least two target images to the display interface template according to the position coordinates of the at least two target images in the display interface template.

[0041] As an optional implementation of an embodiment of the present application, the generation unit is specifically used to obtain the layer order of the at least two target images; and add the at least two target images to the display interface template according to the position coordinates of the at least two target images in the display interface template and the layer order of the at least two target images.

[0042] As an optional implementation of an embodiment of the present application, the generation unit is specifically used to add the at least two target images to the display interface template in sequence according to the position coordinates of the at least two target images in the corresponding image frames, in a layer order from the bottom layer to the top layer.

[0043] As an optional implementation of the embodiment of the present application, the generation unit is specifically used to fuse the at least two target images to generate a fused image based on the position coordinates of the at least two target images in the display interface template and the layer order of the at least two target images, and obtain the position coordinates of the fused image in the display interface template; and add the fused image to the display interface template based on the position coordinates of the fused image in the display interface template.

[0044] As an optional implementation of the embodiment of the present application, the display interface template includes display areas corresponding to each client device participating in the video call; the generation unit is also used to render the display area corresponding to each client device according to the background area of ​​the video frame obtained by video shooting by each client device participating in the video call.

[0045] As an optional implementation of an embodiment of the present application, the display interface template includes a fusion display area; the generation unit is further used to determine the target video frame from the at least two video frames; and render the fusion display area according to the background area of ​​the target video frame.

[0046] As an optional implementation of the embodiment of the present application, the display interface template includes a fusion display area; the generation unit is further used to render the fusion display area according to a preset background image.

[0047] As an optional implementation of the embodiment of the present application, the acquisition unit is specifically used to perform video shooting to obtain a first video frame, and receive at least one second video frame sent by the video link server.

[0048] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to enable the electronic device to implement the microphone connection method described in any of the above embodiments when executing the computer program.

[0049] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which, when the computer program is executed by a computing device, enables the computing device to implement the microphone connection method described in any of the above embodiments.

[0050] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a computer, enables the computer to implement the microphone connection method described in any one of the above embodiments.

[0051] The video conferencing method provided in an embodiment of the present application obtains at least two video frames obtained by respectively shooting videos by at least two client devices participating in the video conferencing, then extracts target objects from the at least two video frames to obtain at least two target images, and generates a video conferencing display interface based on the at least two target images. Because the embodiment of the present application extracts target objects from video frames obtained by respectively shooting videos by the client devices participating in the video conferencing, to obtain at least two target images, and generates a video conferencing display interface based on the at least two extracted target images. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0053] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0054] FIG1 is a scenario architecture diagram of a microphone connection method provided in an embodiment of the present application;

[0055] FIG2 is a flowchart of one of the steps of the method for connecting to a microphone provided in an embodiment of the present application;

[0056] FIG3 is a schematic diagram of a display interface for a video link provided in an embodiment of the present application;

[0057] FIG4 is a second flowchart of the method for connecting to a microphone according to an embodiment of the present application;

[0058] FIG5 is a schematic diagram of the position coordinates of a target image provided in an embodiment of the present application;

[0059] FIG6 is an interactive flow chart of the microphone connection method provided in an embodiment of the present application;

[0060] FIG7 is a schematic diagram of a rendering process of a display interface provided in an embodiment of the present application;

[0061] FIG8 is a second schematic diagram of the rendering process of the display interface provided in an embodiment of the present application;

[0062] FIG9 is a schematic diagram of one of the scene interfaces provided in an embodiment of the present application;

[0063] FIG10 is a second schematic diagram of a scene interface provided in an embodiment of the present application;

[0064] FIG11 is a schematic structural diagram of a microphone connection device provided in an embodiment of the present application;

[0065] FIG12 is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0066] In order to more clearly understand the above-mentioned objectives, features and advantages of the present application, the scheme of the present application will be further described below. It should be noted that, in the absence of conflict, the embodiments of the present application and the features therein can be combined with each other.

[0067] In the following description, many specific details are set forth to facilitate a full understanding of the present application, but the present application may also be implemented in other ways different from those described herein. Obviously, the embodiments in the specification are only part of the embodiments of the present application, not all of the embodiments.

[0068] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete way. In addition, in the description of the embodiments of the present application, unless otherwise specified, the meaning of "multiple" refers to two or more.

[0069] The following first describes the scenario architecture of the microphone connection method provided in the embodiment of the present application.

[0070] 1 , the scenario architecture of the microphone connection method provided in the embodiment of the present application includes: at least two client devices 100 and a video microphone connection server 200. The at least two client devices 100 establish a data transmission link with the video microphone connection server 200. On the one hand, the client device 100 can shoot a video based on a video shooting device, and send the shot video frames to the video microphone connection server 200 through the data transmission link established with the video microphone connection server 200. On the other hand, the client device 100 can obtain the video frames obtained by shooting videos by other client devices 100 from the video microphone connection server 200, and generate a video microphone connection display interface based on the video stream obtained by video acquisition based on the video shooting device and the video stream obtained by shooting videos by other terminal devices 100, and display the video microphone connection display interface.

[0071] It should be noted that FIG1 and subsequent embodiments all illustrate the case where two client devices participate in the video call, but the embodiments of the present application are not limited thereto. The number of client devices participating in the video call can also be other numbers, for example, 3, 4, etc.

[0072] The embodiment of the present application provides a method for connecting microphones to a client device or a server for connecting microphones to a video conference. As shown in FIG2 , the method includes the following steps S21 to S23:

[0073] S21: Obtain at least two video frames captured by at least two client devices participating in the video link.

[0074] In an embodiment of the present application, it can be executed by any participating end participating in the video conferencing, and the at least two video frames obtained by the client through video shooting of at least two client devices participating in the video conferencing include two parts of video frames, one part is the video frames obtained by the client device itself through video shooting, and the other part is the video frames received from other client devices participating in the video conferencing through the video conferencing server; therefore, the above step S12 (obtaining at least two video frames obtained by video shooting of at least two client devices participating in the video conferencing) includes: performing video shooting to obtain a first video frame, and receiving at least one second video frame sent by the video conferencing server.

[0075] The video link in the embodiment of the present application can be any communication service that requires transmitting video information between client devices, such as live link, video call, video conference, etc.

[0076] Exemplarily, the video conferencing communication server may receive the at least one second video frame sent by other clients participating in the video conferencing via the Real-time Transport Protocol (RTP), and send the at least one second video frame to the client device via the Real-time Transport Protocol.

[0077] S22. Extract target objects from the at least two video frames respectively to obtain at least two target images.

[0078] The target object in the embodiment of the present application can be the host or guest connected to the microphone, or it can be the items displayed or promoted in the live broadcast room, etc., and the embodiment of the present application does not limit this.

[0079] In some embodiments, the target objects in the at least two video frames are extracted respectively to obtain at least two target images, including: segmenting the image areas corresponding to the target objects in the at least two video frames based on an image segmentation algorithm, and extracting the image content in the image areas corresponding to the target objects as the at least two target images.

[0080] In other embodiments, respectively extracting the target objects in the at least two video frames to obtain at least two target images includes: respectively extracting the target objects in the at least two video frames based on an image volume extraction model to obtain the at least two target images. The image extraction model can be a model obtained by training a machine learning model such as a Convolutional Neural Networks (CNN) model or a Deep Neural Networks (DNN) model based on sample data, and the sample data includes sample video frames and target images corresponding to the sample video frames. That is, the at least two video frames are respectively input into the image volume extraction model, and the at least two target images are obtained based on the output of the image extraction model.

[0081] S23: Generate a display interface for the video link based on the at least two target images.

[0082] In some embodiments, generating the video link display interface based on the at least two target images includes:

[0083] The at least two target images are added to a display interface template to generate the display interface of the video link. The embodiment of the present application can enhance the link display effect and improve the user experience.

[0084] For example, as shown in FIG3 , FIG3 takes the example of the connection between the first and second microphone clients. In this case, at least two client devices include a first microphone client and a second microphone client; the at least two video frames include a first video frame and a second video frame, the first microphone client performs video shooting to obtain the first video frame, and the second microphone client performs video shooting to obtain the second video frame; the at least two target images include a first target image 301 and a second target image 302, the first target image is an image corresponding to the first target object in the first video frame, and the second target image is an image corresponding to the second target object in the second video frame. The display interface 30 of the video connection can be obtained by adding the first target image 301 and the second target image 302 to the display interface template 303. Since the display interface 30 is obtained by adding the first target image 301 and the second target image 302 to the display interface template 303, the complete target object image information can be retained without cropping, thereby enriching the screen display content, improving the screen display effect, avoiding the limitations of a fixed display area, and improving the interactive effect.

[0085] The video conferencing method provided in the embodiment of the present application obtains at least two video frames obtained by respectively shooting videos by at least two client devices participating in the video conferencing, and then respectively extracts the target objects in the at least two video frames to obtain at least two target images, and generates the video conferencing display interface based on the at least two target images. Since the embodiment of the present application extracts the target objects in the video frames obtained by shooting videos by the client devices participating in the video conferencing to obtain at least two target images, and generates the video conferencing display interface based on the at least two extracted target images, the embodiment of the present application can avoid cropping the target objects in the video frames obtained by shooting videos by the client devices, thereby avoiding the inability to fully display the target objects in the video conferencing display interface.

[0086] It should also be noted that, as shown in FIG3 , since at least two target images will not be cropped and may overlap, the embodiment of the present application can also enable users of video conferencing to interact across display areas.

[0087] As an extension and refinement of the above embodiment, the embodiment of the present application further provides another method for connecting to a microphone. As shown in FIG4 , the method further includes the following steps:

[0088] S401: Obtain at least two video frames captured by at least two client devices participating in a video link.

[0089] S402: Extract target objects from the at least two video frames respectively to obtain at least two target images.

[0090] S403 : Obtain position coordinates of the at least two target images in corresponding image frames respectively.

[0091] Exemplarily, the position coordinates of the target image in the corresponding image frame can be represented by the coordinates of the corner points of the circumscribed rectangular frame of the target image, the coordinates of the geometric center point of the circumscribed rectangular frame of the target image, the direction of the circumscribed rectangular frame of the target image, etc. The embodiment of the present application does not limit this, and the position coordinates of the target image in the corresponding image frame are sufficient to uniquely identify the position information of the target image in the corresponding image frame.

[0092] Exemplarily, as shown in Figure 5, target object extraction can be performed on the first video frame 500 of the at least two video frames to obtain the first image 501, and the position coordinates of the first image 501 in the first video frame 500 can be represented by the coordinates of the upper left corner point (x1, y1) and the lower right corner point coordinates (x2, y2) of the circumscribed rectangular box of the first image 501.

[0093] S404: Add the at least two target images to a display interface template according to the position coordinates of the at least two target images in the corresponding image frames to generate a display interface for the video link.

[0094] In some embodiments, the above step S404 (adding the at least two target images to the display interface template according to the position coordinates of the at least two target images in the corresponding image frames to generate the display interface of the video link) includes the following steps a to c:

[0095] Step a: respectively obtaining the position coordinates of the at least two video frames in the display interface template.

[0096] Exemplarily, the position coordinates of the video frame in the display interface template can be represented by the position coordinates of the geometric center of the video frame in the display interface template.

[0097] For example: the display interface template includes a display frame corresponding to the second video frame of the at least two video frames. During the display interface generation process, the geometric center of the second video frame needs to be aligned with the geometric center of the display frame. If the position coordinates of the geometric center of the display frame are (x3, y3), the position coordinates of the second video frame in the display interface template can be identified as (x3, y3).

[0098] Step b: acquiring the position coordinates of the at least two target images in the display interface template according to the position coordinates of the at least two video frames in the display interface template and the position coordinates of the at least two target images in the corresponding image frames.

[0099] Since the embodiment of the present application has obtained the position coordinates of the at least two video frames in the display interface template and the position coordinates of the at least two target images in the corresponding image frames, the position coordinates of the at least two target images in the display interface template can be obtained by performing addition and subtraction operations on the position coordinates of the video frames in the display interface template and the position coordinates of the target images in the corresponding image frames.

[0100] Step c: adding the at least two target images to the display interface template according to the position coordinates of the at least two target images in the display interface template.

[0101] An embodiment of the present application provides another method for connecting microphones. In the method, the at least two video frames obtained by the client device include: a second video frame obtained by the first client device through video shooting itself, and a second video frame received from the second client device through the connecting microphone server. As shown in FIG6 , the method includes the following steps S601 to S611:

[0102] S601: A second client device obtains a first video frame.

[0103] The first video frame is a video frame captured by a second client device participating in the video link.

[0104] That is, the second client device captures a video through a video capture device to obtain a first video frame.

[0105] S602: The second client device sends the first video frame to the video link server.

[0106] Correspondingly, the video link server receives the first video frame sent by the second client device.

[0107] S603: The video link server sends the first video frame to the first client device.

[0108] Correspondingly, the first client device receives the first video frame sent by the video link server.

[0109] S604: The first client device extracts the target object in the first video frame to obtain a first image.

[0110] S605: The first client device obtains a second video frame.

[0111] The second video frame is a video frame obtained by shooting a video on the first client device.

[0112] S606: The first client device extracts the target object in the first video frame to obtain a second image.

[0113] S607: The first client device obtains the position coordinates of the first image in the first video frame and the position coordinates of the second image in the second video frame respectively.

[0114] S608: The first client device respectively obtains the position coordinates of the first video frame and the second video frame in the display interface template.

[0115] S609: The first client device obtains the position coordinates of the first video frame in the presentation interface template according to the position coordinates of the first video frame in the presentation interface template and the position coordinates of the first image in the first video frame.

[0116] S610. The first client device obtains the position coordinates of the second video frame in the presentation interface template according to the position coordinates of the second video frame in the presentation interface template and the position coordinates of the second image in the second video frame.

[0117] S611. The first client device adds the first image and the second image to the display interface template according to the position coordinates of the first image in the display interface template and the position coordinates of the second image in the display interface template to generate the display interface of the video link.

[0118] In some embodiments, the above steps add the at least two target images to the display interface template based on the position coordinates of the at least two target images in the display interface template and the layer order of the at least two target images, including: obtaining the layer order of the at least two target images; adding the at least two target images to the display interface template based on the position coordinates of the at least two target images in the display interface template and the layer order of the at least two target images.

[0119] Since the at least two target images in the embodiment of the present application may exceed the corresponding display area, there may be an overlapping area between the at least two target images. Therefore, before adding the at least two target images to the display interface template, it is first necessary to obtain the layer order of the at least two target images to determine which target image's content is displayed in the overlapping area.

[0120] In some embodiments, the at least two target images are added to the display interface template according to the position coordinates of the at least two target images in the display interface template and the layer order of the at least two target images, including: adding the at least two target images to the display interface template in sequence according to the layer order from the bottom layer to the top layer according to the position coordinates of the at least two target images in the corresponding image frames.

[0121] That is, based on the layer order, the target image at the lower layer of the at least two target images is first added to the display interface template, and then the target image at the upper layer of the at least two target images is added to the display interface template.

[0122] Continuing from the above embodiment, if the layer order of the first image and the second image is that the first image is located on the upper layer of the second layer, then the above step S611 (adding the first image and the second image to the display interface template according to the position coordinates of the first image in the display interface template and the position coordinates of the second image in the display interface template) includes: first adding the second image to the display interface template according to the position coordinates of the second image in the display interface template, and then adding the first image to the display interface template according to the position coordinates of the first image in the display interface template.

[0123] For example, as shown in FIG7 , FIG7 illustrates an example in which the layer order of a first image 71 and a second image 72 is such that the first image 71 is located on an upper layer of the second image 72. Adding the first image and the second image to the display interface template based on the position coordinates of the first image and the second image in the display interface template includes: first, adding the second image 72 to the display interface template based on the position coordinates of the second image 72 in the display interface to obtain a graphical interface 73; and then, adding the first image 71 to the graphical interface 73 based on the position coordinates of the first image 71 in the display interface to obtain a graphical interface 74.

[0124] In some embodiments, adding the at least two target images to the display interface template according to the position coordinates of the at least two target images in the display interface template and the layer order of the at least two target images includes: fusing the at least two target images to generate a fused image according to the position coordinates of the at least two target images in the display interface template and the layer order of the at least two target images, and obtaining the position coordinates of the fused image in the display interface template; adding the fused image to the display interface template according to the position coordinates of the fused image in the display interface template.

[0125] That is, the at least two target images are first fused into one image based on the position coordinates and the layer order, and then the fused image is added to the display interface.

[0126] Continuing from the above embodiment, if the layer order of the first image and the second image is that the second image is located on the upper layer of the first image, then the above step S611 (adding the first image and the second image to the display interface template according to the position coordinates of the first image in the display interface template and the position coordinates of the second image in the display interface template) includes: fusing the first image and the second image to obtain a fused image according to the position coordinates of the first image in the display interface template, the position coordinates of the second image in the display interface template, and the layer order of the first image and the second image, and adding the fused image to the display interface template.

[0127] For example, as shown in FIG8 , the layer order of first image 81 and second image 82 is shown as an example in which second image 82 is located on top of first image 81. Adding the first image and the second image to the display interface template based on the position coordinates of the first image and the second image in the display interface template includes: fusing the first image 81 and the second image 82 to generate a fused image 83 based on the position coordinates of the first image 81 and the second image 82 in the display interface, as well as the layer order of the first and second images; and then adding the fused image to the display interface to obtain a graphical interface 84.

[0128] In some embodiments, based on the embodiment shown in Figure 6, the microphone connection method provided in the embodiment of the present application also includes: the first client device sends the second video frame to the video microphone connection communication server, so that the video microphone connection communication server sends the second video frame to the second client device, and the second client device generates the video microphone connection display interface corresponding to the target moment based on the microphone connection method provided in the embodiment of the present application.

[0129] In some embodiments, the display interface template includes display areas corresponding to each client device participating in the video call. The method for connecting to the microphone provided in the embodiment of the present application further includes: rendering the display area corresponding to each client device based on the background area of ​​the video frame obtained by the video shooting of each client device participating in the video call.

[0130] As shown in Figure 9, the client devices participating in the video link include a first client device and a second client device. The video frame captured by the first client device is a first video frame, and the video frame captured by the second client device is a second video frame. The display interface 900 includes a first display area 901 corresponding to the first client device and a second display area 902 corresponding to the second client device. The first display area 901 is rendered based on the background area of ​​the first video frame 91, and the second display area 902 is rendered based on the background area of ​​the second video frame 92, to obtain a graphical interface 93.

[0131] In some embodiments, the display interface template includes a fusion display area. The method for connecting microphones provided in an embodiment of the present application further includes: determining a target video frame from the at least two video frames; and rendering the fusion display area according to a background area of ​​the target video frame.

[0132] 10 , the display interface 1000 includes a fusion display area 1001 corresponding to the first client device and the second client device. The fusion display area 1001 is rendered based on the background area 101 of the first video frame to obtain a graphical interface 102.

[0133] In some embodiments, the display interface template includes a fusion display area. The microphone connection method provided in the embodiment of the present application further includes: rendering the fusion display area according to a preset background image.

[0134] The preset background image can be any image, and the embodiment of the present application does not limit this.

[0135] Based on the same inventive concept, as an implementation of the above method, the embodiment of the present application also provides a microphone connection device, which corresponds to the above method embodiment. For the convenience of reading, this embodiment will no longer repeat the details of the above method embodiment one by one, but it should be clear that the microphone connection device in this embodiment can correspond to and implement all the contents of the above method embodiment.

[0136] An embodiment of the present application provides a microphone connection device. FIG11 is a schematic structural diagram of the microphone connection device. As shown in FIG11 , the microphone connection device 1100 includes:

[0137] An acquiring unit 111 is configured to acquire at least two video frames captured by at least two client devices participating in a video link;

[0138] The processing unit 112 is configured to extract the target objects in the at least two video frames respectively to obtain at least two target images;

[0139] The generating unit 113 is configured to generate a display interface for the video link based on the at least two target images.

[0140] As an optional implementation of the embodiment of the present application, the generation unit 113 is specifically used to respectively obtain the position coordinates of the at least two target images in the corresponding image frames; according to the position coordinates of the at least two target images in the corresponding image frames, the at least two target images are added to the display interface template to generate the display interface of the video connection.

[0141] As an optional implementation of the embodiment of the present application, the generation unit 113 is specifically used to respectively obtain the position coordinates of the at least two video frames in the display interface template; obtain the position coordinates of the at least two target images in the display interface template according to the position coordinates of the at least two video frames in the display interface template and the position coordinates of the at least two target images in the corresponding image frames; and add the at least two target images to the display interface template according to the position coordinates of the at least two target images in the display interface template.

[0142] As an optional implementation of an embodiment of the present application, the generation unit 113 is specifically used to obtain the layer order of the at least two target images; and add the at least two target images to the display interface template according to the position coordinates of the at least two target images in the display interface template and the layer order of the at least two target images.

[0143] As an optional implementation of an embodiment of the present application, the generation unit 113 is specifically used to add the at least two target images to the display interface template in sequence according to the layer order from the bottom layer to the top layer based on the position coordinates of the at least two target images in the corresponding image frames.

[0144] As an optional implementation of the embodiment of the present application, the generation unit 113 is specifically used to fuse the at least two target images to generate a fused image based on the position coordinates of the at least two target images in the display interface template and the layer order of the at least two target images, and obtain the position coordinates of the fused image in the display interface template; and add the fused image to the display interface template based on the position coordinates of the fused image in the display interface template.

[0145] As an optional implementation of an embodiment of the present application, the display interface template includes display areas corresponding to each client device participating in the video call; the generation unit 113 is also used to render the display area corresponding to each client device according to the background area of ​​the video frame obtained by video shooting by each client device participating in the video call.

[0146] As an optional implementation of an embodiment of the present application, the display interface template includes a fusion display area; the generation unit 113 is also used to determine the target video frame from the at least two video frames; and render the fusion display area according to the background area of ​​the target video frame.

[0147] As an optional implementation of the embodiment of the present application, the display interface template includes a fusion display area; the generating unit 113 is further configured to render the fusion display area according to a preset background image.

[0148] As an optional implementation of the embodiment of the present application, the acquisition unit 111 is specifically configured to perform video capture to acquire a first video frame, and receive at least one second video frame sent by the video link server.

[0149] The microphone connection device provided in this embodiment can execute the microphone connection method provided in the above method embodiment. Its implementation principle and technical effect are similar and will not be repeated here.

[0150] Based on the same inventive concept, an embodiment of the present application further provides an electronic device. Figure 12 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. As shown in Figure 12, the electronic device provided in this embodiment includes: a memory 121 and a processor 122. The memory 121 is used to store a computer program, and the processor 122 is used to execute the microphone connection method provided in the above embodiment when executing the computer program.

[0151] Based on the same inventive concept, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the computing device implements the microphone connection method provided in the above embodiment.

[0152] Based on the same inventive concept, an embodiment of the present application further provides a computer program product. When the computer program product is run on a computer, the computing device implements the microphone connection method provided in the above embodiment.

[0153] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0154] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0155] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0156] Computer-readable media includes both permanent and non-permanent, removable and non-removable storage media. Storage media can implement any method or technology for storing information, which can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media, such as modulated data signals and carrier waves.

[0157] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some or all of the technical features therein, and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for connecting to a microphone, comprising: Obtain at least two video frames obtained by respectively shooting videos by at least two client devices participating in the video link; Respectively extracting the target objects in the at least two video frames to obtain at least two target images; A display interface for the video link is generated based on the at least two target images.

2. The method according to claim 1, wherein generating the video linking display interface based on the at least two target images comprises: Respectively obtaining position coordinates of the at least two target images in corresponding image frames; According to the position coordinates of the at least two target images in the corresponding image frames, the at least two target images are added to a display interface template to generate a display interface for the video link.

3. The method according to claim 2, wherein the adding the at least two target images to the display interface template according to the position coordinates of the at least two target images in the corresponding image frames comprises: Respectively obtaining the position coordinates of the at least two video frames in the display interface template; Acquire the position coordinates of the at least two target images in the display interface template according to the position coordinates of the at least two video frames in the display interface template and the position coordinates of the at least two target images in the corresponding image frames; According to the position coordinates of the at least two target images in the display interface template, the at least two target images are added to the display interface template.

4. The method according to claim 3, wherein adding the at least two target images to the display interface template according to the position coordinates of the at least two target images in the display interface template comprises: Acquire the layer order of the at least two target images; According to the position coordinates of the at least two target images in the display interface template and the layer order of the at least two target images, the at least two target images are added to the In the display interface template.

5. The method according to claim 4, wherein the adding the at least two target images to the display interface template according to the position coordinates of the at least two target images in the display interface template and the layer order of the at least two target images comprises: According to the position coordinates of the at least two target images in the corresponding image frames, the at least two target images are added to the display interface template in sequence from the bottom layer to the top layer.

6. The method according to claim 4, wherein the adding the at least two target images to the display interface template according to the position coordinates of the at least two target images in the display interface template and the layer order of the at least two target images comprises: According to the position coordinates of the at least two target images in the display interface template and the layer order of the at least two target images, the at least two target images are fused to generate a fused image, and the position coordinates of the fused image in the display interface template are obtained; The fused image is added to the display interface template according to the position coordinates of the fused image in the display interface template.

7. The method according to claim 2, wherein the display interface template includes display areas corresponding to each client device participating in the video link; the method further comprises: The display area corresponding to each client device is rendered according to the background area of ​​the video frame obtained by video shooting of each client device participating in the video connection.

8. The method according to claim 2, wherein the display interface template includes a fusion display area; the method further comprises: determining a target video frame from the at least two video frames; The fused display area is rendered according to the background area of ​​the target video frame.

9. The method according to claim 2, wherein the display interface template includes a fusion display area; the method further comprises: The fusion display area is rendered according to a preset background image.

10. The method according to claim 1, wherein obtaining at least two video frames obtained by respectively shooting videos by at least two client devices participating in the video link comprises: Video shooting is performed to obtain a first video frame, and at least one second video frame sent by the video link server is received.

11. A microphone connection device, comprising: An acquisition unit, used to acquire at least two video frames obtained by respectively shooting videos by at least two client devices participating in the video link; A processing unit, configured to respectively extract the target objects in the at least two video frames to obtain at least two target images; A generating unit is used to generate a display interface of the video connection based on the at least two target images.

12. An electronic device comprising: A memory and a processor, wherein the memory is used to store a computer program and the processor is used to enable the electronic device to implement the microphone connection method described in any one of claims 1-10 when executing the computer program.

13. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a computing device, the computing device implements the microphone connection method according to any one of claims 1 to 10.

Images