Microphone-connected video display method and device

Abstract

The embodiment of the invention provides a microphone connection video display method. The microphone connection video display method comprises the following steps: acquiring a mixed picture live broadcast stream comprising an initial microphone connection picture; executing transcoding operation to obtain a live broadcast code stream carrying supplementary enhancement information; the transcoding operation comprises the following steps: comparing the resolution of the initial microphone-connected picture with a preset resolution; under different conditions, adjusting the initial microphone connection picture to obtain a target microphone connection picture, the target microphone connection picture comprising a target video picture; updating the supplementary enhancement information through the resolution of the target video picture; the live broadcast code stream is pushed to a client, and the client is used for obtaining the target video picture from the live broadcast code stream according to the supplemental enhancement information and displaying the target video picture through a player. According to the technical scheme provided by the embodiment of the invention, the player can accurately identify and play the actual content area, the problems of image proportion distortion and black edges in scenes such as transverse and vertical screen switching and direct turning points are reduced, and the watching experience of live broadcast microphone connection is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of live broadcast technology, and in particular, to a method, apparatus, computer device, computer-readable storage medium, and computer program product for displaying a video with microphones connected. Background Art

[0002] In the field of live broadcasting, live broadcasting with microphone is one of the common interactive methods. It allows multiple users to communicate in real time with video and audio during live broadcasting. It may cause the resolution of the push stream to change, causing compatibility issues with players. Therefore, the resolution can be changed during the merge to keep the resolution consistent.

[0003] However, during the live broadcast process, users may switch between horizontal and vertical screens, resulting in distorted picture ratios, distorted content, or black border areas, affecting the viewing experience.

[0004] It should be noted that the above content is not necessarily prior art, nor is it intended to limit the scope of patent protection of this application. Summary of the invention

[0005] The embodiments of the present application provide a method, apparatus, computer device, computer-readable storage medium, and computer program product for displaying a video with a microphone connected to a microphone, so as to solve or alleviate one or more of the technical problems mentioned above.

[0006] One aspect of an embodiment of the present application provides a method for displaying a video connected to a microphone, which is used in a server. The method includes:

[0007] In response to the microphone connection request, obtaining a mixed-image live stream, where the mixed-image live stream includes an initial microphone connection picture;

[0008] Performing a transcoding operation on the mixed-image live stream to obtain a live stream, wherein the live stream carries the supplementary enhancement information; wherein the transcoding operation includes: obtaining a resolution of the initial connected-microphone screen; comparing the resolution of the initial connected-microphone screen with a preset resolution; if the resolution of the initial connected-microphone screen is different from the preset resolution, adjusting the initial connected-microphone screen to obtain a target connected-microphone screen with the same resolution as the preset resolution, wherein the target connected-microphone screen includes a target video screen; and updating the supplementary enhancement information according to the resolution of the target video screen;

[0009] The live stream is pushed to a client, and the client is used to obtain the target video picture from the live stream according to the supplementary enhancement information, and display it through a preset player.

[0010] Optionally, obtaining the mixed-image live stream includes:

[0011] Get the first live stream of the anchor end and the second live stream of the microphone end;

[0012] Merging the first live stream and the second live stream to obtain the mixed-image live stream;

[0013] The merging includes: merging the video picture of the first live stream and the video picture of the second live stream to obtain the initial joint microphone picture.

[0014] Optionally, the method for displaying a video with microphones connected to the microphone further includes:

[0015] In response to the microphone disconnection request, obtaining the latest first live stream;

[0016] Transcoding the latest first live stream, and updating the supplementary enhancement information according to the resolution of the video picture of the latest first live stream to obtain the latest live stream, wherein the latest live stream carries the updated supplementary enhancement information;

[0017] The latest live stream is pushed to the client, and the client is further used to: obtain the latest video picture of the first live stream from the latest live stream according to the updated supplementary enhancement information, and display it through a preset player.

[0018] Optionally, adjusting the initial microphone connection screen includes:

[0019] Scaling the initial microphone connection picture according to the preset resolution to obtain the target video picture;

[0020] Setting a black border area at the edge of the initial combined microphone picture after scaling to obtain the target combined microphone picture;

[0021] The target microphone connection screen includes the zoomed target video screen and the black border area.

[0022] Optionally, the method for displaying a video with microphones connected to the microphone further includes:

[0023] Recording the live stream and the supplementary enhancement information to generate a video-on-demand file;

[0024] In response to the on-demand request, the on-demand file is pushed to the client.

[0025] Another aspect of an embodiment of the present application provides a method for displaying a video connected to a microphone, which is used in a client. The method includes:

[0026] Acquire a live stream or a video-on-demand file carrying supplementary enhancement information provided by a server, wherein the live stream or the video-on-demand file includes an original video picture, the original video picture includes a target video picture and a black border area, and the supplementary enhancement information includes a resolution of the target video picture;

[0027] According to the supplementary enhancement information, the original video picture is adjusted to obtain the target video picture;

[0028] The target video screen is displayed through a preset player.

[0029] Optionally, adjusting the original video picture according to the supplemental enhancement information to obtain the target video picture includes:

[0030] Determining the resolution of the target video picture according to the supplemental enhancement information;

[0031] According to the resolution of the target video picture, the target video picture in the original video picture is identified and extracted.

[0032] Another aspect of an embodiment of the present application provides a video display device connected to a microphone, used in a server, the device comprising:

[0033] An acquisition module, configured to acquire a mixed-image live stream in response to a microphone connection request, wherein the mixed-image live stream includes an initial microphone connection picture;

[0034] A transcoding module, configured to perform a transcoding operation on the mixed-image live stream to obtain a live stream, wherein the live stream carries the supplementary enhancement information; wherein the transcoding operation comprises: obtaining a resolution of the initial connected-microphone screen; comparing the resolution of the initial connected-microphone screen with a preset resolution; if the resolution of the initial connected-microphone screen is different from the preset resolution, adjusting the initial connected-microphone screen to obtain a target connected-microphone screen with the same resolution as the preset resolution, wherein the target connected-microphone screen comprises a target video screen; and updating the supplementary enhancement information according to the resolution of the target video screen;

[0035] A push module is used to push the live stream to a client, and the client is used to obtain the target video picture from the live stream according to the supplementary enhancement information and display it through a preset player.

[0036] Another aspect of an embodiment of the present application provides a video display device for connecting to microphones, used for a client, the device comprising:

[0037] A first acquisition module is used to acquire a live stream or a video-on-demand file carrying supplementary enhancement information provided by a server, wherein the live stream or the video-on-demand file includes an original video picture, the original video picture includes a target video picture and a black border area, and the supplementary enhancement information includes a resolution of the target video picture;

[0038] A second acquisition module, configured to adjust the original video picture according to the supplementary enhancement information to obtain the target video picture;

[0039] The playing module is used to display the target video screen through a preset player.

[0040] Another aspect of an embodiment of the present application provides a computer device, including:

[0041] at least one processor; and

[0042] a memory communicatively coupled to the at least one processor;

[0043] Wherein: the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method as described above.

[0044] Another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method described above is implemented.

[0045] Another aspect of an embodiment of the present application provides a computer program product, including a computer program, which implements the above-mentioned method when executed by a processor.

[0046] The above technical solution adopted in the embodiment of the present application may have the following advantages:

[0047] During live broadcast and microphone connection, a mixed live stream including an initial microphone connection screen is obtained and a transcoding operation is performed to obtain a live stream carrying supplementary enhancement information. The transcoding operation includes: comparing the resolution of the initial microphone connection screen with the preset resolution. When the two are different, the initial microphone connection screen is adjusted to obtain a target microphone connection screen containing a target video screen, and the resolution of the target microphone connection screen is the same as the preset resolution. And the resolution of the target video screen is added to the supplementary enhancement information of the live broadcast stream. The live broadcast stream is pushed to the client so that the client can obtain the target video screen in the live broadcast stream according to the supplementary enhancement information and display it through the player. It can be seen that the embodiment of the present application adds the resolution of the target video screen to the SEI (supplementary enhancement information) of the live broadcast stream, so that the player can accurately identify the actual content area and play it. This can reduce the distortion and black edge problems of the screen ratio in scenes such as horizontal and vertical screen switching and vertical switching points, thereby improving the viewing experience of live broadcast and microphone connection. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0049] Figure 1 The following schematically shows an operating environment diagram of the method for displaying video through microphones according to the first embodiment of the present application;

[0050] Figure 2 The flowchart of the method for displaying a video with microphones according to the first embodiment of the present application is schematically shown;

[0051] Figure 3 The following schematically shows the microphone connection process according to the first embodiment of the present application;

[0052] Figure 4 Schematically shows Figure 2 Flow chart of sub-steps of step S200;

[0053] Figure 5 Schematically shows Figure 2 Flow chart of sub-steps of step S202;

[0054] Figure 6 The newly added flow chart of the method for displaying video through microphone connection according to the first embodiment of the present application is schematically shown;

[0055] Figure 7 The newly added flow chart of the method for displaying video through microphone connection according to the first embodiment of the present application is schematically shown;

[0056] Figure 8 The flowchart of the method for displaying a video with microphones according to the second embodiment of the present application is schematically shown;

[0057] Fig. 9 A block diagram of a video display device for connecting microphones according to Embodiment 3 of the present application is schematically shown;

[0058] Fig.10 A block diagram of a video display device with microphones connected according to a fourth embodiment of the present application is schematically shown; and

[0059] Fig.11 The hardware architecture diagram of the computer device according to the fifth embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0061] It should be noted that the descriptions involving "first", "second", etc. in the embodiments of the present application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0062] In the description of the present application, it should be understood that the numerical labels before the steps do not indicate the order in which the steps are executed, but are only used to facilitate the description of the present application and to distinguish each step, and therefore should not be understood as a limitation on the present application.

[0063] First, the following terms are explained:

[0064] Live broadcast: Real-time transmission of video, audio and other media content over the Internet. Viewers can watch the content as it happens. It can be used for real-time sharing of events, performances, games and other content.

[0065] On-demand: Select and play media files stored on the server (such as movies, TV series, videos, etc.). Unlike live broadcasts, on-demand content does not occur in real time and viewers can watch it at any time.

[0066] Direct conversion: The process of converting live content into on-demand content after or during the live broadcast. Viewers who missed the live broadcast can watch the content on demand.

[0067] Live broadcast and microphone connection: During the live broadcast, the host and other hosts or guests have real-time video and audio connections to interact with each other, which can increase the fun and interactivity of the live broadcast.

[0068] Switching between horizontal and vertical screens: During video playback or live broadcast, the video screen is rotated from horizontal (wide screen) to vertical (narrow screen), or from vertical to horizontal, depending on the orientation of the content or device. It is mostly used on mobile devices, and viewers can adjust the viewing mode as needed.

[0069] Video transcoding: The process of converting a video file from one encoding format to another, involving video resolution, bit rate, frame rate and other parameters to adapt to different playback environments or meet specific quality requirements. Transcoding can improve the compatibility and playback efficiency of the video.

[0070] Secondly, in order to facilitate those skilled in the art to understand the technical solutions provided in the embodiments of the present application, the relevant technologies are described below:

[0071] In the field of live broadcasting, live broadcasting with microphone connection is one of the common ways of interaction. Microphone connection allows multiple users to communicate in real time with video and audio during live broadcasting. Microphone connection may cause changes in streaming resolution and cause player compatibility issues. To solve the aforementioned problems, the applicant understands that: the resolution can be changed during confluence, for example, black borders can be added above and below the video screen to keep the resolution consistent; sequence parameter set information (SPS) and picture parameter set information (PPS) can also be added to each I frame (key frame) of the transcoding to assist the player in determining resolution changes.

[0072] However, the above method has the following defects: (1) The transcoding process may not be able to respond to the resolution change in time, resulting in content distortion or distorted screen ratio; (2) When switching the player between horizontal and vertical screens, unsightly black edges will appear, affecting the viewing experience; (3) It is not compatible with vertical switching points.

[0073] To this end, the embodiment of the present application provides a video display technology solution for live broadcast. In this technical solution: during the confluence process, the resolution of the actual content area is added to the supplementary enhancement information (SEI) of the bitstream, so that the player can accurately identify and play the actual content area according to the SEI, ignore the black edges, maintain the picture ratio and viewing experience, and be compatible with and improve the live broadcast experience of live broadcast in scenarios such as vertical rotation point and horizontal and vertical screen switching. See below for details.

[0074] Finally, for ease of understanding, an exemplary operating environment is provided below.

[0075] like Figure 1 As shown, the operating environment diagram includes: service platform 2, anchor terminals (4A, 4B, ..., 4M), and audience terminals (6A, 6B, ..., 6N). In the live broadcast scenario, the anchor terminal (4A, 4B, ..., 4M) logs in to the service platform 2, and the service platform 2 pushes the live broadcast data to the audience terminal (6A, 6B, ..., 6N) in real time.

[0076] Service platform 2 can provide live broadcast service, which can be a single server, a server cluster or a cloud computing service center.

[0077] The anchor terminal (4A, 4B, ..., 4M) is used to generate live broadcast data in real time and perform streaming operations on the live broadcast data. The live broadcast data may include audio data or video data. The anchor terminal may be an electronic device such as a smart phone or a tablet computer. Of course, the anchor terminal may be a virtual computing instance in the service platform 2.

[0078] The audience terminal (6A, 6B, ..., 6N) can be configured to receive the live broadcast data of the anchor terminal in real time. The audience terminal (6A, 6B, ..., 6N) can be any type of computing device, such as a smart phone, a tablet device, a laptop computer, a smart TV, a car terminal, etc. The audience terminal (6A, 6B, ..., 6N) can have a built-in browser or a special program, and receive the live broadcast data through the browser or the special program to output content to the user. The content may include video, audio, comments, text data and / or the like.

[0079] The audience terminal (6A, 6B, ..., 6N) may include a player. The player outputs (e.g., displays, presents) content to the user. The content may include video, audio, comments, text data and / or the like. The audience terminal (6A, 6B, ..., 6N) may include an interface, which may include an input element (touch screen). For example, the input element may be configured to receive user instructions, which may enable the audience terminal (6A, 6B, ..., 6N) to perform various operations, such as sending a bullet screen, entering a comment, giving a gift, etc.

[0080] The anchor terminals (4A, 4B, ..., 4M), the audience terminals (6A, 6B, ..., 6N) and the service platform 2 may be connected via a network. The network may include various network devices, such as routers, switches, multiplexers, hubs, modems, bridges, repeaters, firewalls and / or proxy devices, etc. The network may include physical links, such as coaxial cable links, twisted pair cable links, optical fiber links, combinations thereof and / or the like. The network may include wireless links, such as cellular links, satellite links, Wi-Fi links and / or the like.

[0081] It should be noted that the number of the host terminals and the audience terminals in the figure is only for illustration and is not used to limit the scope of patent protection of this application. According to actual conditions, there can be any number of host terminals and audience terminals.

[0082] The following uses the server (service platform 2) as the execution subject to introduce the technical solution of the present application through multiple embodiments. It should be noted that these embodiments can be implemented in a variety of different forms and should not be interpreted as being limited to the embodiments described here.

[0083] Embodiment 1

[0084] The embodiment of the present application takes the server (service platform 2) as the execution entity and exemplarily introduces the method for displaying video via microphone provided by the present application.

[0085] Figure 2 The flowchart of the method for displaying video via microphone connection according to the first embodiment of the present application is schematically shown.

[0086] like Figure 2 As shown, the method for displaying a video with microphones may include steps S200 to S204, wherein:

[0087] Step S200: In response to the microphone connection request, a mixed-image live stream is obtained, where the mixed-image live stream includes an initial microphone connection screen.

[0088] Step S202, performing a transcoding operation on the mixed live stream to obtain a live stream, wherein the live stream carries supplementary enhancement information. The transcoding operation includes: obtaining the resolution of the initial connected microphone screen; comparing the resolution of the initial connected microphone screen with a preset resolution; if the resolution of the initial connected microphone screen is different from the preset resolution, adjusting the initial connected microphone screen to obtain a target connected microphone screen with the same resolution as the preset resolution, wherein the target connected microphone screen includes a target video screen; and updating the supplementary enhancement information according to the resolution of the target video screen.

[0089] Step S204: Push the live stream to the client, and the client is used to obtain the target video picture from the live stream according to the supplementary enhancement information, and display it through a preset player.

[0090] The embodiment provides a method for displaying a video connected with a microphone: during live broadcast, a mixed live stream including an initial connected microphone screen is obtained and a transcoding operation is performed to obtain a live stream carrying supplementary enhancement information. The transcoding operation includes: comparing the resolution of the initial connected microphone screen with the preset resolution. When the two are different, the initial connected microphone screen is adjusted to obtain a target connected microphone screen containing a target video screen, and the resolution of the target connected microphone screen is the same as the preset resolution. And the resolution of the target video screen is added to the supplementary enhancement information of the live stream. The live stream is pushed to the client, so that the client can obtain the target video screen in the live stream according to the supplementary enhancement information and display it through the player. It can be seen that the embodiment of the present application adds the resolution of the target video screen in the SEI (supplementary enhancement information) of the live stream, so that the player can accurately identify the actual content area and play it. This can reduce the distortion and black edge problems of the screen ratio in scenes such as horizontal and vertical screen switching and vertical switching points, thereby improving the viewing experience of live broadcast connected microphones.

[0091] The following combination Figure 2 , each step in steps S200 to S204 and other optional steps are described in detail.

[0092] Step S200 In response to the microphone connection request, a mixed-image live stream is obtained, where the mixed-image live stream includes an initial microphone connection screen.

[0093] During the live broadcast process, the host end (also called the host terminal) can push the generated live stream to the server. The server transcodes the live stream and generates a live stream. The server can push the live stream to the client, and the client can display the host's live screen accordingly.

[0094] During the live broadcast, the host can also interact with other hosts or viewers via microphone. Figure 3 As shown, anchor A (which can be called the anchor end) can initiate a microphone connection request for anchor B (which can be called the microphone connection end) to the server. After receiving the microphone connection request, the server can forward it to anchor B. If anchor B accepts the request, anchor A and anchor B will enter the microphone connection mode. In the microphone connection mode, the live streams of anchor A and anchor B will be displayed simultaneously in the live broadcast room, and the audience can see the live broadcast screens of the two anchors at the same time in the live broadcast room, that is, the microphone connection screen. It should be noted that the microphone connection end can include one or more. The following will take a microphone connection end as an example to exemplarily introduce the microphone connection video display method provided in an embodiment of the present application.

[0095] To display the live broadcast screen, the server can obtain the live broadcast streams of the host and the host, and mix the two live broadcast streams into the same live broadcast stream, i.e., the mixed live broadcast stream. The mixed live broadcast stream can include the initial live broadcast screen. The initial live broadcast screen can be generated by processing the live broadcast screens of the two hosts according to a preset method. The preset processing method may include cropping, splicing, etc. An exemplary solution will be provided below.

[0096] In an optional embodiment, if Figure 4 As shown, step S200 may include:

[0097] Step S300, obtaining a first live stream from the host end and a second live stream from the microphone connected end.

[0098] Step S302: merge the first live stream and the second live stream to obtain the mixed live stream, wherein the merging includes: merging the video picture of the first live stream and the video picture of the second live stream to obtain the initial joint microphone picture.

[0099] In actual applications, after the microphone connection is successful, the server can obtain the first live stream of the anchor end (such as anchor A) and the second live stream of the microphone connection end (such as anchor B). The first live stream and the second live stream are merged to synthesize a mixed live stream. There are many ways to merge, such as picture-in-picture, side-by-side / parallel display, partial overlapping, etc. Exemplarily, the video screen of the second live stream can be embedded in the video screen of the first live stream to obtain an initial microphone connection screen. In the initial microphone connection screen of this mode, the anchor's video screen can fill the player window, while the video screen of the microphone connection person can be displayed as a small window at a specific position of the player. In this case, the resolution of the initial microphone connection screen is the same as that of the video screen of the first live stream, and the resolution has not changed compared to before the microphone connection. For another example, the video screen of the first anchor stream and the video screen of the second live stream can also be horizontally spliced ​​to obtain the initial microphone connection screen. In the initial microphone connection screen of this mode, the anchor and the microphone connection person are displayed side by side. If the resolution of the video screen of the first live stream is 720×640, and the resolution of the video screen of the second live stream is 720×640, then the resolution of the initial co-hosted screen is 1440×640. It can be seen that the resolution of the live stream to be transcoded has changed before and after the co-hosted video (720×640 before the co-hosted video and 1440×640 after the co-hosted video).

[0100] In this embodiment, by merging the first live stream and the second live stream, video images from different sources can be combined into an initial joint-microphone image for simultaneous viewing by viewers.

[0101] It can be seen that the resolution of the generated initial connected microphone screen (including anchors A and B) may change relative to the live broadcast screen before the connection (only anchor A). This resolution change will cause player compatibility issues. The following will exemplarily introduce the connected microphone video display method provided by this application through multiple embodiments to overcome the above problems.

[0102] Step S202 , performing a transcoding operation on the mixed live stream to obtain a live stream, wherein the live stream carries the supplementary enhancement information. The transcoding operation includes: obtaining the resolution of the initial connected microphone screen; comparing the resolution of the initial connected microphone screen with a preset resolution; if the resolution of the initial connected microphone screen is different from the preset resolution, adjusting the initial connected microphone screen to obtain a target connected microphone screen with the same resolution as the preset resolution, wherein the target connected microphone screen includes a target video screen; and updating the supplementary enhancement information according to the resolution of the target video screen.

[0103] As mentioned above, live broadcast and microphone connection may cause the resolution of the live stream to be transcoded before and after the microphone connection to change, and this change may cause player compatibility issues. Therefore, during the transcoding process, the resolution of the initial microphone connection screen can be obtained. The resolution of the initial microphone connection screen is compared with the preset resolution (target output resolution). The preset resolution can be set according to the host settings, client performance, and transcoding requirements. For example, the preset resolution can be the same as the resolution of the video screen of the first live stream, such as 720×640. If the initial microphone connection screen is the same as the preset resolution (picture-in-picture mode), there is no need to adjust the resolution of the initial microphone connection screen, and the initial microphone connection screen is determined as the target microphone connection screen output. If the initial microphone connection screen is not the same as the preset resolution (side by side display, etc.), the initial microphone connection screen can be adjusted according to the preset resolution, such as compression, stretching, cropping, etc., to obtain the target microphone connection screen (720×640) output with the same preset resolution. The target microphone connection screen can include a target video screen, and the target video screen corresponds to the adjusted initial microphone connection screen. The resolution of the target video screen is added to the SEI (Supplemental Enhancement Information) of the live stream.

[0104] In this embodiment, when a resolution change is detected, the initial microphone screen is adjusted according to the target output resolution to obtain a target microphone screen that can be output normally, thereby avoiding transcoding failure. At the same time, the adjusted resolution of the initial microphone screen is added to the SEI of the live stream, that is, the resolution information of the actual content area, which can subsequently assist the player in determining the resolution change of the live stream and accurately identifying the actual content area.

[0105] As mentioned above, there are many methods for adjusting the initial microphone connection screen. An exemplary solution is provided below.

[0106] In an optional embodiment, if Figure 5 As shown, step S202 may include:

[0107] Step S400: scaling the initial microphone connection picture according to the preset resolution to obtain the target video picture.

[0108] Step S402, setting a black border area at the edge of the scaled initial combined microphone picture to obtain the target combined microphone picture. The target combined microphone picture includes the scaled target video picture and the black border area.

[0109] For example, the preset resolution can be set to 720×640, and the resolution of the initial web-connected screen is 1440×640. It can be seen that the resolution of the initial web-connected screen is greater than the preset resolution, so the initial web-connected screen can be scaled. For example, the initial web-connected screen can be compressed proportionally to 720×320 to avoid distortion of the screen ratio. Since the compressed resolution is still different from the preset resolution, black borders can be added to the upper and lower parts of the compressed initial web-connected screen to form black border areas, and the resolution of each black border area can be 720×160. The target web-connected screen can be obtained by merging the initial web-connected screen and the black border area.

[0110] In this embodiment, by filling the black borders, a target microphone screen that meets the target output resolution can be obtained, and the integrity and correct display ratio of the video content can be maintained to avoid distortion of the screen due to stretching or cropping.

[0111] Step S204 , pushing the live stream to the client, the client is used to: obtain the target video picture from the live stream according to the supplementary enhancement information, and display it through a preset player.

[0112] The client can be the anchor terminal, the microphone terminal or the audience terminal. The server can push the live stream to the client and pass the resolution information given in the SEI carried by the live stream. After receiving the live stream, the client player can parse the SEI to obtain the resolution information of the target video screen. The player can first determine the current display mode (portrait mode or landscape mode). If it is landscape mode, the target video screen (actual content area) can be identified and extracted from the target microphone screen of the live stream according to the resolution information, and played and displayed to avoid the black edge area displayed in landscape mode affecting the audience experience. Of course, the player can also directly display the target microphone screen. Since the black edges are filled in advance, the picture will not be distorted. The portrait mode can also play only the actual content area or directly play the target microphone screen according to actual needs, which is not limited here.

[0113] In this embodiment, by parsing the SEI (supplemental enhancement information) of the live broadcast code stream and obtaining the resolution of the target video picture, the player can accurately identify the actual content area and play it, reduce the distortion and black edge problems of the picture ratio, adapt to the horizontal and vertical screen switching scenarios, and effectively improve the viewing experience of live broadcast and microphone connection.

[0114] The above embodiment introduces the video display method of the connected microphone. The following will further exemplarily introduce the video display of the scenarios such as disconnection of the connected microphone and direct switching point.

[0115] In an optional embodiment, if Figure 6 As shown, the method for displaying a video with microphones connected may further include:

[0116] Step S500: In response to the microphone connection disconnection request, obtain the latest first live broadcast stream.

[0117] Step S502, transcoding is performed on the latest first live stream, and the supplementary enhancement information is updated according to the resolution of the video picture of the latest first live stream to obtain the latest live stream, and the latest live stream carries the updated supplementary enhancement information.

[0118] Step S504: Push the latest live stream to the client, and the client is further used to: obtain the latest video picture of the first live stream from the latest live stream according to the updated supplementary enhancement information, and display it through a preset player.

[0119] In actual applications, if the host or microphone connection is disconnected, the server can obtain the latest live stream from the host and perform transcoding operations to obtain the latest live stream. At the same time, the SEI can be refreshed according to the latest video resolution of the first live stream, and the refreshed SEI can be added to the live stream. The latest live stream is pushed to the client, and the client can obtain the latest video of the first live stream by parsing the updated SEI and display it through the player.

[0120] In this embodiment, after the microphone connection is disconnected, the server can return to the initial state from the merged state, that is, re-acquire the live stream of the anchor to transcode and refresh the SEI to obtain the latest live stream. Correspondingly, the client can quickly obtain the video screen of the anchor according to the SEI and display it through the player. It can be seen that the transcoding process of this embodiment can quickly respond to the resolution changes before and after the microphone connection, further improving the viewing experience of the live broadcast.

[0121] In an optional embodiment, if Figure 7 The method for displaying a video with microphones may further include:

[0122] Step S600, recording the live stream and the supplementary enhancement information to generate a video-on-demand file.

[0123] Step S602: In response to the on-demand request, the on-demand file is pushed to the client.

[0124] In order to realize on-demand service, the server can also record the real-time live stream and synchronously record the supplementary enhancement information during the live broadcast to record the resolution changes and the actual content area, so as to provide a richer viewing experience when on demand. After the recording is completed, all the data is encapsulated into an on-demand file. The on-demand file can be a single media file (such as MP4) or a directory structure including multiple files (such as: M3U8 and related TS fragments). During the encapsulation process, the server can associate the live stream and the supplementary enhancement information to ensure accurate presentation during playback. In response to the on-demand request, the server can push the on-demand file to the client. The client parses the SEI in the on-demand file and plays the actual content area according to the resolution information therein, ignoring the black edges.

[0125] In this embodiment, the resolution information of the actual content area is added to the SEI of the live stream, and the live stream and SEI are recorded synchronously to generate a on-demand file, so that the client can accurately identify and play the actual content area according to the SEI when on demand, reducing the picture ratio distortion and black edge problems, being compatible with the direct-to-point scene, and further improving the on-demand viewing experience.

[0126] Embodiment 2

[0127] The embodiment of the present application takes the client (audience terminal, anchor terminal) as the execution body, and exemplarily introduces the method for displaying video with microphones provided by the present application.

[0128] Figure 8 The flowchart of the method for displaying video with microphones connected according to the second embodiment of the present application is schematically shown.

[0129] like Figure 8 As shown, the method for displaying a video with microphones connected may include steps S700 to S704, wherein:

[0130] Step S700, obtaining a live stream or on-demand file carrying supplementary enhancement information provided by a server, wherein the live stream or on-demand file includes an original video picture, the original video picture includes a target video picture and a black border area, and the supplementary enhancement information includes a resolution of the target video picture.

[0131] Step S702: adjusting the original video picture according to the supplemental enhancement information to obtain the target video picture.

[0132] Step S704: display the target video screen through a preset player.

[0133] The method for displaying video with microphone provided in this embodiment is such that the client can obtain the live stream or on-demand file carrying supplementary enhancement information provided by the server. By parsing the supplementary enhancement information, the resolution information of the target video screen can be obtained. Based on the resolution information, the client can adjust the original video screen of the live stream or on-demand file to obtain the target video screen (i.e., the actual content area). The target video screen is displayed through the player. It can be seen that the embodiment of the present application enables the player to accurately identify the actual content area and play it by obtaining the resolution information of the target video screen in the SEI (supplementary enhancement information) carried by the live stream. This can reduce the distortion and black edge problems of the screen ratio in scenes such as switching between horizontal and vertical screens and vertical rotation points, thereby improving the viewing experience of live broadcast with microphone or on-demand video.

[0134] There are many methods for adjusting the original video image, such as cropping, compression, stretching, etc. An exemplary solution is provided below.

[0135] In an optional embodiment, step S702 may include: determining the resolution of the target video picture according to the supplemental enhancement information. Identifying and extracting the target video picture from the original video picture according to the resolution of the target video picture.

[0136] Exemplarily, the client can obtain the resolution of the target video screen by parsing the supplementary enhancement information. The target video screen is the actual content area. According to the resolution of the target video screen, the target video screen and the black border area in the original video screen are identified. The black border area of ​​the original video screen is cropped or the target video screen is extracted from the original video screen.

[0137] In this embodiment, the actual content area can be accurately extracted from the live stream or on-demand file through the resolution information, ignoring the black edges, thereby effectively improving the viewing experience.

[0138] Embodiment 3

[0139] Fig. 9 The block diagram of the video display device with microphone connection according to the third embodiment of the present application is schematically shown. The device can be used in a server and can be divided into one or more program modules. One or more program modules are stored in a storage medium and executed by one or more processors to complete the embodiment of the present application. The program module referred to in the embodiment of the present application refers to a series of computer program instruction segments that can complete specific functions. The following description will specifically introduce the functions of each program module in this embodiment. Fig. 9 As shown, the device 1000 may include: an acquisition module 1100, a transcoding module 1200 and a push module 1300, wherein:

[0140] An acquisition module 1100 is used to acquire a mixed-image live stream in response to a microphone connection request, where the mixed-image live stream includes an initial microphone connection screen;

[0141] The transcoding module 1200 is used to perform a transcoding operation on the mixed-image live streaming stream to obtain a live streaming stream, where the live streaming stream carries supplementary enhancement information;

[0142] The transcoding operation includes: obtaining a resolution of the initial connected-microphone picture; comparing the resolution of the initial connected-microphone picture with a preset resolution; if the resolution of the initial connected-microphone picture is different from the preset resolution, adjusting the initial connected-microphone picture to obtain a target connected-microphone picture with the same resolution as the preset resolution, the target connected-microphone picture including a target video picture; and updating the supplementary enhancement information according to the resolution of the target video picture;

[0143] The push module 1300 is used to push the live stream to the client, and the client is used to obtain the target video picture from the live stream according to the supplementary enhancement information and display it through a preset player.

[0144] As an optional embodiment, obtaining the mixed-image live stream includes:

[0145] Get the first live stream of the anchor end and the second live stream of the microphone end;

[0146] Merging the first live stream and the second live stream to obtain the mixed-image live stream;

[0147] The merging includes: merging the video picture of the first live stream and the video picture of the second live stream to obtain the initial joint microphone picture.

[0148] As an optional embodiment, the microphone-connected video display device 1000 is further used for:

[0149] In response to the microphone disconnection request, obtaining the latest first live stream;

[0150] Transcoding the latest first live stream, and updating the supplementary enhancement information according to the resolution of the video picture of the latest first live stream to obtain the latest live stream, wherein the latest live stream carries the updated supplementary enhancement information;

[0151] The latest live stream is pushed to the client, and the client is further used to: obtain the latest video picture of the first live stream from the latest live stream according to the updated supplementary enhancement information, and display it through a preset player.

[0152] As an optional embodiment, adjusting the initial microphone connection screen includes:

[0153] Scaling the initial microphone connection picture according to the preset resolution to obtain the target video picture;

[0154] Setting a black border area at the edge of the initial combined microphone picture after scaling to obtain the target combined microphone picture;

[0155] The target microphone connection screen includes the zoomed target video screen and the black border area.

[0156] As an optional embodiment, the microphone-connected video display device 1000 is further used for:

[0157] Recording the live stream and the supplementary enhancement information to generate a video-on-demand file;

[0158] In response to the on-demand request, the on-demand file is pushed to the client.

[0159] Embodiment 4

[0160] Fig.10 The block diagram of the video display device with microphone connection according to the fourth embodiment of the present application is schematically shown. The device can be used in a client and can be divided into one or more program modules. One or more program modules are stored in a storage medium and executed by one or more processors to complete the embodiment of the present application. The program module referred to in the embodiment of the present application refers to a series of computer program instruction segments that can complete specific functions. The following description will specifically introduce the functions of each program module in this embodiment. Fig.10 As shown, the device 1000 may include: a first acquisition module 2100, a second acquisition module 2200 and a playback module 2300, wherein:

[0161] The first acquisition module 2100 is used to acquire a live stream or a video-on-demand file carrying supplementary enhancement information provided by a server, wherein the live stream or the video-on-demand file includes an original video picture, the original video picture includes a target video picture and a black border area, and the supplementary enhancement information includes a resolution of the target video picture;

[0162] A second acquisition module 2200 is used to adjust the original video picture according to the supplementary enhancement information to obtain the target video picture;

[0163] The playing module 2300 is used to display the target video screen through a preset player.

[0164] As an optional embodiment, adjusting the original video picture according to the supplemental enhancement information to obtain the target video picture includes:

[0165] Determining the resolution of the target video picture according to the supplemental enhancement information;

[0166] According to the resolution of the target video picture, the target video picture in the original video picture is identified and extracted.

[0167] Embodiment 5

[0168] Fig.11 The schematic diagram of the hardware architecture of a computer device 10000 suitable for implementing the method for displaying video through microphones according to the third embodiment of the present application is shown schematically. In some embodiments, the computer device 10000 may be a terminal device such as a smart phone, a wearable device, a tablet computer, a personal computer, a vehicle-mounted terminal, a game console, a virtual device, a workbench, a digital assistant, a set-top box, a robot, etc. In other embodiments, the computer device 10000 may be a rack server, a blade server, a tower server, or a cabinet server (including an independent server, or a server cluster composed of multiple servers), etc. Fig.11 As shown, the computer device 10000 includes but is not limited to: a memory 10010, a processor 10020, and a network interface 10030 that can communicate with each other through a system bus. Among them:

[0169] The memory 10010 includes at least one type of computer-readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (such as SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 10010 can be an internal storage module of the computer device 10000, such as a hard disk or memory of the computer device 10000. In other embodiments, the memory 10010 can also be an external storage device of the computer device 10000, such as a plug-in hard disk equipped on the computer device 10000, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. Of course, the memory 10010 can also include both the internal storage module of the computer device 10000 and its external storage device. In this embodiment, the memory 10010 is generally used to store an operating system and various application software installed in the computer device 10000, such as program codes of a method for displaying a video with microphones connected, etc. In addition, the memory 10010 can also be used to temporarily store various data that have been output or are to be output.

[0170] In some embodiments, the processor 10020 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other chips. The processor 10020 is generally used to control the overall operation of the computer device 10000, such as performing control and processing related to data interaction or communication with the computer device 10000. In this embodiment, the processor 10020 is used to run the program code stored in the memory 10010 or process data.

[0171] The network interface 10030 may include a wireless network interface or a wired network interface, and the network interface 10030 is generally used to establish a communication link between the computer device 10000 and other computer devices. For example, the network interface 10030 is used to connect the computer device 10000 to an external terminal through a network, and to establish a data transmission channel and a communication link between the computer device 10000 and the external terminal. The network can be a wireless or wired network such as an intranet, the Internet, the Global System of Mobile communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Bluetooth, Wi-Fi, etc.

[0172] It should be pointed out that Fig.11 Only a computer device having components 10010 - 10030 is shown, but it should be understood that implementing all of the components shown is not a requirement, and more or fewer components may alternatively be implemented.

[0173] In this embodiment, the method for displaying video with microphones stored in the memory 10010 may also be divided into one or more program modules and executed by one or more processors (such as the processor 10020) to complete the embodiment of the present application.

[0174] Embodiment 6

[0175] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, wherein when the computer program is executed by a processor, the steps of the method for displaying video with microphones in the embodiment are implemented.

[0176] In this embodiment, the computer-readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the computer-readable storage medium may be an internal storage unit of a computer device, such as a hard disk or memory of the computer device. In other embodiments, the computer-readable storage medium may also be an external storage device of a computer device, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Fl ash Card), etc. equipped on the computer device. Of course, the computer-readable storage medium may also include both the internal storage unit of the computer device and its external storage device. In this embodiment, the computer-readable storage medium is generally used to store an operating system and various application software installed on the computer device, such as the program code of the video display method of the microphone connection in the embodiment. In addition, the computer-readable storage medium may also be used to temporarily store various types of data that have been output or are to be output.

[0177] Embodiment 7

[0178] An embodiment of the present application also provides a computer program product, including a computer program, which implements the method in the above embodiment when executed by a processor.

[0179] Obviously, those skilled in the art should understand that the modules or steps of the above-mentioned embodiments of the present application can be implemented by general-purpose computer devices, they can be concentrated on a single computer device, or distributed on a network composed of multiple computer devices, optionally, they can be implemented by executable program codes of computer devices, so that they can be stored in a storage device and executed by the computer device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0180] It should be noted that the above are only preferred embodiments of the present application, and the patent protection scope of the present application is not limited thereto. Any equivalent structure or equivalent process transformation made using the contents of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for displaying a video with microphones connected, characterized in that: For a server, the method comprises: In response to the microphone connection request, obtaining a mixed-image live stream, where the mixed-image live stream includes an initial microphone connection picture; Performing a transcoding operation on the mixed-image live stream to obtain a live stream, wherein the live stream carries the supplementary enhancement information; wherein the transcoding operation includes: obtaining a resolution of the initial connected-microphone screen; comparing the resolution of the initial connected-microphone screen with a preset resolution; if the resolution of the initial connected-microphone screen is different from the preset resolution, adjusting the initial connected-microphone screen to obtain a target connected-microphone screen with the same resolution as the preset resolution, wherein the target connected-microphone screen includes a target video screen; and updating the supplementary enhancement information according to the resolution of the target video screen; The live stream is pushed to a client, and the client is used to obtain the target video picture from the live stream according to the supplementary enhancement information, and display it through a preset player.

2. The method according to claim 1, characterized in that The obtaining of the mixed-image live stream includes: Get the first live stream of the anchor end and the second live stream of the microphone end; Merging the first live stream and the second live stream to obtain the mixed-image live stream; The merging includes: merging the video picture of the first live stream and the video picture of the second live stream to obtain the initial joint microphone picture.

3. The method according to claim 2, characterized in that Also includes: In response to the microphone disconnection request, obtaining the latest first live stream; Transcoding the latest first live stream, and updating the supplementary enhancement information according to the resolution of the video picture of the latest first live stream to obtain the latest live stream, wherein the latest live stream carries the updated supplementary enhancement information; The latest live stream is pushed to the client, and the client is further used to: obtain the latest video picture of the first live stream from the latest live stream according to the updated supplementary enhancement information, and display it through a preset player.

4. The method according to claim 1, characterized in that: The initial microphone connection screen is adjusted, including: Scaling the initial microphone connection picture according to the preset resolution to obtain the target video picture; Setting a black border area at the edge of the initial combined microphone picture after scaling to obtain the target combined microphone picture; The target microphone connection screen includes the zoomed target video screen and the black border area.

5. The method according to claim 1, characterized in that Also includes: Recording the live stream and the supplementary enhancement information to generate a video-on-demand file; In response to the on-demand request, the on-demand file is pushed to the client.

6. A method for displaying a video with microphones connected, characterized in that: For a client, the method comprises: Acquire a live stream or a video-on-demand file carrying supplementary enhancement information provided by a server, wherein the live stream or the video-on-demand file includes an original video picture, the original video picture includes a target video picture and a black border area, and the supplementary enhancement information includes a resolution of the target video picture; According to the supplementary enhancement information, the original video picture is adjusted to obtain the target video picture; The target video screen is displayed through a preset player.

7. The method according to claim 6, characterized in that Adjusting the original video picture according to the supplemental enhancement information to obtain the target video picture includes: Determining the resolution of the target video picture according to the supplemental enhancement information; According to the resolution of the target video picture, the target video picture in the original video picture is identified and extracted.

8. A video display device with microphone connection, characterized in that: For a server, the device comprises: An acquisition module, configured to acquire a mixed-image live stream in response to a microphone connection request, wherein the mixed-image live stream includes an initial microphone connection picture; A transcoding module, configured to perform a transcoding operation on the mixed-image live stream to obtain a live stream, wherein the live stream carries the supplementary enhancement information; wherein the transcoding operation comprises: obtaining a resolution of the initial connected-microphone screen; comparing the resolution of the initial connected-microphone screen with a preset resolution; if the resolution of the initial connected-microphone screen is different from the preset resolution, adjusting the initial connected-microphone screen to obtain a target connected-microphone screen with the same resolution as the preset resolution, wherein the target connected-microphone screen comprises a target video screen; and updating the supplementary enhancement information according to the resolution of the target video screen; A push module is used to push the live stream to a client, and the client is used to obtain the target video picture from the live stream according to the supplementary enhancement information and display it through a preset player.

9. A video display device with microphones connected, characterized in that: For a client, the device comprises: A first acquisition module is used to acquire a live stream or a video-on-demand file carrying supplementary enhancement information provided by a server, wherein the live stream or the video-on-demand file includes an original video picture, the original video picture includes a target video picture and a black border area, and the supplementary enhancement information includes a resolution of the target video picture; A second acquisition module, configured to adjust the original video picture according to the supplementary enhancement information to obtain the target video picture; The playing module is used to display the target video screen through a preset player.

10. A computer device, characterized in that: include: at least one processor; and a memory communicatively coupled to the at least one processor; wherein: The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to claims 1 to 7 are implemented.

Images