Multi-picture audio and video playing method and device and storage medium

Abstract

The application discloses a multi-picture audio and video playing method and device and a storage medium. The application is related to the technology of blockchains. The application obtains video display data and at least two pieces of audio playing data of a target audio and video. The target audio and video is obtained by mixing at least two pieces of selected audio and video data of selected audio and videos. The at least two pieces of selected audio and video correspond to the at least two pieces of audio playing data. Based on the video display data, a video picture of the target audio and video is displayed. The video picture includes at least two selected video pictures of the at least two pieces of selected audio and video. Target audio playing data corresponding to a target selected video picture in the at least two selected video pictures is played. When a switching instruction for the target selected video picture is detected, switching audio playing data corresponding to a switching selected video picture is played. The switching selected video picture is a selected video picture different from the target selected video picture. The application can improve the data decoding speed.

Description

Technical Field

[0001] This application relates to the field of audio and video, specifically to multi-screen audio and video playback methods, devices, and storage media. Background Technology

[0002] Nowadays, the size of various display screens (such as monitors, smart TVs, smart screens, etc.) is getting larger and larger. Based on these large-sized display screens, some new use cases have gradually emerged. For example, multi-screen viewing can be performed on the display screen at the same time, that is, at least two video frames (one video frame can correspond to one program) can be displayed on one display screen at the same time, and the audio data corresponding to at least one target video frame can be played. In the process of realizing multi-screen viewing, the existing technology will have the terminal system where the display screen is located decode the video data of each program at the same time.

[0003] In the process of researching and practicing the existing technology, the inventors of this application discovered that, limited by the hardware performance of the terminal system where the display screen is located, the existing technology can only achieve a small number of video images to be displayed at the same time, or the clarity of the multiple video images displayed at the same time is low. That is, the existing technology cannot achieve high-quality multi-screen viewing. Summary of the Invention

[0004] This application provides a method, apparatus, and storage medium for multi-screen audio and video playback, which can improve the data decoding speed during multi-screen audio and video playback.

[0005] This application provides a multi-screen audio and video playback method, including:

[0006] Acquire video display data of the target audio and video and at least two audio playback data, wherein the target audio and video is obtained by mixing the selected audio and video data of at least two selected audio and video, and the at least two selected audio and video correspond one-to-one with the at least two audio playback data;

[0007] Based on the video display data, the video frame of the target audio and video is displayed, and the video frame includes at least two selected video frames of the at least two selected audio and video channels;

[0008] Play the target audio playback data corresponding to the target selected video frame in the at least two selected video frames;

[0009] When a switching instruction for the target selected video frame is detected, the switching audio playback data corresponding to the switching selected video frame is played, wherein the switching selected video frame is a selected video frame that is different from the target selected video frame.

[0010] Specifically, this application also provides a multi-screen audio and video playback device, including:

[0011] The acquisition module is used to acquire video display data of the target audio and video and at least two audio playback data. The target audio and video is obtained by mixing the selected audio and video data of at least two selected audio and video, and the at least two selected audio and video correspond one-to-one with the at least two audio playback data.

[0012] The screen display module is used to display the video frame of the target audio and video based on the video display data, wherein the video frame includes at least two selected video frames of the at least two selected audio and video channels;

[0013] The audio playback module is used to play the target audio playback data corresponding to the target selected video frame in the at least two selected video frames;

[0014] The switching module is used to play switching audio playback data corresponding to the selected video frame when a switching instruction for the target selected video frame is detected, wherein the selected video frame to be switched is a selected video frame that is different from the target selected video frame.

[0015] In some embodiments, the multi-screen audio and video playback device further includes a first determining module, which is specifically used for:

[0016] The video display data is subjected to image recognition to determine at least two selected video frames contained in the target audio and video, as well as the image boundary information of each selected video frame.

[0017] In some embodiments, the multi-screen audio / video playback device further includes a second determining module, which includes a query submodule and a determining submodule, wherein...

[0018] The query submodule is used to query the screen display strategy corresponding to the target audio and video based on the quantity of the audio playback data;

[0019] The determination submodule is used to determine, based on the screen display strategy, at least two selected video frames contained in the target audio and video, and the screen boundary information of each selected video frame.

[0020] In some embodiments, the screen display module is specifically used for:

[0021] Based on the video display data and the screen boundary information, the video frame of the target audio and video is displayed. The video frame includes at least two selected video frames and a boundary identifier for each selected video frame. The at least two selected video frames include the target selected video frame, and the boundary identifier of the target selected video frame is different from the boundary identifiers of the other selected video frames.

[0022] In some embodiments, the audio playback module is specifically used for:

[0023] The receiver sends a video-audio mapping relationship, which includes a mapping relationship between at least two selected video frames of the target audio-video and at least two audio playback frequency data.

[0024] From the image-audio mapping relationship, determine the target audio playback data corresponding to the selected video image; play the target audio playback data.

[0025] In some embodiments, the selected video frame corresponds to a target boundary marker, and the switching module includes a detection submodule, a determination submodule, and a playback submodule, wherein...

[0026] The detection submodule is used to detect the currently selected video frame corresponding to the target boundary identifier;

[0027] The determination submodule is used to determine that the currently selected video screen is the switching selected video screen when the currently selected video screen is different from the target selected video screen, and to generate a switching instruction for the target selected video screen and the switching selected video screen.

[0028] The playback submodule is used to play the switching audio playback data corresponding to the selected video frame according to the switching instruction.

[0029] In some embodiments, the playback submodule is specifically used for:

[0030] According to the switching instruction, the playback of the target audio data is stopped via the audio playback interface;

[0031] The audio playback interface is used to play the switching audio playback data corresponding to the selected video frame.

[0032] In some embodiments, the multi-screen audio and video playback device further includes:

[0033] The acquisition module is used to acquire the identifiers of at least two selected audio and video channels determined by the object;

[0034] The sending module is used to send the at least two selected audio and video identifiers to the server, so that the server performs mixed streaming processing on the at least two selected audio and video data corresponding to the selected audio and video identifiers to obtain the video display data of the target audio and video and the at least two audio playback data.

[0035] In some embodiments, the acquisition module is specifically used for:

[0036] Display an audio / video selection page, which includes multiple audio / video icons to be selected;

[0037] In response to the selection operation for the plurality of candidate audio and video identifiers, at least two selected audio and video identifiers determined by the object are obtained.

[0038] In some embodiments, the acquisition module is specifically used for:

[0039] Receive the target audio and video data sent by the server;

[0040] The audio and video data are separated to obtain video encoded data and at least two audio encoded data.

[0041] The video encoded data and the audio encoded data are decoded respectively to obtain the video display data of the target audio and video and at least two audio playback data.

[0042] This application also provides a method for playing multi-screen audio and video, including:

[0043] The selected audio and video identifiers of at least two selected audio and video channels sent by the receiving terminal;

[0044] Based on the at least two selected audio and video identifiers, obtain at least two selected audio and video data corresponding to the at least two selected audio and video;

[0045] The selected audio and video data are mixed to obtain the target audio and video video display data and at least two audio playback data, so that the terminal can perform multi-screen audio and video playback based on the video display data and at least two audio playback data.

[0046] Accordingly, this application also provides a multi-screen audio and video playback device, including:

[0047] The receiving module is used to receive the selected audio and video identifiers of at least two selected audio and video channels sent by the terminal;

[0048] The data acquisition module is used to acquire at least two selected audio and video data corresponding to the at least two selected audio and video based on the at least two selected audio and video identifiers;

[0049] The mixing module is used to perform mixing processing on the at least two selected audio and video data to obtain video display data of the target audio and video and at least two audio playback data, so that the terminal can perform multi-screen audio and video playback based on the video display data and at least two audio playback data.

[0050] In some embodiments, the data acquisition module of the multi-screen audio / video playback device is specifically used for:

[0051] Based on the at least two selected audio and video identifiers, send data acquisition requests to at least two service servers corresponding to the at least two selected audio and video;

[0052] Receive at least two selected audio and video data returned by the at least two service servers, wherein the selected audio and video data is returned by the service servers based on the data acquisition request.

[0053] Accordingly, this application also provides a storage medium storing a computer program adapted for loading by a processor to execute any of the multi-screen audio and video playback methods provided in this application.

[0054] Accordingly, this application also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the multi-screen audio and video playback methods provided in this application.

[0055] This application acquires video display data of the target audio and video and at least two audio playback data. The target audio and video is obtained by mixing the selected audio and video data of at least two selected audio and video streams, and the at least two selected audio and video streams correspond one-to-one with the at least two audio playback data streams. Based on the video display data, the application displays the video frame of the target audio and video, which includes at least two selected video frames of the at least two selected audio and video streams. The application plays the target audio playback data corresponding to the target selected video frame in the at least two selected video frames. When a switching command for the target selected video frame is detected, the application plays the switching audio playback data corresponding to the switching selected video frame, and the switched selected video frame is a selected video frame different from the target selected video frame.

[0056] This application can receive selected audio / video identifiers for at least two selected audio / video channels sent by a terminal; obtain at least two selected audio / video data corresponding to the at least two selected audio / video channels based on the at least two selected audio / video identifiers; and perform mixed streaming processing on the at least two selected audio / video data to obtain video display data and at least two audio playback data for the target audio / video channel, enabling the terminal to perform multi-screen audio / video playback based on the video display data and at least two audio playback data. This application can also perform mixed streaming of selected audio / video data from multiple selected audio / video channels on a server to obtain video display data and at least two audio playback data for the target audio / video channel. In the process of implementing multi-screen audio / video playback for the target audio / video channel, the terminal only needs to decode the video display data. Compared to the prior art, which requires decoding the data for each selected audio / video channel, this application can significantly improve the data decoding speed in the multi-screen audio / video playback process. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0058] Figure 1 This is a scene diagram of the multi-screen audio and video playback system of this application;

[0059] Figure 2 This is a flowchart illustrating the multi-screen audio and video playback method of this application;

[0060] Figure 3 This is another flowchart illustrating the multi-screen audio and video playback method of this application;

[0061] Figure 4 This is another flowchart illustrating the multi-screen audio and video playback method of this application;

[0062] Figure 5 This is a partial flowchart of the multi-screen audio and video playback method of this application;

[0063] Figure 6 This is another part of the flowchart of the multi-screen audio and video playback method of this application;

[0064] Figure 7 This is another part of the flowchart of the multi-screen audio and video playback method of this application;

[0065] Figure 8 This is a schematic diagram of the structure of the multi-screen audio and video playback device of this application;

[0066] Figure 9 This is another structural schematic diagram of the multi-screen audio and video playback device of this application;

[0067] Figure 10 This is a schematic diagram of the computer device described in this application. Detailed Implementation

[0068] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described in this application are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0069] The multi-screen audio and video playback method of this application can be integrated into a multi-screen audio and video playback device, which can be integrated into a multi-screen audio and video playback system. This system can include one or more computer devices, such as terminals or servers. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. Terminals can be smartphones, tablets, laptops, desktop computers, smart speakers, smartwatches, smart TVs, smart screens, etc., but are not limited to these. Terminals and servers can be directly or indirectly connected via wired or wireless communication, which is not limited herein.

[0070] See Figure 1 The multi-screen audio and video playback system may include a terminal 0001 and a server 0002. The multi-screen audio and video playback device may be integrated into the terminal 0001. Specifically, the multi-screen audio and video playback device may acquire video display data of the target audio and video and at least two audio playback data. The target audio and video is obtained by mixing the selected audio and video data of at least two selected audio and video streams, and the at least two selected audio and video streams correspond one-to-one with the at least two audio playback data streams. Based on the video display data, the device may display the video frame of the target audio and video, which includes at least two selected video frames of the at least two selected audio and video streams. The device may play the target audio playback data corresponding to the target selected video frame among the at least two selected video frames. When a switching command for the target selected video frame is detected, the device may play the switching audio playback data corresponding to the switching selected video frame, and the switched selected video frame is a selected video frame different from the target selected video frame.

[0071] Terminal 0001 can also obtain at least two selected audio and video identifiers for the target and send them to server 0002. Server 0002 can perform mixed streaming processing on the at least two selected audio and video data corresponding to the received selected audio and video identifiers to obtain the target audio and video video display data and at least two audio playback data. Server 0002 then sends the video display data and at least two audio display data to terminal 0001, enabling terminal 0001 to perform multi-screen audio and video playback.

[0072] It should be noted that, Figure 1The schematic diagram of the multi-screen audio and video playback system shown is merely an example. The multi-screen audio and video playback system and scenario described in this application embodiment are for the purpose of more clearly illustrating the technical solutions of this application embodiment and do not constitute a limitation on the technical solutions provided in this application embodiment. As those skilled in the art will know, with the evolution of multi-screen audio and video playback devices and the emergence of new business scenarios, the technical solutions provided in this application embodiment are also applicable to similar technical problems.

[0073] The following sections provide detailed explanations. In this embodiment, a multi-screen audio and video playback method will be described in detail. This method can be executed by a terminal or a server, or by both. This embodiment uses the example of the multi-screen audio and video playback method being executed by a terminal for illustration. Figure 2 As shown, Figure 2 This is a flowchart illustrating a multi-screen audio and video playback method provided in an embodiment of this application. The multi-screen audio and video playback method may include:

[0074] 101. Obtain the video display data of the target audio and video and at least two audio playback data. The target audio and video is obtained by mixing the selected audio and video data of at least two selected audio and video, and at least two selected audio and video correspond one-to-one with at least two audio playback data.

[0075] In this application, the target audio and video can be a collective term for multiple selected audio and video from different sources simultaneously displayed on the display screen. The selected audio and video can be multimedia data containing at least one of video and sound. For example, the target audio and video can be a collective term for news programs and TV dramas simultaneously displayed on the display screen. The target audio and video can contain multiple selected audio and video from various sources, such as video software (e.g., short video playback software, movie playback software), live streaming software (e.g., game live streaming, e-commerce live streaming), TV channels, etc. The multiple selected audio and video from a single target audio and video can have the same or different sources, and the selection can be flexibly chosen according to the needs of the user.

[0076] The video display data can include the data required to display video images on the display screen. For example, if the target audio and video includes three sources: online game live stream, news TV, and sports TV, then the video display data can be the data required to simultaneously display the video images of the online game live stream, the news TV, and the sports TV on the display screen. Through this video display data, selected video images of multiple selected audio and video included in the target audio and video can be displayed on the display screen. In this application, the target audio and video corresponds to at least one video display data. The video display data can be sent from a computer device (such as a server) to the terminal where the display screen is located. The terminal where the display screen is located only needs to process this video display data and display the video image of the target audio and video together with the audio playback data on the display screen. Since the hardware performance of the terminal where the display screen is located is limited, and processing video display data with higher display quality (such as higher resolution) requires better hardware performance and network environment, while this application only needs to process one video display data, it can process video display data with higher display quality under limited hardware performance conditions, thereby realizing high-quality multi-screen viewing. In addition, in some embodiments, the video display data can also be generated by the terminal where the display screen is located. The specific settings can be flexibly configured in actual applications, and will not be elaborated here.

[0077] The audio playback data may include the audio data corresponding to each selected audio / video file in the target audio / video file.

[0078] Specifically, the terminal where the display screen is located sends a data acquisition request to the server and receives the target audio and video data returned by the server based on the data acquisition request. The target audio and video data includes video display data and at least two audio playback data.

[0079] For example, target audio / video 1 can be a collective term for four audio / video files, namely: video 1, video 2, video 3 and video 4. It can receive video display data 1 and four audio playback data (including audio playback data 1, audio playback data 2, audio playback data 3 and audio playback data 4) sent by the server.

[0080] In some embodiments, the multi-screen audio and video playback method may further include the following steps:

[0081] Obtain the identifiers of at least two selected audio and video channels determined by the object;

[0082] Send at least two selected audio and video identifiers to the server so that the server can perform mixed streaming processing on the at least two selected audio and video data corresponding to the selected audio and video identifiers to obtain the video display data of the target audio and video and at least two audio playback data.

[0083] In this application, the object may include users who use the terminal where the display screen is located to view multiple screens simultaneously (such as the user object), as well as developers / testers who test and verify the multi-screen audio and video playback process of the terminal where the display screen is located during the development and testing process, and test cases, intelligent testing equipment, etc.

[0084] In this application, the user of the display screen can determine at least two audio and video files that they want to view simultaneously on multiple screens. The selected audio and video file identifier can include unique identification information of the selected audio and video file determined by the user. After receiving the selected audio and video file identifier, the terminal where the display screen is located can send the selected audio and video file identifier to the computer device (such as a server), so that the computer device can perform related operations based on the selected audio and video file identifier.

[0085] The selected audio and video data can include relevant data of the audio and video corresponding to the selected audio and video identifier. This data can be obtained by the server from other servers providing the selected audio and video data, signal generating devices, etc., via a transmission medium. Compared to the terminal where the display device is located, the server's powerful performance allows it to process multiple selected audio and video data streams efficiently, quickly, and with high quality. For example, it can decode each selected audio and video data stream to obtain the video and audio playback data for each stream. Then, the server performs mixed streaming processing on the video data of all the audio and video data to obtain the video display data for the target audio and video.

[0086] This process integrates multiple video data into at least one video display data, enabling multi-screen viewing by allowing the terminal on the display screen to process only the video display data. This eliminates the need for the terminal on the display screen to decode multiple audio and video data. It takes into account the actual performance of the terminal on the display screen, while the powerful performance of the server side can efficiently process more selected audio and video data. This also allows users to see clearer video images and more video images during multi-screen viewing, thus effectively improving the viewing quality of multi-screen viewing.

[0087] For example, the terminal where the display screen is located can receive the user's specified "LA Live", "XX News", "News Channel 3" and "Movie Channel 6". Then, the terminal where the display screen is located sends these four selected audio and video identifiers to the server. The server can obtain the selected audio and video data corresponding to these four selected audio and video identifiers, namely Live 1, TV Series 2, News 3 and Movie 4, and perform decoding, mixing and other processing on these four selected audio and video data to obtain the video display data of the target audio and video 1 and four audio playback data. Then, the data can be encoded, multiplexed and other processing to obtain the video display data of the target audio and video and four audio playback data.

[0088] In some embodiments, the step "obtaining at least two selected audio / video identifiers determined by the object" may include:

[0089] Displays an audio / video selection page, which includes multiple audio / video icons to be selected;

[0090] In response to a selection operation for multiple candidate audio and video identifiers, at least two selected audio and video identifiers determined by the object are obtained.

[0091] To make it easier for users to identify the audio and video identifiers they want to view simultaneously on multiple screens, a selection of audio and video identifiers can be displayed on the display screen, allowing users to choose from them and send the selected audio and video identifier to the terminal where the display screen is located.

[0092] The audio and video selection page may include a page that allows selection of audio and video identifiers. The audio and video selection page may include multiple candidate audio and video identifiers. The candidate audio and video identifiers may include audio and video identifiers that can be selected by the user. The user can select from multiple candidate audio and video identifiers. The selected audio and video identifier may include the identifier of the audio and video to be viewed on multiple screens by the user.

[0093] The selection of objects can be done in various ways. For example, you can set up an edit box, select controls, check boxes, buttons, sliders, etc. on the audio and video selection page. When using objects, you can enter or select the audio and video identifier in the edit box, or check the check box corresponding to the selected audio and video identifier, and so on.

[0094] In some embodiments, the step "acquiring video display data of the target audio / video and at least two audio playback data" may include:

[0095] Receive the target audio and video data sent by the server;

[0096] The audio and video data are separated to obtain video encoded data and at least two audio encoded data streams;

[0097] The video encoding data and audio encoding data are decoded separately to obtain the video display data of the target audio and video and at least two audio playback data.

[0098] In order to facilitate data transmission via wired and wireless means, the terminal where the display screen is located needs to process the target audio and video data sent by the server in order to obtain the video display data of the target audio and video and at least two audio playback data.

[0099] The audio and video data can include target audio and video data received from computer devices (such as servers). In order to transmit efficiently, the data sender will combine the relevant data of the target audio and video together. In order to ultimately achieve multi-screen viewing, the audio and video data first needs to be separated to obtain the different types of data contained therein. The separation processing method can be flexibly selected according to the actual use needs. For example, the audio and video data can be parsed, or the demultiplexing interface in the video player can be called, etc.

[0100] The video encoded data and at least two audio encoded data can be the data obtained after separating and processing the audio and video data. Since the video display data and audio playback data actually used for audio and video playback usually occupy a lot of memory, in order to facilitate transmission, save network resources, and ensure data security, the video display data and audio playback data can be encoded before data transmission to obtain video encoded data and audio encoded data. Correspondingly, after obtaining the video encoded data and audio encoded data, the terminal where the display screen is located needs to decode them separately to obtain the target audio and video video display data and at least two audio playback data. The decoding process can be based on a certain compression and decoding standard. For example, audio compression and decoding standards can include AAC (an audio compression and decoding standard), MP3 (an audio compression and decoding standard), AC-3 (an audio compression and decoding standard), etc., and video compression and decoding standards can include H.264 (a video compression and decoding standard), MPEG2 (a video compression and decoding standard), VC-1 (a video compression and decoding standard), etc. The actual decoding process can include various methods, such as calling the decoding interface to perform audio decoding or video decoding.

[0101] Because all selected audio and video data within the target audio and video are mixed, and the video data of each selected audio and video is combined into a single video display data, in order for the target audio and video to play normally, the user can switch between multiple screens to display the selected target video. After receiving the video display data and audio playback data, the terminal where the display screen is located needs to determine the number of selected audio and video data contained in the target audio and video based on this data. Specifically, there are several ways to confirm the number of selected audio and video data, such as sending a request to the server and receiving the number of selected audio and video data contained in the target audio and video returned by the server, etc.

[0102] The video frame can include the frame of the target audio and video displayed on the display screen. Typically, the display screen will show the selected video frame of each selected audio and video. Therefore, the number of selected audio and video frames of the target audio and video can be equivalent to the number of selected video frames.

[0103] Each selected video frame corresponds to an audio playback data point. There are several ways to determine the correspondence between selected video frames and audio playback data. For example, the correspondence between selected video frames and audio playback data within the target audio / video file can be obtained from the server.

[0104] For example, based on the configuration data of the target audio and video obtained, it can be determined that the target audio and video 1 includes four video frames, which are located in area 1, area 2, area 3 and area 4 of the display screen, respectively. The video frames in these four areas correspond to audio playback data 1, audio playback data 2, audio playback data 3 and audio playback data 4, respectively.

[0105] In some embodiments, the multi-screen audio and video playback method may include the following steps:

[0106] Perform image recognition on the video display data to determine at least two selected video frames contained in the target audio and video, as well as the image boundary information of each selected video frame.

[0107] Specifically, image recognition technology can be used to identify the video frames presented by the video display data, and to identify at least two selected video frames contained in the target audio and video, as well as the position information of each selected video frame within the video frame.

[0108] The image boundary information can include the boundary position information of a single selected video frame within the video frame presented by the video display data. The image boundary information can be obtained through image recognition, or it can be determined based on the position information after the position information of each selected video frame has been determined.

[0109] For example, the video display data can be identified using image recognition model 1 to obtain the four video frames contained therein, as well as the position and boundary information of each video frame.

[0110] In some embodiments, the multi-screen audio and video playback method may further include:

[0111] Based on the amount of audio playback data, query the corresponding screen display strategy for the target audio and video;

[0112] Based on the display strategy, determine at least two selected video frames contained in the target audio and video, as well as the frame boundary information of each selected video frame.

[0113] In some embodiments, the server can perform streaming according to a strategy. Specifically, different display strategies can be used depending on the number of selected audio and video clips within the target audio and video. The display strategy determines the arrangement of all selected video clips on the target audio and video's video frame, thereby determining the position information of each selected video clip. For example, with four selected audio and video clips, they can be evenly arranged, displaying four selected video clips of equal area on the target audio and video's video frame. Or, with eight selected video clips, three selected video clips of equal area can be displayed in the left and right columns of the frame, and two selected video clips of equal area can be displayed in the middle column. The areas of these eight selected video clips can vary, for example, the area in the middle column can be relatively larger, and so on.

[0114] Correspondingly, on the terminal side where the display screen is located, the display strategy of the target audio and video can be determined by the number of selected audio and video within the target audio and video, and then the screen boundary information of each selected video frame contained in the target audio and video can be determined directly according to the screen display strategy.

[0115] For example, after obtaining four audio playback data, the screen display strategy 1 corresponding to the target audio and video can be queried based on the quantity 4, thereby directly determining the four video screens of the target audio and video, and the screen boundary information of each video screen.

[0116] 102. Based on the video display data, display the video frame of the target audio and video, the video frame including at least two selected video frames of at least two selected audio and video channels.

[0117] 103. Play the target audio playback data corresponding to the target selected video frame from at least two selected video frames.

[0118] During multi-screen audio and video playback, in order to ensure the viewing experience, although multiple selected video frames are displayed at the same time, only the audio playback data corresponding to one of the selected video frames will be played. The target selected video frame can include the selected video frame for audio playback. The target selected video frame can be preset by the system, such as the selected video frame in the upper left corner of the display screen. The target selected video frame can also be selected by the user of the display screen, such as the selected video frame in the lower right corner of the display screen, and so on.

[0119] For example, based on video display data 1, four video frames of target audio and video 1 can be displayed, and target audio playback data corresponding to video frame 2 can be played: audio playback data 2.

[0120] In some embodiments, the step of "displaying the video frame of the target audio and video" may include:

[0121] Based on video display data and screen boundary information, the target audio and video video frame is displayed. The video frame includes at least two selected video frames and a boundary identifier for each selected video frame. Among them, at least two selected video frames include the target selected video frame, and the boundary identifier of the target selected video frame is different from the boundary identifiers of the other selected video frames.

[0122] Specifically, the video display can show the target audio and video frame, which includes at least two selected video frames. The boundaries of the selected video frames can be marked using boundary markers based on the frame boundary information. The boundary markers can include information to prompt the user about the boundaries of the selected video frames. Generally, the boundaries between different selected video frames are obvious, and there is no need to use boundary markers for overly obvious boundary marking. However, when multiple selected video frames are displayed simultaneously, it is usually necessary to highlight the target selected video frame so that the viewer can quickly identify the video frame corresponding to the playing sound. Therefore, in some embodiments, only the target selected video frame may be marked with boundaries.

[0123] For example, see [link / reference] Figure 1 In the video, video frame 1 is marked with a relatively obvious black border (a type of boundary marker), while video frames 2, 3, and 4 are marked with gray borders (a type of boundary marker).

[0124] In some embodiments, the step "playing the target audio playback data corresponding to the target selected video frame in at least two selected video frames" may include:

[0125] The system receives the video-audio mapping relationship sent by the server. The video-audio mapping relationship includes the mapping relationship between at least two selected video frames of the target audio and video and at least two audio playback data. From the video-audio mapping relationship, the system determines the target audio playback data corresponding to the selected video frames. The system then plays the target audio playback data.

[0126] The video-audio mapping relationship includes the mapping relationship between at least two selected video frames of the target audio and video and at least two audio playback data. For example, selected video frame 1 corresponds to audio playback data 1, and selected video frame 2 corresponds to audio playback data 2. The video-audio mapping relationship can be determined and recorded by the server during the mixing process. After determining the target selected video frame on the display screen, the target audio playback data corresponding to the target selected video frame can be determined according to the mapping relationship, and the audio playback data can be played synchronously with the video frame.

[0127] 104. When a switching command for the target selected video screen is detected, play the switching audio playback data corresponding to the selected video screen, and switch the selected video screen to a selected video screen that is different from the target selected video screen.

[0128] During the multi-screen video playback process of this application, the user can switch back and forth between selected video screens. The selected video screen to be switched is a selected video screen that is different from the target selected video screen among the multiple selected video screens displayed on the screen. The selected video screen to be switched can be a new target selected video screen. The terminal where the display screen is located can synchronously play the switching audio playback data corresponding to the switched selected video screen. The switching audio playback data is audio playback data that is different from the target audio playback data among the multiple audio playback data.

[0129] The switching instructions can include those generated by the terminal where the display screen is located. These instructions can be automatically generated by the terminal, for example, switching sequentially between four video frames of the target audio / video at set time intervals, with the terminal playing the corresponding audio data for each of the four video frames in turn. Alternatively, the switching instructions can be generated by the terminal based on the switching operation of the user. The switching operation varies depending on the type of display screen. For instance, on a touch screen, the switching operation can be tapping (light touch, double tap, etc.), swiping, or it can be implemented using a mouse, remote control, etc. For example, when a switching instruction to switch from video frame 1 to video frame 2 is detected, the terminal plays the audio data corresponding to video frame 2.

[0130] In some embodiments, the target video frame corresponds to a target boundary marker, and the step "when a switching instruction for the target selected video frame is detected, play the switching audio playback data corresponding to the switching selected video frame, and switch the selected video frame to a selected video frame different from the target selected video frame" may include:

[0131] Detect the currently selected video frame corresponding to the target boundary marker;

[0132] When the currently selected video frame is different from the target selected video frame, determine that the currently selected video frame is the switching selected video frame, and generate a switching instruction for the target selected video frame and the switching selected video frame;

[0133] Based on the switching command, play the switching audio data corresponding to the selected video frame.

[0134] Specifically, the target boundary marker can move automatically or as the user operates. The terminal where the display screen is located can detect whether the target boundary marker has changed at a set time interval. Specifically, it can detect the currently selected video frame corresponding to the target boundary marker at the detection time. If the currently selected video frame is different from the target selected video marker, it is considered that the target boundary marker has moved, and it is necessary to switch to play the audio playback data corresponding to the currently selected video frame.

[0135] During multi-screen viewing, multiple switching operations can be performed, meaning the currently selected video frame will change multiple times. By comparing the currently selected video frame with the selected video frame, it is determined whether the target boundary marker has changed. In fact, the selected video frame where the target boundary marker was previously located can be either the target selected video frame or not. Therefore, after each time the target boundary marker is determined to change, the currently selected video frame can be determined as the selected video frame to switch to.

[0136] Specifically, the step "play the switching audio playback data corresponding to the selected video frame according to the switching instruction" may further include:

[0137] The system continuously detects the currently selected video frame corresponding to the target boundary marker. When the currently selected video frame is different from the switchable selected video frame, the system determines that the currently selected video frame is the new switchable selected video frame and generates a switching instruction for the target selected video frame and the new switchable selected video frame. Based on the switching instruction, the system plays the switching audio playback data corresponding to the new switchable selected video frame until the multi-screen video playback ends.

[0138] In some embodiments, the step "playing the switching audio playback data corresponding to the selected video frame according to the switching instruction" may include:

[0139] According to the switching command, stop playing the target audio playback data through the audio playback interface;

[0140] The audio playback interface plays the corresponding audio playback data for switching video frames.

[0141] Specifically, in order to ensure that different audio playback data can be switched quickly during multi-screen video playback and to avoid phenomena such as audio dropout, the target audio playback data or the switching audio playback data corresponding to the switched video screen can be stopped through the audio interface according to the switching command, and then the switching audio playback data corresponding to the switched video screen or the new switching audio playback data corresponding to the new switched video screen can be played through the audio interface.

[0142] This embodiment can also be illustrated by taking the multi-screen audio and video playback method executed by the server as an example, such as... Figure 3 As shown, Figure 3This is a flowchart illustrating a multi-screen audio and video playback method provided in an embodiment of this application. The multi-screen audio and video playback method may include:

[0143] 201. Selected audio and video identifiers for at least two selected audio and video channels sent by the receiving terminal.

[0144] The selected audio / video identifier may include unique identification information of the selected audio / video determined by the user. After receiving the selected audio / video identifier, the terminal where the display screen is located can send the selected audio / video source identifier to the computer device (such as a server), so that the computer device can perform related operations based on the selected audio / video identifier.

[0145] 202. Based on the identifiers of at least two selected audio and video channels, obtain the data of at least two selected audio and video channels corresponding to at least two selected audio and video channels.

[0146] The selected audio and video data can include relevant data of the audio and video corresponding to the selected audio and video identifier. This data can be obtained by the server from other servers providing the selected audio and video data, signal generating devices, etc., via a transmission medium. Compared to the terminal where the display device is located, the server's powerful performance allows it to process multiple selected audio and video data streams efficiently, quickly, and with high quality. For example, it can decode each selected audio and video data stream to obtain the video and audio playback data for each stream. Then, the server performs mixed streaming processing on the video data of all the audio and video data to obtain the video display data for the target audio and video.

[0147] In some embodiments, the step "obtaining at least two selected audio and video data corresponding to at least two selected audio and video based on at least two selected audio and video identifiers" may include:

[0148] Based on at least two selected audio and video identifiers, send data acquisition requests to at least two business servers corresponding to at least two selected audio and video; receive at least two selected audio and video data returned by at least two business servers, wherein the selected audio and video data is returned by the business servers based on the data acquisition requests.

[0149] The business server may include a server that provides selected audio and video data. After receiving the selected audio and video identifier, it can send a data acquisition request to the business server corresponding to the selected audio and video identifier. The business server will return the selected audio and video data corresponding to the selected audio and video identifier based on the data acquisition request.

[0150] 203. Mix at least two selected audio and video data streams to obtain the target audio and video video display data and at least two audio playback data streams, so that the terminal can perform multi-screen audio and video playback based on the video display data and at least two audio playback data streams.

[0151] This process integrates multiple video data into at least one video display data, enabling multi-screen viewing by allowing the terminal on the display screen to process only the video display data. This eliminates the need for the terminal on the display screen to decode multiple audio and video data. It takes into account the actual performance of the terminal on the display screen, while the powerful performance of the server side can efficiently process more selected audio and video data. This also allows users to see clearer video images and more video images during multi-screen viewing, thus effectively improving the viewing quality of multi-screen viewing.

[0152] This application can mix selected audio and video data from multiple selected audio and video streams on a server to obtain video display data of the target audio and video and at least two audio playback data. When the terminal implements multi-screen audio and video playback for the target audio and video, it only needs to decode the video display data. Compared with the prior art, which requires decoding the data of each selected audio and video stream, this application can significantly improve the data decoding speed of the multi-screen audio and video playback process.

[0153] Based on the methods described in the above embodiments, the following examples will provide further detailed explanations.

[0154] This application will use a multi-screen audio and video playback system integrated with a server and a terminal as an example to introduce a multi-screen audio and video playback method. The multi-screen audio and video playback system may include a terminal and a server, such as... Figure 4 As shown, Figure 4 This is a flowchart illustrating a multi-screen audio and video playback method provided in an embodiment of this application. The multi-screen audio and video playback method may include:

[0155] 301. The terminal obtains at least two selected audio and video identifiers determined by the object and sends the at least two selected audio and video identifiers to the server.

[0156] For example, the terminal can display a multi-screen viewing page, which contains many stream identifiers (audio and video identifiers to be selected) for the user to choose from. The user can select the target stream identifier that they want to view on multiple screens (i.e., select the audio and video identifier). The user can trigger the multi-screen viewing button on the multi-screen viewing page, and the terminal can receive the multiple target stream identifiers selected by the user. The terminal can then send these multiple target stream identifiers to the server.

[0157] 302. The server performs mixed streaming processing on at least two selected audio and video data corresponding to at least two selected audio and video identifiers received, to obtain the video display data of the target audio and video, at least two audio playback data, and the picture and audio mapping relationship.

[0158] Specifically, after receiving multiple target bitstream identifiers, the server mixes the video frames of the multiple bitstreams corresponding to the target bitstream identifiers selected by the user into a single overall frame. The server then sends the resulting video display data and multiple audio display data to the terminal. During the mixing process, the video frames of different bitstreams correspond to different areas in the overall frame. The server then maps the video frames of different areas to their corresponding audio display data to obtain the target audio-visual frame-audio mapping relationship.

[0159] The streaming stream can be sent by the push-stream module, and the multi-screen audio and video playback system can receive the streaming stream sent by the push-stream module. The streaming stream can be in the form of popular live-streaming e-commerce or television channels from major satellite TV stations. The push-stream module can run on a computer device, such as a server.

[0160] The bitstream processing module is responsible for processing the acquired bitstream. Processing can include demultiplexing, decoding, mixing, etc. Mixing can generate a bitstream from multiple bitstreams. The final bitstream contains a total frame (such as a total frame composed of 4 / 6 / 8 frames) and multiple audio channels.

[0161] For example, see Figure 5 A multi-screen audio and video playback system can include a server and a terminal. The server receives and processes the bitstreams sent to it by four streaming ends to obtain the video display data and audio playback data of the target audio and video. The client is installed on the terminal, which can receive the video display data and audio playback data and perform multi-screen audio and video playback.

[0162] 303. Based on the video display data and at least two audio playback data received from the server, the terminal determines that the video frame of the target audio and video includes at least two selected video frames.

[0163] Specifically, there are many ways. For example, the number of selected video frames and the position of each selected video frame can be determined in the target audio and video video frame based on the image-audio mapping relationship. Another example is that image recognition can be performed on the video frame to obtain the number of selected video frames contained therein and their respective positions, and so on.

[0164] 304. The terminal displays a video frame of the target audio and video based on the video display data. The video frame includes at least two selected video frames, a boundary marker for each selected video frame, and target audio playback data corresponding to the target selected video frame among the at least two selected video frames. The target selected video frame corresponds to the target boundary marker.

[0165] 305. The terminal detects the currently selected video frame corresponding to the target boundary identifier.

[0166] 306. When the currently selected video screen is different from the target selected video screen, the terminal determines that the currently selected video screen is the switching selected video screen and generates a switching instruction for the target selected video screen and the switching selected video screen.

[0167] 307. The terminal plays the switching audio playback data corresponding to the selected video frame according to the switching instruction.

[0168] This application allows for the installation of a client on the terminal, enabling multi-screen audio and video playback on the terminal side via the client. (See also...) Figure 6 If a user chooses to watch multiple screens simultaneously on the client, they can add their favorite programs on the client page (i.e., confirm the selection of audio and video). The client can send data to the server and receive data returned by the server based on the user's operation. The client can initialize the multi-screen page and play multi-screen audio and video based on the data returned by the server.

[0169] This application allows switching between video feeds during multi-screen audio and video playback. For details, please refer to... Figure 7 The server can establish a correspondence between the focus and the audio channel (i.e., the image-audio mapping relationship). The server can send this correspondence to the terminal. The terminal can obtain the current focus on the display screen (i.e., the target boundary identifier) ​​and determine whether the focus has changed. If it has changed, the audio of the current focus is turned off (i.e., the target audio playback data or the switching audio playback data), and the server switches to the changed audio channel and turns on the audio of the changed focus for audio playback (i.e., playing the switched audio playback data or the new switched audio playback data through the audio playback interface). If the focus has not changed, the previous audio and video playback continues.

[0170] This application can mix selected audio and video data from multiple selected audio and video streams on a server to obtain video display data of the target audio and video and at least two audio playback data. When the terminal implements multi-screen audio and video playback for the target audio and video, it only needs to decode the video display data. Compared with the prior art, which requires decoding the data of each selected audio and video stream, this application can significantly improve the data decoding speed of the multi-screen audio and video playback process.

[0171] To facilitate better implementation of the multi-screen audio and video playback method provided in this application, this application also provides an apparatus based on the above-described multi-screen audio and video playback method. The meanings of the terms used are the same as in the above-described multi-screen audio and video playback method, and specific implementation details can be found in the descriptions within the method embodiments.

[0172] like Figure 8 Show, Figure 8This is a schematic diagram of the structure of a multi-screen audio and video playback device provided in an embodiment of this application. The multi-screen audio and video playback device may include an acquisition module 401, a screen display module 402, an audio playback module 403, and a switching module 404.

[0173] The acquisition module 401 is used to acquire the video display data of the target audio and video and at least two audio playback data. The target audio and video is obtained by mixing the selected audio and video data of at least two selected audio and video, and the at least two selected audio and video correspond one-to-one with the at least two audio playback data.

[0174] The screen display module 402 is used to display the video frame of the target audio and video based on the video display data. The video frame includes at least two selected video frames of at least two selected audio and video channels.

[0175] The audio playback module 403 is used to play target audio playback data corresponding to the target selected video frame in at least two selected video frames;

[0176] The switching module 404 is used to play the switching audio playback data corresponding to the selected video frame when a switching instruction for the target selected video frame is detected, and the selected video frame to be switched is a different selected video frame from the target selected video frame.

[0177] In some embodiments, the multi-screen audio and video playback device further includes a first determining module, which is specifically used for:

[0178] Perform image recognition on the video display data to determine at least two selected video frames contained in the target audio and video, as well as the image boundary information of each selected video frame.

[0179] In some embodiments, the multi-screen audio / video playback device further includes a second determining module, which includes a query submodule and a determining submodule, wherein...

[0180] The query submodule is used to query the screen display strategy corresponding to the target audio and video based on the amount of audio playback data.

[0181] The determination submodule is used to determine, based on the screen display strategy, at least two selected video frames contained in the target audio and video, as well as the screen boundary information of each selected video frame.

[0182] In some embodiments, the screen display module is specifically used for:

[0183] Based on video display data and screen boundary information, the target audio and video video frame is displayed. The video frame includes at least two selected video frames and a boundary identifier for each selected video frame. Among them, at least two selected video frames include the target selected video frame, and the boundary identifier of the target selected video frame is different from the boundary identifiers of the other selected video frames.

[0184] In some embodiments, the audio playback module is specifically used for:

[0185] The receiving server sends a picture-audio mapping relationship, which includes the mapping relationship between at least two selected video frames of the target audio and video and at least two audio playback frequency data.

[0186] From the image-audio mapping relationship, determine the target audio playback data corresponding to the selected video image;

[0187] Play target audio playback data.

[0188] In some embodiments, the target selected video frame corresponds to a target boundary marker, and the switching module includes a detection submodule, a determination submodule, and a playback submodule, wherein...

[0189] The detection submodule is used to detect the currently selected video frame corresponding to the target boundary marker;

[0190] The determination submodule is used to determine the currently selected video frame as the switching selected video frame when the currently selected video frame is different from the target selected video frame, and to generate switching instructions for the target selected video frame and the switching selected video frame;

[0191] The playback submodule is used to play the corresponding audio playback data for the selected video frame according to the switching command.

[0192] In some embodiments, the playback submodule is specifically used for:

[0193] According to the switching command, stop playing the target audio playback data through the audio playback interface;

[0194] The audio playback interface plays the switching audio data corresponding to the selected video frame.

[0195] In some embodiments, the multi-screen audio and video playback device further includes:

[0196] The acquisition module is used to acquire the identifiers of at least two selected audio and video channels determined by the object;

[0197] The sending module is used to send at least two selected audio and video identifiers to the server, so that the server can perform mixed streaming processing on the at least two selected audio and video data corresponding to the selected audio and video identifiers to obtain the video display data of the target audio and video and at least two audio playback data.

[0198] In some embodiments, the acquisition module is specifically used for:

[0199] Displays an audio / video selection page, which includes multiple audio / video icons to be selected;

[0200] In response to a selection operation for multiple candidate audio and video identifiers, at least two selected audio and video identifiers determined by the object are obtained.

[0201] In some embodiments, the acquisition module is specifically used for:

[0202] Receive the target audio and video data sent by the server;

[0203] The audio and video data are separated to obtain video encoded data and at least two audio encoded data streams;

[0204] The video encoding data and audio encoding data are decoded separately to obtain the video display data of the target audio and video and at least two audio playback data.

[0205] In this embodiment, the acquisition module 401 acquires the video display data of the target audio and video and at least two audio playback data. The target audio and video is obtained by mixing the selected audio and video data of at least two selected audio and video. The at least two selected audio and video correspond one-to-one with the at least two audio playback data. The screen display module 402 displays the video frame of the target audio and video based on the video display data. The video frame includes at least two selected video frames of the at least two selected audio and video. The audio playback module 403 plays the target audio playback data corresponding to the target selected video frame in the at least two selected video frames. When the switching module 404 detects a switching command for the target selected video frame, it plays the switching audio playback data corresponding to the switching selected video frame. The switching selected video frame is a selected video frame different from the target selected video frame.

[0206] This application can mix selected audio and video data from multiple selected audio and video streams on a server to obtain video display data of the target audio and video and at least two audio playback data. When the terminal implements multi-screen audio and video playback for the target audio and video, it only needs to decode the video display data. Compared with the prior art, which requires decoding the data of each selected audio and video stream, this application can significantly improve the data decoding speed of the multi-screen audio and video playback process.

[0207] This application also provides an apparatus based on the above-described multi-screen audio and video playback method. The meanings of the terms used are the same as in the above-described multi-screen audio and video playback method, and specific implementation details can be found in the descriptions within the method embodiments.

[0208] like Figure 9 Show, Figure 9 This is a schematic diagram of the structure of a multi-screen audio and video playback device provided in an embodiment of this application. The multi-screen audio and video playback device may include a receiving module 501, a data acquisition module 502, and a mixing module 503.

[0209] The receiving module 501 is used to receive the selected audio and video identifiers of at least two selected audio and video channels sent by the terminal;

[0210] The data acquisition module 502 is used to acquire at least two selected audio and video data corresponding to at least two selected audio and video based on at least two selected audio and video identifiers;

[0211] The mixing module 503 is used to perform mixing processing on at least two selected audio and video data to obtain video display data of the target audio and video and at least two audio playback data, so that the terminal can perform multi-screen audio and video playback based on the video display data and at least two audio playback data.

[0212] In some embodiments, the data acquisition module of the multi-screen audio / video playback device is specifically used for:

[0213] Based on at least two selected audio and video identifiers, send data acquisition requests to at least two business servers corresponding to at least two selected audio and video;

[0214] Receive at least two selected audio and video data streams returned by at least two business servers. The selected audio and video data are returned by the business servers based on the data acquisition request.

[0215] In this embodiment, the receiving module 501 receives the selected audio and video identifiers of at least two selected audio and video channels sent by the terminal. The data acquisition module 502 acquires the selected audio and video data corresponding to the selected audio and video channels based on the selected audio and video identifiers. The mixing module 503 performs mixing processing on the selected audio and video data to obtain the video display data of the target audio and video and the audio playback data of at least two channels, so that the terminal can perform multi-screen audio and video playback based on the video display data and the audio playback data of at least two channels.

[0216] This application can mix selected audio and video data from multiple selected audio and video streams on a server to obtain video display data of the target audio and video and at least two audio playback data. When the terminal implements multi-screen audio and video playback for the target audio and video, it only needs to decode the video display data. Compared with the prior art, which requires decoding the data of each selected audio and video stream, this application can significantly improve the data decoding speed of the multi-screen audio and video playback process.

[0217] Furthermore, embodiments of this application also provide a computer device, which can be a terminal or a server, such as... Figure 10 As shown, it illustrates a structural schematic diagram of the computer device involved in the embodiments of this application, specifically:

[0218] The computer device may include components such as a processor 601 with one or more processing cores, a memory 602 with one or more computer-readable storage media, a power supply 603, and an input unit 604. Those skilled in the art will understand that... Figure 10 The computer device structure shown does not constitute a limitation on the computer device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0219] The processor 601 is the control center of the computer device. It connects various parts of the computer device via various interfaces and lines, and performs various functions and processes data by running or executing software programs and / or modules stored in the memory 602, and by calling data stored in the memory 602, thereby providing overall monitoring of the computer device. Optionally, the processor 601 may include one or more processing cores; preferably, the processor 601 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user page, and application programs, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 601.

[0220] The memory 602 can be used to store software programs and modules. The processor 601 executes various functional applications and data processing by running the software programs and modules stored in the memory 602. The memory 602 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 602 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 602 may also include a memory controller to provide the processor 601 with access to the memory 602.

[0221] The computer device also includes a power supply 603 that supplies power to the various components. Preferably, the power supply 603 can be logically connected to the processor 601 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 603 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0222] The computer device may also include an input unit 604, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0223] Although not shown, the computer device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 601 in the computer device loads the executable files corresponding to the processes of one or more application programs into the memory 602 according to the following instructions, and the processor 601 runs the application programs stored in the memory 602 to realize various functions, as follows:

[0224] The system acquires video display data of the target audio / video and at least two audio playback data streams. The target audio / video is obtained by mixing the selected audio / video data of at least two selected audio / video streams, and there is a one-to-one correspondence between the at least two selected audio / video streams and the at least two audio playback data streams. Based on the video display data, the system displays the video frame of the target audio / video, which includes at least two selected video frames from the at least two selected audio / video streams. The system plays the target audio playback data corresponding to the target selected video frame from the at least two selected video frames. When a switching command for the target selected video frame is detected, the system plays the switching audio playback data corresponding to the switching selected video frame, and the switched selected video frame is a different selected video frame from the target selected video frame.

[0225] The system receives at least two selected audio / video identifiers from the receiving terminal; based on these identifiers, it obtains at least two selected audio / video data corresponding to each identifier; and it performs mixed-stream processing on the selected audio / video data to obtain video display data and at least two audio playback data for the target audio / video, enabling the terminal to perform multi-screen audio / video playback based on the video display data and the at least two audio playback data. Specific implementation details for each of these operations can be found in the preceding embodiments and will not be repeated here.

[0226] As disclosed in this application, the multi-screen audio and video playback method or apparatus may include multiple servers forming a blockchain, with the servers being nodes on the blockchain.

[0227] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various optional implementations of the above embodiments.

[0228] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by a computer program, or by a computer program controlling related hardware. The computer program can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0229] Blockchain is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and cryptographic algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying platform, a platform product service layer, and an application service layer.

[0230] The underlying blockchain platform can include processing modules such as user management, basic services, smart contracts, and operational monitoring. The user management module is responsible for managing the identity information of all blockchain participants, including maintaining public and private key generation (account management), key management, and maintaining the correspondence between user real identities and blockchain addresses (access management). Furthermore, under authorization, it monitors and audits transactions of certain real identities and provides risk control rule configuration (risk control audit). The basic services module is deployed on all blockchain node devices to verify the validity of business requests. After consensus is reached on valid requests, they are recorded in storage. For a new business request, the basic services first perform interface adaptation parsing and authentication (interface adaptation), and then encrypt the business information through a consensus algorithm (consensus management). After encryption, the data is transmitted completely and consistently to the shared ledger (network communication) and recorded and stored. The smart contract module is responsible for contract registration, issuance, triggering, and execution. Developers can define contract logic using a programming language and publish it to the blockchain (contract registration). According to the contract terms, the key or other events are invoked to trigger execution and complete the contract logic. It also provides functions for contract upgrades and cancellations. The operation monitoring module is mainly responsible for deployment, configuration modification, contract settings, cloud adaptation, and real-time status visualization output during product release, such as alarms, monitoring network conditions, and monitoring the health status of node devices.

[0231] The platform's product service layer provides the basic capabilities and implementation frameworks for typical applications. Developers can leverage these basic capabilities, along with the specific characteristics of their business needs, to implement blockchain-based business logic. The application service layer provides blockchain-based application services to business stakeholders.

[0232] Therefore, embodiments of this application also provide a storage medium storing a computer program that can be loaded by a processor to execute the steps of any of the multi-screen audio / video playback methods provided in embodiments of this application. For example, the computer program can execute the following steps:

[0233] The system acquires video display data of the target audio / video and at least two audio playback data streams. The target audio / video is obtained by mixing the selected audio / video data of at least two selected audio / video streams, and there is a one-to-one correspondence between the at least two selected audio / video streams and the at least two audio playback data streams. Based on the video display data, the system displays the video frame of the target audio / video, which includes at least two selected video frames from the at least two selected audio / video streams. The system plays the target audio playback data corresponding to the target selected video frame from the at least two selected video frames. When a switching command for the target selected video frame is detected, the system plays the switching audio playback data corresponding to the switching selected video frame, and the switched selected video frame is a different selected video frame from the target selected video frame.

[0234] The system receives at least two selected audio and video identifiers from the terminal; obtains at least two selected audio and video data corresponding to the at least two selected audio and video based on the at least two selected audio and video identifiers; performs mixed streaming processing on the at least two selected audio and video data to obtain video display data and at least two audio playback data of the target audio and video, so that the terminal can perform multi-screen audio and video playback based on the video display data and at least two audio playback data.

[0235] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0236] Since the computer program stored in the storage medium can execute the steps of any of the multi-screen audio and video playback methods provided in the embodiments of this application, the beneficial effects that any of the multi-screen audio and video playback methods provided in the embodiments of this application can achieve can be realized. For details, please refer to the previous embodiments, which will not be repeated here.

[0237] The multi-screen audio and video playback method, apparatus, and storage medium provided in the embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A multi-picture audio-video playing method, characterized in that, The method comprises: acquiring at least two selected audio-video identifiers determined by an object, the selected audio-video identifier being unique identifier information of a selected audio-video; sending the selected audio-video identifier to a server, so that the server requests each selected audio-video data based on the selected audio-video identifier and decodes it, respectively, to obtain video data and audio playback data of each selected audio-video data, query a picture display strategy corresponding to a target audio-video according to the number of audio playback data, perform mixed flow processing on the video data in all the audio-video data based on the picture display strategy to obtain video display data of the target audio-video, so that the arrangement of the selected video picture on the video picture of the target audio-video conforms to the picture display strategy, and perform mixed flow processing on the audio playback data in the audio-video data; obtaining audio-video data of the target audio-video from the server; performing separation processing on the audio-video data to obtain video encoding data and at least two audio encoding data; decoding the video encoding data and the audio encoding data, respectively, to obtain video display data of the target audio-video and at least two audio playback data, the target audio-video being obtained by mixed flow of selected audio-video data of at least two selected audio-videos, the at least two selected audio-videos corresponding to the at least two audio playback data one by one; querying a picture display strategy corresponding to the target audio-video according to the number of audio playback data, the picture display strategy representing the arrangement of the selected video picture on the video picture of the target audio-video, and the picture audio mapping relationship including the mapping relationship between at least two selected video pictures of the target audio-video and at least two audio playback data; determining at least two selected video pictures included in the target audio-video and picture boundary information of each selected video picture based on the picture display strategy; displaying the video picture of the target audio-video based on the video display data and the picture boundary information, the video picture including at least two selected video pictures of the at least two selected audio-videos and boundary identifiers of each selected video picture, wherein the at least two selected video pictures include a target selected video picture, the target selected video picture corresponding to a target boundary identifier, and the target boundary identifier being different from the boundary identifiers of other selected video pictures; playing target audio playback data corresponding to the target selected video picture in the at least two selected video pictures based on the picture audio mapping relationship; detecting a current selected video picture corresponding to the target boundary identifier; when the current selected video picture is different from the target selected video picture, determining that the current selected video picture is a switching selected video picture, and generating a switching instruction for the target selected video picture and the switching selected video picture. When a switching instruction for the target selected video picture is detected, the switching selected video picture corresponding switching audio playing data is played, the switching selected video picture is a selected video picture different from the target selected video picture.

2. The method of claim 1, wherein, The method further comprises: Picture recognition is performed on the video display data to determine at least two selected video pictures contained in the target audio video and picture boundary information of each selected video picture.

3. The method of claim 1, wherein, The playing of the target audio playing data corresponding to the target selected video picture in the at least two selected video pictures comprises: The picture audio mapping relationship is received, and the picture audio mapping relationship comprises a mapping relationship between at least two selected video pictures of the target audio video and at least two audio playing data; The target audio playing data corresponding to the target selected video picture is determined from the picture audio mapping relationship; The target audio playing data is played.

4. The method of claim 1, wherein, The playing of the switching audio playing data corresponding to the switching selected video picture comprises: According to the switching instruction, the target audio playing data is stopped from being played through an audio playing interface; The switching audio playing data corresponding to the switching selected video picture is played through the audio playing interface.

5. The method of claim 1, wherein, The at least two selected audio video identifiers determined by the object comprise: An audio video selection page is displayed, and the audio video selection page comprises a plurality of to-be-selected audio video identifiers; In response to a selection operation on the plurality of to-be-selected audio video identifiers, at least two selected audio video identifiers determined by the object are obtained.

6. A multi-picture audio-video playing method, characterized in that, Comprise: The terminal sends at least two selected audio video identifiers of selected audio videos; Based on the selected audio video identifiers, each selected audio video data is requested from a service server corresponding to the at least two selected audio videos and is decoded to obtain video data and audio playing data of each selected audio video data, according to a number of the audio playing data, a picture display strategy corresponding to a target audio video is queried, based on the picture display strategy, the video data in all the audio video data is mixed and processed to obtain video display data of the target audio video, so that a selected video picture arrangement mode on a video picture of the target audio video conforms to the picture display strategy, and the audio playing data in the audio video data is mixed and processed; The terminal is sent the audio video data of the target audio video, so that the terminal performs multi-picture audio video playing based on the video display data and at least two audio playing data.

7. A multi-picture audio-video playing device, characterized in that, Comprise: An acquisition module is configured to acquire at least two selected audio video identifiers determined by an object, the selected audio video identifier being unique identifier information of a selected audio video; sending the selected audio-video identifiers to the server, so that the server requests each selected audio-video data from the service server corresponding to the at least two selected audio-videos based on the selected audio-video identifiers, decodes the selected audio-video data, and obtains video data and audio playback data of each selected audio-video data, queries a picture display strategy corresponding to the target audio-video according to the number of audio playback data, performs mixed flow processing on the video data in all audio-video data based on the picture display strategy, and obtains video display data of the target audio-video, so that the arrangement of the selected video pictures on the video picture of the target audio-video conforms to the picture display strategy, and performs mixed flow processing on the audio playback data in the audio-video data; obtaining audio-video data of the target audio-video from the server, and performing separation processing on the audio-video data to obtain video encoding data and at least two audio encoding data; decoding the video encoding data and the audio encoding data respectively to obtain video display data of the target audio-video and at least two audio playback data, the target audio-video being obtained by mixed flow of selected audio-video data of at least two selected audio-videos, the at least two selected audio-videos corresponding to the at least two audio playback data one by one; a picture display module configured to query a picture display strategy corresponding to the target audio-video according to the number of audio playback data, the picture display strategy representing an arrangement of the selected video pictures on the video picture of the target audio-video, the picture-audio mapping relationship including a mapping relationship between at least two selected video pictures of the target audio-video and at least two audio playback data, and determine at least two selected video pictures included in the target audio-video and picture boundary information of each selected video picture based on the picture display strategy, and display the video picture of the target audio-video based on the video display data and the picture boundary information, the video picture including at least two selected video pictures of the at least two selected audio-videos and a boundary identifier of each selected video picture, wherein the at least two selected video pictures include a target selected video picture, the target selected video picture corresponds to a target boundary identifier, and the target boundary identifier is different from boundary identifiers of other selected video pictures; an audio playback module configured to play target audio playback data corresponding to a target selected video picture in the at least two selected video pictures based on the picture-audio mapping relationship, and detect a current selected video picture corresponding to the target boundary identifier, and determine the current selected video picture as a switching selected video picture when the current selected video picture is different from the target selected video picture, and generate a switching instruction for the target selected video picture and the switching selected video picture; a switching module configured to play switching audio playback data corresponding to a switching selected video picture when the switching instruction for the target selected video picture is detected, the switching selected video picture being a selected video picture different from the target selected video picture.

8. A multi-picture audio-video playing device, characterized in that, comprising: The receiving module is configured to receive selected audio-video identifiers of at least two selected audios and videos sent by the terminal. The data obtaining module is configured to request and decode each selected audio-video data based on the selected audio-video identifiers, to obtain video data and audio playing data of each selected audio-video data, by respectively sending a request to a service server corresponding to each selected audio-video. The mixed flow module is configured to query a picture display strategy corresponding to a target audio-video based on the number of the audio playing data, to perform mixed flow processing on the video data in all the audio-video data based on the picture display strategy, to obtain video display data of the target audio-video, so that an arrangement of the selected video picture on a video picture of the target audio-video conforms to the picture display strategy, to perform mixed flow processing on the audio playing data in the audio-video data, and to send audio-video data of the target audio-video to the terminal, so that the terminal performs multi-picture audio-video playing based on the video display data and at least two audio playing data.

9. A storage medium, characterized by The storage medium stores a plurality of computer programs, and the computer programs are adapted to be loaded by the processor to perform the steps in the method of any one of claims 1 to 6.

10. A computer device, comprising: The processor executes the computer program to implement the steps in the method of any one of claims 1 to 6.

11. A computer program product, comprising computer instructions stored in a computer readable storage medium; a processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to perform the steps in the method of any one of claims 1 to 6.

Images