Method, device, medium and electronic equipment for playing interactive multimedia information

Abstract

The application discloses a playing method and device of interactive multimedia information, a medium and an electronic device. The method comprises the following steps: when detecting an interactive operation for a current multimedia segment in a current multimedia information code stream, generating description information of a target multimedia segment in a target multimedia information code stream according to the interactive operation; the current multimedia information code stream and the target multimedia information code stream are two different multimedia information code streams corresponding to the same multimedia information; the target multimedia segment is continuous with the current multimedia segment in time; searching for target access information of the target multimedia segment in a manifest file corresponding to the multimedia information according to the description information of the target multimedia segment; and playing the target multimedia segment according to the target access information. The application can realize smooth switching between different code streams in an interactive video playing process, and effectively avoid the occurrence of a freezing phenomenon in the interactive process.

Description

Technical Field

[0001] This application belongs to the field of multimedia technology, and specifically relates to an interactive multimedia information playback method, device, medium and electronic device. Background Technology

[0002] With the rapid development of internet technology, multimedia information disseminated via the internet has also evolved, giving rise to a new type of multimedia information: interactive multimedia information, such as interactive video. Interactive video allows viewers to interact with the video during playback, enhancing their immersion. This type of interactive video playback is typically segment-by-segment, where an interactive action is triggered while playing one video segment, and the next segment is determined based on the specific interaction. However, when switching between video segments is frequent, existing interactive video playback methods are prone to causing playback stuttering.

[0003] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0004] The purpose of this application is to provide a method, apparatus, medium, and electronic device for playing interactive multimedia information, so as to optimize the problem of video playback stuttering that easily occurs during interactive video playback in related technologies.

[0005] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part from practice of this application.

[0006] According to one aspect of the embodiments of this application, a method for playing interactive multimedia information is provided, including:

[0007] When an interactive operation is detected targeting the current multimedia segment in the current multimedia information stream, description information of the target multimedia segment in the target multimedia information stream is generated based on the interactive operation; the current multimedia information stream and the target multimedia information stream are two different multimedia information streams corresponding to the same multimedia information, and the target multimedia segment and the current multimedia segment are continuous in time;

[0008] Based on the description information of the target multimedia segment, the target access information of the target multimedia segment is searched in the manifest file corresponding to the multimedia information; the manifest file includes access information of each multimedia segment of the multimedia information;

[0009] The target multimedia segment is obtained and played based on the target access information.

[0010] According to one aspect of the embodiments of this application, an interactive multimedia information playback device is provided, comprising:

[0011] An interactive operation module is used to generate description information of a target multimedia segment in a target multimedia information stream based on an interactive operation detected for a current multimedia segment in the current multimedia information stream; the current multimedia information stream and the target multimedia information stream are two different multimedia information streams corresponding to the same multimedia information, and the target multimedia segment and the current multimedia segment are continuous in time.

[0012] The target access information determination module is used to search for the target access information of the target multimedia segment in the manifest file corresponding to the multimedia information based on the description information of the target multimedia segment; the manifest file includes the access information of each multimedia segment of the multimedia information;

[0013] The target multimedia segment playback module is used to obtain the target multimedia segment based on the target access information and play it.

[0014] In one embodiment of this application, the description information of the target multimedia segment includes a segment identifier of the target multimedia segment and a multimedia information stream identifier corresponding to the target multimedia segment; the manifest file includes a first manifest file and a second manifest file; the target access information determination module includes:

[0015] The first manifest file lookup unit is used to look up the second manifest file corresponding to the target multimedia segment in the first manifest file corresponding to the multimedia information according to the multimedia information bitstream identifier corresponding to the target multimedia segment; the first manifest file includes the correspondence between the multimedia information bitstream identifier and the second manifest file.

[0016] The second manifest file lookup unit is used to look up the target access information of the target multimedia segment in the second manifest file according to the segment identifier of the target multimedia segment; the second manifest file includes the correspondence between the segment identifier of the multimedia segment and the access information of the multimedia segment.

[0017] In one embodiment of this application, the apparatus further includes:

[0018] A multimedia information acquisition module is used to acquire multiple multimedia information bitstreams corresponding to the multimedia information;

[0019] The multimedia information parsing module is used to parse each multimedia information bitstream to obtain multiple multimedia segments included in each multimedia information bitstream; wherein, each multimedia information bitstream includes the same number of multimedia segments.

[0020] The second manifest file generation module is used to generate the second manifest file for each multimedia information bitstream based on the segment identifier and access information of each multimedia segment of the multimedia information bitstream.

[0021] The first manifest file generation module is used to generate the first manifest file based on the second manifest file corresponding to each multimedia information stream and the multimedia information stream identifier corresponding to each multimedia information stream.

[0022] In one embodiment of this application, the apparatus further includes:

[0023] The sorting module is used to sort the multiple multimedia segments of each multimedia information stream according to the playback time order;

[0024] The segment identifier generation module is used to generate a segment identifier for the multimedia segment based on the multimedia information stream identifier of the multimedia information stream and the sort number of the multimedia segment; wherein, in different multimedia information streams, the multimedia segments corresponding to the same sort number have the same playback duration.

[0025] In one embodiment of this application, the multimedia information acquisition module is specifically used for:

[0026] Obtain the multi-channel encoded data corresponding to the multimedia information from the server;

[0027] The multi-channel encoded data of the multimedia information is parsed to obtain multiple multimedia information bitstreams, wherein one channel of encoded data corresponds to one multimedia information bitstream.

[0028] In one embodiment of this application, the multimedia information acquisition module is specifically used for:

[0029] Obtain the encoded data of the multimedia information from the server;

[0030] The encoded data of the multimedia information is parsed to obtain multiple multimedia information streams.

[0031] In one embodiment of this application, the target multimedia segment playback module includes:

[0032] A target encoded data acquisition unit is used to acquire target encoded data of the target multimedia segment based on the target access information.

[0033] A decoding unit is used to perform a decoding operation on the target encoded data to obtain target decoded data corresponding to the target multimedia segment;

[0034] The rendering unit is used to render and display the target decoded data in order to play the target multimedia segment.

[0035] According to one aspect of the embodiments of this application, a computer-readable medium is provided, on which a computer program is stored, which, when executed by a processor, implements the interactive multimedia information playback method as described in the above technical solutions.

[0036] According to one aspect of the embodiments of this application, an electronic device is provided, the electronic device comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor executes the executable instructions to cause the electronic device to perform an interactive multimedia information playback method as described in the above technical solutions.

[0037] According to one aspect of the embodiments of this application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the interactive multimedia information playback method as described in the above technical solutions.

[0038] In the technical solution provided in this application embodiment, when an interactive operation is detected on the current multimedia segment in the current multimedia information bitstream, description information of the target multimedia segment in the target multimedia information bitstream is generated based on the interactive operation; then, the target access information of the target multimedia segment is searched in the manifest file corresponding to the multimedia information based on the description information of the target multimedia segment; finally, the target multimedia segment is obtained and played based on the target access information. Since the current multimedia information bitstream and the target multimedia information bitstream are two different multimedia information bitstreams corresponding to the same multimedia information, and the target multimedia segment and the current multimedia segment are continuous in time, the technical solution of this application can realize the switching between different bitstreams during multimedia information playback. During the bitstream switching process, since the manifest file includes the access information of each multimedia segment of the multimedia information, compared with the related technology where the terminal device needs to re-send a request to the server to obtain the access information of the target multimedia, this application can quickly obtain the access information of the target multimedia segment based on the manifest file, thereby making the playback of multimedia information smoother and more fluid during bitstream switching, effectively avoiding or reducing the occurrence of stuttering during interaction.

[0039] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0040] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0041] Figure 1 An exemplary system architecture block diagram illustrating the application of the technical solution of this application is shown schematically.

[0042] Figure 2 A schematic diagram illustrating an application scenario of the technical solution of this application is shown.

[0043] Figure 3 A flowchart illustrating an embodiment of the interactive multimedia information playback method provided in this application is shown schematically.

[0044] Figure 4 This illustration shows a schematic diagram of multiple multimedia information streams corresponding to multimedia information provided in one embodiment of this application.

[0045] Figure 5 This illustration shows a schematic diagram of multiple multimedia information streams corresponding to multimedia information provided in one embodiment of this application.

[0046] Figure 6 A flowchart illustrating an embodiment of the interactive multimedia information playback method provided in this application is shown schematically.

[0047] Figure 7 A schematic diagram of a manifest file provided in one embodiment of this application is shown.

[0048] Figure 8 A schematic diagram illustrating the structure of an interactive multimedia information playback system provided in an embodiment of this application is shown.

[0049] Figure 9 A schematic block diagram of the interactive multimedia information playback device provided in the embodiments of this application is shown.

[0050] Figure 10 A schematic block diagram of a computer system architecture for implementing an electronic device according to embodiments of the present application is shown. Detailed Implementation

[0051] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0052] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0053] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0054] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily need to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0055] Figure 1 An exemplary system architecture block diagram illustrating the application of the technical solution of this application is shown schematically.

[0056] like Figure 1 As shown, the system architecture 100 may include terminal devices 110, a network 120, and a server 130. Terminal devices 110 may include mobile phones, computers, smart voice interaction devices, smart home appliances, vehicle terminals, aircraft, etc. Server 130 may be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. Network 120 may be a communication medium of various connection types capable of providing a communication link between terminal devices 110 and server 130, such as a wired communication link or a wireless communication link.

[0057] Depending on the implementation requirements, the system architecture in this application embodiment can have any number of terminal devices, networks, and servers. For example, server 130 can be a server group composed of multiple server devices. In addition, the technical solutions provided in this application embodiment can be applied to terminal device 110, or to server 130, or can be implemented jointly by terminal device 110 and server 130. This application does not impose any special limitations on this.

[0058] For example, the technical solution provided in this application embodiment is implemented by terminal device 110. When terminal device 110 detects an interactive operation on a current multimedia segment in the current multimedia information stream, it generates description information of the target multimedia segment in the target multimedia information stream based on the interactive operation. The multimedia information includes multiple multimedia information streams, and terminal device 110 can obtain the required multimedia information stream from server 130 via network 120. The current multimedia information stream and the target multimedia information stream are two different multimedia information streams corresponding to the same multimedia information, and the target multimedia segment and the current multimedia segment are sequential in time. Then, terminal device 110 searches for target access information of the target multimedia segment in the manifest file corresponding to the multimedia information based on the description information of the target multimedia segment; the manifest file includes access information for each multimedia segment of the multimedia information. Finally, terminal device 110 obtains the target multimedia segment for playback based on the target access information.

[0059] For example, video as a multimedia information example will be used to illustrate the implementation process of the technical solution of this application. Figure 2 As shown, the video provided in this embodiment is obtained by shooting the same subject P (the same subject can also refer to the same shooting scene, which will not be elaborated hereafter), and different shooting angles are used to shoot the subject P during the shooting process, thereby generating multiple video streams. Figure 2 As shown, a shooting angle is a shooting position. In this embodiment, the video includes three video streams: video stream s0 corresponding to shooting position 0, video stream s1 corresponding to shooting position 1, and video stream s2 corresponding to shooting position 2.

[0060] Assume the video is played via a mobile phone. Currently playing is the video stream s0 corresponding to camera position 0. At time T1, the user interacts with the currently playing video via the mobile phone, wanting to adjust the camera angle to camera position 1. Based on this interaction, it can be determined that the current video segment is video segment s0-1 within video stream s0 corresponding to camera position 0, and the target video segment is video segment s1-2 within video stream s1 corresponding to camera position 1. This leads to the generation of descriptive information for the target video segment s1-2. This descriptive information describes the target video segment s1-2 to accurately identify which specific video segment it is. For example, this descriptive information could be the segment identifier "s1-2". Clearly, the current video segment s0-1 and the target video segment s1-2 are temporally continuous.

[0061] Next, the phone searches for the access information of the target multimedia segment in the manifest file based on the description information of the target multimedia segment. The phone stores the manifest file corresponding to the video, which may be provided by the application (APP) playing the video. The manifest file includes access information for video segments in each video stream of the video. The access information for the video segments is used to obtain the corresponding video segments, such as the storage address of the video segments.

[0062] Finally, the mobile phone obtains the target video segment based on the determined target access information and plays it, switching the video to be played from camera angle 0 to camera angle 1.

[0063] For example, such as Figure 2 As shown, assuming the object wants to switch the camera angle from 1 to 2 at time T2, the mobile phone detects the interactive operation and then performs the above operation again, that is, it obtains the target video segment s2-5 and plays it through the interactive multimedia information playback method provided in this application embodiment.

[0064] The technical solution of this application is applicable to the playback of any video information, such as online video program playback. It is suitable for application scenarios with high-frequency interaction, continuous response to object operations, and seamless quality changes. It can be widely used in application scenarios such as brand advertising videos, wide-angle or panoramic videos, continuously zoomable or zoomable screen resolution, and continuously increasing or decreasing video frame rate, bringing a more impactful video interactive experience. For example, when playing a product brand display video, the user can zoom in or out of the video image by clicking the "+" and "-" keys on the keyboard, or the volume "+" and "-" keys on a mobile phone or tablet, to achieve a more detailed or global view of the product display video.

[0065] The following detailed description of the playback of interactive multimedia information provided in this application, in conjunction with specific implementation methods, provides a detailed explanation.

[0066] Figure 3 The flowchart illustrating an embodiment of the interactive multimedia information playback method provided in this application is shown in the illustration. Figure 3 As shown, the method includes steps 310 to 330, as detailed below:

[0067] Step 310: When an interactive operation is detected targeting the current multimedia segment in the current multimedia information stream, a description of the target multimedia segment in the target multimedia information stream is generated based on the interactive operation; the current multimedia information stream and the target multimedia information stream are two different multimedia information streams corresponding to the same multimedia information, and the target multimedia segment and the current multimedia segment are continuous in time.

[0068] In this embodiment of the application, the multimedia information includes multiple multimedia information streams. Each multimedia information stream displays the multimedia information from a certain aspect. Therefore, the multimedia information corresponding to different multimedia information streams is the same.

[0069] In one embodiment of this application, multiple multimedia information streams are displayed from different quality levels. Assuming the multimedia information is video, the quality level of the multimedia information can be reflected from multiple aspects such as image resolution, image frame rate, texture fidelity, and sharpness. For example, the multimedia information may include smooth multimedia information streams, high-definition multimedia information streams, ultra-high-definition multimedia information streams, etc.

[0070] In one embodiment of this application, a layered coding technique is employed to provide multiple multimedia information bitstreams of different quality levels for the same multimedia information. For example, taking video as an example, Scalable Video Coding (SVC) technology is used as an extended coding algorithm tool within the video compression framework. This technology can provide several output bitstreams of variable quality for an input image sequence. The variable quality referred to here refers to the different quality levels mentioned above. For example, for dimensions such as image resolution, image frame rate, texture fidelity, and sharpness, different quality levels can be variations in one dimension or variations in multiple dimensions. For example, Figure 4 This diagram schematically illustrates a multi-channel multimedia information bitstream corresponding to multimedia information provided in one embodiment of this application. For example... Figure 4As shown, the base layer 0 represents the multimedia information bitstream 410 with the lowest quality level, the enhancement layer 1 represents the multimedia information bitstream 420 with increased frame rate after the base layer 0 is superimposed, and the enhancement layer 2 represents the multimedia information bitstream 430 with increased frame rate and image resolution after the base layer 0 and enhancement layer 1 are superimposed.

[0071] In one embodiment of this application, multiple multimedia information streams are displayed from different camera angles, where each camera angle refers to the angle of the camera device capturing the multimedia information. The multimedia information may include a multimedia information stream from a first camera angle, a multimedia information stream from a second camera angle, a multimedia information stream from a third camera angle, and so on. For example, Figure 5 This diagram schematically illustrates a multi-channel multimedia information bitstream corresponding to multimedia information provided in one embodiment of this application. For example... Figure 5 As shown, by shooting the same object P from three different camera positions, three multimedia information streams can be obtained: multimedia information stream s0 corresponding to camera position 0, multimedia information stream s1 corresponding to camera position 1, and multimedia information stream s2 corresponding to camera position 2.

[0072] In this embodiment, the current multimedia information stream refers to the multimedia information stream being played at the current moment. It is one of the multiple multimedia information streams included in the multimedia information. It can be the default multimedia information stream corresponding to the multimedia information, or it can be the multimedia information stream determined after the last interaction operation. One multimedia information stream includes multiple multimedia segments. Therefore, the current multimedia segment of the multimedia information is one of the multimedia segments in the multimedia information stream being played at the current moment.

[0073] Interactive operations on the current multimedia segment refer to operations performed by an object through a terminal device playing multimedia information, switching the current multimedia information stream to a target multimedia information stream. Examples include screen gestures, virtual buttons on the UI (User Interface), and gyroscope rotation on the terminal device. Figure 2 As shown, at time T1, the object wants to switch the video viewing angle from camera position 0 to camera position 1. The object can interact with the camera by using the virtual buttons on the video playback interface of the smartphone, or by rotating the smartphone.

[0074] When an interactive operation is detected targeting the current multimedia segment, it indicates that the object needs to switch the multimedia information stream. At this time, a description of the target multimedia segment is generated based on the interactive operation. This description is used to determine which specific multimedia segment needs to be switched. In this embodiment, the interactive operation is an operation generated for switching the multimedia information stream. Therefore, the current multimedia segment and the target multimedia segment are segments in different multimedia information streams. At the same time, the current multimedia segment and the target multimedia segment are continuous in time, thereby ensuring continuous playback of multimedia information.

[0075] In one embodiment of this application, when generating description information of a target multimedia segment based on an interactive operation, the target multimedia stream containing the target multimedia segment can first be determined based on the interactive operation, that is, the target multimedia stream to be switched as indicated by the interactive operation. Then, the multimedia segment in the target multimedia stream that is consecutive to the current multimedia segment in terms of playback time is taken as the target multimedia segment. For example, a reference multimedia segment with the same playback time as the current multimedia segment is found in the target multimedia stream, and then the multimedia segment in the target multimedia stream that is adjacent to and follows the reference multimedia segment is taken as the target multimedia segment.

[0076] Step 320: Based on the description information of the target multimedia segment, search for the target access information of the target multimedia segment in the manifest file corresponding to the multimedia information; the manifest file includes the access information of each multimedia segment of the multimedia information.

[0077] Specifically, the manifest file for multimedia information records access information for the multimedia segments included in each multimedia information stream. This access information is the information needed to retrieve the multimedia segment; for example, it may be the storage address of the multimedia segment, its ID information, etc. For instance, if the manifest file records the correspondence between the segment identifier and the storage address of a multimedia segment, then by searching for a match in the manifest file based on the segment identifier in the target multimedia description information, the storage address of the target multimedia segment can be obtained. This storage address is the target access information for the target multimedia segment.

[0078] Step 330: Obtain the target multimedia segment based on the target access information and play it.

[0079] Specifically, after determining the target access information, the corresponding target multimedia segment can be obtained based on the target access information, and then the target multimedia segment can be played, thereby realizing the process of switching from the current multimedia segment of the current multimedia information stream to the target multimedia segment of the target multimedia information stream.

[0080] In the technical solution provided in this application embodiment, when an interactive operation is detected on the current multimedia segment in the current multimedia information bitstream, description information of the target multimedia segment in the target multimedia information bitstream is generated based on the interactive operation; then, the target access information of the target multimedia segment is searched in the manifest file corresponding to the multimedia information based on the description information of the target multimedia segment; finally, the target multimedia segment is obtained and played based on the target access information. Since the current multimedia information bitstream and the target multimedia information bitstream are two different multimedia information bitstreams corresponding to the same multimedia information, and the target multimedia segment and the current multimedia segment are continuous in time, the technical solution of this application can realize the switching between different bitstreams during multimedia information playback. During the bitstream switching process, since the manifest file includes the access information of each multimedia segment of the multimedia information, compared with the related technology where the terminal device needs to re-send a request to the server to obtain the access information of the target multimedia, this application can quickly obtain the access information of the target multimedia segment based on the manifest file, thereby making the playback of multimedia information smoother and more fluid during bitstream switching, effectively avoiding or reducing the occurrence of stuttering during interaction.

[0081] Figure 6 A flowchart illustrating an embodiment of the interactive multimedia information playback method provided in this application is shown schematically. This application is a further refinement of the above embodiment. Figure 6 As shown, the method includes steps 610 to 660, as detailed below:

[0082] Step 610: When an interactive operation is detected targeting the current multimedia segment in the current multimedia information bitstream, a description of the target multimedia segment in the target multimedia information bitstream is generated based on the interactive operation; the description of the target multimedia segment includes the segment identifier of the target multimedia segment and the multimedia information bitstream identifier corresponding to the target multimedia segment.

[0083] Specifically, the multimedia information stream identifier corresponding to the target multimedia segment is the identifier of the target multimedia information stream in which the target multimedia segment resides. This multimedia information stream identifier is generated based on the identifier of the target multimedia information stream after the switch indicated by the interactive operation. For example, such as... Figure 2 As shown, when the interactive operation switches the camera position from 0 to 1, the multimedia information stream identifier corresponding to the target multimedia segment can be determined to be "s1".

[0084] The segment identifier of a target multimedia segment is used to identify the position of the target multimedia segment within the target multimedia information stream. For example, the multimedia segments included in the multimedia information stream can be numbered according to their playback time sequence, and then the number can be used as the segment identifier. When generating the segment identifier of the target multimedia segment based on the interactive operation, the segment identifier of the current multimedia segment is first determined, that is, the number of the current multimedia segment is determined, and then this number is incremented by 1, which is the segment identifier of the target multimedia segment. For example, if the segment identifier of the current multimedia segment is 1, then the segment identifier of the target multimedia segment is 2.

[0085] In the first embodiment of this application, the number of the multimedia segment in the multimedia information stream and the identifier of the multimedia information stream can be used together as the segment identifier of the multimedia segment. For example, the segment identifier of the multimedia segment is in the form of "multimedia information stream identifier-number". For example, the segment identifier of the multimedia segment is s0-0, which represents the multimedia segment with number 0 in the multimedia information stream s0. Then, when generating the segment identifier of the target multimedia segment, the target multimedia information stream identifier is first determined, and then the number representing the number in the segment identifier of the current multimedia segment is incremented by 1 to obtain the target number of the target multimedia segment. Then, the segment identifier of the target multimedia segment is generated according to the target multimedia information stream identifier and the target number. For example, if the segment identifier of the current multimedia segment is s0-1 and the target multimedia information stream identifier is s1, then the segment identifier of the target multimedia segment is s1-2.

[0086] Step 620: Based on the multimedia information stream identifier corresponding to the target multimedia segment, search for the second manifest file corresponding to the target multimedia segment in the first manifest file corresponding to the multimedia information; the first manifest file includes the correspondence between the multimedia information stream identifier and the second manifest file.

[0087] Step 630: Locate the target access information of the target multimedia segment in the second list file based on the segment identifier of the target multimedia segment; the second list file includes the correspondence between the segment identifier of the multimedia segment and the access information of the multimedia segment.

[0088] Specifically, the first manifest file defines the correspondence between multimedia information stream identifiers and the second manifest file, while the second manifest file defines the correspondence between multimedia segment identifiers and access information for those multimedia segments. When searching for a manifest file based on the description information of a target multimedia segment, the first manifest file is first searched based on the multimedia information stream identifier in the description information to determine the second manifest file corresponding to the target multimedia segment. This second manifest file is used to record the relationship between the segment identifiers and access information of each multimedia segment in the target multimedia information stream. Then, the target access information for the target multimedia segment is obtained by searching the second manifest file based on the segment identifier of the target multimedia segment.

[0089] For example, Figure 7 A schematic diagram of a manifest file provided in one embodiment of this application is shown. Figure 7 As shown, the manifest file provided in this embodiment includes a parent manifest file 710 and a sub-manifest file 720, wherein the parent manifest file 710 is the first manifest file, and the sub-manifest file 720 is the second manifest file. The parent manifest file 710 is mainly used to describe how many sub-manifest files 720 it contains, and the quality level (i.e., quality grade) corresponding to each sub-manifest file 720. Figure 7 (Quality profile in the file), where the parent manifest file 710 describes each contained sub-manifest file 720 by recording the addresses of each sub-manifest file 720. For example... Figure 7 As shown, “Quality File 0”, “Quality File 1” and “Quality File 2” are used to represent multimedia information stream identifiers, and “Address of Sub-List File 0”, “Address of Sub-List File 1” and “Address of Sub-List File 2” represent the storage addresses of the sub-list file 720 corresponding to the multimedia information stream identifier.

[0090] Figure 7 The sub-manifest file 720 shown is the detailed content of sub-manifest file 1 corresponding to quality level 1. Sub-manifest file 720 is used to specifically describe the specific file data segment information contained in the multimedia information stream at the corresponding quality level. The file data segment information includes at least descriptive information directly used by the player (a component or functional part used to play multimedia information). For example, sub-manifest file 720 includes: the CDN (Content Delivery Network) server URL (Uniform Resource Locator) of the file data segment, which is the storage address of the multimedia segment, used by the player to retrieve the data of that multimedia segment from the corresponding address. For example, Figure 7 The terms "File Data Slice 0", "File Data Slice 1", and "File Data Slice 2" in the text represent segment identifiers for multimedia segments. Figure 7 The “duration” in the text refers to the playback duration of the corresponding multimedia segment, and the playback duration of different multimedia segments can be different. Figure 7 The “address” in the text refers to the storage address of the corresponding multimedia segment.

[0091] For example, with Figure 7 Taking the manifest file as an example, assume the segment identifier of the target multimedia segment is s1-2, and the multimedia information stream identifier corresponding to the target multimedia segment is s1. First, based on the multimedia information stream identifier s1, search in the parent manifest file 710 to find the address of the sub-manifest file 1 corresponding to "quality file" 1. Then, based on the segment identifier s1-2, search in the sub-manifest file 720 to obtain the storage address corresponding to "file data segment 2", which is the target access information for the target multimedia segment s1-2.

[0092] In one embodiment of this application, the manifest file can describe different levels of manifest files in a nested manner according to actual application needs. That is, the manifest file not only includes the first manifest file and the second manifest file, but may also include lower-level third manifest files, fourth manifest files, and so on. This application embodiment does not limit the hierarchy and number of manifest files.

[0093] In one embodiment of this application, before searching for the manifest file based on the description information, a manifest file generation step is further included, specifically including: obtaining multiple multimedia information streams corresponding to the multimedia information; parsing each multimedia information stream to obtain multiple multimedia segments included in each multimedia information stream; for each multimedia information stream, generating a second manifest file based on the segment identifiers of each multimedia segment and the access information of each multimedia segment; and generating a first manifest file based on the second manifest file corresponding to each multimedia information stream and the multimedia information stream identifier corresponding to each multimedia information stream.

[0094] In this embodiment, the manifest file generation step can be performed by the terminal device, or the server can generate the manifest file and then send it to the terminal device. The following description uses the server as the executing entity to illustrate the manifest file generation process in this application.

[0095] First, multiple multimedia information streams corresponding to the multimedia information are acquired, and each stream is assigned a multimedia information stream identifier. Then, each multimedia information stream is parsed; for each stream, multiple multimedia segments can be obtained, and the number of multimedia segments is the same for each stream. Next, each multimedia segment in each stream is assigned a segment identifier, and a second manifest file is generated based on the segment identifiers and storage addresses of each multimedia segment in each stream. Finally, a first manifest file is generated based on the second manifest files and the multimedia information stream identifiers for each stream.

[0096] In one embodiment of this application, when the entity executing the manifest file generation step is a terminal device, the process of parsing each multimedia information stream to generate a second manifest file and a first manifest file is the same as when the server is the entity executing the step, and will not be described in detail here. The difference is that when the entity executing the step is a terminal device, the terminal device needs to obtain multiple multimedia information streams from the server.

[0097] In one embodiment of this application, multiple multimedia information streams are uniformly encoded into one encoded data stream. The terminal device obtains the encoded data from the server and parses it to obtain multiple multimedia information streams.

[0098] In one embodiment of this application, multiple multimedia information streams are independently encoded and transmitted. One multimedia information stream is encoded to obtain one encoded data stream. The terminal device obtains the multiple encoded data streams from the server and parses them separately to obtain multiple multimedia information streams. When multiple multimedia information streams are independently encoded and transmitted, standard encoding tools can be used to encode each multimedia information stream, without the need for SVC algorithm tools.

[0099] In one embodiment of this application, the process of assigning a segment identifier to a multimedia segment includes: sorting multiple multimedia segments of each multimedia information stream according to the playback time order; generating a segment identifier for the multimedia segment based on the sorting number of the multimedia information stream and the multimedia segment; wherein, in different multimedia information streams, the multimedia segments corresponding to the same sorting number have the same playback duration.

[0100] Specifically, for each multimedia information stream corresponding to multiple multimedia segments, these segments are first sorted according to their playback time order, with the first-played segment appearing first and the last-played segment appearing last. Since the number of multimedia segments corresponding to each multimedia information stream is equal, each multimedia information stream has a corresponding multimedia segment for the same sequence number. To ensure a smooth transition of the multimedia information played during stream switching, the playback duration of multimedia segments corresponding to the same sequence number is the same in each multimedia information stream; for example, the playback duration of multimedia segments with sequence number 0 is 2 seconds. However, within the same multimedia information stream, the playback duration of multimedia segments corresponding to different sequence codes can be the same or different. For example, in multimedia information stream s0, the playback duration of multimedia segment with sequence number 0 is 2 seconds, and the playback duration of multimedia segment with sequence number 1 is 2.5 seconds.

[0101] For example, such as Figure 2 As shown, the video stream s0 corresponding to camera position 0 includes video segments s0-0 to s0-7, the video stream s1 corresponding to camera position 1 includes video segments s1-0 to s1-7, and the video stream s2 corresponding to camera position 2 includes video segments s2-0 to s2-7. It should be noted that... Figure 2 The video clip identifier shown is in the style of "video stream identifier-number.encoding format", that is, "fmp4" is the encoding format of the video clip. For ease of description, this embodiment simplifies the video clip identifier to the style of "video stream identifier-number" and omits the ".fmp4" field in the clip identifier, which will not be elaborated on further.

[0102] Multimedia segments with the same ID in different multimedia information streams have the same playback duration. For example, video segments s0-0 in video stream s0, s1-0 in video stream s1, and s2-0 in video stream s2 all have the same playback duration, such as 2 seconds. Multimedia segments with different IDs in the same multimedia information stream can have different playback durations. For example, video segments s0-0, s0-1, and s0-2 in video stream s0 can correspond to different playback durations.

[0103] After sorting the multimedia segments, a segment identifier is generated based on the multimedia information stream identifier corresponding to the multimedia segment and the sort number of the multimedia segment. For example, if the sort number is 0 and the multimedia information stream identifier is s0, then the segment identifier of the multimedia segment is s0-0.

[0104] Step 640: Obtain the target encoding data of the target multimedia segment based on the target access information.

[0105] Specifically, the target access information is the storage address of the target multimedia segment. Based on this target access information, the target encoded data of the target multimedia segment can be obtained.

[0106] In one embodiment of this application, the terminal device can pre-download the encoded data of multiple multimedia information streams from the server. When obtaining the target encoded data, the target access information points to the local storage address of the terminal device, which can speed up the acquisition of the target encoded data by the terminal device and improve the smoothness of stream switching.

[0107] In one embodiment of this application, the encoding format of multimedia information is not limited, and it can be MP4, fMP4, MKV, etc.

[0108] In one embodiment of this application, the terminal device can first request the target multimedia stream from the server based on the multimedia information stream identifier corresponding to the target multimedia segment, and then extract the target encoded data of the target multimedia segment corresponding to the target access information from the target multimedia stream. That is, the terminal device does not simultaneously receive multiple multimedia information streams transmitted by the server, but requests the corresponding multimedia information stream from the server according to the current needs, so as to avoid wasting storage resources by receiving multimedia information streams that are not currently needed.

[0109] Step 650: Decode the target encoded data to obtain the target decoded data corresponding to the target multimedia segment.

[0110] Specifically, decoding is the reverse process of encoding, used to restore the target encoded data to its unencoded state, decoding the compressed audio and video stream into the original image and sound data to obtain the target decoded data. The specific decoding method is determined according to the corresponding encoding rules, and this application does not limit it. For example, the image in the target decoded data can be raw data in YUV, RGB, or other formats, and the sound can be raw data in PCM format.

[0111] Step 660: Render and display the target decoded data to play the target multimedia segment.

[0112] Specifically, rendering is the process of converting the target decoded data into an image that is actually drawn on the display screen for display, thereby enabling the playback of the target multimedia segment.

[0113] This application embodiment realizes the switching playback of different multimedia information bitstreams, and provides interactive multimedia information with variable quality. By querying the target access information of the target multimedia segment through the first list file and the second list file, the speed of obtaining the target multimedia segment is effectively improved, so that the multimedia segments can be played smoothly and seamlessly during the interaction, and the playback experience of multimedia information is enhanced.

[0114] For example, the implementation process of the technical solution of this application will be illustrated below using video as an example of multimedia information. Figure 5 As shown, the video in this embodiment is obtained by shooting the same subject (or scene) from three different camera positions. For example, camera position 0 corresponds to a wide-angle field of view of -45° to 45°, camera position 1 corresponds to a wide-angle field of view of -90° to 0°, and camera position 2 corresponds to a wide-angle field of view of 0° to 90°. Therefore, the video includes three video streams: video stream s0 corresponding to camera position 0, video stream s1 corresponding to camera position 1, and video stream s2 corresponding to camera position 2. This embodiment uses... Figure 7 The manifest file shown records information about the above three video streams, among which, Figure 7 The quality setting corresponds to the camera position.

[0115] Based on the above video information, the video playback process in this embodiment is as follows: Figure 2 As shown, assuming the video starts playing at time T0, and the initial viewing angle is assumed to be camera position 0. During the time interval T0-T1, the video content corresponding to camera position 0 is played.

[0116] At time T1, the object interacts with the currently playing video segment s0-1 through the terminal device, adjusting the viewing angle, for example, through screen gestures, virtual buttons on the UI interface, or device gyroscope rotation. Since the main list file 710 records and describes the viewing angle range covered by the video streams of each camera position, the desired angle range can be found to belong to camera position 1, thus determining the storage address of sub-list file 1. Because the video streams corresponding to all camera positions are strictly aligned on the timeline, after time T1, the player only needs to sequentially play the next sequenced video segment file s1-2 (one segment file corresponds to one video segment) to ensure that the played audio and video content is completely seamless and continuous. The playback address of video segment file number 2 in the video stream s1 corresponding to camera position 1 is recorded in its corresponding sub-list file 1, which the player can directly obtain for switching.

[0117] During the time interval T1-T2, the video content corresponding to camera position 1 is played. At time T2, the user interacts with the currently playing video segments s1-4 via a terminal device to adjust the viewing angle. Assuming the desired angle is camera position 2, then, following the same method as the previous step, the next video segment file s2-5 is found in the sub-list file 2 corresponding to camera position 2 and played seamlessly and smoothly.

[0118] During the time interval T2-T3, the video content corresponding to camera position 2 is played. At time T3, the subject again interacts with the currently playing video segment s2-5 through the terminal device to adjust the viewing angle. The adjustment and playback process is similar to the aforementioned steps, so it will not be described again.

[0119] In one embodiment of this application, Figure 2 The three video streams in the video playback process shown can also be implemented using layered encoding, such as encoding with SVC technology to obtain three video streams of different quality levels. Taking a product brand demonstration video as an example, viewers can zoom in or out of the video image by clicking the "+" and "-" keys on the keyboard, or the volume "+" and "-" keys on a mobile phone or tablet, to achieve a more detailed or overall view of the product demonstration video. For example, in Figure 2 In the playback flow shown, starting from time T0, the default playback is the product video corresponding to video stream S0. At time T1, the user switches the video to the product video corresponding to video stream S1 using the volume "+" button on the terminal device. This video stream S1 is a video for zooming in on the product. At time T2, the user switches the video to the product video corresponding to video stream S2 using the volume "-" button on the terminal device. This video stream S2 is a video for viewing the product globally. The specific details of the switching process are similar to those of the camera position details mentioned earlier, so they will not be repeated here.

[0120] In interactive video playback using related technologies, video information and interactive description information are often stored separately, especially the video information files prepared to respond to different object interaction selections, which are generally independent. Therefore, the player in the video playback device needs to acquire different video information files and splice them together to achieve a complete playback process. The player assembles the complete playback content based on object selection by splicing video information. This process involves repeatedly acquiring video information data, starting playback, stopping playback, and acquiring new video information, repeating this cycle until all video information has been played.

[0121] Starting, stopping, and acquiring video information all take time, and switching between these interactive video segments can cause the player to stutter. The root cause is that, generally, video frame rates are around 24-25 fps, meaning the longest possible interval between two adjacent frames is about 40 milliseconds. The higher the frame rate, the shorter this interval. However, the time it takes for a player to switch video segments—including stopping, acquiring new material, and restarting—often significantly exceeds 40 milliseconds, resulting in stuttering during playback. In particular, in applications with high-frequency interactive operations where the player needs to continuously respond to interactions and adjust playback content, this video segmentation solution, due to player stuttering, cannot provide a seamless playback experience. Playback may exhibit continuous interruptions in video or audio, thus degrading the user experience during continuous interactive responses.

[0122] According to the interactive multimedia information playback method provided in this application embodiment, the access information of video segments is described through a manifest file. During the video segment switching process based on object interaction, the player directly and sequentially plays the next video segment indicated by the access address according to the manifest file, without the need for video information splicing or repeated starting and stopping of the player, thereby improving the efficiency of video segment switching and achieving smooth and seamless video switching. Furthermore, the video information in this application embodiment adopts a layered encoding and multi-stream transmission method, realizing the playback of interactive videos with variable quality. For example, interactive videos can continuously change in terms of image resolution, video frame rate, and field of view (FOV). Especially in high-frequency interaction scenarios, it can quickly switch between videos of different qualities, providing a high-quality playback experience.

[0123] It should be noted that although the steps of the method in this application are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0124] The following describes an embodiment of the apparatus or system of this application, which can be used to execute the interactive multimedia information playback method in the above embodiments of this application.

[0125] Figure 8 A schematic block diagram illustrating the structure of an interactive multimedia information playback system provided in an embodiment of this application is shown. Figure 8As shown, the system 810 includes: a video material acquisition module 811, a video format parsing module 812, a video interaction point acquisition module 813, a user interaction command processing module 814, a playback data slice selection module 815, a playback data decoding module 816, and an image and sound rendering module 817.

[0126] The video material acquisition module 811 connects to the video server 820 to obtain video material file data, i.e., video stream, from the video server 820. The acquisition and data transmission of video materials generally use application-layer streaming media protocols based on HTTP (Hypertext Transfer Protocol), such as Apple HLS (HTTP Live Streaming) or MPEG DASH (Dynamic Adaptive Streaming over HTTP). During the acquisition of video material file data, it is not required that the entire video material file be downloaded and transmitted. Instead, a portion of the data is downloaded and transmitted. The subsequent video format parsing module 812, playback data segment selection module 815, playback data decoding module 816, and image and sound rendering module 817 can then begin operation. The subsequent download and transmission of the video file and the playback of the video file in the downstream modules are carried out in parallel and synchronously.

[0127] The video format parsing module 812 parses and records the video materials acquired by the video material acquisition module 811 according to the format of the manifest file provided in this application, and generates a corresponding manifest file. Please refer to the description in the preceding method embodiments for details, which will not be elaborated upon here.

[0128] The video interaction point acquisition module 813 is used to acquire information describing the interaction method and content between the object and the video, i.e., to acquire interaction description information. The video interaction point acquisition module 813 can connect to the video server 820, meaning it can acquire interaction description information from the video server 820. It should be noted that the video server storing the interaction description information and the video server storing the video materials can be the same server or different servers. The interaction description information generally includes at least the time point at which the interaction occurs, used to align the interactive interface with the video content it is bound to; and elements that the object can interact with, for example, in an interactive drama, it needs to describe all the video content branches selectable at this time point. Specifically, based on the interactive multimedia information playback method provided in this application embodiment, the object can be allowed to interact for a maximum time comparable to the video duration. That is, the object can continuously interact to change image quality, viewing angle, etc., throughout the entire video viewing process.

[0129] The user interaction instruction processing module 814 provides an actual interactive operation interface based on the interaction description information obtained by the video interaction point acquisition module 813. This interface may take the form of a list selection box, a row of buttons, or an image that can be clicked to navigate to a specific page on the program's UI. The user interaction instruction processing module 814 ultimately translates the object's interaction operation into a specific description of the operation on the video material. For example, if the object rotates the terminal device in a certain way, the user interaction instruction processing module 814 detects this interaction operation and outputs the corresponding instruction to perform a specific operation on the video material. For instance, rotating the terminal device corresponds to adjusting the video viewing angle, so the module outputs the URL (or access ID, etc.) of the video material at the adjusted target viewing angle. Other interaction operations may also output: the timeline progress of the currently playing video file, operations on the player (pause, change resolution, stop, etc.), etc.

[0130] The playback data slice selection module 815 is used to find suitable quality-level video material (i.e., find the corresponding video bitstream) based on the instructions corresponding to the interactive operation and the manifest file, and then address the specific video data slice according to the corresponding sub-manifest file. The specific process can be referred to the relevant description in the previous method embodiment, and will not be repeated here.

[0131] The playback data decoding module 816 is used to decode the audio and video compressed bitstream into raw image and sound data, where the image can be raw data in YUV, RGB or other formats, and the sound can be raw data in PCM format.

[0132] The image and sound rendering module 817 is used to display the raw image and sound data output by S6 through peripherals.

[0133] Figure 9 A schematic block diagram of an interactive multimedia information playback device provided in an embodiment of this application is shown. Figure 9 As shown, the device includes:

[0134] The interactive operation module 910 is used to generate description information of the target multimedia segment in the target multimedia information stream based on the interactive operation when an interactive operation is detected for the current multimedia segment in the current multimedia information stream; the current multimedia information stream and the target multimedia information stream are two different multimedia information streams corresponding to the same multimedia information, and the target multimedia segment and the current multimedia segment are continuous in time.

[0135] The target access information determination module 920 is used to search for the target access information of the target multimedia segment in the manifest file corresponding to the multimedia information based on the description information of the target multimedia segment; the manifest file includes the access information of each multimedia segment of the multimedia information;

[0136] The target multimedia segment playback module 930 is used to obtain the target multimedia segment and play it according to the target access information.

[0137] In one embodiment of this application, the description information of the target multimedia segment includes a segment identifier of the target multimedia segment and a multimedia information stream identifier corresponding to the target multimedia segment; the manifest file includes a first manifest file and a second manifest file; the target access information determination module 920 includes:

[0138] The first manifest file lookup unit is used to look up the second manifest file corresponding to the target multimedia segment in the first manifest file corresponding to the multimedia information according to the multimedia information bitstream identifier corresponding to the target multimedia segment; the first manifest file includes the correspondence between the multimedia information bitstream identifier and the second manifest file.

[0139] The second manifest file lookup unit is used to look up the target access information of the target multimedia segment in the second manifest file according to the segment identifier of the target multimedia segment; the second manifest file includes the correspondence between the segment identifier of the multimedia segment and the access information of the multimedia segment.

[0140] In one embodiment of this application, the apparatus further includes:

[0141] A multimedia information acquisition module is used to acquire multiple multimedia information bitstreams corresponding to the multimedia information;

[0142] The multimedia information parsing module is used to parse each multimedia information bitstream to obtain multiple multimedia segments included in each multimedia information bitstream; wherein, each multimedia information bitstream includes the same number of multimedia segments.

[0143] The second manifest file generation module is used to generate the second manifest file for each multimedia information bitstream based on the segment identifier and access information of each multimedia segment of the multimedia information bitstream.

[0144] The first manifest file generation module is used to generate the first manifest file based on the second manifest file corresponding to each multimedia information stream and the multimedia information stream identifier corresponding to each multimedia information stream.

[0145] In one embodiment of this application, the apparatus further includes:

[0146] The sorting module is used to sort the multiple multimedia segments of each multimedia information stream according to the playback time order;

[0147] The segment identifier generation module is used to generate a segment identifier for the multimedia segment based on the multimedia information stream identifier of the multimedia information stream and the sort number of the multimedia segment; wherein, in different multimedia information streams, the multimedia segments corresponding to the same sort number have the same playback duration.

[0148] In one embodiment of this application, the multimedia information acquisition module is specifically used for:

[0149] Obtain the multi-channel encoded data corresponding to the multimedia information from the server;

[0150] The multi-channel encoded data of the multimedia information is parsed to obtain multiple multimedia information bitstreams, wherein one channel of encoded data corresponds to one multimedia information bitstream.

[0151] In one embodiment of this application, the multimedia information acquisition module is specifically used for:

[0152] Obtain the encoded data of the multimedia information from the server;

[0153] The encoded data of the multimedia information is parsed to obtain multiple multimedia information streams.

[0154] In one embodiment of this application, the target multimedia segment playback module 930 includes:

[0155] A target encoded data acquisition unit is used to acquire target encoded data of the target multimedia segment based on the target access information.

[0156] A decoding unit is used to perform a decoding operation on the target encoded data to obtain target decoded data corresponding to the target multimedia segment;

[0157] The rendering unit is used to render and display the target decoded data in order to play the target multimedia segment.

[0158] The specific details of the interactive multimedia information playback device provided in the various embodiments of this application have been described in detail in the corresponding method embodiments, and will not be repeated here.

[0159] Figure 10 A schematic block diagram of a computer system architecture for implementing an electronic device according to embodiments of the present application is shown.

[0160] It should be noted that, Figure 10The computer system 1000 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0161] like Figure 10 As shown, the computer system 1000 includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 1002 or programs loaded from storage section 1008 into random access memory (RAM). The RAM 1003 also stores various programs and data required for system operation. The CPU 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An input / output interface 1005 (I / O interface) is also connected to the bus 1004.

[0162] The following components are connected to the input / output interface 1005: an input section 1006 including a keyboard, mouse, etc.; an output section 1007 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a local area network card, modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the input / output interface 1005 as needed. A removable medium 1011, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 1010 as needed so that computer programs read from it can be installed into the storage section 1008 as needed.

[0163] Specifically, according to embodiments of this application, the processes described in the various method flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1009, and / or installed from removable medium 1011. When the computer program is executed by central processing unit 1001, it performs various functions defined in the system of this application.

[0164] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0165] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0166] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0167] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the method according to the embodiments of this application.

[0168] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0169] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for playing interactive multimedia information, characterized in that, include: When an interactive operation is detected targeting a current multimedia segment in the current multimedia information stream, description information of a target multimedia segment in the target multimedia information stream is generated based on the interactive operation. The current multimedia information stream and the target multimedia information stream are two different multimedia information streams corresponding to the same multimedia information, displaying the multimedia information from different quality levels or camera angles. The camera angle refers to the angle of the camera device capturing the multimedia information. The target multimedia segment and the current multimedia segment are continuous in time. The description information of the target multimedia segment includes the segment identifier of the target multimedia segment and the multimedia information stream identifier corresponding to the target multimedia segment. The multimedia information is interactive video; Based on the multimedia information bitstream identifier corresponding to the target multimedia segment, the second manifest file corresponding to the target multimedia segment is searched in the first manifest file corresponding to the multimedia information; the first manifest file includes the correspondence between the multimedia information bitstream identifier and the second manifest file; The target access information of the target multimedia segment is searched in the second manifest file based on the segment identifier of the target multimedia segment; the second manifest file includes the correspondence between the segment identifier of the multimedia segment and the access information of the multimedia segment; The target multimedia segment is obtained and played based on the target access information.

2. The interactive multimedia information playback method according to claim 1, characterized in that, Before searching for the target access information of the target multimedia segment in the manifest file corresponding to the multimedia information based on the description information of the target multimedia segment, the method further includes: Obtain the multi-channel multimedia information bitstream corresponding to the multimedia information; Each multimedia information stream is parsed to obtain multiple multimedia segments included in each multimedia information stream; wherein, each multimedia information stream includes the same number of multimedia segments. For each multimedia information stream, the second manifest file is generated based on the segment identifiers and access information of each multimedia segment of the multimedia information stream. The first manifest file is generated based on the second manifest file corresponding to each multimedia information stream and the multimedia information stream identifier corresponding to each multimedia information stream.

3. The interactive multimedia information playback method according to claim 2, characterized in that, Before generating the second manifest file based on the segment identifiers and access information of each multimedia segment of the multimedia information stream, the method further includes: The multimedia segments of each multimedia information stream are sorted according to the playback time order; Based on the multimedia information stream identifier of the multimedia information stream and the sort number of the multimedia segment, a segment identifier of the multimedia segment is generated; wherein, in different multimedia information streams, the multimedia segments corresponding to the same sort number have the same playback duration.

4. The interactive multimedia information playback method according to claim 2, characterized in that, Obtaining the multi-channel multimedia information bitstream corresponding to the multimedia information includes: Obtain the multi-channel encoded data corresponding to the multimedia information from the server; The multi-channel encoded data of the multimedia information is parsed to obtain multiple multimedia information bitstreams, wherein one channel of encoded data corresponds to one multimedia information bitstream.

5. The interactive multimedia information playback method according to claim 2, characterized in that, Obtaining the multi-channel multimedia information bitstream corresponding to the multimedia information includes: Obtain the encoded data of the multimedia information from the server; The encoded data of the multimedia information is parsed to obtain multiple multimedia information streams.

6. The method for playing interactive multimedia information according to any one of claims 1-5, characterized in that, Retrieving and playing the target multimedia segment based on the target access information includes: Obtain the target encoded data of the target multimedia segment based on the target access information; The target encoded data is decoded to obtain the target decoded data corresponding to the target multimedia segment; The target decoded data is rendered and displayed to play the target multimedia segment.

7. An interactive multimedia information playback device, characterized in that, include: An interactive operation module is used to generate description information of a target multimedia segment in a target multimedia information stream based on an interactive operation detected for a current multimedia segment in the current multimedia information stream. The current multimedia information stream and the target multimedia information stream are two different multimedia information streams corresponding to the same multimedia information, displaying the multimedia information from different quality levels or camera angles. The camera angle refers to the angle of the camera device capturing the multimedia information. The target multimedia segment and the current multimedia segment are continuous in time. The description information of the target multimedia segment includes a segment identifier and a corresponding multimedia information stream identifier. The multimedia information is interactive video; The target access information determination module is used to search for the second list file corresponding to the target multimedia segment in the first list file corresponding to the multimedia information based on the multimedia information bitstream identifier corresponding to the target multimedia segment; the first list file includes the correspondence between the multimedia information bitstream identifier and the second list file; The target access information of the target multimedia segment is searched in the second manifest file based on the segment identifier of the target multimedia segment; the second manifest file includes the correspondence between the segment identifier of the multimedia segment and the access information of the multimedia segment; The target multimedia segment playback module is used to obtain the target multimedia segment based on the target access information and play it.

8. A computer-readable medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the interactive multimedia information playback method according to any one of claims 1 to 6.

9. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor executes the executable instructions to cause the electronic device to perform the interactive multimedia information playback method according to any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product or computer program includes computer instructions stored in a computer-readable storage medium; The processor of the computer device reads the computer instructions from a computer-readable storage medium, and executes the computer instructions to cause the computer device to perform the interactive multimedia information playback method according to any one of claims 1 to 6.

Images