Data processing method and device, equipment and storage medium

Abstract

The embodiment of the invention discloses a data processing method and device, equipment and a storage medium, which can be applied to the technical field of computers, and the method comprises the following steps: obtaining streaming media data, the streaming media data comprising media data of at least one initial track; determining a selection state of each initial track, and playing the media data of the initial track of which the selection state is the selected state; in the media data playing process, in response to the change of the streaming media attribute of the streaming media data at the first moment, determining the selection state of each first track of the streaming media data after the first moment according to the selection state of each initial track; and playing the media data of the first track of which the selection state is the selected state after the first moment. By adopting the embodiment of the invention, the playing stability and playing efficiency of the streaming media data can be improved, and the applicability is high.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data processing method, apparatus, device, and storage medium. Background Technology

[0002] With the rapid development of computer and internet technologies, various real-time streaming media data playback technologies have brought great convenience to people's daily lives, such as live streaming, online classes, online meetings, group live streaming, etc. It has become increasingly popular for people to obtain information or shop online by watching live streams.

[0003] In practical applications, playback devices can enable or disable different tracks by default to play different media data, and adjust the playback of different media data based on the browsing user's settings during playback. When the streaming media attributes change, it may lead to changes at the track level, such as adding or removing tracks, or changing a default disabled track to a default enabled track. Existing technology will play different tracks based on the default enabling or disabling of tracks after the streaming media attributes change, which cannot maintain consistency in type and content with the media data played before the streaming media attributes changed, thus affecting playback efficiency. Summary of the Invention

[0004] This application provides a data processing method, apparatus, device, and storage medium that can improve media data playback efficiency, maintain the continuity of media data playback, and has high applicability.

[0005] On one hand, embodiments of this application provide a data processing method, the method comprising: Acquire streaming media data, which includes media data from at least one initial track; Determine the selection status of each of the above initial tracks, and play the media data of the initial track whose selection status is selected; During media data playback, in response to changes in the streaming media attributes of the aforementioned streaming media data at the first moment, the selection status of each first track of the aforementioned streaming media data after the aforementioned first moment is determined based on the selection status of each of the aforementioned initial tracks. Play the media data of the first track that is selected after the first moment mentioned above.

[0006] On the other hand, embodiments of this application provide a data processing apparatus, which includes: The data acquisition module is used to acquire streaming media data, which includes media data from at least one initial track. The data playback module is used to determine the selection status of each of the above initial tracks and play the media data of the initial track whose selection status is selected. The state determination module is used to determine the selection state of each first track of the streaming media data after the first moment in response to the change of the streaming media attributes of the streaming media data at the first moment during the media data playback process, based on the selection state of each of the initial tracks. The aforementioned data playback module is used to play media data of the first track whose selection state is selected after the aforementioned first moment.

[0007] On the other hand, embodiments of this application provide an electronic device, including a processor and a memory, which are interconnected; The aforementioned memory is used to store computer programs; The processor described above is used to execute the data processing method provided in the embodiments of this application when the computer program described above is invoked.

[0008] On the other hand, embodiments of this application provide a computer-readable storage medium storing a computer program that is executed by a processor to implement the data processing method provided in embodiments of this application.

[0009] On the other hand, embodiments of this application provide a computer program product, which includes a computer program that, when executed by a processor, implements the data processing method provided in embodiments of this application.

[0010] In this embodiment, when the streaming media attributes change at the first moment, the selection state of the first track after the first moment can be determined based on the initial track selection state before the first moment, thereby ensuring that the track selection state of the streaming media data always remains continuous. This avoids playing streaming media data based on the default on or off settings of each track, thus improving the playback efficiency of streaming media data and making it highly applicable. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic diagram of a scenario illustrating the data processing method provided in an embodiment of this application; Figure 2 This is a flowchart illustrating the data processing method provided in an embodiment of this application; Figure 3 This is a schematic diagram of a scenario for determining a selection state provided in an embodiment of this application; Figure 4 This is one of the scenario diagrams for determining the first combination provided in the embodiments of this application; Figure 5 This is a second schematic diagram of the scenario for determining the first combination provided in the embodiments of this application; Figure 6 This is the third schematic diagram of the scenario for determining the first combination provided in the embodiments of this application; Figure 7 This is another scenario diagram illustrating the determination of the selection state provided in the embodiments of this application; Figure 8 This is a schematic diagram of the structure of the data processing apparatus provided in the embodiments of this application; Figure 9 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0013] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0014] The data processing method provided in this application embodiment can be applied to the field of computer technology, specifically to the field of media information, in order to improve the playback efficiency of streaming media data.

[0015] See Figure 1 , Figure 1 This is a schematic diagram illustrating a scenario of the data processing method provided in an embodiment of this application. For example... Figure 1 As shown, terminal 11 can obtain streaming media data sent by server 12, which includes media data of at least one initial track.

[0016] For terminal 11, after receiving the streaming media data sent by server 12, terminal 11 can first determine the selection status of each initial track and play the media data of the initial track whose selection status is selected. For example, if terminal 11 determines that the selection status of the video track in the initial track is selected, terminal 11 can play the media data of the video track, and the media content presented on the display interface of terminal 11 is screen 13.

[0017] During media data playback, in response to the change in streaming media attributes of the streaming media data at the first moment, the terminal 11 determines the selection status of each first track of the streaming media data after the first moment (after the change in streaming media attributes) based on the selection status of each initial track before the first moment, and plays the media data of the first track whose selection status is selected after the first moment.

[0018] For example, if the media attributes of the streaming media data change at the first moment, terminal 11 determines, based on the selection status of each initial track before the first moment, that the video track and a certain subtitle track in the first track after the first moment are selected. Terminal 11 then plays the media data of the corresponding video track and subtitle track after the first moment, and the media content presented on the display interface of terminal 11 is screen 14. That is, after the first moment, the media content (screen 14) presented on the display interface of terminal 11 includes video content and subtitle content.

[0019] Optionally, server 12 can acquire streaming media data, which includes media data for at least one initial track. Further, server 12 can determine the selection state of each initial track and send the media data of the initial tracks whose selection state is selected to terminal 11, so that terminal 11 can play the media data of the initial tracks whose selection state is selected; or, server 12 can mark the selection state of each initial track in the streaming media data so that terminal 11, after receiving the streaming media data, can play the media data of the initial tracks whose selection state is selected.

[0020] During media data playback, server 12 continuously acquires streaming media data. When the streaming media attributes of the streaming media data change at the first moment, server 12 determines the selection status of each first track of the streaming media data after the first moment (after the streaming media attributes change) based on the selection status of each initial track before the first moment. Server 12 can then send the media data of the first track with the selected status to terminal 11, so that terminal 11 can play the media data of the first track with the selected status after the first moment, or mark the selection status of each first track of the streaming media data so that terminal 11 can play the media data of the first track with the selected status after receiving the streaming media data.

[0021] Among them, server 12 can be an independent physical server, such as a video server or streaming media server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0022] The terminal 11 can be a smartphone, tablet, laptop, desktop computer (including monitor), smartwatch, vehicle terminal, aircraft, smart home appliance (such as smart TV), or wearable device with display function.

[0023] It should be noted that the data processing method provided in this application embodiment can be executed by the server or terminal alone, or by the server and terminal jointly; no limitation is imposed here. See also Figure 2 , Figure 2 This is a flowchart illustrating the data processing method provided in an embodiment of this application. Figure 2 As shown, the data processing method provided in this application embodiment can be described using terminal execution as an example. The method may include the following steps: Step S21: Obtain streaming media data, which includes media data from at least one initial track.

[0024] In some feasible implementations, streaming media data can be a continuous media data stream transmitted and played sequentially over a network. Users can watch and play the streaming media data simultaneously with data transmission, without waiting for the data to download completely. Examples include watching movies online, listening to music, or conducting video conferences.

[0025] The streaming media data may include one or more of the following: video data, audio data, or subtitle data. The specific data may be determined based on the actual application scenario requirements and is not limited here.

[0026] Optionally, the streaming media data in this application embodiment can be media data based on the Hypertext Transfer Protocol (HTTP) Live Streaming (HLS) protocol, that is, media data based on the HTTP adaptive bitrate streaming media transmission protocol, or streaming media data based on other streaming media data transmission protocols. The specific method can be determined based on the actual application scenario requirements, and no restrictions are imposed here.

[0027] HLS is a streaming media transmission protocol primarily used to transmit audio and video streams over the Internet via the HTTP protocol.

[0028] In some feasible implementations, in addition to supporting video tracks, streaming media data can also support at least one audio track and at least one subtitle track, thereby meeting the different audio playback and subtitle requirements of the browsing audience, such as supporting the playback of English or Chinese subtitles.

[0029] After acquiring the streaming media data, the streaming media data includes at least one initial track, and the media data of each initial track corresponds to a data type, such as video type, subtitle type, or audio type.

[0030] The acquisition process of streaming media data is a real-time acquisition process, meaning that the media data to be played later can be acquired in real time during the media playback process.

[0031] Step S22: Determine the selection status of each initial track and play the media data of the initial track whose selection status is selected.

[0032] In some feasible implementations, after acquiring the streaming media data, it is necessary to determine the initial tracks corresponding to the streaming media data and the selection status of each initial track.

[0033] The aforementioned initial track refers to the initial track corresponding to the streaming media data when the streaming media data is acquired, or it can be understood as the preset track corresponding to the streaming media data when the media data starts playing.

[0034] The selection states mentioned above include selected and unselected states. For any track of any streaming media data (such as any initial track), when the selection state of the track is selected, it means that the media data corresponding to the track is the media data that needs to be played. When the selection state of the track is unselected, it means that the media data corresponding to the track is the media data that does not need to be played (prohibited).

[0035] As an example, for any initial track, if the initial track is a subtitle track, when the initial track is selected, it means that the subtitle data corresponding to the initial track needs to be played. When the initial track is not selected, it means that the subtitle data corresponding to the initial track does not need to be played, that is, the subtitle data corresponding to the initial track is not included in the final media display.

[0036] Specifically, the streaming media data may include a status label for each initial track, which indicates the selection status of the corresponding initial track. When acquiring streaming media data and starting to play media data, the initial track with the selected status can be determined from among the initial tracks based on the status labels of each initial track, and the media data of the initial track with the selected status can be played.

[0037] Optionally, when determining the selection state of each initial track, the default playback state of each initial track can also be determined. For each initial track, when the default playback state of the initial track is enabled, it means that the media data of the initial track is the media data that is played by default after receiving streaming media data, and the selection state of the initial track can be determined as the selected state; when the default playback state of the initial track is not enabled, it means that the media data of the initial track is the media data that is not required to be played by default after receiving streaming media data. Since the media data of the initial track does not need to be played when the media data starts playing, the selection state of the initial track can be determined as the unselected state.

[0038] As an example, the default playback state of each initial track can be represented as follows: or ,when When this happens, it indicates that the default playback status of the corresponding initial track has been enabled. This indicates that the default playback status of the corresponding initial track is not enabled.

[0039] Based on this, after acquiring streaming media data and determining the selection status of each initial track, streaming media data of the initial track whose selection status is selected can be played.

[0040] As an example, if the initial tracks corresponding to the streaming media data are , , , as well as The default playback status of each initial track is shown below.

[0041] in, , as well as Default playback is enabled. ), and The default playback mode is not enabled. You can then... , as well as The selection state is set to the selected state, and and The selection status is set to unselected.

[0042] In this case, the streaming data outputs a single video by default. video ) track, one for audio ( audio ) track and a subtitle ( subtitle (track, that is, the playback after acquiring streaming media data) , as well as The media data, so that the final output media content includes Corresponding video content The corresponding audio content and The corresponding subtitle content.

[0043] In the above embodiments, after acquiring streaming media data, the selection status of the initial track with the default playback status enabled can be set to selected, and the selection status of the initial track without the default playback status enabled can be set to unselected. Thus, when streaming media data starts playing, the media data of the initial track with the default playback status enabled can be played, realizing automatic playback of streaming media data and improving the playback efficiency of media.

[0044] Step S23: During media data playback, in response to changes in the streaming media attributes of the streaming media data at the first moment, the selection status of each first track of the streaming media data after the first moment is determined based on the selection status of each initial track.

[0045] In some feasible implementations, the streaming media attributes of streaming media data include, but are not limited to, one or more of the following: media encapsulation format, data type of media data corresponding to a track, number of tracks, timestamp sequence, encoding format of media data (such as H.264 / AVC, H.265 / HEVC, AV1, etc.), and encoding sequence of media data. The specific attributes can be determined based on the actual application scenario requirements and are not limited here.

[0046] The media container formats include, but are not limited to, Transport Stream (TS) and freume-mp4 formats, etc., and are not restricted here. Among them, the TS media container format is also known as MPEG2-TS, which is a widely used multimedia file format.

[0047] In this context, the timestamp sequence refers to the sequence of timestamps attached to each media data packet during the streaming media data transmission process. These timestamps record the generation or sampling time of the data packets, ensuring synchronized playback and correct decoding of the media data, thereby reconstructing continuous output content.

[0048] Any change in any of the aforementioned streaming media attributes may lead to a change in the initial track selection state. To ensure consistency in playback content before and after the streaming media attribute change, during media data playback, if the streaming media attribute changes at any moment, it is necessary to redetermine the tracks and the selection state of each track after the moment the streaming media attribute change occurs.

[0049] For example, after initially acquiring streaming media data, the initial tracks selected include the video track, audio track, and the subtitle track corresponding to Chinese subtitles. At this point, the output content includes video content, audio content, and Chinese subtitles. When the streaming media attributes change, the default media data played is the media data corresponding to the video track, the media data corresponding to the audio track, and the media data corresponding to the subtitle track with English subtitles. If the default media data is played after the streaming media attributes change, the final output content will include English subtitles but not Chinese subtitles, thus affecting the playback continuity and browsing experience. Therefore, to ensure consistency in the output content based on the streaming media data before and after the streaming media attributes change, it is necessary to redetermine the tracks and the selection status of each track after the streaming media attributes change.

[0050] Based on this, during the playback of media data on the initial track with the selected state, when the streaming media attributes of the streaming media data change, the selection state of each track (hereinafter referred to as the first track) after the first moment can be determined according to the selection state of each initial track.

[0051] During the real-time acquisition of streaming media data, it is determined in real time whether attribute change markers appear in the streaming media data. Attribute change markers are used to indicate that the streaming media attributes have changed before and after the appearance of the attribute change marker.

[0052] In other words, if an attribute change marker is detected in the streaming media data at any time during the media data playback process, then the streaming media attribute of the streaming media data has changed at that time.

[0053] When streaming data is transmitted using the HLS protocol, the aforementioned attribute change identifier can be the #EXT-X-DISCONTINUITY tag. This tag indicates a break or discontinuity between the current media segment and the previous media segment. The #EXT-X-DISCONTINUITY tag can be used to indicate changes in streaming attributes, such as encoding format, number of tracks, or other key parameters. For example, in live streaming, this tag is used when switching from advertisements to actual program content, or when the video stream switches from one encoding format to another.

[0054] In some feasible implementations, when determining the selection status of each first track of streaming media data after the first moment based on the selection status of each initial track, the track attributes of each initial track and the track attributes of each first track can be determined first, and then the track status of each first track can be determined based on the track attributes of each initial track, the track attributes of each first track, and the selection status of each initial track.

[0055] Specifically, after determining the orbital attributes of each initial orbit and the orbital attributes of each first orbit, the matching degree between each first orbit and each initial orbit can be determined, and then based on all the determined matching degrees, it can be determined whether there is at least one first combination.

[0056] Each first combination includes an initial orbit and a first orbit, and the matching degree between the first orbit and the initial orbit in each first combination meets the matching requirements.

[0057] In this combination, the first orbit and the initial orbit are different in any two first combinations. For example, for any two first combinations: combination A and combination B, the first orbit in combination A is different from the first orbit in combination B, and the initial orbit in combination A is different from the initial orbit in combination B.

[0058] In other words, for any given first orbit, if multiple initial orbits match the matching requirements with that first orbit, only one initial orbit and that first orbit form a first combination. Similarly, if multiple first orbits match the matching requirements with the same initial orbit, only one first orbit and that initial orbit form a first combination.

[0059] In this case, when there is at least one first combination, that is, when at least one first combination is determined, the selection state of the initial track in each first combination can be determined as the selection state of the corresponding first track, and the selection state of each first track that does not exist in any first combination can be determined as the unselected state.

[0060] When there is no first combination, that is, when no first combination has been determined, the selection state of each first track can be set to unselected.

[0061] As an example, see Figure 3 , Figure 3 This is a schematic diagram of a scenario for determining a selection state provided in an embodiment of this application. For example... Figure 3 As shown, if the initial tracks of the streaming media data before the first moment are initial track A, initial track B, initial track C, and initial track D, and the selection states of initial track A and initial track B are selected ( selection= trueThe initial orbitals C and D are currently unselected. selection=false ).

[0062] Given that the first tracks of the streaming data after the first moment are the first track E, the first track F, the first track G, and the first track H, the matching degree between the first track E and the initial track AD, the matching degree between the first track F and the initial track AD, the matching degree between the first track G and the initial track AD, and the matching degree between the first track H and the initial track AD can be determined based on the track attributes of each initial track and the track attributes of each first track.

[0063] Based on the matching degrees mentioned above, the first combination that meets the matching requirements is determined to be the initial track B and the first track E. Since the initial track B is in the selected state, the first track E can be determined to be in the selected state. selection=true ).

[0064] Similarly, based on the matching degrees mentioned above, the first combination that meets the matching requirements also includes the combination of initial track C and first track F, and the combination of initial track A and first track G. Since the initial track C is in an unselected state, the first track F can be determined to be in an unselected state. selection= false Since the initial selection state of track A is selected, the selection state of the first track G can be determined as selected. selection=true ).

[0065] If, based on the aforementioned matching degrees, it is determined that there is no initial track in the initial track AD that matches the matching degree of the first track H, then it can be determined that the first track H does not exist in any first combination. In this case, the selection state of the first track H can be determined as the unselected state. selection=false ).

[0066] In some feasible implementations, the first combination described above is determined based on a cyclic process.

[0067] Specifically, the set of matching degrees between each first track and each initial track at the start of the first cycle is called the matching degree set.

[0068] For the first iteration, first determine whether there exists a matching degree in the matching degree set that meets the matching requirements.

[0069] If there is a matching degree in the matching degree set that meets the matching requirements, then the initial track and the first track corresponding to each target matching degree are determined as a first combination, and all matching degrees associated with the first track and the initial track in each first combination are deleted from the matching degree set, and the next loop process is entered. The target matching degree is the highest matching degree among all matching degrees that meet the matching requirements.

[0070] Among them, when there are multiple first orbits with the same initial orbit whose matching degree (for ease of description, it is called the first matching degree) is the highest among all matching degrees that meet the matching requirements, any one of the first matching degrees is determined as the target matching degree.

[0071] Among them, when there are multiple initial orbits with the same first orbit whose matching degree (for ease of description, it is called the second matching degree) is the highest among all matching degrees that meet the matching requirements, any one of the second matching degrees is determined as the target matching degree.

[0072] If there is no matching degree in the matching degree set that meets the matching requirements, or if the matching degree set is empty, then the loop process stops.

[0073] For each iteration after the initial iteration, it can be determined whether there exists a matching degree that meets the matching requirements in the matching degree set obtained from the previous iteration.

[0074] If there are matching degrees that meet the matching requirements in the matching degree set obtained in the previous loop, then the initial track and the first track corresponding to each target matching degree are determined as a first combination, and all matching degrees associated with the first track and the initial track in each first combination are deleted from the matching degree set, and the next loop process is entered. The target matching degree is the highest matching degree among the matching degrees that meet the matching requirements in the matching degree set obtained in the previous iteration.

[0075] In this case, if there are multiple first orbits in the matching degree set obtained in the previous cycle that have the highest matching degree that meets the matching requirements (for ease of description, they are called first matching degrees), then any one of the first matching degrees will be determined as the target matching degree.

[0076] In this case, if there are multiple initial orbits in the matching degree set obtained in the previous cycle that have the highest matching degree that meets the matching requirements with the same first orbit (for ease of description, they are called second matching degrees), then any one of the second matching degrees will be determined as the target matching degree.

[0077] If the matching degree set obtained in the previous iteration does not contain a matching degree that meets the matching requirements, or if the matching degree set obtained in the previous iteration is empty, then the iteration process stops.

[0078] Based on the above cyclic process, the matching degree between the first orbit and the initial orbit in each first combination can meet the matching requirements, and the first orbit and the initial orbit in any two first combinations are different.

[0079] As an example, see Figure 4 , Figure 4 This is one of the scenario diagrams for determining the first combination provided in the embodiments of this application. For example... Figure 4 As shown, if the initial tracks of streaming media data before the first moment are initial track A and initial track B, and the first tracks after the first moment are first track E and first track F.

[0080] Before the first loop begins, the matching degree 1 (the matching degree between the initial orbit A and the first orbit E), matching degree 2 (the matching degree between the initial orbit A and the first orbit F), matching degree 3 (the matching degree between the initial orbit B and the first orbit E), and matching degree 4 (the matching degree between the initial orbit B and the first orbit F) are determined as the matching degree set.

[0081] At the start of the first loop, it is determined whether there is a matching degree among matching degrees 1-4 that meets the matching requirements. If matching degrees 1-4 all meet the matching requirements, and matching degree 1 is the highest matching degree among matching degrees 1-4, then matching degree 1 is determined as the target matching degree. Further, the initial track A and the first track E corresponding to matching degree 1 can be determined as a first combination, and all matching degrees associated with the initial track A (matching degree 1 and matching degree 2) and all matching degrees associated with the first track E (matching degree 3) are deleted from the matching degree set to obtain the matching degree set used in the next loop, that is, the matching degree set used in the second loop includes matching degree 4.

[0082] As an example, see Figure 5 , Figure 5 This is a second schematic diagram of a scenario for determining the first combination provided in an embodiment of this application. For example... Figure 5 As shown, if in Figure 4 Based on the previous iteration, a second loop is performed. At the beginning of the second loop, the matching degree set only includes matching degree 4. If matching degree 4 meets the matching requirements, the initial orbit B and the first orbit F corresponding to matching degree 4 are determined as the first combination, and all matching degrees (i.e., matching degree 4) associated with the initial orbit B and the first orbit F are deleted from the matching degree set.

[0083] After the second iteration, the matching set is empty, at which point the loop can be stopped.

[0084] As an example, see Figure 6 , Figure 6 This is the third schematic diagram of the scenario for determining the first combination provided in the embodiments of this application. For example... Figure 6As shown, if the initial tracks of streaming media data before the first moment are initial track A and initial track B, and the first tracks after the first moment are first track E and first track F.

[0085] Before the first loop begins, the matching degree 1 (the matching degree between the initial orbit A and the first orbit E), matching degree 2 (the matching degree between the initial orbit A and the first orbit F), matching degree 3 (the matching degree between the initial orbit B and the first orbit E), and matching degree 4 (the matching degree between the initial orbit B and the first orbit F) are determined as the matching degree set.

[0086] At the start of the first loop, it is determined whether there exists a matching degree among matching degrees 1-4 that meets the matching requirements. If matching degrees 1-3 all meet the matching requirements, but matching degree 4 does not, and when matching degrees 1 and 2 are both the highest matching degrees, since both matching degrees 1 and 2 are associated with the initial orbit A, either matching degree 1 or matching degree 2 can be determined as the target matching degree. Figure 6 As shown, a matching degree of 1 is determined as the target matching degree, and the initial orbit A and the first orbit E corresponding to the matching degree of 1 are determined as a first combination.

[0087] Further, all matching degrees (matching degree 1 to matching degree 3) associated with at least one of the initial orbit A or the first orbit E are removed from the matching degree set, and the process proceeds to the next loop.

[0088] For the second loop, since the current matching set only includes matching degree 4, and matching degree 4 does not meet the matching requirements, the loop can be stopped.

[0089] The process of determining the selection state of the first track will be further explained below with specific examples.

[0090] When media data begins to play, each initial track of the streaming data can be represented as: , This represents the set of initial orbits, where N is the number of initial orbits. The selection state of each initial orbit can be represented as follows: , When the value is true, it indicates that the corresponding initial track is selected. When the value is false, it means that the selection state of the corresponding initial track is not selected.

[0091] Similarly, each first orbit can be represented as , This represents the set of orbits formed by the first orbit, where M is the initial number of orbits.

[0092] Step 1, for For each first orbit, determine the relationship between that first orbit and... The matching degree of each initial orbit in the data, thus obtaining the following: A set of matching degrees with a matching degree of _____.

[0093] Step 2: Determine the highest matching degree that meets the matching degree requirement from the matching degree set. , , and the highest matching degree The initial orbit selection state is determined as the corresponding first orbit selection state, that is... .

[0094] Step 3, will be with All associated matches will be removed from the match set, and will be associated with All associated matches are removed from the match set.

[0095] Step 4: Iterate through steps 2-3 until the matching degree set is empty or there is no matching degree in the matching degree set that meets the matching degree requirements.

[0096] In the above embodiments, when the streaming media attributes of the streaming media data change at the first moment, for each first track after the first moment, if there is an initial track that matches the matching requirements, the selection state of the first track can be determined as selected; if there is no initial track that matches the matching requirements, the selection state of the first track can be determined as unselected. In this case, when playing media data of the first track that is selected after the first moment, the media data played after the first moment can maintain a certain similarity and continuity with the media data played before the first moment. For example, if the media data of the video track and the audio track were played before the first moment, and the media data of the video track and the audio track exist after the first moment, the media data of the audio track can also be played after the first moment, thereby improving the playback continuity and efficiency of the media data while enhancing the browsing experience.

[0097] In some feasible implementations, the track attributes of any track of streaming media data include a first attribute item, which is used to characterize the data type of the media data corresponding to the corresponding track.

[0098] The aforementioned data types include video types, audio types, and subtitle types.

[0099] In other words, for each track of streaming media data, the media content presented when playing the media data corresponding to that track is video content, audio content, or subtitle information.

[0100] Among them, the first attribute item in the orbital properties of any orbit can be adopted. type The fields are used for representation. This indicates that the corresponding media data is of the video type. This indicates that the corresponding media data is of the audio type. The time indicates that the data type of the corresponding media data is subtitle.

[0101] When determining the matching degree of any two tracks, the data types of the two tracks can be compared. If they are inconsistent, the matching degree of the two tracks is determined as the first value; if they are consistent, the matching degree of the two tracks is determined as the second value.

[0102] In some feasible implementations, when the data type of the media data corresponding to any track is audio, the track attribute of the track also includes at least one second attribute item from the language type or encoding method of the corresponding media data.

[0103] Specifically, when the data type of the media data corresponding to any track is audio, the second attribute item in the track attributes that represents the language type can be used as follows: Field representation. For example, At that time, the language type used to represent the corresponding media data is Chinese, that is, the corresponding audio language is Chinese. At that time, the language type used to represent the corresponding media data is English, that is, the corresponding audio language is English.

[0104] Specifically, when the data type of the media data corresponding to any track is audio, the second attribute item in the track attributes that represents the encoding method can be used... Field representation. For example, At that time, the encoding method used to represent the corresponding media data is That is, the corresponding output audio is (Dolby) audio. At that time, it is used to indicate that the corresponding media data is encoded using Advanced Audio Coding (ACC).

[0105] As an example, for any track, if the data type of the media data corresponding to that track is audio, the language type is Chinese, and the encoding method is ACR, the track attributes of that track can be represented in the following way:

[0106] When the data type of the media data corresponding to any track is subtitle type, the track attribute of the track also includes at least one second attribute item from the language type or function attribute of the corresponding media data.

[0107] Among them, the functional attributes are used to indicate the display method of the subtitles, such as top display, bottom display, etc.

[0108] Specifically, when the data type of the media data corresponding to any track is subtitle type, the second attribute item in the track attributes that represents the language type can be used as follows: Field representation. For example, At that time, the language type used to represent the corresponding media data is Chinese, that is, the corresponding subtitles are in Chinese. At that time, the language type used to represent the corresponding media data is English, that is, the corresponding subtitles are in English.

[0109] Specifically, when the data type of the media data corresponding to any track is subtitle type, the second attribute item in the track attributes that represents the functional attribute can be used as follows: Field representation. For example, At this time, the display method used to represent the corresponding media data (subtitles) is to display them at the top. At this time, the display method used to represent the corresponding media data (subtitles) is bottom display. At the same time, the display method used to represent the corresponding media data (subtitles) is coordinated. dolby (Dolby) mode display.

[0110] As an example, for any track, if the data type of the media data corresponding to that track is subtitle, the language type is Chinese, and the display mode is bottom display, the track attributes of that track can be represented in the following way:

[0111] When determining the matching degree of any two tracks, the data types of the two tracks can be compared. If they are inconsistent, the matching degree of the two tracks is determined as the first value. If they are consistent and the data type is video, the matching degree of the two tracks is determined as the second value.

[0112] For any two tracks whose corresponding data type is not video, if their data types are the same but not video, the matching degree of the two tracks can be determined as the third value.

[0113] The third value is the sum of the second and fourth values, and the fourth value is the number of overlaps in the second attribute items corresponding to the two tracks.

[0114] For example, when determining the matching degree between the initial track and the first track, if the data type corresponding to the initial track is inconsistent with the data type corresponding to the first track, the matching degree between the initial track and the first track is determined to be the first value.

[0115] If the data type corresponding to the initial track is the same as the data type corresponding to the first track, and the data type is video, then the matching degree between the initial track and the first track is determined to be the second value, which is greater than the first value.

[0116] If the data type corresponding to the initial track is consistent with the data type corresponding to the first track, and the data type is not a video type, the matching degree between the initial track and the first track is determined to be the third value. The third value is the sum of the second value and the fourth value. The fourth value is the number of second attribute items that overlap between the second attribute items corresponding to the initial track and the second attribute items corresponding to the first track.

[0117] As an example, suppose the first attribute is TYPE, and the second attribute includes LANGUAGE and NAME. Assume two tracks. and The three attribute items TYPE, LANGUAGE, and NAME are respectively represented as and , and , and Then their matching degree The calculations follow these rules: (1). If and If they match, the matching degree is... a And perform steps (2) and (3); if they do not match, the matching degree is b .

[0118]

[0119] in, a Greater than b .

[0120] (2). If and If they match, the matching degree is increased. k :

[0121] (3). If and If they match, the matching degree is increased. k :

[0122] in, k Greater than 0.

[0123] As an example, the above a , b as well as k These can be 0, -1, or 1. That is... b As the first value, a The second value, and Consistent and and When they are consistent, the fourth value is 2. k , and Consistent and and Inconsistent, or and Inconsistent and and When they are consistent, the fourth value is k , and Inconsistent and and When there is a discrepancy, the fourth value is 0.

[0124] Based on the above description, when determining whether there exists a matching degree in the matching degree set that meets the matching requirements, for any two tracks, if the matching degree is not the first value, then the matching degree is determined to meet the matching requirements. If the matching degree is the first value, then the matching degree is determined to not meet the matching requirements.

[0125] Optionally, when determining the matching degree between any two orbitals, the orbital feature vector of the corresponding orbital can also be determined based on the attribute items in the orbital attributes of each orbital. For example, a tuple (such as a triple) can be determined based on the corresponding attribute items of each orbital, and this tuple can be used as the orbital feature vector of the corresponding orbital.

[0126] Furthermore, the similarity of the orbital feature vectors of the two orbits can be determined, and the similarity can be defined as the matching degree.

[0127] In determining the similarity between the orbital feature vectors of two orbits, the orbital feature vectors can be input into a pre-trained computer model, and the computer model can output the similarity between the two, thereby obtaining the matching degree between the two orbits.

[0128] The aforementioned computer model can be trained based on artificial intelligence (AI) machine learning (ML) and cloud technology technologies such as cloud computing and artificial intelligence cloud services.

[0129] Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence.

[0130] Machine learning is the study of how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence.

[0131] In determining whether a matching degree exists in the matching degree set that meets the matching requirements, for any two tracks, if the matching degree is higher than a preset threshold, then the matching degree is determined to meet the matching requirements. If the matching degree is less than or equal to the preset threshold, then the matching degree is determined to not meet the matching requirements.

[0132] In the above embodiments, the selection state of each first track is determined by the track matching degree between each first track after the first moment and each initial track before the first moment. This ensures that when playing media data of a certain data type before the first moment, if media data of the same data type exists after the first moment, media data of the same data type can be played after the first moment. For example, if video and audio media data are played before the first moment, and video and audio media data exist after the first moment, the audio media data of the video media data can also be played after the first moment. This improves the playback continuity and efficiency of media data while enhancing the browsing experience.

[0133] Furthermore, for the track corresponding to the subtitle type, the matching degree is determined by the second attribute item of language type and / or functional attribute. If an initial track of data type "subtitle" exists before the first moment and a first track of data type "subtitle" exists after the first moment, the track data of the subtitle track selected after the first moment will be more matched with the track data of the subtitle track selected before the first moment in terms of language type and / or functional attribute. This further improves the continuity of the subtitle playback effect. For example, if English subtitles are played before the first moment, and both English and Chinese subtitles exist after the first moment, based on the embodiments of this application, English subtitles can still be played after the first moment. This helps maintain the continuity of the subtitle playback effect and allows for the continuation of the browsing user's subtitle selection habits, thus improving the browsing experience.

[0134] Similarly, for tracks corresponding to audio types, the matching degree is determined by the second attribute item of language type and / or encoding method. If an initial track of data type audio exists before the first moment and a first track of data type audio exists after the first moment, the track data of the audio track selected after the first moment will be more matched with the track data of the audio track selected before the first moment in terms of language type and / or encoding method. This further improves the continuity of audio playback. For example, if English dubbing is played before the first moment, and both English and Chinese dubbing exist after the first moment, based on the embodiments of this application, English dubbing can still be played after the first moment. This helps maintain the continuity of audio playback and allows the browsing user's audio selection habits to be followed, thus improving the browsing experience.

[0135] S24. Play the media data of the first track whose selection state is selected after the first moment.

[0136] In some feasible implementations, for each first track, when the first track is selected, it indicates that the media data corresponding to the first track is associated with the media data played before the first moment.

[0137] For example, for each first track, if the data type of the media data corresponding to that first track is video, and there exists a first combination including that first track, it means that the streaming media data also had an initial track with an audio data type before the first moment. When that initial track is played before the first moment, both the initial track and the first track are selected. If that first track is played after the first moment, video content can still be output after the first moment, thus ensuring that the media content output before and after the first moment remains consistent in terms of data type.

[0138] Similarly, for each first track, if the data type of the media data corresponding to that first track is audio or subtitle, and there exists a first combination including that first track, it means that the streaming media data also had an initial track of the same data type before the first moment. When that initial track is played before the first moment, both the initial track and the first track are selected. If that first track is played after the first moment, media information of the same data type can also be output after the first moment, thus ensuring that the media content output before and after the first moment remains consistent at the data type level.

[0139] Based on this, after determining the selection status of each first track, media data of the first track with the selected status can be played, thereby improving the continuity of media content output and enhancing the browsing experience.

[0140] In some feasible implementations, during media data playback, when the streaming media attributes change at a second time after the first time, the number of tracks and track attributes of the streaming media data after the second time may also change. To maintain the continuity of media data playback, it is necessary to redetermine the selection state of each track of the streaming media data after the second time (hereinafter referred to as the second track for ease of description), and play the second track that is selected after the second time.

[0141] Specifically, when there is no third orbit in the initial orbit, the selection state of the second orbit can be determined based on the selection state of the first orbit. The specific determination method is as described above, which determines the orbit state of the first orbit based on the selection state of the initial orbit, and will not be repeated here.

[0142] The third track is the initial track that does not exist in any of the first combinations and is in the selected state.

[0143] If a third track exists in the initial track, it means that there is no track with the same data type as the third track in the first track after the first moment. If the selection state of the second track is determined directly based on the selection state of the first track, the selection state of the second track with the same data type as the third track will be determined as unselected, and thus media content with the same data type as the third track cannot continue to be played after the second moment.

[0144] For example, suppose the initial track corresponding to the streaming media data is:

[0145] in, For video tracks, For audio tracks.

[0146] When each initial track is selected, the media data played includes both audio and video data.

[0147] If the first track of the streaming data after the first moment is (e.g., a segment without audio ads) will have its first track selected based on the initial track selection status. At this point, the media data played after the first moment will include video data.

[0148] If the second track corresponding to the streaming data after the second moment reverts to the initial track before the first moment, then based on the aforementioned method, the selection state of the second track corresponding to the audio type can be determined as unselected, and the selection state of the second track corresponding to the video type can be determined as selected. Therefore, the media data played after the second moment will only include video data and not audio data. For example, after an advertisement finishes playing, only video content will be played, and audio content will not.

[0149] Based on this, when there is a third track in the initial track, the selection status of each second track after the second moment can be determined according to the selection status of each third track and the selection status of each first track, and then the media data of the second track with the selection status selected can be played after the second moment.

[0150] Specifically, the third track and the first track can both be regarded as the aforementioned initial track, and the second track can be regarded as the aforementioned first track. Then, the selection state of the first track can be determined in the same way as the selection state of the initial track. The selection state of the second track can be determined according to the selection states of the third track and the first track, which will not be elaborated here.

[0151] Optionally, the track attributes of each second track can be determined, and the matching degree between each second track and each first track can be determined based on the track attributes of each first track and the track attributes of each second track.

[0152] Furthermore, based on all the determined matching degrees, it can be determined whether at least one second combination exists. Each second combination includes a first track and a second track, and the matching degree of the first track and the second track in each second combination meets the matching requirements.

[0153] In this combination, the first and second orbitals in any two second combinations are different. For example, for any two second combinations: combination A and combination B, the first orbital in combination A is different from the first orbital in combination B, and the second orbital in combination A is different from the second orbital in combination B.

[0154] In other words, for any given second track, if multiple first tracks match the matching requirements with that second track, only one first track and that second track constitute a second combination. Similarly, if multiple second tracks match the matching requirements with the same first track, only one second track and that first track constitute a second combination.

[0155] In this case, when at least one second combination exists, that is, when at least one second combination is determined, the selection state of the first track in each second combination can be determined as the selection state of the corresponding second track. When a fourth track exists, the selection state of each fourth track is determined based on the selection state of each third track.

[0156] At this point, the fourth track is a second track that does not exist in any second combination, that is, there is no second track in the first track that matches the matching requirements.

[0157] When there is no second combination, that is, when no second combination is determined, each second track can be regarded as a fourth track, and then the selection state of the fourth track can be determined according to the selection state of each third track.

[0158] Furthermore, when determining the selection state of each fourth track based on the selection state of each third track, the track attributes of each third track and the track attributes of each fourth track can be determined first, and then the matching degree between each fourth track and each third track can be determined. Then, based on all the determined matching degrees, it can be determined whether there is at least one third combination.

[0159] Each third combination includes a third track and a fourth track, and the matching degree between the fourth track and the third track in each third combination meets the matching requirements.

[0160] In this case, the fourth orbital and the third orbital are different in any two third combinations. For example, for any two third combinations: combination A and combination B, the fourth orbital in combination A is different from the fourth orbital in combination B, and the third orbital in combination A is different from the third orbital in combination B.

[0161] In other words, for any fourth orbital, if multiple third orbitals match the matching requirements with the fourth orbital, only one third orbital and the fourth orbital form a third combination. Similarly, if multiple fourth orbitals match the matching requirements with the same third orbital, only one fourth orbital and the third orbital form a third combination.

[0162] In this case, when there is at least one third combination, that is, when at least one third combination is determined, the selection state of the third track in each third combination can be determined as the selection state of the corresponding fourth track, and the selection state of each fourth track that does not exist in any third combination can be determined as an unselected state.

[0163] When there is no third combination, that is, when no third combination has been determined, the selection state of each fourth track can be set to unselected.

[0164] It should be noted that the methods for determining the second and third combinations are the same as those for determining the first combination, and will not be repeated here.

[0165] As an example, see Figure 7 , Figure 7 This is a schematic diagram of another scenario for determining the selection state provided in an embodiment of this application. For example... Figure 7 As shown, if the initial tracks corresponding to the streaming media data are track A, track B, and track C when the streaming media data starts playing, and it is determined that the selection status of track A and track B is selected, while the selection status of track C is unselected, then the media data corresponding to track A and track B will be played.

[0166] When the first track corresponding to the streaming media data after the first moment only includes track A, the implementation method of determining the selection state of the first track based on the initial track selection state can determine that the selection state of track A after the first moment is selected, and then the media data corresponding to the track is played after the first moment.

[0167] After the second moment, the second track corresponding to the streaming media data is restored to track A, track B, and track E. Based on the selection status of track B (the first track) after the first moment, the selection status of track B after the second moment can be predetermined. Then, track A before the first moment is considered the third track, and tracks A and E after the second moment are considered the fourth track. The selection status of the fourth track is determined based on the selection status of the third track; for example, if track A after the second moment is selected, and track E after the second moment is unselected. Based on this, media data from tracks B and A can be played after the second moment, achieving the same playback as the media data played before the first moment, thus maintaining the continuity and consistency of media data playback.

[0168] In the above embodiments, when the streaming media attributes of the streaming media data change at the second moment, if there is no first track in the first track that matches the matching requirements of the initial track in a certain selected state, it means that there is no first track in the first track with the same data type as the initial track. Therefore, media data of the data type corresponding to that data track will not be played after the first moment. In this case, the selection state of each second track after the second moment can be determined by the selection state of the initial track and the selection state of each first initial track. Thus, even if there is no first track with the same data type as the initial track in the first track, and there is a second track with the same data type as the initial track in the second track after the second moment, media data of the same data type as the initial track can still be played at the second moment. This makes the data type of the media data played after the second moment more complete, avoiding the situation where some data type media information is missing due to the absence of tracks corresponding to some data types, thereby improving the continuity and consistency of media data playback at the data type level.

[0169] For example, when initially acquiring streaming media data, the streaming media data includes media data for the initial track corresponding to the audio type, media data for the initial track corresponding to the video type, and media data for the initial track corresponding to the subtitle type. When each initial track is in default playback mode, the selection state of each initial track is set to selected, and the media data of each initial track is played. At this time, the presented media content includes audio, video, and subtitles.

[0170] When the streaming media attributes of the streaming media data change at a first moment, and the streaming media data after the first moment includes at least one first track of media data corresponding to an audio type and at least one first track of media data corresponding to a video type, the data processing method provided in this application embodiment can determine the selection state of one first track corresponding to an audio type as selected, and determine the selection state of one first track corresponding to a video type as selected, thereby playing the media data of the first track in the selected state at the first moment. In this case, the final presented media content includes both audio and video.

[0171] Furthermore, when the streaming media attributes of the streaming media data change at a second time after the first time, and the streaming media data after the second time includes at least one second track of media data corresponding to the audio type, at least one second track of media data corresponding to the video type, and at least one second track of media data corresponding to the subtitle type, since the media information presented after the first time does not include subtitle information, the selection state of one second track corresponding to the audio type, one second track corresponding to the video type, and one second track corresponding to the subtitle type can be determined as selected based on the initial selection state (selected state) of the track corresponding to the subtitle type and the selection state of each first track, and the media data of the second track in the selected state can be played after the second time. In this case, the final presented media content is consistent with the media content presented at the beginning, including audio, video, and subtitles.

[0172] Understandably, during media data playback, when the streaming media attributes change at a third time after the second time, the selection status of the track after the third time (which can be considered the initial track) can be determined based on the track selection status between the first and second times (which can be considered the initial track) and the track selection status between the second and third times (which can be considered the first track). Then, media data on the selected track will be played after the third time. Similarly, when the streaming media attributes change at a fourth time after the third time, the selection status of the track after the fourth time (which can be considered the second track) can be determined based on the track selection status between the second and third times (which can be considered the initial track) and the track selection status between the third and fourth times (which can be considered the first track). Then, media data on the selected track will be played after the fourth time.

[0173] In some feasible implementations, during media data playback, in response to the browsing object's close operation on the fifth track, the selection state of the fifth track is changed from selected to unselected, and the playback of the media data on the fifth track is terminated.

[0174] The aforementioned close operation is used to stop playing the media data of the fifth track, which is the track corresponding to the media data currently being played.

[0175] For example, if the media data corresponding to the fifth track is a subtitle, and the subtitle is currently being played, the selection status of the fifth track can be changed from selected to unselected in response to the browsing object's close operation on the fifth track, thereby stopping the playback of the subtitle.

[0176] In some feasible implementations, during media data playback, in response to the browsing object's activation operation for the sixth track, the selection state of the sixth track is changed from unselected to selected, and the media data of the sixth track is started to play.

[0177] The aforementioned start operation is used to begin playing the media data of the sixth track, which is the track corresponding to the currently unplayed media data.

[0178] For example, if the media data corresponding to the sixth track is audio data, when a video is currently playing, the system can respond to the browsing object's activation of the sixth track, thereby changing the selection status of the sixth track from unselected to selected, and thus start playing the media data of the sixth track, realizing the output of audio data.

[0179] In the above embodiments, during the playback of streaming media data, the selection status of the corresponding track can be changed in real time according to the browsing object's opening or closing operation of the track, thereby realizing playback control of the corresponding media data, which is conducive to improving the flexibility of streaming media data playback and the browsing experience of the browsing object.

[0180] In this application embodiment, the track attributes of any track, the matching degree between any two tracks, and the selection status of any track can be stored in a media data playback device or in an associated storage space. This storage space includes, but is not limited to, cloud storage, databases, blockchains, etc. The specific storage space can be determined based on the actual application scenario requirements and is not limited here.

[0181] In short, a database can be viewed as an electronic filing cabinet—a place to store electronic files. It can be a relational database (SQL database) or a non-relational database (NoSQL database), without limitation. In this application, it can be used to store the track attributes of any track, the matching degree between any two tracks, and the selection status of any track. Blockchain is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. In the embodiments of this application, each data block in the blockchain can store the track attributes of any track, the matching degree between any two tracks, and the selection status of any track. Cloud storage is a new concept extended and developed from the concept of cloud computing. It refers to the use of cluster applications, grid technology, and distributed storage file systems to aggregate a large number of various types of storage devices (also called storage nodes) in a network through application software or application interfaces to work together to store the track attributes of any track, the matching degree between any two tracks, and the selection status of any track.

[0182] The process of determining the selection state and calculating the matching degree in the embodiments of this application can be implemented based on cloud computing. Cloud technology refers to a hosting technology that unifies hardware, software, network, and other resources within a wide area network (WAN) or local area network (LAN) to achieve data computation, storage, processing, and sharing. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, and application technology applied in the cloud computing business model. It can form resource pools, be used on demand, and is flexible and convenient.

[0183] Cloud computing is a computing paradigm, a product of the convergence of traditional computer and network technologies such as grid computing, distributed computing, parallel computing, utility computing, network storage technologies, virtualization, and load balancing. Cloud computing distributes computing tasks across a resource pool composed of numerous computers, enabling various application systems to access computing power, storage space, and information services as needed. The network providing these resources is called the "cloud," and the resources in the "cloud" are infinitely scalable, readily available, on-demand, expandable, and pay-as-you-go.

[0184] In this embodiment, by determining the previous matching degree to determine the aforementioned first combination, second combination, and third combination, it is possible to determine the track combination whose data type remains consistent before and after the change of streaming media attributes. In this way, the selection status of the two tracks within each track combination remains consistent, so that even if the streaming media attributes change, the browsing object can still continue to watch the media data of its preferred data type, thereby improving the browsing object's viewing experience.

[0185] Furthermore, according to Figure 2 The implementation shown, as well as the aforementioned implementation of determining the selection state of the second track based on the initial track and the selection state of the first track, can maintain the stability and continuity of media data playback even when changes in streaming media attributes cause changes or loss of some tracks, thereby improving the quality and efficiency of media data playback.

[0186] See Figure 8 , Figure 8 This is a schematic diagram of the structure of the data processing apparatus provided in the embodiments of this application. The data processing apparatus provided in the embodiments of this application includes: The data acquisition module 81 is used to acquire streaming media data, which includes media data of at least one initial track; The data playback module 82 is used to determine the selection status of each of the above-mentioned initial tracks and play the media data of the initial track whose selection status is selected. The state determination module 83 is used to determine the selection state of each first track of the streaming media data after the first moment in response to a change in the streaming media attributes of the streaming media data at the first moment during the media data playback process, based on the selection state of each initial track. The aforementioned data playback module 82 is used to play media data of the first track whose selection state is selected after the aforementioned first moment.

[0187] In some feasible implementations, the aforementioned state determination module 83 is used for: Determine the track attributes of each of the aforementioned initial tracks, and the track attributes of each of the aforementioned first tracks of streaming media data after the aforementioned first moment; Based on the orbital attributes of each of the aforementioned initial orbits and the orbital attributes of each of the aforementioned first orbits, determine the matching degree between each of the aforementioned first orbits and each of the aforementioned initial orbits; Determine whether there exists at least one first combination, each of the first combinations including one first orbit and one initial orbit, wherein the matching degree of the first orbit and the initial orbit in each of the first combinations meets the matching requirements, and the first orbit and the initial orbit in any two of the first combinations are different; When at least one of the above first combinations exists, the selection state of the initial track in each of the above first combinations is determined as the selection state of the corresponding first track, and the selection state of each of the above first tracks that does not exist in any of the above first combinations is determined as an unselected state. When the first combination mentioned above does not exist, the selection state of each of the first tracks mentioned above is determined to be unselected.

[0188] In some feasible implementations, the aforementioned state determination module 83 is further configured to: During media data playback, in response to the change in the streaming media attributes of the streaming media data at a second time after the first time, the selection status of each second track of the streaming media data after the second time is determined based on the selection status of each third track and the selection status of each first track; wherein, each third track is an initial track that does not exist in any of the first combinations and whose selection status is selected. After the second moment mentioned above, play the media data of the second track, which is in the selected state.

[0189] In some feasible implementations, the aforementioned state determination module 83 is used for: Determine the track attributes of each second track of the aforementioned streaming media data after the aforementioned second moment; Based on the track attributes of each of the first tracks and the track attributes of each of the second tracks, determine the matching degree between each of the second tracks and each of the first tracks. Determine whether there exists at least one second combination, each of the second combinations including one first track and one second track, wherein the matching degree of the first track and the second track in each second combination meets the matching requirements, and the first track and the second track in any two of the second combinations are different; When at least one of the above second combinations exists, the selection state of the first track in each of the above second combinations is determined as the selection state of the corresponding second track; when a fourth track exists, the selection state of each fourth track is determined according to the selection state of each of the above third tracks. When the second combination mentioned above does not exist, the selection state of each of the fourth tracks mentioned above is determined according to the selection state of each of the third tracks mentioned above; Wherein, when at least one of the above second combinations exists, each of the above fourth orbits is a second orbit that does not exist in any of the above second combinations; when no of the above second combinations exists, each of the above fourth orbits is a second orbit.

[0190] In some feasible implementations, the aforementioned state determination module 83 is used for: Based on the orbital attributes of each of the aforementioned third orbitals and the orbital attributes of each of the aforementioned fourth orbitals, determine the matching degree between each of the aforementioned fourth orbitals and each of the aforementioned third orbitals; Determine whether there exists at least one third combination, each of the above third combinations including one of the above third orbitals and one of the above fourth orbitals, the matching degree of the third orbital and the fourth orbital in each of the above third combinations meets the matching requirements, and the third orbital and the fourth orbital in any two of the above third combinations are different; When at least one of the above third combinations exists, the selection state of the fourth track in each of the above third combinations is determined to be selected, and the selection state of the other above fourth tracks is determined to be unselected. When the third combination mentioned above does not exist, the selection status of each of the fourth tracks mentioned above is determined to be unselected.

[0191] In some feasible implementations, the first combination described above is determined by the data playback module 82 based on a loop process, and each loop process is implemented by the data playback module 82 in the following manner: Determine if there is a matching score in the set of matching scores obtained in the previous loop that meets the matching requirements; If there is a matching degree that meets the matching requirements in the matching degree set obtained in the previous loop, then the initial track and the first track corresponding to each target matching degree are determined as a first combination. All matching degrees associated with the first track and the initial track in each of the above first combinations are deleted from the matching degree set obtained in the previous loop, and the next loop process is entered. The target matching degree mentioned above is the highest matching degree among the matching degrees that meet the matching requirements. When there are multiple first orbits and the same initial orbit with the highest first matching degree, any one of the first matching degrees is determined as the target matching degree. When there are multiple initial orbits and the same first orbit with the highest second matching degree, any one of the second matching degrees is determined as the target matching degree. If the matching degree set obtained in the previous iteration does not contain a matching degree that meets the matching requirements, or if the matching degree set obtained in the previous iteration is empty, then the iteration process stops. The matching degree set at the beginning of the first cycle includes the matching degree between each of the first tracks and each of the initial tracks.

[0192] In some feasible implementations, the track attributes of any track include a first attribute item, which is used to characterize the data type of the media data corresponding to the track. The data type includes at least one of video type, audio type, or subtitle type.

[0193] In some feasible implementations, when the data type of the media data corresponding to any track is audio, the track attribute of the track also includes at least one second attribute item used to characterize the language type or encoding method of the corresponding media data; when the data type of the media data corresponding to any track is subtitle, the track attribute of the track also includes at least one second attribute item used to characterize the language type or functional attribute of the corresponding media data.

[0194] In some feasible implementations, for any of the aforementioned initial tracks and any of the aforementioned first tracks, the aforementioned data playback module 82 is configured to: When the data type corresponding to the initial track is inconsistent with the data type corresponding to the first track, the matching degree between the initial track and the first track is determined to be the first value; When the data type corresponding to the initial track is consistent with the data type corresponding to the first track, and the data type is video, the matching degree between the initial track and the first track is determined to be the second value, which is greater than the first value. When the data type corresponding to the initial track is consistent with the data type corresponding to the first track, and the data type is not a video type, the matching degree between the initial track and the first track is determined to be the third value. The third value is the sum of the second and fourth values. The fourth value is the number of second attribute items that overlap between the second attribute items corresponding to the initial track and the second attribute items corresponding to the first track.

[0195] In some feasible implementations, the data playback module 82 described above is further used for: At any point during media data playback, in response to the browsing object's close operation on the fifth track, the selection state of the fifth track is changed from selected to unselected, and the playback of the media data on the fifth track is terminated. The close operation is used to stop the playback of the media data on the fifth track, where the fifth track is any track corresponding to the media data being played. At any point during media data playback, in response to the aforementioned browsing object's activation operation for the sixth track, the selection state of the sixth track is changed from unselected to selected, and the media data of the sixth track is played simultaneously with the media data of other tracks. The aforementioned activation operation is used to activate the playback of the media data of the sixth track, which is the track corresponding to any unplayed media data.

[0196] In some feasible implementations, the data playback module 82 described above is used for: Determine the default playback state for each of the initial tracks mentioned above; Set the initial track that has enabled default playback to the selected state, and set the initial track that does not have default playback to the unselected state.

[0197] In specific implementation, the aforementioned data processing device can perform the above-described actions through its built-in functional modules. Figure 2 The implementation methods provided for each step are detailed in the above-mentioned implementation methods, and will not be repeated here.

[0198] See Figure 9 , Figure 9 This is a schematic diagram of the structure of the electronic device provided in an embodiment of this application. For example... Figure 9As shown, the electronic device 900 in this embodiment may include: a processor 901, a network interface 904, and a memory 905. Furthermore, the electronic device 900 may also include: an object interface 903, and at least one communication bus 902. The communication bus 902 is used to implement communication between these components. The object interface 903 may include a display screen and a keyboard; optionally, the object interface 903 may also include a standard wired interface or a wireless interface. The network interface 904 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 905 may be a high-speed RAM or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory 905 may also be at least one storage device located remotely from the aforementioned processor 901. Figure 9 As shown, the memory 905, which is a computer-readable storage medium, may include an operating system, a network communication module, an object interface module, and a device control application.

[0199] in, Figure 9 The electronic device can be Figure 1 The server shown can also be Figure 1 The terminal shown can also be any other device with streaming media data processing and playback capabilities; no restrictions are imposed here.

[0200] exist Figure 9 In the illustrated electronic device 900, the network interface 904 provides network communication functionality; the object interface 903 is primarily used to provide an input interface for objects; and the processor 901 can be used to call the device control application stored in the memory 905 to implement the steps in the aforementioned method embodiments of this application. For example, in a specific implementation, the electronic device 900 can execute the above-described steps through its built-in functional modules. Figure 2The implementation methods provided for each step are not detailed here. It should be understood that in some feasible implementations, the processor 901 described above can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The memory can include read-only memory and random access memory, and provides instructions and data to the processor. A portion of the memory can also include non-volatile random access memory. For example, the memory can also store device type information.

[0201] This application also provides a computer-readable storage medium storing a computer program that is executed by a processor to implement... Figure 2 The methods provided in each step are detailed in the implementation methods provided in the above steps, and will not be repeated here.

[0202] The aforementioned computer-readable storage medium can be an internal storage unit of the data processing apparatus or electronic device provided in any of the foregoing embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device. The aforementioned computer-readable storage medium can also include magnetic disks, optical disks, read-only memory (ROM), or random access memory (RAM), etc. Furthermore, the computer-readable storage medium can include both internal storage units and external storage devices of the electronic device. The computer-readable storage medium is used to store the computer program and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0203] This application provides a computer program product, which includes a computer program that is executed by a processor. Figure 2The methods provided for each step in the process.

[0204] The terms "first," "second," etc., used in the claims, description, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or electronic device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or electronic devices. References to "embodiment" herein mean that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The presentation of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments. The term "and / or" as used in this application's description and appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0205] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Those skilled in the art can implement the described functions using different methods for each specific application, but such implementations should not be considered beyond the scope of this application.

[0206] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A data processing method, characterized in that, The method includes: Acquire streaming media data, which includes media data from at least one initial track; Determine the selection status of each initial track, and play the media data of the initial track whose selection status is selected; During media data playback, in response to a change in the streaming media attributes of the streaming media data at a first moment, the selection status of each first track of the streaming media data after the first moment is determined based on the selection status of each initial track. Play the media data of the first track whose selection state is selected after the first moment.

2. The method according to claim 1, characterized in that, The step of determining the selection status of each first track of streaming media data after the first moment, based on the selection status of each initial track, includes: Determine the track attributes of each initial track and the track attributes of each first track of the streaming media data after the first moment; Based on the orbital attributes of each initial orbit and the orbital attributes of each first orbit, determine the matching degree between each first orbit and each initial orbit; Determine whether there exists at least one first combination, each first combination including a first orbit and an initial orbit, wherein the matching degree of the first orbit and the initial orbit in each first combination meets the matching requirements, and the first orbit and the initial orbit in any two first combinations are different; When at least one of the first combinations exists, the selection state of the initial track in each of the first combinations is determined as the selection state of the corresponding first track, and the selection state of each of the first tracks that does not exist in any of the first combinations is determined as an unselected state. When the first combination does not exist, the selection state of each of the first tracks is determined to be unselected.

3. The method according to claim 2, characterized in that, The method further includes: During media data playback, in response to a change in the streaming media attributes of the streaming media data at a second time after the first time, the selection status of each second track of the streaming media data after the second time is determined based on the selection status of each third track and the selection status of each first track; wherein, each third track is an initial track that does not exist in any of the first combinations and whose selection status is selected. After the second moment, play the media data of the second track, which is in the selected state.

4. The method according to claim 3, characterized in that, Determining the selection status of each second track of the streaming media data after the second moment, based on the selection status of each third track and the selection status of each first track, includes: Determine the track attributes of each second track of the streaming media data after the second moment; Based on the track attributes of each first track and the track attributes of each second track, determine the matching degree between each second track and each first track; Determine whether there exists at least one second combination, each second combination including one first track and one second track, the matching degree of the first track and the second track in each second combination meets the matching requirements, and the first track and the second track in any two second combinations are different; When at least one of the second combinations exists, the selection state of the first track in each of the second combinations is determined as the selection state of the corresponding second track; when a fourth track exists, the selection state of each fourth track is determined according to the selection state of each of the third tracks. When the second combination does not exist, the selection state of each of the fourth tracks is determined according to the selection state of each of the third tracks; Wherein, when at least one of the second combinations exists, each of the fourth orbits is a second orbit that does not exist in any of the second combinations; when no second combination exists, each of the fourth orbits is a second orbit.

5. The method according to claim 4, characterized in that, Determining the selection state of each of the fourth tracks based on the selection state of each of the third tracks includes: Based on the orbital attributes of each third orbit and the orbital attributes of each fourth orbit, determine the matching degree between each fourth orbit and each third orbit; Determine whether there exists at least one third combination, each third combination including one third track and one fourth track, wherein the matching degree of the third track and the fourth track in each third combination meets the matching requirements, and the third track and the fourth track in any two third combinations are different; When at least one of the third combinations exists, the selection state of the fourth track in each third combination is determined to be selected, and the selection state of the other fourth tracks is determined to be unselected. When the third combination does not exist, the selection status of each of the fourth tracks is determined to be unselected.

6. The method according to claim 2, characterized in that, The first combination is determined based on a loop process, and each loop process is implemented in the following way: Determine if there is a matching score in the set of matching scores obtained in the previous loop that meets the matching requirements; If there is a matching degree that meets the matching requirements in the matching degree set obtained in the previous loop, then the initial track and the first track corresponding to each target matching degree are determined as a first combination, and all matching degrees associated with the first track and the initial track in each first combination are deleted from the matching degree set obtained in the previous loop, and the next loop process is entered. Wherein, the target matching degree is the highest matching degree among the matching degrees that meet the matching requirements. When there are multiple first orbits and the same initial orbit with the highest first matching degree, any one of the first matching degrees is determined as the target matching degree; when there are multiple initial orbits and the same first orbit with the highest second matching degree, any one of the second matching degrees is determined as the target matching degree. If the matching degree set obtained in the previous iteration does not contain a matching degree that meets the matching requirements, or if the matching degree set obtained in the previous iteration is empty, then the iteration process stops. The matching degree set at the start of the first loop includes the matching degree between each of the first tracks and each of the initial tracks.

7. The method according to claim 2, characterized in that, The track attributes of any track include a first attribute item, which is used to characterize the data type of the media data corresponding to the track. The data type includes at least one of video type, audio type, or subtitle type.

8. The method according to claim 7, characterized in that, When the data type of the media data corresponding to any track is audio, the track attribute of the track also includes at least one second attribute item used to characterize the language type or encoding method of the corresponding media data; when the data type of the media data corresponding to any track is subtitle, the track attribute of the track also includes at least one second attribute item used to characterize the language type or functional attribute of the corresponding media data.

9. The method according to claim 8, characterized in that, For any initial orbit and any first orbit, the matching degree between the initial orbit and the first orbit is determined based on the orbital attributes of the initial orbit and the first orbit, including: When the data type corresponding to the initial track is inconsistent with the data type corresponding to the first track, the matching degree between the initial track and the first track is determined to be the first value; When the data type corresponding to the initial track is the same as the data type corresponding to the first track, and the data type is video, the matching degree between the initial track and the first track is determined to be a second value, and the second value is greater than the first value; When the data type corresponding to the initial track is consistent with the data type corresponding to the first track, and the data type is not a video type, the matching degree between the initial track and the first track is determined to be a third value. The third value is the sum of the second value and the fourth value. The fourth value is the number of second attribute items that overlap between the second attribute items corresponding to the initial track and the second attribute items corresponding to the first track.

10. The method according to claim 1, characterized in that, The method further includes at least one of the following: At any point during media data playback, in response to the browsing object's close operation on the fifth track, the selection state of the fifth track is changed from selected to unselected, and the playback of the media data on the fifth track is terminated. The close operation is used to stop the playback of the media data on the fifth track, where the fifth track is any track corresponding to the media data being played. At any point during media data playback, in response to the browsing object's activation operation for the sixth track, the selection state of the sixth track is changed from unselected to selected, and the media data of the sixth track is played while the media data of other tracks are being played. The activation operation is used to start playing the media data of the sixth track, which is the track corresponding to any unplayed media data.

11. The method according to claim 1, characterized in that, Determining the selection state of each of the initial orbits includes: Determine the default playback state for each of the initial tracks; Set the initial track that has enabled default playback to the selected state, and set the initial track that does not have default playback to the unselected state.

12. A data processing apparatus, characterized in that, The device includes: A data acquisition module is used to acquire streaming media data, which includes media data from at least one initial track; The data playback module is used to determine the selection state of each initial track and play the media data of the initial track whose selection state is selected. The state determination module is used to determine the selection state of each first track of the streaming media data after the first moment in response to a change in the streaming media attributes of the streaming media data at the first moment during media data playback, based on the selection state of each initial track. The data playback module is used to play media data of the first track whose selection state is selected after the first moment.

13. An electronic device, characterized in that, It includes a processor and a memory, which are interconnected; The memory is used to store computer programs; The processor is configured to execute the method according to any one of claims 1 to 11 when the computer program is invoked.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that is executed by a processor to implement the method of any one of claims 1 to 11.

15. A computer program product, characterized in that, The computer program product includes a computer program that is executed by a processor to implement the method of any one of claims 1 to 11.

Images