Method for processing streaming data, electronic device and storage medium

Abstract

The application provides a streaming media data processing method, an electronic device and a storage medium, and belongs to the technical field of streaming media file protection. The streaming media data processing method of the application embodiment determines a plaintext position according to the playing habit of a user, thereby improving the probability that a target segment specified by the user hits the plaintext when the user fast forwards and fast rewinds, saving decryption time, and setting a preset range when a mobile phone acquires a starting segment. The plaintext or ciphertext that has been downloaded is acquired as much as possible within the preset range, the probability of hitting the plaintext or the downloaded ciphertext is improved, network transmission delay and decryption time are reduced as much as possible, and the playing experience of the user is improved.

Description

Technical Field

[0001] This application relates to the field of streaming media data protection technology, and in particular to methods for processing streaming media data, electronic devices, and storage media. Background Technology

[0002] Currently, with the development of computer and network technology, more and more people are playing streaming media files on electronic devices, such as playing the same song or video on mobile phones, computers, or watches. As streaming media files represent the intellectual work of their creators, it is crucial to consider how to better protect the creators' labor as they are widely transmitted online. Typically, encryption (ciphertext) is used, requiring users to pay to download and play the song. Simultaneously, only when the work is widely published and disseminated, gaining public recognition and acceptance, can the creator reap greater benefits. To achieve a balance between protection and dissemination, content providers (e.g., those who own or license the copyright as creators) often allow users to play a certain amount of content for free. This free content is usually distributed unencrypted (plaintext) to ensure that this playback method reaches as many users as possible.

[0003] During the playback of streaming media files, if there are excessively long start-up delays or playback stutters, it will reduce the user experience, causing users to abandon the current song and easily leading to a significant loss of users for the content distribution platform. Summary of the Invention

[0004] In view of this, the present invention provides a method for processing streaming media data, an electronic device, and a storage medium, which can better balance product protection and product dissemination, and improve user experience.

[0005] This application provides a streaming media data processing method, an electronic device, a storage medium, and a computer program product, among other embodiments. The application is described below from multiple perspectives, and the embodiments and beneficial effects described below can be referenced interchangeably.

[0006] In a first aspect, the present invention provides a method for processing streaming media data, applied to a system including a server and a first electronic device. The method includes: the server dividing a streaming media file into several streaming media segments of a predetermined time length; the server determining, according to a preset rule, the streaming media segments to be plaintext from the several streaming media segments, wherein the preset rule is set based on the playback habits of a preset user group, and the plaintext is an unencrypted streaming media segment; the server encrypting the streaming media segments other than the plaintext to obtain ciphertext; the server setting a pre-defined strategy for the first electronic device to acquire streaming media segments, and generating a description file based on the pre-defined strategy to record the position information of the plaintext and ciphertext in the streaming media file; wherein the pre-defined strategy includes a priority order of the plaintext and ciphertext being acquired by the first electronic device; the first electronic device acquiring the description file, and sequentially acquiring streaming media segments from the streaming media file based on the description file for processing by the first electronic device. It should be noted that the steps of dividing the streaming media file into several streaming media segments of a predetermined time length, determining the plaintext from the several streaming media segments, and encrypting the streaming media segments other than the plaintext to obtain ciphertext in this application can also be performed by the first electronic device. The following description uses a server as an example and is not intended to limit the scope of the application.

[0007] The streaming media data processing method according to embodiments of this application combines a large amount of historical playback data from users to determine the playback habits of most users. It treats the streaming media segments most likely to be specified by the user when fast-forwarding, rewinding, or pausing playback as plaintext. This ensures that when the user resumes playback after fast-forwarding, rewinding, or pausing, the corresponding streaming media segment at the progress position during these actions is as plaintext as possible, thereby saving the time spent decrypting the streaming media segments. By employing this processing method, the probability of the user-specified target segment hitting plaintext is increased when playing streaming media files. This minimizes decryption and download time, avoids stuttering, and improves the user experience.

[0008] In one possible implementation of the first aspect described above, the server determines the streaming media segments that are plaintext from a plurality of streaming media segments according to a preset rule, including: the server calculating the total number of plaintext segments in the streaming media file based on the total duration of plaintext in the streaming media file that can be provided to the user and the predetermined duration of each streaming media segment; the server calculating the number of plaintext segments in the header and the number of plaintext segments in other positions based on the total number of plaintext segments and a first proportional relationship, wherein the first proportional relationship is expressed as the preset ratio of the number of plaintext segments in the header of the streaming media file to the number of plaintext segments in other positions of the streaming media file. The plaintext segment in the header of the streaming media file refers to the first plaintext segment that starts counting from zero seconds in the streaming media file according to the playback order, or includes the first plaintext segment and a plurality of plaintext segments that are consecutively arranged adjacent to the first plaintext segment; the server determines the streaming media segments that are plaintext segments in the header of the streaming media file based on the number of plaintext segments in the header, and determines the streaming media segments that are plaintext segments in other positions based on the number of plaintext segments in other positions of the streaming media file. It can determine the amount of plaintext when the streaming media file is played at the beginning, and also ensure the amount of plaintext in other positions. This ensures that the plaintext is hit in both normal and abnormal playback scenarios (fast forward, rewind, or pause), thus improving the user experience.

[0009] In one possible implementation of the first aspect described above, the server prioritizes the plaintext of the header as the highest priority for selecting plaintext. When a user plays a streaming media file, the plaintext of the header is the most likely text the user wants to play. Therefore, prioritizing the plaintext of the header improves the user's playback experience.

[0010] In one possible implementation of the first aspect described above, the server determines the location of plaintext at other locations based on the amount of plaintext in other locations of the streaming media file. This includes: the server acquiring the target segment selected by each user in a preset user group when controlling the progress of the streaming media file; the server statistically analyzing the set of target segments specified by users in the preset user group, and the number of users corresponding to each target segment in the set, and calculating the user ratio corresponding to each target segment, where the user ratio refers to the proportion of the number of users corresponding to each target segment in the target segment set to the total number of users in the user group corresponding to that set; and the server determining the location of plaintext at other locations from several streaming media segments based on the user ratio and the amount of plaintext at other locations. By using the user ratio to determine the plaintext, the server can identify the target segment that the user is most likely to select, thereby increasing the probability that the target segment is plaintext in several scenarios in which the user plays the video.

[0011] In one possible implementation of the first aspect above, the user controls the progress of the streaming media file, including fast-forwarding or rewinding the streaming media file.

[0012] In one possible implementation of the first aspect above, the server determines the position of plaintext at other locations based on the amount of plaintext at other locations in the streaming media file, including: the server separately counts the target segment set for fast-forwarding and the target segment set for rewinding for users within a preset user group; the server separately calculates the percentage of fast-forwarding users corresponding to each target segment in the target segment set for fast-forwarding and the percentage of rewinding users corresponding to each target segment in the target segment set for rewinding; the server determines the position of plaintext during fast-forwarding and rewinding from several streaming media segments based on the percentage of fast-forwarding users, the percentage of rewinding users, and the amount of plaintext at other locations.

[0013] In one possible implementation of the first aspect above, the server determines the positions of plaintext during fast-forwarding and rewinding from several streaming media segments based on the proportion of users who fast-forward, the proportion of users who rewind, and the amount of plaintext at other locations. This further includes: the server determining the positions of plaintext during fast-forwarding from several streaming media segments based on a preset second ratio between the amount of plaintext during fast-forwarding and the amount of plaintext during rewinding for each user on the streaming media file, the amount of plaintext at other locations, and the proportion of users who fast-forward; and the server determining the positions of plaintext during rewinding from several streaming media segments based on the second ratio, the amount of plaintext at other locations, and the proportion of users who rewind.

[0014] In one possible implementation of the first aspect described above, when the server determines the location of plaintext during fast forward and rewind, the priority of selecting target segments from the target segment set during fast forward as plaintext is higher than the priority of target segments from the target segment set during rewind. Since fast forward is the most likely operation when a user plays a streaming media file, prioritizing the selection of target segments from the set during fast forward as plaintext ensures that plaintext is present during fast forward, further increasing the probability of the user hitting plaintext during streaming media playback.

[0015] In one possible implementation of the first aspect above, in the set of target segments during fast forward, the server determines the target segments that meet the second ratio relationship of plaintext quantity at other positions according to the rule that the higher the proportion of fast-forwarding users, the higher the priority of the target segments corresponding to that proportion of fast-forwarding users being selected as plaintext, and the position of the target segment is taken as the position of the plaintext during fast forward; or, in the set of target segments during rewind, the server determines the target segments that meet the second ratio relationship of plaintext quantity at other positions according to the rule that the higher the proportion of rewinding users, the higher the priority of the target segments corresponding to that proportion of rewinding users being selected as plaintext, and the position of the target segment is taken as the position of the plaintext during rewind.

[0016] In one possible implementation of the first aspect above, when the same target segment belongs to both the target segment set during fast forward and the target segment set during rewind, the server sets the target segment to the priority of the target segment set during fast forward, and the proportion of users corresponding to the target segment is the sum of the proportion of users who fast forward and the proportion of users who rewind.

[0017] In one possible implementation of the first aspect above, when there are target segments with the same percentage of users appearing in the target segment set during fast forward, the server randomly removes one of the two target segments with the closest position until the plaintext count during fast forward is met; or, when there are target segments with the same percentage of users appearing in the target segment set during rewind, the server randomly removes one of the two target segments with the closest position until the plaintext count during rewind is met.

[0018] In one possible implementation of the first aspect above, the server obtains the target segment selected by each user in the preset user group when controlling the progress of the streaming media file, including: the server obtains the target position selected by each user on the screen when the users in the preset user group play the streaming media file on screens of different sizes, and the streaming media segment corresponding to the target position is the target segment.

[0019] In one possible implementation of the first aspect described above, user control of the streaming media file's progress further includes: pausing playback; the server determining the position of plaintext at other locations in the streaming media file based on the amount of plaintext at other locations; and the server obtaining the target segment for pausing playback selected by each user in the user group when playing the streaming media file, and obtaining a set of target segments at the time of pausing; the server calculating the percentage of users pausing for each target segment in the set of target segments at the time of pausing; and the server determining the position of plaintext at the time of pausing from several streaming media segments based on the percentage of users pausing and the amount of plaintext at other locations. This can increase the probability that the pause position is plaintext when the user pauses playback.

[0020] In one possible implementation of the first aspect described above, the target fragment in the set of target fragments at the time of pause is selected as the plaintext with the lowest priority.

[0021] In one possible implementation of the first aspect described above, the pre-defined strategy includes a priority ranking of plaintext and ciphertext acquired by the first electronic device. This includes: the server setting a priority for streaming media segments acquired by the first electronic device, arranging the acquisition priorities of the streaming media segments from high to low, in the following order: the user-specified target segment is plaintext already downloaded by the first electronic device or ciphertext already downloaded and decrypted by the first electronic device; the user-specified target segment is ciphertext already downloaded but not decrypted by the first electronic device; and the user-specified target segment is plaintext not yet downloaded. This priority ranking ensures that when the user is not playing the streaming media file normally, the downloaded plaintext or decrypted ciphertext is acquired first, thereby saving download and decryption time, avoiding stuttering, and improving the user experience.

[0022] In one possible implementation of the first aspect described above, when the target segment specified by the user is undownloaded ciphertext, the server arranges the acquisition priorities of streaming media segments from high to low. This further includes: the server arranging the acquisition priorities of streaming media segments within a preset range from high to low, in the following order: plaintext or ciphertext downloaded by the first electronic device that is closest to the user-specified target segment within the preset range; ciphertext downloaded by the first electronic device but not yet decrypted within the preset range; and undownloaded plaintext closest to the user-specified target segment within the preset range. The preset range is the difference between the location of the acquired streaming media segment and the location of the user-selected target segment that the user can tolerate. By setting a preset range, the probability of hitting downloaded plaintext can be further increased, thus improving the user experience.

[0023] In one possible implementation of the first aspect described above, when the system also includes a second electronic device, and the second electronic device is connected to the first electronic device as an execution device for playing streaming media files, the method further includes: the first electronic device acquiring parameters of the second electronic device's data processing capabilities and sending the parameters to a server; the server determining whether to further segment the streaming media segments in the streaming media file based on the parameters, and generating a corresponding segmentation scheme when segmentation is deemed necessary; the first electronic device segmenting each streaming media segment in the streaming media file according to the segmentation scheme, and re-encrypting the ciphertext of the segmented streaming media segments. This method combines the data processing capabilities of the second electronic device to process the streaming media file, generating a protection scheme suitable for the second electronic device, and further processing the streaming media file to facilitate playback by the second electronic device, avoiding playback stuttering due to the weak processing capabilities of the second electronic device, and improving the user experience.

[0024] Secondly, this application also provides an electronic device, comprising:

[0025] The segmentation module is used to divide a streaming media file into several streaming media segments with a predetermined time length;

[0026] The processing module is used to determine the streaming media segment to be used as plaintext from a number of streaming media segments according to preset rules. The preset rules are set according to the playback habits of a preset user group. The plaintext is an unencrypted streaming media segment.

[0027] The encryption module is used to encrypt streaming media segments other than plaintext to obtain ciphertext;

[0028] The processing module sets a pre-defined strategy for the first electronic device to acquire streaming media segments, and generates a description file based on the pre-defined strategy to record the position information of plaintext and ciphertext in the streaming media file; wherein, the pre-defined strategy includes the priority order of plaintext and ciphertext being acquired by the first electronic device; the first electronic device acquires the description file, and acquires streaming media segments from the streaming media file in sequence based on the description file for processing by the first electronic device.

[0029] The electronic device according to embodiments of this application incorporates a large amount of historical playback data from users to determine the playback habits of most users. It uses the streaming media segment most likely to be specified by the user when fast-forwarding, rewinding, or pausing playback as plaintext. This approach aims to maximize the probability that the user-specified target segment will match the plaintext when playing streaming media files. It also minimizes decryption and download time, avoids buffering, and improves the user experience.

[0030] In one possible implementation of the second aspect described above, the processing module is specifically used to: calculate the total number of plaintexts in the streaming media file based on the total duration of plaintexts in the streaming media file available to the user and the predetermined duration of each streaming media segment; and calculate the number of plaintexts in the header and other positions based on the total number of plaintexts and a first proportional relationship. The first proportional relationship refers to the ratio of the number of plaintexts in the header of the streaming media file to the number of plaintexts in other positions of the streaming media file. The plaintext in the header of the streaming media file refers to the first plaintext, starting from the zero-second mark of the streaming media file in playback order, or a sequence containing the first plaintext and multiple consecutive plaintexts adjacent to it. Based on the number of plaintexts in the header, the processing module determines the streaming media segments in the header that are plaintexts, and based on the number of plaintexts in other positions of the streaming media file, determines the streaming media segments in other positions that are plaintexts. This method can determine the number of plaintexts in the header of the streaming media file and ensure the number of plaintexts in other positions, thus ensuring that plaintexts are detected in both normal and abnormal playback scenarios (fast forward, rewind, or paused playback), improving the user experience.

[0031] In one possible implementation of the second aspect described above, the processing module prioritizes the plaintext in the header as the highest priority for selection. When a user plays a streaming media file, the plaintext in the header is the most likely text the user wants to play. Therefore, prioritizing the plaintext in the header improves the user's playback experience.

[0032] In one possible implementation of the second aspect described above, the processing module obtains the target segment selected by each user in a preset user group when controlling the progress of a streaming media file; the processing module counts the set of target segments specified by users in the preset user group, and the number of users corresponding to each target segment in the set, and calculates the user proportion corresponding to each target segment, where the user proportion refers to the ratio of the number of users corresponding to each target segment in the target segment set to the number of users in the user group corresponding to that set; based on the user proportion and the amount of plaintext at other positions, the processing module determines the position of plaintext at other positions from several streaming media segments. By using the user proportion to determine the plaintext, the target segment most likely to be selected by the user can be identified, thereby increasing the probability that the target segment is plaintext in several scenarios in which the user plays the video.

[0033] In one possible implementation of the second aspect above, the user controls the progress of the streaming media file, including fast-forwarding or rewinding the streaming media file.

[0034] In one possible implementation of the second aspect above, the processing module respectively counts the target segment set for fast-forwarding and the target segment set for rewinding of the streaming media file for users within a preset user group; the processing module respectively calculates the percentage of fast-forwarding users corresponding to each target segment in the target segment set for fast-forwarding and the percentage of rewinding users corresponding to each target segment in the target segment set for rewinding; the processing module determines the position of plaintext during fast-forwarding and rewinding from several streaming media segments based on the percentage of fast-forwarding users, the percentage of rewinding users, and the amount of plaintext at other positions.

[0035] In one possible implementation of the second aspect above, the processing module is further configured to determine the position of the plaintext during fast-forward from several streaming media segments based on a preset second ratio relationship between the number of plaintexts during fast-forward and the number of plaintexts during rewind for a user on a streaming media file, the number of plaintexts at other positions, and the proportion of users who fast-forward; the processing module determines the position of the plaintext during rewind from several streaming media segments based on the second ratio relationship, the number of plaintexts at other positions, and the proportion of users who rewind.

[0036] In one possible implementation of the second aspect described above, when the processing module determines the location of plaintext during fast forward and rewind, it prioritizes selecting target segments from the target segment set during fast forward as plaintext over those from the target segment set during rewind. Since fast forward is the most likely operation when a user plays a streaming media file, prioritizing plaintext in the target segment set during fast forward ensures that plaintext is present during fast forward, further increasing the probability of hitting plaintext during playback.

[0037] In one possible implementation of the second aspect above, in the target segment set during fast forward, the processing module determines the target segment that meets the plaintext quantity and second ratio relationship at other positions according to the rule that the higher the proportion of fast-forwarding users, the higher the priority of the target segment corresponding to that proportion of fast-forwarding users being selected as plaintext, and uses the position of the target segment as the position of the plaintext during fast forward; or, in the target segment set during rewind, the processing module determines the target segment that meets the plaintext quantity and second ratio relationship at other positions according to the rule that the higher the proportion of rewinding users, the higher the priority of the target segment corresponding to that proportion of rewinding users being selected as plaintext, and uses the position of the target segment as the position of the plaintext during rewind.

[0038] In one possible implementation of the second aspect above, when the same target segment belongs to both the target segment set during fast forward and the target segment set during rewind, the processing module sets the target segment to the priority of the target segment set during fast forward, and the proportion of users corresponding to the target segment is the sum of the proportion of users who fast forward and the proportion of users who rewind.

[0039] In one possible implementation of the second aspect above, when there are target segments with the same percentage of users appearing in the target segment set during fast forward, the processing module randomly removes one of the two target segments with the closest position until the plaintext count during fast forward is met; or, when there are target segments with the same percentage of users appearing in the target segment set during rewind, the processing module randomly removes one of the two target segments with the closest position until the plaintext count during rewind is met.

[0040] In one possible implementation of the second aspect above, the processing module obtains the target position selected by each user on the screen when users in the preset user group play streaming media files on screens of different sizes, and the streaming media segment corresponding to the target position is the target segment.

[0041] In one possible implementation of the first aspect described above, user control of the streaming media file's progress further includes: pausing playback. The processing module is also used to obtain the target segment for pausing playback selected by each user in the user group when playing the streaming media file, and to obtain a set of target segments at the time of pause. The processing module calculates the percentage of users pausing for each target segment in the set of target segments at the time of pause. Based on the percentage of users pausing and the amount of plaintext at other positions, the processing module determines the position of the plaintext at the time of pause from several streaming media segments. Determining the plaintext position using this method can increase the probability that the pause position is plaintext when the user pauses playback.

[0042] In one possible implementation of the first aspect described above, the target fragment in the set of target fragments at the time of pause is selected as the plaintext with the lowest priority.

[0043] In one possible implementation of the first aspect described above, the pre-defined strategy includes a priority ranking of plaintext and ciphertext acquired by the first electronic device. This includes: the processing module setting a priority for streaming media segments acquired by the first electronic device, arranging the acquisition priorities of the streaming media segments from high to low, in the following order: the user-specified target segment is plaintext already downloaded by the first electronic device or ciphertext already downloaded and decrypted by the first electronic device; the user-specified target segment is ciphertext already downloaded but not decrypted by the first electronic device; and the user-specified target segment is plaintext not yet downloaded. This priority ranking ensures that when the user is not playing the streaming media file normally, the downloaded plaintext or decrypted ciphertext is acquired first, thereby saving download and decryption time, avoiding stuttering, and improving the user experience.

[0044] In one possible implementation of the second aspect described above, the processing module is further configured to prioritize the acquisition of streaming media segments within a preset range, arranging them from high to low priority: plaintext or ciphertext downloaded and decrypted by the first electronic device that is closest to the user-specified target segment within the preset range; ciphertext downloaded and not decrypted by the first electronic device within the preset range; and undownloaded plaintext closest to the user-specified target segment within the preset range. The preset range is the difference between the location of the acquired streaming media segment and the location of the user-selected target segment that the user can tolerate. By setting the preset range, the probability of hitting downloaded plaintext can be further increased, thus improving the user experience.

[0045] In one possible implementation of the second aspect described above, the processing module further determines whether to further segment the streaming media segments in the streaming media file based on parameters. When segmentation is deemed necessary, a corresponding segmentation scheme is generated. This method combines the data processing capabilities of the second electronic device to process the streaming media file, generating a protection scheme suitable for the second electronic device. It further processes the streaming media file to facilitate playback on the second electronic device, avoiding playback stuttering or other issues caused by the second electronic device's weak processing capabilities, thus improving the user experience.

[0046] Thirdly, this application provides a terminal device, including:

[0047] Memory, used to store instructions executed by one or more processors of the device, and

[0048] The processor is used to perform the following steps:

[0049] A description file is obtained, and streaming media segments are sequentially obtained from the streaming media file based on the description file for processing by the first electronic device. The description file refers to the description file used in the first and second aspects.

[0050] Fourthly, this application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the method described in the first aspect embodiment.

[0051] Fifthly, this application discloses a computer program product containing instructions that, when run on a computer, causes the computer to perform the method disclosed in the first aspect of the embodiments described above.

[0052] Sixthly, this application discloses an apparatus including a processor, a memory, and a communication module. The processor, memory, and communication module communicate with each other via internal links to transmit control and / or data signals, causing the apparatus to perform the method disclosed in the first aspect embodiment described above.

[0053] In a seventh aspect, this application also discloses a system comprising a server and a first electronic device, the system being used to perform the method of the first aspect embodiment described above.

[0054] Eighthly, this application also discloses a system comprising a server, a first electronic device, and a second electronic device, wherein the system is used to perform the method of the first aspect embodiment described above. Attached Figure Description

[0055] Figure 1a This is a schematic diagram illustrating an exemplary process of playing a streaming media file using existing technology.

[0056] Figure 1bThis is another exemplary schematic diagram of the process of playing a streaming media file using existing technology;

[0057] Figure 2 This is a system architecture diagram of one embodiment of this application;

[0058] Figure 3 This is a block diagram of a device according to one embodiment of this application;

[0059] Figure 4 This is a schematic diagram of an electronic device structure according to an embodiment of this application;

[0060] Figure 5 This is a flowchart of a streaming media data processing method according to an embodiment of this application;

[0061] Figure 6 This is a schematic diagram of the description code of a description file according to an embodiment of this application;

[0062] Figure 7 This is a schematic diagram illustrating the structure of a streaming media file after determining the plaintext header, according to an embodiment of this application.

[0063] Figure 8a This is a schematic diagram illustrating the distribution of target locations operated by users in a user group on screens of different sizes, according to one embodiment of this application.

[0064] Figure 8b This is a schematic diagram illustrating the structure of plaintext determined based on the proportion of user numbers according to an embodiment of this application;

[0065] Figure 9 This is a flowchart illustrating a mobile phone acquiring a streaming media segment according to an embodiment of this application;

[0066] Figure 10a A scene diagram illustrating a mobile phone acquiring a streaming media segment according to an embodiment of this application;

[0067] Figure 10b This is a scene diagram illustrating a mobile phone acquiring a streaming media segment during fast forward, according to one embodiment of this application.

[0068] Figure 11a This is a flowchart illustrating another method for processing streaming media data according to an embodiment of this application;

[0069] Figure 11b This is a schematic diagram showing the before-and-after comparison structure of a streaming media file according to an embodiment of this application;

[0070] Figure 12 This is a schematic diagram of the structure of a terminal device according to an embodiment of this application;

[0071] Figure 13 This is a block diagram of a system-on-a-chip (SoC) according to some embodiments of this application. Detailed Implementation

[0072] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0073] The prior art is described below with reference to the accompanying drawings.

[0074] Figure 1a This diagram illustrates an exemplary process for playing a streaming media file using existing technology. (For example...) Figure 1a As shown in the diagram, this includes a server 101 and a player 102. The server 101 splits the streaming media file into two sub-files, such as... Figure 1a The image shows the plaintext at the beginning of the streaming media file and the ciphertext excluding the header. Server 101 provides the user with the plaintext for free, according to a fixed duration set by the content provider, for example, 30 seconds. The remaining files are encrypted; the user must pay to play the ciphertext, which is 153 seconds long. Figure 1a As shown, the normal playback process of the streaming media file by player 102 includes: S1, player 102 downloads plaintext from server 101; S2, plays the plaintext; S3, synchronized with S2, player 102 downloads ciphertext from server 101; S4, decrypts the ciphertext; S5, after the plaintext playback ends, plays the ciphertext to complete the streaming media playback process. However, when users fast-forward, rewind, or pause the streaming media file, interrupted playback cannot be resumed. When the specified position during fast-forwarding, rewinding, or pausing falls on an encrypted segment, the entire ciphertext must be downloaded and decrypted before playback. Due to the long download and decryption time, this can easily lead to prolonged playback time and a poor playback experience.

[0075] The player 102 mentioned above can be a standalone player or a player installed in an electronic device; there is no limitation here.

[0076] Figure 1b This diagram illustrates another exemplary process for playing a streaming media file using existing technology. (Example:) Figure 1b As shown in the diagram, this includes a server 103 and a player 104. The server 103 divides the streaming media file into several streaming media segments, and the player 104 downloads these segments from the server and plays them. The server 103 can select the streaming media segments to be encrypted according to certain rules, including: selecting segments based on their parity (odd or even) sequence numbers, randomly selecting segments, or encrypting p segments at fixed intervals, where p is a natural number greater than or equal to 1. For example, as... Figure 1bAs shown, server 103 divides the song, which has a total length of 197 seconds, into 20 streaming media segments according to a set time length of 10 seconds. The streaming media segments with even-numbered sequence numbers are encrypted to obtain ciphertext, while those with odd-numbered sequence numbers are set as plaintext. For example... Figure 1b As shown, the normal playback process of the streaming media file by player 103 includes: S10, player 104 downloads the first plaintext from server 103; S20, plays the first plaintext; S30, synchronized with S2, player 104 downloads the first ciphertext from server 103; S40, decrypts the first ciphertext; S50, after the first plaintext finishes playing, plays the first ciphertext, and so on, completing the streaming media playback process. This solution can solve the problem of long download time for the entire ciphertext segment. However, when the user fast-forwards, rewinds, or pauses the streaming media file, if the user-specified position falls into the ciphertext, it is still necessary to download the entire file and decrypt each ciphertext, increasing the decryption time of the streaming media segment and extending the start-up time. Furthermore, because the total duration of the plaintext content is not considered, the maximum plaintext segment duration allowed by the content provider cannot be guaranteed.

[0077] To address the aforementioned technical problems, this application provides a method for processing streaming media data. Before explaining the technical solution of this application, the terms mentioned herein will be explained below for better understanding.

[0078] A streaming media segment refers to a streaming media file that has been divided into several segments according to a preset duration. Each segment is a streaming media segment. The preset duration can be a fixed length or a length determined based on the data processing capabilities of the playback device, such as a duration measured in seconds, like 6 seconds or 10 seconds.

[0079] Plaintext refers to unencrypted streaming media clips, which electronic devices with players can directly access and play. Furthermore, this plaintext is typically provided to users free of charge by the content provider.

[0080] Ciphertext refers to encrypted streaming media segments that a player cannot play directly; it requires decryption before playback. Typically, ciphertext is only accessible to users after payment or within a specified timeframe granted by the content provider.

[0081] The target position refers to the position of the progress bar relative to the screen after the user drags it on the playback device screen.

[0082] A target segment refers to a segment of a streaming media file that is played on the screen at the target location.

[0083] A pre-defined user group refers to a specified number of users who meet certain characteristics. For example, users who like to listen to classical music, users who like to listen to pop music, or users who access a predetermined number of audio files more than a predetermined number of times.

[0084] The target segment set refers to the target segments obtained by all users in a predefined user group for a specific scenario, such as fast forwarding or rewinding. For example, in a fast forward scenario, it is the set of historical target segments obtained by all users in the predefined user group when fast forwarding a streaming media file.

[0085] User share refers to the proportion of users corresponding to each target segment in the target segment set for each scenario, relative to the total number of users in the corresponding preset user group. For example, in a scenario where users fast-forward through a streaming media file, this refers to the proportion of users corresponding to each target segment in the target segment set for that scenario, relative to the total number of users in the corresponding preset user group. This data is stored on the server as historical data.

[0086] The starting segment refers to the streaming media segment that the playback device selects as the starting point for playback after the user drags the progress bar on the playback device screen.

[0087] refer to Figure 2 , Figure 2 An exemplary system architecture diagram of this application is shown. Figure 2As shown, the system includes a server 210 and a mobile phone 220. The server 210 stores a large amount of historical data generated by users playing streaming media files. This includes historical data on users fast-forwarding, rewinding, and pausing streaming media files played on screens of different sizes. For example, historical data such as the progress bars on the screens of watches, mobile phones, and televisions operated by different users, the streaming media segment (target segment) corresponding to the position reached by the progress bar, and the number of users corresponding to the target segment. The copyright protection scheme generation unit 211 of the server 210 determines the plaintext position from the streaming media files based on this historical data. After determining the plaintext position, the server 210 assigns numbers to the streaming media files that need encryption (excluding plaintext) and assigns a priority to the streaming media segments for acquisition by the mobile phone 220 to generate a file protection scheme. The copyright protection scheme generation unit 211 sends the file protection scheme to the processing unit 212. The processing unit 212 encrypts the streaming media segments with encryption numbers according to the requirements of the file protection scheme to obtain ciphertext. Processing unit 212 generates a description file corresponding to the streaming media file. This description file may include information such as the encryption status, sequence number, address, and acquisition priority of the streaming media segment. Simultaneously, server 210 stores the plaintext, ciphertext, and description file of the streaming media file in corresponding storage modules. For example, the plaintext is stored in storage module 214, the ciphertext in storage module 215, and the description file in storage module 216. Server 210 sends the description file of the streaming media file to mobile phone 220 via distribution unit 213. Mobile phone 220 downloads the streaming media file from server 210 according to the description file and plays it. When playing the streaming media file, mobile phone 220 prioritizes playing the plaintext stored in mobile phone 220 or the ciphertext already downloaded to mobile phone 220, based on the acquisition priority of the streaming media segment.

[0088] The solution in this application combines a large amount of historical playback data from users to determine the playback habits of most users. It uses the streaming media segment most likely to be specified by the user when fast-forwarding, rewinding, or pausing playback as plaintext. This ensures that when the user resumes playback after fast-forwarding, rewinding, or pausing, the corresponding streaming media segment is as plaintext as possible, thus reducing latency in network transmission of the streaming media segment. Alternatively, it guides the playback of the segment after fast-forwarding, rewinding, or pausing to the locally stored streaming media segment closest to the current progress position. By implementing this solution, the probability of the user-specified target segment hitting plaintext when playing streaming media files is increased. This minimizes decryption and download time, avoids stuttering, and improves the user experience.

[0089] In the following embodiments, the process by which the server 210 determines the plaintext location and the mobile phone 220 acquires the streaming media segment will be described in detail. The streaming media data processing method of this application will be described below in conjunction with the specific structure of the device.

[0090] Now for reference Figure 3 , Figure 3 The diagram illustrates a device 300 according to one embodiment of this application. Device 300 may include one or more processors 301 coupled to a controller hub 303. In at least one embodiment, the controller hub 303 communicates with the processor 301 via a multi-branch bus such as a Front Side Bus (FSB), a point-to-point interface such as a Quick Path Interconnect (QPI), or a similar connection 306. The processor 301 executes instructions controlling general-type data processing operations. In one embodiment, the controller hub 303 includes, but is not limited to, a Graphics Memory Controller Hub (GMCH) (not shown) and an Input / Output Hub (IOH) (which may be on a separate chip) (not shown), wherein the GMCH includes memory and a graphics controller and is coupled to the IOH.

[0091] Device 300 may also include a coprocessor 302 and a memory 304 coupled to a controller hub 303. Alternatively, one or both of the memory and the GMCH may be integrated within the processor (as described in this application), with memory 304 and coprocessor 302 directly coupled to processor 301 and controller hub 303, which resides on a single chip with the IOH. Memory 304 may be, for example, Dynamic Random Access Memory (DRAM), Phase Change Memory (PCM), or a combination of both. In one embodiment, coprocessor 302 is a dedicated processor, such as, for example, a high-throughput MIC processor (Many Integerated Core, MIC), a network or communication processor, a compression engine, a graphics processor, a general-purpose graphics processor (GPGPU), or an embedded processor, etc. Optional properties of coprocessor 302 are indicated by dashed lines. Figure 3 middle.

[0092] Memory 304, as a computer-readable storage medium, may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. For example, memory 304 may include any suitable non-volatile memory such as flash memory and / or any suitable non-volatile storage device, such as one or more hard-disk drives (HDD(s)), one or more compact disc (CD) drives, and / or one or more digital versatile disc (DVD) drives.

[0093] According to some embodiments of this application, memory 304, which is a computer-readable storage medium, stores instructions that, when executed on a computer, cause system 300 to perform the streaming media data processing method of this application.

[0094] In some embodiments, the memory 304 may store a large amount of historical data on user operations related to streaming media files. For example, a large number of users may be playing a song on electronic devices with screens of different sizes. When a user manipulates a progress bar on the screen to control the playback progress of the song, for example, by moving the progress bar backward on the screen, the position of the progress bar on the screen and the position of the corresponding streaming media segment (target segment) will be stored in the memory.

[0095] In one embodiment, the processor 301 retrieves a large amount of historical data on user operations related to streaming media files from the memory 304. These operations may include fast-forwarding, rewinding, or pausing playback on a progress bar on the screen. Based on this historical data, the processor 301 calculates the percentage of users within a preset user group who fast-forwarded the target segment of the streaming media file (the percentage of fast-forwarding users), rewinding the target segment (the percentage of rewinding users), or pausing the target segment (the percentage of pausing users). Simultaneously, the processor 301 selects streaming media segments as plaintext based on the maximum limit on the amount of plaintext provided by the content provider to the user and according to the requirements for setting plaintext positions. For example, the header of the streaming media file needs to be set with plaintext, as do other positions besides the header. The plaintext in the header is the streaming media segment counted from second 0 of the streaming media file. For example, assuming the header plaintext is set to two, then following the normal playback order of the streaming media file, the first and second streaming media segments are used as the header plaintext. This determines the position of the plaintext in the streaming media file header. Additionally, it is necessary to determine the plaintext in other positions besides the header. The processor 301 can select streaming media segments as plaintext from the target segment sets for fast-forwarding, rewinding, and / or pause playback according to a certain ratio, based on the number of plaintexts allocated to other positions in the streaming media file. For example, assuming the number of plaintexts in other positions is four, and the ratio of plaintexts for fast-forwarding, rewinding, and pause playback is 2:1:1, then two streaming media segments are selected from the target segment set for fast-forwarding, and the segments with the largest proportion of fast-forwarding users are selected as plaintext, prioritizing those with higher proportions of fast-forwarding users. Similarly, one streaming media segment is selected from the target segment set for rewinding, and the segment with the largest proportion of rewinding users is selected as plaintext. One streaming media segment is selected from the target segment set when playback is paused, and the segment with the largest percentage of paused users is taken as plaintext. This determines the plaintext in the streaming media file, records its position, and saves the plaintext position information to memory 304. The processor 301 then encrypts all other streaming media segments except the plaintext to obtain ciphertext, and stores the ciphertext's position information and other descriptive files in memory 304.

[0096] In some embodiments, when a user performs several operations on a streaming media file, such as fast forward, rewind, and pause, and the number of plaintext segments is insufficient to ensure that the target segment corresponding to each operation can be selected, the segments can be prioritized according to their priority, with the streaming media segments from the target segment set corresponding to the operation with the highest priority being selected first. Specifically, the priority of the streaming media segments in the target segment set during fast forward is greater than the priority of the streaming media segments in the target segment set during rewind, which is greater than the priority of the streaming media segments in the target segment set during pause. For example, after the plaintext setting at the beginning is completed, if there are two plaintext segments in other positions, the two streaming media segments in the target segment set during fast forward are selected first as plaintext.

[0097] In some embodiments, the memory 304 may also store a pre-defined strategy for acquiring streaming media segments. This pre-defined strategy includes, based on the principle of proximity to the user-selected target segment, arranging the streaming media segments in descending order of acquisition priority: plaintext within the playback device, ciphertext within the playback device, online plaintext, and online ciphertext. Further, it may arrange the streaming media segments within a preset range in descending order of acquisition priority: plaintext within the playback device, ciphertext within the playback device, online plaintext, and online ciphertext. For example, when the user fast-forwards, the progress bar on the screen is dragged backward. Within a range of three streaming media segments before and after the target segment corresponding to the position where the progress bar stops, the plaintext stored in the electronic device that is closest to the target segment is preferentially selected. If no plaintext is stored in the playback device within this range, the ciphertext stored in the playback device that is closest to the target segment is selected. Based on the acquisition priority order of the streaming media segments by the playback device, a playable streaming media segment is finally determined. By prioritizing streaming media segments for playback devices, the system ensures that the device receives the plaintext stored within the segment first, followed by the ciphertext. This avoids stuttering issues that can occur when fast-forwarding or rewinding if the target segment hasn't been downloaded, as the device would need to download the streaming media file.

[0098] In one embodiment, device 300 may further include a Network Interface Controller (NIC) 306. The network interface 306 may include a transceiver for providing a radio interface for device 300 to communicate with any other suitable device (such as a front-end module, antenna, etc.). In various embodiments, the network interface 306 may be integrated with other components of device 300. The network interface 306 can implement the functionality of the communication unit in the above embodiments.

[0099] Device 300 may further include input / output (I / O) device 305. I / O 305 may include: a user interface designed to enable a user to interact with device 300; a peripheral component interface designed to enable peripheral components to also interact with device 300; and / or sensors designed to determine environmental conditions and / or location information related to device 300.

[0100] It is worth noting that, Figure 3 This is merely an example. That is, although... Figure 3 The diagram shows that device 300 includes multiple devices such as processor 301, controller hub 303, and memory 304. However, in actual applications, devices using the methods of this application may include only a portion of the devices in device 300. For example, it may include only processor 301 and NIC 306. Figure 3 The properties of the optional devices are shown by dashed lines.

[0101] The following describes the streaming media data processing method of this application in conjunction with the specific structure of the mobile phone.

[0102] For example, Figure 4 A schematic diagram of an electronic device is shown. This electronic device 100 processes streaming media data as a playback device in the above embodiments. (Reference) Figure 4 Electronic device 100 may include processor 110, external memory interface 120, internal memory 121, universal serial bus (USB) connector 130, charging management module 140, power management module 141, battery 142, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0103] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0104] Processor 110 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors.

[0105] Processor 110 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors.

[0106] The processor 110 can generate operation control signals based on the instruction opcode and timing signals to control the instruction fetching and execution.

[0107] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0108] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0109] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with tuning switches.

[0110] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0111] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0112] The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 100, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0113] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling electronic device 100 to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or satellite-based augmentation systems (SBAS).

[0114] In some embodiments, the electronic device 100 communicates with a server via an antenna 2 and a wireless communication module 160, the server being, for example, Figure 3 The device structure is shown. Electronic device 100 downloads a streaming media file from a server. This streaming media file is a file processed by the server, along with a pre-defined strategy for acquiring streaming media segments. For details, please refer to the process of determining the plaintext and ciphertext positions of the streaming media file by device 300 in the above embodiment, and the description of the pre-defined strategy. Processor 110 obtains streaming media segments from the server according to the pre-defined strategy, so that when the user fast-forwards, rewinds, or pauses playback, the streaming media segment corresponding to the starting position (starting segment) is as plaintext as possible. This shortens the electronic device's playback response time in fast-forwarding, rewinding, or paused playback scenarios, improving the user's playback experience.

[0115] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0116] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniature LED, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0117] In some embodiments, the display screen 194 is used to display a progress bar for playing a streaming media file. Users can control the playback progress of the streaming media file by moving the progress bar. The position of the streaming media segment corresponding to the progress bar is the target segment. When different users play streaming media files using display screens of different sizes, the server calculates the target position that the user is most likely to move the progress bar to when fast-forwarding, rewinding, or pausing playback based on the positions of the progress bars moved by a large number of users (target positions), and treats the streaming media segment corresponding to that target position as plaintext. This application takes into account the habits of most users when playing streaming media files, thereby increasing the probability that the target segment corresponding to fast-forwarding, rewinding, and pausing playback is plaintext, thus shortening the playback start time.

[0118] Internal memory 121 can be used to store computer executable program code, which includes instructions. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of electronic device 100 (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc. Processor 110 executes various functional applications and data processing of electronic device 100 by running instructions stored in internal memory 121 and / or instructions stored in memory located in the processor.

[0119] In some embodiments, the internal memory 121 may store a pre-defined strategy for a user to acquire streaming media segments received by the electronic device 100. This allows the processor 110 to acquire streaming media segments from the server according to the pre-defined strategy. The acquired streaming media segments are also stored in the internal memory 121.

[0120] Electronic device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0121] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110. In some embodiments, the audio module 170 plays the streaming media file in the order of the acquired streaming media segments.

[0122] Touch sensor 180K, also known as a "touch device," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touchscreen." Touch sensor 180K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of electronic device 100, in a different position than display screen 194.

[0123] In some embodiments, a user can touch a progress bar displayed on the display screen 194 to control the playback progress of the streaming media file.

[0124] Furthermore, in some embodiments of this application, the electronic device 100 may also be communicatively connected to another electronic device, so that the other electronic device can obtain a pre-made strategy from the electronic device 100, and obtain streaming media segments from the electronic device 100 according to the pre-made strategy, and play the streaming media file.

[0125] In some embodiments, the electronic device 100 also acquires parameters of the data processing capabilities of another connected electronic device, such as storage capacity. The electronic device calculates whether to further segment the streaming media file based on these parameters. When secondary segmentation is required, the electronic device 100 segments the plaintext and ciphertext of the streaming media file separately. The segmented ciphertext is then re-encrypted to make it easier and faster for the other electronic device to receive and play the streaming media file.

[0126] In some embodiments of this application, parameters of the data processing capabilities of another electronic device are obtained, and a secondary segmentation copyright protection scheme is generated for the streaming media file based on these parameters. This can also be achieved in scenarios such as... Figure 3 The process is performed in device 300. Electronic device 100 is connected to device 300, obtains the copyright protection scheme from device 300, and performs the secondary segmentation operation. Alternatively, the secondary segmentation can also be performed by device 300, which then sends the segmented streaming media file to electronic device 100. This is not intended to be limiting.

[0127] The following description, in conjunction with the accompanying drawings and using an architecture consisting of a server and a mobile phone as an example, illustrates the method for processing streaming media data in this application.

[0128] refer to Figure 5 , Figure 5 A flowchart illustrating an exemplary method for processing streaming media data is provided. In this process, the server first processes the streaming media file to obtain a processed streaming media file. The mobile phone retrieves the processed streaming media segment from the server according to a preset strategy and plays it. This is done to increase the probability of hitting plaintext at the playback start position in scenarios where the user fast-forwards, rewinds, and pauses the streaming media file. This reduces playback latency or stuttering, improving the user's playback experience. Figure 5 As shown, the method for processing streaming media data includes the following steps.

[0129] S510: The server divides the streaming media file into several streaming media segments.

[0130] In the embodiments of this application, the server can divide the streaming media into several segments according to a preset time length. For example, the preset time length can be 5 seconds, and the total time length of the streaming media file is 200 seconds. Then, the server can divide the streaming media file into 40 streaming media segments according to the preset time length of 5 seconds.

[0131] In some embodiments, the preset duration can be a fixed duration, or it can be dynamically set by the server based on the data processing capabilities of the electronic device playing the streaming media file. For example, if a watch's data transmission capability is lower than that of a mobile phone, and the streaming media file is played on a watch, the server can set the preset duration to be shorter than the preset duration used when transmitting to a mobile phone, based on the watch's data transmission capability. This allows the watch to easily acquire the streaming media segment and play it smoothly, avoiding stuttering when downloading excessively large streaming media segments on the watch.

[0132] S520: The server determines the plaintext from several streaming media segments.

[0133] The server first determines the amount of plaintext that can be provided to the user. The server can determine the amount of plaintext based on the total duration of the unencrypted streaming media file set by the content provider, and also based on this duration and the preset duration of each streaming media segment. Taking a 200-second streaming media file in step S510 as an example, if the total unencrypted duration of the streaming media file is set to 30 seconds, and the preset duration of each streaming media segment is 5 seconds, then the amount of plaintext is 6. After calculating the amount of plaintext, the server needs to determine the positions of these 6 plaintext segments from the 40 streaming media segments.

[0134] The server determines the plaintext positions based on the user's playback habits for streaming media files. This considers not only the scenario where the user plays the streaming media file normally on an electronic device, but also the user's habits when fast-forwarding, rewinding, and / or pausing playback. This determines the positions of six plaintext elements. The following embodiments will explain in detail how the plaintext positions are determined in these scenarios. See the following embodiments for the process of determining the six plaintext positions.

[0135] S530, the server encrypts the streaming media segments other than plaintext to obtain ciphertext, thereby determining the positions of the plaintext and ciphertext in the streaming media file. The method used to encrypt these streaming media segments is not limited in this application. S540, the server sets a pre-defined strategy for the mobile phone to acquire streaming media segments. This pre-defined strategy can be acquired and executed by the mobile phone, including the priority of streaming media segments acquired by the mobile phone, for example, ranking the priority of plaintext and ciphertext acquired by the mobile phone. The server marks the priority of each streaming media segment so that the mobile phone can acquire the best streaming media segment as the starting streaming media segment (starting playback segment) according to the marked priority.

[0136] The S550 server generates a description file for the streaming media file based on the positions of the plaintext and ciphertext and a pre-defined strategy. This description file records the position of each streaming media segment or its sequence number according to playback time, the duration of the streaming media segment, whether the streaming media segment is encrypted, the acquisition priority of the streaming media segment, and the total duration of the streaming media file, etc.

[0137] refer to Figure 6 , Figure 6 A schematic diagram of the description code for the description file is shown. Figure 6 In this format, PN represents the sequence number of the streaming media segment (counting from 1, where 1 indicates the first streaming media segment starting from the beginning of the streaming media file). URL represents the network storage path of the streaming media segment. HashCode is the hash value of the file (used to verify file integrity and prevent tampering). Encrypt indicates whether the streaming media segment is encrypted (ciphertext). Prior indicates the priority when retrieving streaming media segments online (the smaller the value, the higher the priority). SubPrior is the sub-priority of the segment; when multiple streaming media segments have the same Prior, it tells the device retrieving the file which segment to prioritize.

[0138] In the S560, the mobile phone obtains a streaming media profile and retrieves streaming media segments from the server based on the profile. The phone processes these segments sequentially according to the order specified in the profile. Processing a streaming media segment can include playing it, editing it, or applying effects. The pre-defined strategy in the profile can be obtained before the phone plays the streaming media file, during normal playback of the file's header plaintext, or temporarily when the user fast-forwards or rewinds. The phone executes this pre-defined strategy to retrieve and play the streaming media segments, thereby improving the user's playback experience.

[0139] The following is a detailed description of the process by which the server in the S520 determines six plaintexts from several streaming media segments.

[0140] This scenario describes a user playing a streaming media file normally.

[0141] For example, when a user first switches to a song and starts playing it, their usual practice is to play the beginning segment of the song first, i.e., starting playback from 0 seconds in the song. Therefore, prioritizing the normal playback order, the first streaming segment of the song's header, or several consecutive streaming segments containing that first segment, is treated as plaintext to determine the location of the plaintext in the streaming media file's header. For example, ... Figure 7 As shown, assuming the total duration of the streaming media file is 200 seconds, the preset duration of each streaming media segment is 5 seconds, the server divides the streaming media segment into 40 segments, sets the number of plaintext segments to 6, and sets the header of the streaming media file to 2 plaintext segments according to the set ratio. Then, as... Figure 6 As shown, the server treats the first and second streaming media segments (two consecutive streaming media segments including the first segment) as plaintext to determine the plaintext position in the header of the streaming media file. The server then further determines the positions of four other plaintext segments besides the plaintext based on several user scenarios involving fast-forwarding, rewinding, and pausing playback of the streaming media file.

[0142] The determination of the four plaintext locations other than the header of the streaming media file will be explained in detail below with reference to specific embodiments.

[0143] In the embodiments of this application, the plaintext positions of locations other than the header can be determined by the habits of most users when playing streaming media files out of order.

[0144] In embodiments of this application, the server acquires a large number of operation samples of streaming media files played by users and selects the target positions of users from a preset user group when "fast forward," "rewind," and "pause" playback on the progress bar on the screen. The preset user group can be based on the type of streaming media file, such as fans of classical music or pop music (based on user preference tags). This application is not limited to this and can determine the preset user group based on any criteria. Because the amplitude of the gesture operation corresponding to "fast forward" and "rewind" will be different on different screen sizes, one embodiment of this application also considers the different situations on different screens when determining the target positions of users when "fast forward," "rewind," and "pause" playback on the progress bar on the screen.

[0145] refer to Figure 8a , Figure 8a An exemplary diagram illustrates the distribution of target locations operated by users within a user group on screens of different sizes. For example... Figure 8a As shown, Figure 8a(a) in the image represents the distribution of the number of users corresponding to the target location on the small screen. Figure 8a (b) in the image represents the user number distribution map corresponding to the target location on the middle screen. Figure 8a (c) in the diagram represents the user number distribution map corresponding to the target position on the large screen. Line 1 represents the user number curve corresponding to the target position during fast forward, line 2 represents the user number curve corresponding to the target position during rewind, and line 3 represents the user number curve corresponding to the target position during pause. The server finds the streaming media segment (target segment) corresponding to the specified streaming media file based on the target position of the progress bar. The server separately counts the target segment sets (set A) during fast forward, (set B) during rewind, and (set C) during pause for users in the preset user group, as well as the percentage of users corresponding to each target segment in set A (fast forward user percentage), the percentage of users corresponding to each target segment in set B (rewind user percentage), and the percentage of users corresponding to each target segment in set C (pause user percentage). The user percentage refers to the proportion of users corresponding to each target segment in the target segment set in each scenario to the total number of users in the preset user group corresponding to that set. A higher user percentage indicates a higher probability that the target segment corresponding to that user percentage is designated as the target segment by the user. Therefore, the server sorts users by percentage from highest to lowest, with higher percentages receiving higher priority as plaintext. For example, assuming the plaintext allocation ratio for fast-forward, rewind, and pause playback is 2:1:1, then with a total of 4 plaintext segments, set A selects 2 target segments as plaintext, set B selects 1, and set C selects 1. Furthermore, prioritizing by user percentage from highest to lowest, set A selects the top 2 target segments by user percentage. Set B selects the target segment with the largest user percentage. Set C selects the target segment with the largest user percentage, thus determining the 4 plaintext segments in the streaming media file excluding the header.

[0146] In embodiments of this application, when a streaming media segment belongs to the plaintext position of the header and is located within any one or more sets, then the streaming media segment is considered part of the header plaintext. In this case, the streaming media segment can be removed from the set it belongs to, and then the percentage of users corresponding to other target segments in that set can be compared. For example, ... Figure 7As shown, once the plaintext position is determined, when calculating the set B during backtracking, the first plaintext belonging to the header also belongs to set B. Therefore, when calculating the plaintext position within set B, the arrangement of the percentage of users corresponding to that plaintext will no longer be considered.

[0147] In one embodiment of this application, when the number of plaintexts cannot be allocated according to a predetermined ratio—for example, when the number of plaintexts cannot be guaranteed to be allocated to every scenario according to a set ratio—it is necessary to consider possible user operations and further increase the probability of hitting plaintexts during user operations. This can be achieved by prioritizing several scenarios of user operations. For example, fast-forwarding is the most likely user operation, followed by rewinding, and pausing playback is the least likely. Therefore, according to the priority of selecting target segments, set A has a higher priority than set B, which in turn has a higher priority than set C. Plaintext is selected first from the sets with higher priorities. For instance, if the ratio of fast-forwarding, rewinding, and pausing playback scenarios is set to: fast-forward: rewind: pausing playback = 2:1:1, and there are only 3 plaintexts, then according to the priority order, 2 target segments are selected as plaintexts from set A, and 1 target segment is selected as plaintext from set B. The selection of the 2 target segments from set A can be based on the priority method described in the above embodiment, selecting the 2 target segments with the higher percentage of users as plaintexts.

[0148] In another embodiment of this application, when multiple target fragments within the same set have the same percentage of users, and the number of target fragments corresponding to the same percentage of users exceeds the set number of plaintext fragments, it is necessary to remove the excess target fragments from the several target fragments with the same percentage of users. For example, in set A, two target fragments need to be selected as plaintext. However, in set A, there are three target fragments with the same percentage of users. In this case, one of every two closest target fragments can be randomly removed until the number of selectable target fragments equals two, thus satisfying the plaintext quantity requirement in each set.

[0149] refer to Figure 8b , Figure 8b This diagram illustrates the structure of plaintext determined based on the percentage of users. (For example...) Figure 8b As shown, line 3 represents the user count curve corresponding to the target position during fast forward, and line 4 represents the user count curve corresponding to the target position during rewind. When the server determines three streaming media segments with equal user counts in set A during fast forward, it can set one of the adjacent 12th and 13th streaming media segments as ciphertext to select two plaintext segments from set A, as shown below. Figure 8bThe 12th and 18th streaming segments are shown. In the rewind sequence B, the 18th streaming segment, corresponding to the highest percentage of users, is selected as the plaintext.

[0150] In one embodiment of this application, when the same target segment belongs to both set A and set B, the priority level of the target segment is the same as that of the higher priority set A, and the user proportion of the target segment is the sum of the user proportion in set A and the user proportion in set B.

[0151] Based on the plaintext position determination rules of the above embodiments of this application, user playback habits are taken into account, thereby ensuring that the target segment corresponding to the target position of the progress bar is as plaintext as possible in scenarios where the user performs fast forward, rewind, and pause playback. This increases the probability that the target segment corresponding to the target position is plaintext.

[0152] The S560 is described below with reference to a specific embodiment of a mobile phone acquiring streaming media segments.

[0153] refer to Figure 9 , Figure 9 A flowchart illustrating how a mobile phone acquires a streaming media segment is shown. Figure 9 As shown, the flowchart includes the following steps:

[0154] The S910 phone creates a separate thread to retrieve the first streaming segment of a streaming media file.

[0155] In the embodiments of this application, considering the normal playback order, most users start from 0 seconds of the streaming media file. Therefore, the plaintext located at the beginning of the streaming media file is set as the highest priority for acquiring streaming media segments. When there are more than two plaintext entries in the beginning, the streaming media segments closer to 0 seconds are acquired according to the normal playback time.

[0156] In another embodiment of this application, when the user plays the media files in the normal order, the media files are obtained sequentially according to the time order starting from 0 seconds.

[0157] S920: The phone acquires the first streaming media segment and starts playback. Following the acquisition priority described above, this first streaming media segment is the segment of the streaming media file starting from second 0.

[0158] S930: The phone creates a second thread. While the phone is playing the current streaming media segment, the second thread is executed simultaneously to ensure that the phone can smoothly and continuously play the streaming media file after the current one has finished playing.

[0159] In the embodiments of this application, when the mobile phone is playing content in the normal order, the mobile phone executes S940. When the user fast-forwards or rewinds the streaming media file, the mobile phone executes S950.

[0160] In the S940, during normal playback, while the phone is playing the current streaming media segment, a second thread is executed to retrieve the most recent streaming media segment. If this most recent segment is plaintext, it is stored directly in memory for later playback. If this most recent segment is ciphertext, the phone decrypts it and stores it in memory for later playback.

[0161] refer to Figure 10a , Figure 10a This diagram illustrates a scenario where a mobile phone acquires a streaming media clip. Figure 10a As shown, this scenario includes a server 1010 and a mobile phone 1020. The server 1010 stores the song "My Love" 1030, which is divided into 40 streaming media segments. Based on the above steps S520 and S530, the server 1010 determines the locations of the plaintext and ciphertext of the song. Figure 10a As shown, following the normal playback order from front to back, the plaintext streaming media segments are the 1st, 2nd, 12th, 18th, 19th, and 24th streaming media segments. The remaining streaming media segments are encrypted by server 1010 to obtain ciphertext. When the user starts playing the song for the first time, phone 1020 executes S910 and S920 to obtain the first plaintext and play it. Simultaneously, phone 1020 executes S930-S940 to create a second thread, run the second thread, and obtain the second plaintext. After the first streaming media file finishes playing, the phone plays the second streaming media segment. At the same time, the second thread continues to run to obtain the third streaming media segment (ciphertext), decrypts the ciphertext, and stores it in the phone, waiting for playback. This process continues until the entire song is played normally.

[0162] When users fast-forward or rewind streaming media, the phone executes S950-S960 to retrieve the starting segment. S950-S960 is a further description of S560. The following is a detailed description of S950-S960.

[0163] In the S950, the mobile phone obtains the target position when the user fast-forwards or rewinds the progress bar, and retrieves the description status of the target segment corresponding to that position from the description file. During playback, the user operates the progress bar on the phone screen to adjust the playback progress of the streaming media file, including fast-forwarding and rewinding. The description status of the target segment includes: whether the target segment is plaintext or ciphertext, whether it has been downloaded by the phone, and if so, whether the downloaded ciphertext has been decrypted.

[0164] In S960, the mobile phone obtains the starting segment based on the description status of the target segment. This step also further explains the pre-made strategy in S540.

[0165] In one embodiment of this application, the priority order of the mobile phone in selecting streaming media segments as the starting segment is as follows from high to low: the target segment is downloaded plaintext or the target segment is downloaded and decrypted ciphertext, the target segment is downloaded but not decrypted ciphertext, and the target segment is not downloaded plaintext.

[0166] Furthermore, when the target segment is undownloaded ciphertext, the mobile phone also needs to know the description status of streaming media segments within a preset range. This includes whether the streaming media segments within the preset range contain plaintext, whether they have been downloaded by the phone, whether the downloaded ciphertext has been decrypted, and the location information of these streaming media segments. Based on this description status, the mobile phone selects the streaming media segments within the preset range as the starting segments for playback, arranging them in descending order of priority: the nearest downloaded plaintext or downloaded and decrypted ciphertext within the preset range to the target segment; the downloaded but undecrypted ciphertext within the preset range; and the nearest undownloaded plaintext within the preset range to the target segment.

[0167] The phone sets the lowest priority for acquiring the starting segment to: the target segment is an undownloaded ciphertext.

[0168] When the mobile phone selects the starting segment according to the above priority sorting, if the mobile phone determines that the target segment has been downloaded into its internal memory, it will use the target segment as the starting segment and play the target segment directly.

[0169] When the phone determines that the target segment has not been downloaded, but the target segment is in plaintext, it downloads the plaintext and uses it as the starting segment, and then plays the plaintext.

[0170] When the phone determines that the target segment is encrypted and has not been downloaded, it obtains the description status of streaming media segments within a preset range from the description file, and selects a suitable streaming media segment as the starting segment for playback based on these description statuses. For example, the phone may select downloaded plaintext or downloaded decrypted ciphertext within the preset range as the starting segment for playback.

[0171] In some embodiments of this application, the preset range may include two streaming media segments before and after the target segment, or it may include three streaming media segments before or after the target segment; the specific range is not limited. Specifically, the mobile phone first determines whether there is any downloaded plaintext or downloaded and decrypted ciphertext within the preset range. If so, it selects the downloaded plaintext or downloaded and decrypted ciphertext closest to the target segment as the starting segment according to the proximity principle. When the mobile phone selects according to the proximity principle, if two streaming media segments are equidistant from the target segment, the streaming media segment before the target segment can be selected as the starting segment to play the user-specified target segment. When the mobile phone determines that there is no plaintext within the preset range, but only downloaded and uncracked ciphertext, it selects the ciphertext closest to the target segment, decrypts the ciphertext, and uses it as the starting segment.

[0172] If the phone determines that the target segment has not been downloaded, and there is no plaintext or already downloaded ciphertext within the preset range of the target segment, the phone will directly download the ciphertext corresponding to the target location, decrypt it, and then play it.

[0173] While the phone is playing the initial segment, the phone continues to execute S940.

[0174] In one embodiment of this application, when the phone is not executing S930, and the user fast-forwards or rewinds a streaming media file, the phone directly executes S950-S960. When the phone is playing the initial segment, the phone simultaneously executes S930-S940.

[0175] refer to Figure 10b , Figure 10b The diagram illustrates a scenario where a mobile phone captures a streaming media segment during fast-forwarding. Combined with... Figure 10a and 10b As shown, during playback on the phone 1020, the user adjusts the progress bar position to, as shown in the image. Figure 10b The target location is shown. Mobile phone 1020 first determines whether the plaintext 20 (target segment) corresponding to the target location has been downloaded. If downloaded, mobile phone 1020 directly plays plaintext 20. If not downloaded, mobile phone 1020 judges the target segment. If the target segment 20 is determined to be ciphertext, the description status of streaming media segments within a preset range is obtained. Assuming the preset range is one streaming media segment before and after it, the description status of these two streaming media segments is judged, i.e., plaintext 19 and ciphertext 21. The phone further determines whether they have been downloaded. If neither has been downloaded, the priority of the undownloaded plaintext is higher than that of the undownloaded ciphertext, so the phone directly downloads plaintext 19 and plays it.

[0176] The streaming media data processing method according to the embodiments of this application determines the plaintext position based on the user's playback habits, thereby increasing the probability that the target segment specified by the user hits the plaintext when the user fast-forwards or rewinds, saving decryption time. Furthermore, when the mobile phone acquires the starting segment, it acquires as much downloaded plaintext or ciphertext as possible, and sets a preset range, increasing the probability of hitting plaintext or downloaded ciphertext, minimizing network transmission latency and decryption time, and improving the user's playback experience.

[0177] In another scenario of this application, after the aforementioned mobile phone obtains the streaming media file, if the user wants to connect the watch to the mobile phone and play the streaming media file from the watch, the mobile phone also needs to determine whether the streaming media file needs to be split again based on the data processing of the watch.

[0178] refer to Figure 11a , Figure 11a A flowchart of another streaming media data processing method is shown. This processing method is executed by server 1101, mobile phone 1102, and watch 1103. Mobile phone 1102 is connected to both server 1101 and watch 1103. Mobile phone 1102 determines whether a secondary segmentation of the streaming media file obtained in the above embodiment is needed based on the watch's data processing capabilities, including the following steps:

[0179] S1110 allows the mobile phone to obtain parameters of the watch's data processing capabilities from the watch, such as the watch model and storage capacity.

[0180] S1120, the mobile phone sends a request to the server for a protection scheme for the streaming media files of the watch. This request includes parameters of the watch's data processing capabilities, as well as the address of the streaming media file, file type, and watch model.

[0181] S1130, the server calculates a protection scheme for the streaming media files applicable to the watch based on the request.

[0182] The server determines the preset duration of the streaming media segment suitable for the watch based on the streaming media file sent by the mobile phone and the watch's data processing capabilities.

[0183] S1140, the server sends the protection scheme for the streaming media files applicable to the watch to the mobile phone. S1150, the mobile phone determines whether it needs to perform secondary splitting of the streaming media files stored inside it based on the protection scheme.

[0184] In one embodiment of this application, the mobile phone determines whether secondary segmentation is required based on the duration of the streaming media segment suitable for playback on the watch calculated by the server and the duration of the streaming media segment stored in the mobile phone. When the duration of the streaming media segment suitable for playback on the watch calculated by the server is less than the duration of the original streaming media segment stored in the mobile phone, the mobile phone determines that secondary segmentation of the streaming media file stored therein is required and executes S1170-S1190. When the duration of the streaming media segment suitable for playback on the watch calculated by the server is greater than or equal to the duration of the original streaming media segment stored in the mobile phone, the mobile phone determines that secondary segmentation of the streaming media file stored therein is not required and executes S1160. For example, if the original duration of the streaming media segment is 6 seconds, and the protection scheme sent by the server specifies that the duration of the streaming media segment suitable for playback on the watch is 4 seconds, then secondary segmentation of the streaming media file is required.

[0185] In S1160, the phone directly sends the existing streaming media file and its corresponding description file to the watch. In S1170, the phone performs a second segmentation of the streaming media file, encrypts the resulting ciphertext, and generates a new description file for the streaming media file. The segmentation process can be based on the duration of the streaming media segment provided by the server suitable for watch playback, or it can be shorter than that duration.

[0186] In the S1180, the phone sends the split streaming media file and the corresponding description file to the watch.

[0187] In the S1190, the watch directly stores the new streaming media file or updates an already stored streaming media file.

[0188] In embodiments of this application, the mobile phone can also generate a pre-defined strategy, which includes a priority ordering of the segmented plaintext and ciphertext for the watch to acquire. The mobile phone can send a description file and the pre-defined strategy to the watch, and the watch can acquire the segmented streaming media segments from the mobile phone according to the pre-defined strategy and the description file. The process of acquiring the streaming media segments can be referred to... Figure 5 The description of the S560 in the document will not be repeated here.

[0189] In one embodiment of this application, the server may also redetermine the plaintext position of the streaming media file based on the user's watch usage habits.

[0190] refer to Figure 11b , Figure 11bA schematic diagram comparing the structure of the streaming media file before and after is shown. This diagram includes the original, already generated streaming media file 1104, and the streaming media file 1105 to be generated. Considering that users primarily use the pause function during watch use, the server can set the plaintext based on the scenario where most users pause playback. Furthermore, the set of target segments during pause playback can be used as the highest level for selecting plaintext. For example... Figure 11b As shown, plaintext positions are selected only from the target segment set during the temporary playback scenario, with the 10 streaming media segments having the highest user participation rate being used as plaintext segments. The server can send the regenerated streaming media file and description file to the mobile phone, which then synchronizes the regenerated streaming media file and description file to the watch.

[0191] According to the method in this application embodiment, the streaming media file is segmented based on the data processing capability of the playback device, which improves the acquisition time of a single streaming media segment and reduces the decryption time, effectively reducing the start-up time. Furthermore, since the plaintext setting is based on the user's playback habits, the probability that the user-specified target segment is in plaintext is increased when the user fast-forwards, rewinds, or pauses playback. This reduces decryption time and improves the watch's start-up speed.

[0192] This application also provides an electronic device, which includes:

[0193] Memory, used to store instructions executed by one or more processors of the device, and

[0194] Processor, for executing the above embodiments Figure 5 The methods shown in S510-S560 in Figure 11, and S1130 and S1140 in Figure 11.

[0195] refer to Figure 12 This application also provides a terminal device. Figure 12 A schematic diagram of the terminal device is shown. (For example...) Figure 12 As shown, the terminal device includes:

[0196] Memory 1210 is used to store instructions executed by one or more processors of the device, and

[0197] Processor 1220 is used to execute the above embodiments. Figure 5 S560 and Figure 9 The method shown.

[0198] This application also provides a computer-readable storage medium storing a computer program, which, when run by a processor, causes the processor to perform the streaming media data processing method of the above embodiments.

[0199] Now for reference Figure 13 The diagram shown is a block diagram of a SoC (System on Chip) 1300 according to an embodiment of this application. Figure 13 In the diagram, similar components share the same reference numerals. Additionally, dashed boxes are an optional feature for more advanced SoCs. Figure 13 In this SoC 1300, the following are included: an interconnect unit 1350 coupled to an application processor 1310; a system proxy unit 1380; a bus controller unit 1390; an integrated memory controller unit 1340; a group or one or more coprocessors 1320, which may include integrated graphics logic, an image processor, an audio processor, and a video processor; a static random access memory (SRAM) unit 1330; and a direct memory access (DMA) unit 1360. In one embodiment, the coprocessor 1320 includes a dedicated processor, such as, for example, a network or communication processor, a compression engine, a GPGPU, a high-throughput MIC processor, or an embedded processor.

[0200] The static random access memory (SRAM) cell 1330 may include one or more computer-readable media for storing data and / or instructions. The computer-readable storage medium may store instructions, specifically, temporary and permanent copies of those instructions. These instructions may include, when executed by at least one unit in the processor, causing the SoC 1300 to perform a streaming media data processing method according to the above embodiments, as detailed in the methods described above, which will not be repeated here.

[0201] The various embodiments of the mechanisms disclosed in this application can be implemented in hardware, software, firmware, or a combination of these implementation methods. Embodiments of this application can be implemented as computer programs or program code executable on a programmable system, the programmable system including at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0202] Program code can be applied to input instructions to execute the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, the processing system includes any system having a processor such as, for example, a Digital Signal Processor (DSP), a microcontroller, an Application Specific Integrated Circuit (ASIC), or a microprocessor.

[0203] The program code can be implemented using a high-level procedural language or an object-oriented programming language to communicate with the processing system. Assembly language or machine language can also be used when needed. In fact, the mechanisms described in this application are not limited to any particular programming language. In either case, the language can be a compiled language or an interpreted language.

[0204] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored thereon on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, the instructions may be distributed via a network or via other computer-readable media. Therefore, machine-readable media may include any mechanism for storing or transmitting information in a machine-readable (e.g., computer-readable) form, including but not limited to floppy disks, optical disks, CD-ROMs, compact disc read-only memory (CD-ROMs), magneto-optical disks, read-only memory (ROM), random access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic cards or optical cards, flash memory, or tangible machine-readable storage for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in the form of electrical, optical, acoustic, or other forms of propagated signals. Therefore, machine-readable media includes any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a machine-readable (e.g., computer-readable) form.

[0205] In the accompanying drawings, some structural or methodological features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be necessary. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the accompanying drawings. Furthermore, including structural or methodological features in a particular figure does not imply that such features are required in all embodiments, and in some embodiments, these features may be omitted or may be combined with other features.

[0206] It should be noted that all units / modules mentioned in the device embodiments of this application are logical units / modules. Physically, a logical unit / module can be a physical unit / module, a part of a physical unit / module, or a combination of multiple physical units / modules. The physical implementation of these logical units / modules themselves is not the most important factor; the combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed in this application. Furthermore, to highlight the innovative aspects of this application, the above-described device embodiments of this application have not introduced units / modules that are not closely related to solving the technical problems proposed in this application. This does not mean that the above-described device embodiments do not contain other units / modules.

[0207] It should be noted that in the examples and description of this patent, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0208] Although this application has been illustrated and described with reference to certain preferred embodiments thereof, those skilled in the art should understand that various changes in form and detail may be made thereto without departing from the spirit and scope of this application.

Claims

1. A method for processing streaming media data, applied to a system including a server and a first electronic device, characterized in that, The method includes: The server divides the streaming media file into several streaming media segments according to a predetermined time length; The server determines the streaming media segment as plaintext from the plurality of streaming media segments according to a preset rule. The preset rule is set according to the playback habits of a preset user group. The plaintext is an unencrypted streaming media segment. The server encrypts streaming media segments other than the plaintext to obtain ciphertext; The server sets a pre-defined strategy for the first electronic device to acquire streaming media segments, and generates a description file based on the pre-defined strategy to record the location information of the plaintext and ciphertext in the streaming media file; wherein, the pre-defined strategy includes the priority order of the plaintext and ciphertext being acquired by the first electronic device. The first electronic device acquires the description file and sequentially acquires streaming media segments from the streaming media file based on the description file for processing by the first electronic device.

2. The method according to claim 1, characterized in that, The server determines the streaming media segment to be plaintext from the plurality of streaming media segments according to a preset rule, including: The server calculates the total number of plaintexts in the streaming media file based on the total duration of plaintexts in the streaming media file that can be provided to the user and the predetermined duration of each streaming media segment. The server calculates the number of plaintexts in the header and the number of plaintexts in other positions based on a preset first ratio between the number of plaintexts in the header of the streaming media file and the number of plaintexts in other positions of the streaming media file, and the total number of plaintexts. The plaintext in the header of the streaming media file refers to the first plaintext that starts counting from zero seconds of the streaming media file in the playback order, or includes the first plaintext and a series of plaintexts that are consecutively arranged next to the first plaintext. The server determines the plaintext streaming media segments in the header of the streaming media file based on the amount of plaintext in the header, and determines the plaintext streaming media segments in other locations of the streaming media file based on the amount of plaintext in other locations of the streaming media file.

3. The method according to claim 2, characterized in that, The server uses the plaintext in the header as the highest priority for selecting plaintext.

4. The method according to claim 2 or 3, characterized in that, The server determines the location of plaintext at other locations based on the amount of plaintext at other locations in the streaming media file, including: The server obtains the target segment selected by each user in the preset user group when controlling the progress of the streaming media file. The server counts the set of target segments specified by users in the preset user group, and the number of users corresponding to each target segment in the set, and calculates the proportion of the number of users corresponding to each target segment. The proportion of the number of users refers to the ratio of the number of users corresponding to each target segment in the target segment set to the number of users in the user group corresponding to the set. The server determines the location of plaintext in other locations from the plurality of streaming media segments based on the proportion of users and the amount of plaintext in other locations.

5. The method according to claim 4, characterized in that, The user controls the progress of the streaming media file, including fast forwarding or rewinding the streaming media file.

6. The method according to claim 5, characterized in that, The server determines the location of plaintext at other locations based on the amount of plaintext at other locations in the streaming media file, including: The server respectively calculates the target segment set for fast-forwarding and the target segment set for rewinding the streaming media file for users within the preset user group. The server calculates the percentage of users who fast-forwarded for each target segment in the target segment set during fast-forwarding, and the percentage of users who rewind for each target segment in the target segment set during rewinding. The server determines the positions of the plaintext during fast-forwarding and rewinding from the plurality of streaming media segments based on the proportion of users who fast-forward, the proportion of users who rewind, and the amount of plaintext at other positions.

7. The method according to claim 6, characterized in that, The server determines the positions of the plaintext during fast-forwarding and rewinding from the plurality of streaming media segments based on the proportion of users who fast-forward, the proportion of users who rewind, and the amount of plaintext at other positions, further including: The server determines the position of the plaintext during fast-forwarding from the plurality of streaming media segments based on a preset second ratio of the number of plaintexts during fast-forwarding to the number of plaintexts during rewinding for the user in relation to the streaming media file, the number of plaintexts at other positions, and the proportion of the number of users who fast-forward. The server determines the position of the plaintext during rewind from the plurality of streaming media segments based on the second ratio, the amount of plaintext at other positions, and the proportion of users who rewind.

8. The method according to claim 7, characterized in that, include: In the set of target segments during fast-forwarding, the server determines the target segments that meet the second ratio relationship based on the rule that the higher the proportion of fast-forwarding users, the higher the priority of the target segments corresponding to that proportion of fast-forwarding users being selected as plaintext. The position of this target segment is then used as the position of the plaintext during fast-forwarding; or... In the set of target fragments during rewind, the server determines the target fragments that meet the second ratio relationship based on the rule that the higher the proportion of users who rewind, the higher the priority of the target fragments corresponding to that proportion of users who rewind. The position of the target fragment is then used as the position of the plaintext during rewind.

9. The method according to claim 8, characterized in that, When the same target segment belongs to both the target segment set during fast forward and the target segment set during rewind, the server sets the target segment to the priority of the target segment set during fast forward, and the proportion of users corresponding to the target segment is the sum of the proportion of users who fast forward and the proportion of users who rewind.

10. The method according to claim 8, characterized in that, When there are target segments in the target segment set that have the same proportion of users to choose from during fast forward, the server randomly removes one of the two target segments that are closest in position until the plaintext quantity during fast forward is met; or, When there are target segments in the target segment set that have the same proportion of users to choose from during rewind, the server randomly removes one of the two closest target segments until the plaintext count during rewind is met.

11. The method according to claim 4, characterized in that, The server obtains the target segment selected by each user in the preset user group when controlling the progress of the streaming media file, including: When the server obtains the target position selected by each user on the screen of a user group playing the streaming media file on screens of different sizes, the streaming media segment corresponding to the target position is the target segment.

12. The method according to claim 5, characterized in that, The user control of the streaming media file's progress also includes: pausing playback. The server determines the location of plaintext at other locations based on the amount of plaintext at other locations in the streaming media file, and further includes: The server obtains the target segment selected by each user in the user group to pause playback when playing the streaming media file, and obtains the set of target segments at the time of pause; The server calculates the percentage of paused users corresponding to each target segment in the target segment set at the time of pause. The server determines the location of the plaintext during the pause from the plurality of streaming media segments based on the percentage of paused users and the amount of plaintext at other locations.

13. The method according to claim 12, characterized in that, The target fragments in the target fragment set during the pause are selected as plaintext with the lowest priority.

14. The method according to claim 1, characterized in that, The pre-formulation strategy includes the priority order of the plaintext and ciphertext being acquired by the first electronic device, including: The server sets the priority for the acquisition of the streaming media segment by the first electronic device, and arranges the acquisition priorities of the streaming media segments from high to low, in the following order: the target segment specified by the user is plaintext that has been downloaded by the first electronic device or ciphertext that has been downloaded by the first electronic device and decrypted; the target segment specified by the user is ciphertext that has been downloaded by the first electronic device but not decrypted; and the target segment specified by the user is plaintext that has not been downloaded.

15. The method according to claim 14, characterized in that, When the target segment specified by the user is undownloaded ciphertext, the server prioritizes the acquisition of streaming media segments from high to low, and further includes: The server will sort streaming media segments that fall within a preset range according to the priority of their acquisition, and then sort them from highest to lowest priority as follows: plaintext or ciphertext that has been downloaded by the first electronic device and has been downloaded and decrypted by the first electronic device, which is closest to the target segment specified by the user within the preset range; ciphertext that has been downloaded by the first electronic device but has not been decrypted within the preset range; and plaintext that has not been downloaded and is closest to the target segment specified by the user within the preset range. The preset range is the difference between the location of the acquired streaming media segment and the location of the target segment selected by the user that the user can tolerate.

16. The method according to claim 1, characterized in that, When the system further includes a second electronic device, and the second electronic device is connected to the first electronic device as an execution device for playing the streaming media file, the method further includes: The first electronic device acquires parameters of the second electronic device's data processing capability and sends the parameters to the server; The server determines whether to further segment the streaming media segments in the streaming media file based on the parameters. When it is determined that segmentation is necessary, a corresponding segmentation scheme is generated. The first electronic device segments each of the streaming media files according to the segmentation scheme, and re-encrypts the segmented ciphertext in the streaming media files.

17. An electronic device, characterized in that, include: Memory, used to store instructions executed by one or more processors of the device, and A processor for performing the method according to any one of claims 1-16.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to perform the method according to any one of claims 1-16.

Images