Video stream loading method, electronic device and storage medium

Abstract

The present application discloses a video stream loading method, an electronic device and a storage medium, and relates to the technical field of video stream loading. The method is applied to the server of a communication system, and the communication system also includes a client, and the server is connected to the client in communication; first, the original video stream is obtained, and then the audio file and the video file of the original video stream are extracted, and then the video file and the audio file are segmented according to a preset time period to generate a plurality of file fragments, wherein each file fragment includes time information, and then an access request based on the original video stream sent by the client is received, wherein the access request carries a play mode and a timestamp, and finally the file fragments corresponding to the audio file and / or the video file are sent to the client according to the play mode and the timestamp. The present application has the advantage of saving the traffic consumed by video stream loading.

Description

Technical Field

[0001] The present invention relates to the technical field of video stream loading, and in particular to a video stream loading method, electronic equipment and storage medium. Background Art

[0002] Video streaming refers to the transmission of video data. Currently, with the rise of short video platforms, video streaming is receiving more and more attention.

[0003] During the video stream loading process, a large amount of traffic is generally required for loading, resulting in a large amount of traffic consumption when users play the video stream.

[0004] In summary, the existing technology has the problem that video streaming playback requires a large amount of traffic. Summary of the invention

[0005] Therefore, the object of the present invention is to provide a video stream loading method, an electronic device and a storage medium, which have the effect of improving the large amount of traffic consumed by video stream playback.

[0006] In a first aspect, a video stream loading method is provided, the method being applied to a server of a communication system, the communication system further comprising a client, the server being in communication connection with the client; the method comprising:

[0007] Get the original video stream;

[0008] Extracting the audio file and the video file of the original video stream;

[0009] The video file and the audio file are segmented according to preset time periods to generate a plurality of file segments, wherein each file segment includes time information;

[0010] Receiving an access request based on the original video stream sent by the client, wherein the access request carries a play mode and a timestamp;

[0011] The file segments corresponding to the audio file and / or the video file are sent to the client according to the play mode and the timestamp.

[0012] Optionally, sending the file segments corresponding to the audio file and / or the video file to the client according to the play mode and the timestamp includes:

[0013] When the playback mode is a normal mode without mute or static screen, the file segments corresponding to the audio file and the video file are sent to the client;

[0014] When the playback mode is a silent mode, sending a file segment corresponding to the video file to the client;

[0015] When the playing mode is the silent mode, the file segment corresponding to the audio file is sent to the client.

[0016] Optionally, sending the file segments corresponding to the audio file and / or the video file to the client according to the play mode and the timestamp includes:

[0017] When no jump instruction is received, the file segment of the current time period and / or the file segment of the next time period corresponding to the timestamp are sent to the client.

[0018] Optionally, sending the file segments corresponding to the audio file and / or the video file to the client according to the play mode and the timestamp includes:

[0019] When a jump broadcast instruction is received, determining a current time period corresponding to the timestamp;

[0020] The file segments of the current period are sent to the client.

[0021] Optionally, sending the file segments of the current period to the client includes:

[0022] Sending the file segment of the current time period at the first resolution to the client;

[0023] The sending of the file segments corresponding to the audio file and / or the video file to the client according to the play mode and the timestamp further includes:

[0024] Sending a file segment of a next time period at a second resolution to the client; wherein the second resolution is higher than the first resolution.

[0025] Optionally, sending the file segments corresponding to the audio file and / or the video file to the client according to the play mode and the timestamp includes:

[0026] When a jump broadcast instruction is received, determining a current time period corresponding to the timestamp;

[0027] Segmenting the file segment of the current period based on the timestamp to obtain at least two file sub-segments, wherein each file sub-segment includes time information;

[0028] According to the time information of each of the file sub-segments, the file sub-segments located after the timestamp are sent to the client.

[0029] Optionally, the sending the file segments corresponding to the audio file and / or the video file to the client according to the play mode and the timestamp further includes:

[0030] Sending a file segment of a next time period at a second resolution to the client, wherein the file segment sent to the client has a first resolution, and the second resolution is higher than the first resolution.

[0031] In a second aspect, a video stream loading method is provided, the method being applied to a client of a communication system, the communication system further comprising a server, the server being communicatively connected to the client; the method comprising:

[0032] In response to user operations, an access request carrying a play mode and a timestamp is generated;

[0033] Determine the original video stream from the server according to the access request, and pull the file segments corresponding to the audio file and / or the video file in the original video stream according to the play mode and the timestamp; wherein the file segments are generated by the server by segmenting the audio file and the video file according to a preset time period;

[0034] When the audio file and the video file are pulled at the same time, the audio file and the video file are synthesized.

[0035] Optionally, determining the original video stream from the server according to the access request, and pulling the file segments corresponding to the audio file and / or the video file in the original video stream according to the play mode and the timestamp, includes:

[0036] When the playback mode is a normal mode that is neither silent nor silent, the file segments corresponding to the audio file and the video file are pulled, and the file segments of the audio file and the video file are synthesized;

[0037] When the playback mode is the silent mode, pulling a file segment corresponding to the video file;

[0038] When the playing mode is the silent mode, the file segment corresponding to the audio file is played.

[0039] Optionally, determining the original video stream from the server according to the access request, and pulling the file segments corresponding to the audio file and / or the video file in the original video stream according to the play mode and the timestamp, includes:

[0040] Pull the file fragment of the current time period and the file fragment of the next time period corresponding to the timestamp.

[0041] Optionally, determining the original video stream from the server according to the access request, and pulling the file segments corresponding to the audio file and / or the video file in the original video stream includes:

[0042] When receiving a jump operation from the user, obtaining a target timestamp corresponding to the jump operation;

[0043] Pull the corresponding file segment according to the target timestamp.

[0044] In a third aspect, an electronic device is provided, comprising a processor and a memory storing a computer program, wherein the processor is configured to execute the above-mentioned video stream loading method when running the computer program.

[0045] In a fourth aspect, a storage medium is provided, wherein the storage medium stores a computer program, and the computer program is configured to execute the above-mentioned video stream loading method when executed.

[0046] The embodiments of the present invention have the following beneficial effects:

[0047] The present invention provides a video stream loading method, an electronic device and a storage medium. The method is applied to a server of a communication system. The communication system also includes a client. The server is connected to the client in communication. First, an original video stream is obtained, and then an audio file and a video file of the original video stream are extracted. Then, the video file and the audio file are segmented according to a preset time period to generate a plurality of file fragments, wherein each file fragment includes time information, and then an access request based on the original video stream sent by the client is received, wherein the access request carries a play mode and a timestamp, and finally, the file fragments corresponding to the audio file and / or the video file are sent to the client according to the play mode and the timestamp. On the one hand, by extracting the video file and the audio file from the original video stream, the client can select the play mode when loading the video stream, and can also load only the audio file or only the video file, so as to achieve the purpose of saving traffic. On the other hand, by segmenting the video file and the audio file, the video file and the audio file can be segmented into a plurality of file fragments, and then when the video is loaded, only the corresponding file fragment needs to be selected according to the timestamp for loading, without directly loading the entire video, so as to achieve the purpose of saving loading traffic.

[0048] Optional features and other effects of the embodiments of the present invention are partially described below and partially understood by reading this document. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The embodiments of the present invention are described in detail with reference to the accompanying drawings. The elements shown are not limited by the proportions shown in the accompanying drawings. The same or similar reference numerals in the accompanying drawings represent the same or similar elements, wherein:

[0050] Figure 1 A first exemplary flow chart of a video stream loading method provided by an embodiment of the present invention is shown.

[0051] Figure 2 A schematic diagram of server-side preprocessing provided by an embodiment of the present invention is shown.

[0052] Figure 3 Another schematic diagram of server-side preprocessing provided by an embodiment of the present invention is shown.

[0053] Figure 4 A schematic diagram of interaction between a server and a client provided in an embodiment of the present invention is shown.

[0054] Figure 5 A second exemplary flow chart of the video stream loading method provided by an embodiment of the present invention is shown.

[0055] Figure 6 A third exemplary flow chart of the video stream loading method provided by an embodiment of the present invention is shown.

[0056] Figure 7 A fourth exemplary flow chart of the video stream loading method provided by an embodiment of the present invention is shown.

[0057] Figure 8 Another schematic diagram of interaction between a server and a client provided in an embodiment of the present invention is shown.

[0058] Fig. 9 A module schematic diagram of a video stream loading device provided by an embodiment of the present invention is shown.

[0059] Fig.10 An exemplary flow chart of another video stream loading method provided by an embodiment of the present invention is shown.

[0060] Fig.11 An exemplary flow chart of another video stream loading method provided by an embodiment of the present invention is shown.

[0061] Fig.12 A schematic diagram showing an exemplary hardware structure of a mobile terminal capable of implementing the method according to an embodiment of the present invention;

[0062] Fig.13 A schematic diagram showing an exemplary operating system structure of a mobile terminal capable of implementing the method according to an embodiment of the present invention is shown;

[0063] Fig.14 A schematic diagram of an exemplary operating system structure of a mobile terminal capable of implementing the method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific implementation methods and drawings. Here, the exemplary implementation methods of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0065] The embodiment of the present invention provides a video stream loading method, device, and related electronic equipment and storage medium. The video stream loading method can be implemented with the aid of one or more computers, such as terminals, such as mobile terminals, such as smart phones. In some embodiments, the video stream loading method can be implemented by software, hardware, or a combination of software and hardware.

[0066] The following is an exemplary description of the video stream loading method provided by this application:

[0067] As an optional implementation method, the video stream loading method is applied to the server of a communication system, and the communication system also includes a client, and the server is connected to the client in communication. Among them, the server refers to a terminal that provides video streaming services, such as a video website terminal. The client refers to an intelligent terminal used by the user, such as a mobile phone, a tablet computer, a wearable mobile device, etc., which is not specifically limited here. The client can access the server to load the video stream. For example, an APP of a video website A is installed on the client. When the user opens the APP, he can watch the video on the video website A, and then load the video stream through the client.

[0068] Optionally, see Figure 1 , the video stream loading method includes:

[0069] S102, obtaining an original video stream.

[0070] S104: extracting the audio file and the video file of the original video stream.

[0071] S106, dividing the video file and the audio file into preset time periods to generate a plurality of file segments, wherein each file segment includes time information.

[0072] S108: Receive an access request based on the original video stream sent by the client, wherein the access request carries a play mode and a timestamp.

[0073] S110: Send file segments corresponding to the audio file and / or the video file to the client according to the play mode and the timestamp.

[0074] Among them, see Figure 2, on the server side, the original video stream in the database can be preprocessed by separating the audio track from the video track in the original video stream, and then extracting the audio file and the video file. The audio file only contains the audio information corresponding to the original video stream, and the video file only contains the video information corresponding to the original video stream. It should be noted that the audio track and the video track are in a one-to-one correspondence, that is, the time point of the audio track corresponds to the time point of the video track. For example, for an original video stream with a total length of 1 minute, the duration of the video track and the audio track is also 1 minute, and the video track of the 1S corresponds to the audio track of the 1S, so that when the video stream is loaded, the synchronous playback of the video and audio is achieved. It can be understood that for an original video stream with a total length of 1 minute, after extracting the audio file and the video file, the duration of the audio file and the video file is 1 minute.

[0075] It should be noted that, generally, on the server side, the audio file and the video file can be extracted from the entire original video stream, but in some other optional implementations, the audio file and the video file can be extracted from only a part of the original video stream. For example, in one implementation, the video stream loading method provided by the present application can be used to load data for a video stream within a certain time period in the video stream, thereby satisfying the need for some users to quickly watch the video in a low data mode but in a near real-time mode.

[0076] By splitting the original video stream, when the client loads the video stream, the audio file and / or the video file can be selected to be loaded, thereby achieving the purpose of saving traffic.

[0077] For example, when a user is watching a video normally, the video file and the audio file can be loaded at the same time; when the user is in some special scenarios, only the video file or the audio file can be loaded. For example, if the user is in a public environment and plays the sound out loud, it may affect others. At this time, the user can choose to play only the video file and not the audio file; or if the user is on a running train and watches a video, he may get carsick. At this time, the user can choose to receive only the audio file and not watch the video. By selectively receiving video files and audio files, the purpose of saving user traffic can be achieved in specific scenarios, and the user experience can be improved.

[0078] Here, optionally, the playback mode may include a first playback mode, which is defined based on the playback mode (display or broadcast). For example, the playback mode may include a silent mode, a static screen mode, and a normal mode that is not silent and not static screen, and the user can select a mode based on the current scenario through the client. When the user selects the normal mode, the server will send both the audio file and the video file to the client, and the client will synthesize the received audio file and video file locally; when the user selects the silent mode, the server will only send the video file to the client, and play the video file through the client. At this time, the client cannot load the audio file; when the user selects the static screen mode, the server will only send the audio file to the client, and play the audio file through the client. Understandably, when in silent mode or static screen mode, the effect of saving user traffic can be achieved.

[0079] In order to further save traffic, during the video stream loading process, the video file and the audio file can be sliced ​​and the corresponding segment can be selected for loading during the loading process.

[0080] For example, see Figure 3 After extracting the video and audio files, the server will also segment the video and audio files according to the preset time periods. Figure 3 In the example, the preset time period is 2S. Of course, the preset time period can also be other values, such as setting the preset time period to 1S or 3S. Please continue to refer to Figure 3 ,Depend on Figure 3 It can be seen that when the original video stream has a duration of 1 minute, after the audio file and the video file are extracted, the duration of the audio file and the video file is also 1 minute, and then the audio file and the video file are segmented in 2 seconds, and then the video file is segmented into 30 file segments, and the audio file is also segmented into 30 file segments. It can be understood that in order to facilitate the loading of the video stream, each file clip includes time information.

[0081] Taking a video file as an example, after being cut into 30 file segments, if the file segments are arranged from left to right, the time information of the first file segment is 0 to 2S, the time information of the second file segment is 2 to 4S, the time information of the third file segment is 4 to 6S, and so on.

[0082] On this basis, when the client loads the video stream, the access request carries the play mode and timestamp, where the play mode is used to determine the video file and / or audio file to be loaded, and the timestamp is used to determine which time segments to load. Therefore, the server will send the file segments corresponding to the audio file and / or video file to the client based on the play mode and timestamp.

[0083] Further, the playback mode may include a second playback mode, which is defined based on the playback position (time point) operated by the user. Specifically, the second playback mode may include a sequential playback mode and a jump play (jump playback) mode. In an embodiment of the present invention, the jump play mode may include a variety of situations, such as a fast forward mode, a fast rewind mode, and a drag mode. Therefore, those skilled in the art will understand that these jump play modes will have corresponding jump play operations, such as a click fast forward button operation corresponding to the fast forward mode, a click fast rewind button operation corresponding to the fast rewind mode, and a drag progress bar operation corresponding to the drag mode (dragging the progress bar can achieve fast forward and fast rewind effects). Since jump play is performed based on user operation instructions, in some embodiments of the present invention, the jump play mode may also be referred to as a jump play instruction, such as a fast forward instruction, or directly referred to as its operation or effect, such as fast forward.

[0084] Those skilled in the art will understand that the first playback mode and the second playback mode can be combined with each other, for example, the video can be played sequentially or skipped (for example, by pulling the progress bar) in a normal mode that is not silent and not still screen, a silent mode, or a still screen mode.

[0085] Among them, when the playback mode is a normal mode without mute or static screen, the server will send the file segments corresponding to the audio file and the video file to the client.

[0086] When the playback mode is silent mode, the server will send the file segment corresponding to the video file to the client.

[0087] When the playback mode is the silent mode, the server will send the file segment corresponding to the audio file to the client.

[0088] For example, when the user opens the APP, selects silent mode, and after opening a video, directly fast-forwards to the 7th second, the server determines that it only needs to send the video file to the client. At the same time, the timestamp of the client's access request is the 7th second, then the server will directly send the file segments of 6 to 8 seconds to the client for playback. Of course, the server will also send the file segments after 6 to 8 seconds to the client. It can be understood that by dividing the video file and the audio file into segments, loading useless file segments can be avoided. Taking the above example, if the timestamp of the client's access request is the 7th second, the server does not need to send the three file segments of 0 to 2 seconds, 2 to 4 seconds, and 4 to 6 seconds to the client, thereby saving the traffic consumed by the client to load the video.

[0089] If the user selects the normal playback mode, the server will send the video file and audio file segments to the client at the same time, and the client will synthesize the file segments with the same time information locally.

[0090] For example, see Figure 4 When a 6-8S file segment needs to be synthesized, the server will send the file segment corresponding to the 6-8S video file and the file segment corresponding to the audio file to the client at the same time, and then the client will synthesize the file segments locally to synthesize the 6-8S video stream segment for playback. It can be understood that when it is necessary to synthesize file segments corresponding to other time information, the same method is used, which will not be repeated here.

[0091] Through the video stream loading method provided by the present application, on the one hand, the user can select the playback mode, and then can choose to load only the audio file or only the video file, so as to achieve the purpose of saving traffic. On the other hand, by segmenting the video file and the audio file, the video file and the audio file can be segmented into multiple file segments, and then when the video is loaded, only the corresponding file segment needs to be selected according to the timestamp for loading, without directly loading the entire video, thus avoiding the loading of unnecessary file segments and achieving the purpose of saving loading traffic.

[0092] In addition, as an optional implementation method, in this application, a preloading method can be used to ensure the smoothness of user video playback. At the same time, preloading is only performed according to file fragments, so it will not consume too much traffic. Even if the user uses the fast-forward method, there will not be too many file fragments that will not be played, thereby achieving the purpose of saving traffic.

[0093] For example, when a user needs to play a 6-8S video file and an audio file, the server will send the 6-8S video file and the audio file to the client. When the client synthesizes and plays the 6-8S file, the server will also send the 8-10S video file and the audio file to the client. This allows the client to naturally transition to 8-10S after playing 6-8S, thus avoiding playback freezes and the like.

[0094] Please note that Figure 5 , S110 includes:

[0095] S1101, determine whether a jump instruction is received, if not, execute S1102, if yes, execute S1104.

[0096] S1102: Send the file fragments of the current period and / or the file fragments of the next period corresponding to the timestamp to the client.

[0097] S1104, determining the current time period corresponding to the timestamp.

[0098] S1105: Send the file segments of the current period to the client.

[0099] The judgment of the skip instruction can be based on whether the time stamps carried in two adjacent access requests appear in a time interval. For example, when the user does not fast forward, the time stamp carried in the first access request is the 6th second, and after the playback is completed, the time stamp carried in the next access request is the 8th second, which means that the user is playing the video in sequence and is not fast forwarding.

[0100] However, if the timestamp carried by the first access request is the 6th second, and the timestamp carried by the second access request is the 30th second, it means that the user did not play the video in order, and the video skipped. Alternatively, if the timestamp carried by the first access request is the 6th second, and the timestamp carried by the second access request is the 2nd second, the video skipped as well.

[0101] Of course, in actual operation, a skip instruction can also be directly generated for the client. For example, when the client recognizes that the user drags the progress bar, it generates a skip instruction and sends the skip instruction and the access request to the server at the same time. After receiving the skip instruction, the server can send the corresponding file segment to the client for playback.

[0102] Among them, when no jump instruction is received, it means that the user is watching the video in the order of the progress bar at this time. On this basis, this application provides three possible loading methods:

[0103] The first one is to send the file fragment of the current period corresponding to the timestamp to the client.

[0104] That is, the server sends the file segments of the current time period directly to the client for playback according to the client's request. For example, in the normal mode of mute, if the timestamp carried by the access request sent by the client is 8S, the server sends the corresponding file segments to the client for loading according to the timestamp. In this example, the server sends the file segments of 8 to 10S to the client. After the client finishes playing the segment, it will continue to send request instructions at 10S, and the server will continue to send the file segments of 10 to 12S to the client, and so on.

[0105] The second method is to send the file fragment of the next period corresponding to the timestamp to the client.

[0106] That is, in this implementation, a preloading method is adopted. After receiving the request instruction, the server will not send the file fragment of the current time period to the client, but will send the file fragment of the next time period to the client. For example, if the timestamp carried by the access request sent by the client is 8S, the current file fragment corresponding to the event stamp is the file fragment of 8~10S, and at this time the server will send the file fragment of 10~12S to the client, and the file fragment of 8~10S has been sent to the client in the previous time period. Through this implementation, it is possible to send the file fragment to the client in advance, so that the client will not experience line freeze when playing video and / or audio. It should be noted that in this implementation, when in the first time period, the server will send the file fragment corresponding to the first time period and the file fragment of the next time period to the client. For example, when the user starts watching the video from the beginning, the server will send both the 0~2S and 2~4S file fragments to the client.

[0107] The third method is to send the file fragments of the current period and the file fragments of the next period corresponding to the timestamp to the client.

[0108] That is, in this implementation, the preloading method is also adopted. When the file segments are specifically sent, the file segments of the current period and the file segments of the next period are sent to the client to implement preloading. This can be implemented, for example, when the user starts watching a video, such as when the user clicks to enter the video.

[0109] Therefore, if preloading is required, after the server sends the file fragments of the time period corresponding to the timestamp to the client, it will send the file fragments of the next time period to the client before the end of the current time period to ensure the smoothness of video and audio playback.

[0110] For example, when the user clicks to enter the video, the memory function directly locates the timestamp to 6.2S, and after sending the 6-8S file segment to the client, as described in the third method, the server will also synchronously send the 8-10S file segment to the client, so that the client can continue to smoothly play the 8-10S file segment after playing the 6-8S file segment. And, when playing the 8-10S file segment, as described in the second method, the server will continue to send the 10-12S file segment to the client.

[0111] As an optional implementation method, in the specific implementation process, if the user does not jump to play, and the server receives the access request sent by the client, it will send the next file segment to the client. For example, when the client plays the file segment of 6 to 8S, the server has sent the file segment of 8 to 10S to the client. When the 8S ends, the client will send an access request to the server, and the access request includes a timestamp between 8 and 10S. At this time, since the file segment of 8 to 10S has been sent to the client, the server will send the file segment of 10 to 12S to the client.

[0112] When the server receives the jump instruction, it will determine the current time period corresponding to the timestamp and send the file segment of the current time period corresponding to the timestamp to the client. For example, when the server receives the jump instruction and the access request, and the timestamp carried by the access request is 7S, the server will send the file segment of 6~8S to the client. Moreover, the server will not send the file segment of 8~10S to the client at this time, because the user may just be looking for the segment he wants to watch at this time, so the user may fast forward again. At this time, if the file segment of 8~10S is directly sent to the client, it may cause a waste of loading traffic. Therefore, the server will only send the file segment of 6~8S to the client at this time, and if the user is not jumping at this time, when the client plays to the 8S, it will send the access request to the server again, and the server will send the file segment of 8~10S to the client. At this time, it can be determined that the server has not received the jump instruction, and can be processed according to the above-mentioned execution steps of not receiving the jump instruction. For example, in order to achieve preloading, the server can also send 10-12S file segments to the client at the same time. In the embodiment of the present invention, the slices are not equivalent to data packets transmitted by the network, but the corresponding video files / audio files are segmented based on the predetermined duration at the video track / audio track level.

[0113] By differentiating between receiving a skip instruction and not receiving a skip instruction, it is possible to avoid wasting traffic caused by users fast-forwarding multiple times.

[0114] Additionally, in one implementation, see Figure 6 , S110 includes:

[0115] S1106: Send the file segments of the current time period at the first resolution to the client.

[0116] S1107, sending the file segments of the next time period at a second resolution to the client; wherein the second resolution is higher than the first resolution.

[0117] That is, based on the above implementation, when receiving the jump instruction, the server will send the file segment corresponding to the time information to the client, and the file segment sent at this time is a low-resolution file segment. If the user does not fast forward at this time, the server will send the high-resolution file segment when sending the file segment of the next time period to the client.

[0118] For example, when the user fast-forwards to the 7th second, the server matches the file segment of 6 to 8 seconds and sends the low-resolution file segment of 6 to 8 seconds to the client. When the client plays to the 8th second, it sends an access request to the server again, and the server sends the high-resolution file segment of 8 to 10 seconds to the client. After that, if the user continues to watch without fast-forwarding, the server will continue to send the high-resolution file segment of 10 to 12 seconds to the client when it receives the next access request, and so on.

[0119] By setting the fast-forward function to send low-resolution file segments to the client, the user can avoid wasting traffic caused by multiple fast-forwards. For example, when a user wants to find a segment he wants to watch in a video, he may jump to play multiple times, such as dragging the progress bar or fast-forwarding. For example, dragging the progress bar or fast-forwarding to the 10th second, when the 10th second is still not the segment he wants to watch, the user will continue to fast-forward, such as dragging the progress bar or fast-forwarding to the 20th second, until he finds the segment he wants to watch. In this process, since the server only sends low-resolution file segments to the client, and the file segments obtained after dragging the progress bar or fast-forwarding once or multiple times are actually segments that the user may not want to watch, the high-resolution method of loading these segments is avoided, thereby achieving the purpose of saving loading traffic.

[0120] At the same time, since the time period of each file segment is actually relatively long, for example, the time period of each file segment is 2S, when the user drags the progress bar or fast-forwards, useless file segments may also be loaded. For example, when the user fast-forwards to the 7th second, the server will send the file segments of 6 to 8S to the client, and then, when the client plays, it will actually only play the file segments after the time point when the user fast-forwards. That is, in this example, although the server sends the file segments of 6 to 8S to the client, the client will only play the file segments of 7 to 8S when playing. Therefore, the file segments of 6 to 7S are not actually played by the client, resulting in a waste of traffic.

[0121] In view of this, in order to avoid wasting bandwidth when skipping playback, such as dragging the progress bar or fast forwarding, please refer to Figure 7 , S110 may include:

[0122] S1108: When a jump instruction is received, determine the current time period corresponding to the timestamp.

[0123] S1109, split the file segment of the current period based on the timestamp to obtain at least two file sub-segments, wherein each file sub-segment includes time information.

[0124] S1110 , sending the file sub-segment after the timestamp to the client according to the time information of each file sub-segment.

[0125] That is, after the user drags the progress bar or fast-forwards, the server does not immediately send the corresponding file segment to the client, but first splits the corresponding file segment and then sends the split sub-segments to the client.

[0126] For example, see Figure 8 When the timestamp is received as 7S, the server determines that the target file segment is a segment of 6 to 8S. At this time, the server divides the file segment of 6 to 8S. In this application, 1S is used as the time period for segmentation, and the file segment of 6 to 8S will be divided into two file sub-segments of 6 to 7S and 7 to 8S. Then the server sends the file sub-segment of 7 to 8S to the server. At the same time, when preloading is required, the file segment of 8 to 10S can also be sent to the client.

[0127] It can be understood that this implementation avoids the client receiving 6-7S file sub-segments, thereby reducing the traffic consumed by loading.

[0128] It should be noted that when the file segments are segmented, they can also be segmented according to other time periods, for example, the file segments can be segmented according to 0.5S time periods, so that the file sub-segments sent by the server to the client are more accurate, which is not limited here.

[0129] It should also be noted that Figure 8 Although it is shown that the audio file and the video file are loaded simultaneously, it is understandable that when the user selects the silent mode or the silent screen mode, the server may also only split a certain segment in the audio file or the video file into file sub-segments.

[0130] In addition, in order to avoid excessive complexity in data processing on the server side, when receiving a jump instruction, the server side will only split the file segment corresponding to the timestamp, and will not process the file segment after the timestamp.

[0131] On this basis, after S1110, the method further includes:

[0132] S1111: When the current time is the end time of the file sub-segment, the file segment of the next time period is sent to the client.

[0133] The current time refers to the time when the client is currently playing the video stream. For example, when the client plays a file segment of 7 to 8 seconds, if the current time is the 8th second, the server will send the file segment of 8 to 10 seconds to the client, and the server will not split the subsequent file segments.

[0134] The video stream loading method provided in the present application can reduce the waste of loading traffic as much as possible in different application scenarios, saving the traffic consumed by users when watching videos.

[0135] Of course, based on the segmentation of the file segments, the resolution can also be combined to reduce the traffic consumption required for video loading.

[0136] Optionally, S1110 further includes:

[0137] The file segment of the next time period at the second resolution is sent to the client, wherein the file sub-segment sent to the client has the first resolution, and the second resolution is higher than the first resolution.

[0138] For example, when the client plays a 7-8S file segment, the 7-8S file segment sent by the server is a low-resolution file segment, and when the server sends an 8-10S file segment to the client, it sends a high-resolution file segment.

[0139] In such Fig. 9 In the exemplary embodiment shown, the present application also provides a video stream loading device 900. The video stream loading device may include:

[0140] The data acquisition unit 910 is used to acquire the original video stream.

[0141] It can be understood that S102 can be executed by the data acquisition unit 910 .

[0142] The data extraction unit 920 is used to extract the audio file and the video file of the original video stream.

[0143] It can be understood that S104 can be executed by the data extraction unit 920 .

[0144] The data segmentation unit 930 is used to segment the video file and the audio file according to a preset time period to generate a plurality of file segments, wherein each file segment includes time information.

[0145] It can be understood that S106 can be executed by the data segmentation unit 930.

[0146] The signal receiving unit 940 is used to receive an access request based on the original video stream sent by the client, wherein the access request carries a play mode and a timestamp.

[0147] It can be understood that S108 can be executed by the signal receiving unit 940 .

[0148] The signal sending unit 950 is used to send the file segments corresponding to the audio file and / or the video file to the client according to the play mode and the timestamp.

[0149] It can be understood that S110 can be executed by the signal sending unit 950 .

[0150] It can be understood that each step in the above implementation corresponds to a virtual device, which will not be described in detail here.

[0151] Based on the above implementation, please refer to Fig.10 The present application also provides another video stream loading method, which is applied to a client of a communication system, wherein the communication system also includes a server, and the server is connected to the client in communication; the method includes:

[0152] S202, in response to the user operation, generating an access request carrying a play mode and a timestamp.

[0153] S204, determining the original video stream from the server according to the access request, and pulling the file segments corresponding to the audio file and / or video file in the original video stream according to the playback mode and timestamp; wherein the file segments are generated by the server by segmenting the audio file and the video file according to a preset time period.

[0154] S206, when the audio file and the video file are pulled at the same time, the audio file and the video file are synthesized.

[0155] The operation request refers to the user operating the client. For example, the user can control the client through the touch screen or mouse, such as clicking on a video, or dragging the progress bar, fast forward, rewind, etc. of the currently watched video.

[0156] After receiving the corresponding instructions, the client will pull the corresponding file fragments from the server and play them. When the user is selecting a mode, when the playback mode is a normal mode that is neither silent nor still, the client will pull the file fragments corresponding to the audio file and the video file, as well as the file fragments of the synthesized audio file and video file; when the playback mode is silent mode, the client will pull the file fragments corresponding to the video file; when the playback mode is still mode, the client will pull the file fragments corresponding to the audio file.

[0157] Among them, S204 includes:

[0158] S2041, pull the file fragment of the current period and the file fragment of the next period corresponding to the timestamp.

[0159] That is, when the client pulls data, it will preload file segments to ensure smooth video playback.

[0160] Optionally, in response to a user's skip operation, the client determines a target timestamp corresponding to a skip operation such as dragging a progress bar or fast forwarding, and pulls a corresponding file segment according to the timestamp.

[0161] Since the above implementation has already elaborated on the video stream loading method in detail, this application will not repeat it again.

[0162] Based on the above implementation, please refer to Fig.11 The present application also provides a video stream loading method, which is applied to a communication system, and the method includes:

[0163] S302: The server obtains the original video stream.

[0164] S304, the server extracts the audio file and the video file of the original video stream.

[0165] S306, the server divides the video file and the audio file into preset time periods to generate multiple file segments, wherein each file segment includes time information.

[0166] S308: The client responds to the user operation and generates an access request carrying a play mode and a timestamp.

[0167] S310, when the playback mode is the normal mode, the client pulls the file segments corresponding to the audio file and the video file, and synthesizes the file segments of the audio file and the video file.

[0168] S312: When the playback mode is the silent mode, the client pulls the file segment corresponding to the video file.

[0169] S314: When the playback mode is the silent mode, the client pulls the file segment corresponding to the audio file.

[0170] S316, the client determines whether a jump instruction is received, if yes, executes S318, if not, executes S320.

[0171] S318, the client pulls the corresponding file segment according to the timestamp.

[0172] S320: The client sends the corresponding file segment and / or the file segment of the next time period pulled up according to the timestamp to the client.

[0173] Since the above embodiment has described in detail the specific implementation of the video stream loading method, it will not be repeated here.

[0174] In an embodiment of the present invention, an electronic device is provided, which includes: a processor and a memory storing a computer program, wherein the processor is configured to implement any method according to an embodiment of the present invention when running the computer program. In addition, a video stream loading device according to an embodiment of the present invention can also be provided.

[0175] In a preferred embodiment of the present invention, the electronic device may be a server device or a client device.

[0176] In a preferred embodiment, the client device may include a mobile terminal, preferably a mobile phone. As an exemplary implementation scheme only, Fig.12 1 shows a schematic diagram of the hardware structure of a specific embodiment of an electronic device, such as a mobile terminal device, for example, a mobile terminal 1200; and Fig.13 and Fig.14 A schematic diagram of the system structure of a specific embodiment of an electronic device, such as a mobile terminal, is shown.

[0177] In the illustrated embodiment, the mobile terminal 1200 may include a processor 1201, an external memory interface 1212, an internal memory 1210, a universal serial bus (USB) interface 1213, a charging management module 1214, a power management module 1215, a battery 1216, a mobile communication module 1240, a wireless communication module 1242, antennas 1239 and 1241, an audio module 1234, a speaker 1235, a receiver 1236, a microphone 1237, an earphone interface 1238, a button 1209, a motor 1208, an indicator 1207, a subscriber identification module (SIM) card interface 1211, a display screen 1205, a camera device 1206, and a sensor module 1220, etc.

[0178] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the mobile terminal 1200. In other embodiments of the present application, the mobile terminal 1200 may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0179] In some embodiments, the processor 1201 may include one or more processing units. In some embodiments, the processor 1201 may include one of the following or a combination of at least two of the following: an application processor (AP), a modem processor, a baseband processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, a neural network processor (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0180] The controller may be the nerve center and command center of the mobile terminal 1200. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0181] A memory may also be provided in the processor for storing instructions and data. In some embodiments, the memory in the processor is a cache memory. The memory may store instructions or data that the processor has just used or circulated. If the processor needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 1201, and thus improves the efficiency of the system.

[0182] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn.

[0183] The GPU is a microprocessor for image processing that connects the display screen and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor may include one or more GPUs that execute program instructions to generate or change display information.

[0184] A digital signal processor (ISP) is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals.

[0185] In some embodiments, the processor 1201 may include one or more interfaces. The interface may include an integrated circuit (I2C) interface, an integrated circuit internal sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver and transmitter (UART) interface, a mobile industry processor interface (MIPI), a general purpose input and output (GPIO) interface, a subscriber identity module (SIM) interface, a universal serial bus (USB) interface, etc.

[0186] It is understandable that the interface connection relationship between the modules illustrated in the embodiments of the present application is only a schematic illustration and does not constitute a structural limitation on the mobile terminal. In other embodiments of the present application, the mobile terminal may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0187] The wireless communication function of the mobile terminal 1200 can be implemented through antennas 1239 and 1241, a mobile communication module 1240, a wireless communication module 1242, a modem processor or a baseband processor, etc.

[0188] Video codecs are used to compress or decompress digital video.

[0189] The mobile terminal 1200 can implement audio functions such as music playing and recording through an audio module, a speaker, a receiver, a microphone, an earphone interface, and an application processor.

[0190] The audio module is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals.

[0191] A microphone is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can put their mouth close to the microphone and speak, inputting the sound signal into the microphone.

[0192] The sensor module 1220 may include one or more of the following sensors:

[0193] The pressure sensor 1223 is configured to sense the pressure signal and convert the pressure signal into an electrical signal.

[0194] The air pressure sensor 1224 is used to measure air pressure.

[0195] The magnetic sensor 1225 includes a Hall sensor.

[0196] The gyro sensor 1227 may be used to determine the motion posture of the mobile terminal 1200 .

[0197] The acceleration sensor 1228 can detect the magnitude of acceleration of the mobile terminal 1200 in various directions.

[0198] The distance sensor 1229 may be configured to measure distance.

[0199] The proximity light sensor 1221 may include, for example, a light emitting diode (LED) and a light detector such as a photodiode.

[0200] The ambient light sensor 1222 is used to sense the brightness of ambient light.

[0201] The fingerprint sensor 1231 may be configured to collect fingerprints.

[0202] The touch sensor 1232 may be disposed on a display screen, and the touch sensor and the display screen form a touch screen, also known as a "touch screen". The touch sensor is used to detect a touch operation applied thereto or thereabout. The touch sensor may transmit the detected touch operation to an application processor to determine the type of touch event, such as single click, double click, long press, tap, directional slide, gather, etc.

[0203] The bone conduction sensor 1233 can acquire a vibration signal.

[0204] The software operating system of an electronic device (computer), such as a mobile terminal, may adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture.

[0205] The embodiments shown in this article take iOS and Android operating system platforms of layered architecture as examples to illustrate the software structure of the mobile terminal. However, it is conceivable that the embodiments of this article can be implemented in different software operating systems.

[0206] exist Fig.13 In the illustrated embodiment, the solution of the embodiment of the present invention can adopt the iOS operating system. The iOS operating system adopts a four-layer architecture, which includes, from top to bottom, a touchable layer (Cocoa Touch layer) 1310, a media layer (Medialayer) 1320, a core service layer (Core Services layer) 1330, and a core operating system layer (Core OSlayer) 1340. The touch layer 1310 provides various commonly used frameworks for application development and most of the frameworks are related to the interface, which is responsible for the user's touch interaction operations on the iOS device. The media layer provides audio-visual technologies in the application, such as graphics and images, sound technology, video, and audio and video transmission-related frameworks. The core service layer provides the basic system services required by the application. The core operating system layer contains most of the low-level functions close to the hardware.

[0207] In the embodiment of the present invention, UIKit is a user interface framework of the touchable layer 1310, which can be supported by many image frameworks in the media layer 1320, including but not limited to: Fig.13 The core library (Core Graphics), core animation (Core Animation), open library ES (OPEN GL ES), core image (Core Image), image IO (ImageIO), and library package (GLKit) shown in.

[0208] Fig.14 14 is a schematic diagram of the structure of the Android operating system. The solution of the embodiment of the present invention can adopt the Android operating system. The layered architecture divides the software into several layers, and the layers communicate through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, namely, the application layer 1410, the application framework layer 1420, the Android runtime (Runtime) and system library 1430, and the kernel layer 1440.

[0209] The application layer 1410 may include a series of application packages.

[0210] The application framework layer 1420 provides an application programming interface (API) and a programming framework for the applications of the application layer. The application framework layer includes some predefined functions.

[0211] A window manager is used to manage window programs.

[0212] Content providers are used to store and retrieve data and make it accessible to applications.

[0213] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0214] The phone manager is used to provide the communication function of the mobile terminal.

[0215] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0216] The notification manager enables applications to display notification information in the status bar, which can be used to convey informational messages and disappear automatically after a short stay without user interaction.

[0217] Android Runtime includes core libraries and virtual machines. Android Runtime is responsible for scheduling and management of the Android system. The core library consists of two parts: one is the function to be called by the Java language, and the other is the Android core library. The application layer and the framework layer run in the virtual machine.

[0218] The system library can include multiple functional modules. The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications.

[0219] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0220] The kernel layer 1440 is a layer between hardware and software. The kernel layer may include display driver, camera driver, audio interface, sensor driver, power management and GPS interface. In some embodiments of the present invention, the display may call the display driver.

[0221] The systems, devices, modules or units described in the above embodiments may be implemented by an electronic device (computer) or its associated components, preferably by a mobile terminal. The mobile terminal may be, for example, a smart phone, a laptop computer, a vehicle-mounted human-computer interaction device, a personal digital assistant, a media player, a navigation device, a game console, a tablet computer, a wearable device or a combination thereof.

[0222] Although not shown, in some embodiments, a storage medium is further provided, storing a computer program, which is configured to execute any method of the embodiments of the present invention when executed.

[0223] The storage media in the embodiments of the present invention include permanent and non-permanent, removable and non-removable items that can be implemented by any method or technology to store information. Examples of storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0224] Those skilled in the art should understand that the embodiments of the present specification can be provided as methods, systems or computer program products. Therefore, those skilled in the art can imagine that the implementation of the functional modules / units or controllers and related method steps described in the above embodiments can be implemented in software, hardware or a combination of software / hardware.

[0225] Unless explicitly stated, the actions or steps of the methods, programs, and embodiments of the present invention do not have to be performed in a specific order and can still achieve the desired results. In some implementations, multitasking and parallel processing are also possible or may be advantageous.

[0226] In this article, multiple embodiments of the present invention are described, but for the sake of brevity, the description of each embodiment is not exhaustive, and the same or similar features or parts between the embodiments may be omitted. In this article, "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" are meant to be applicable to at least one embodiment or example according to the present invention, but not all embodiments. The above terms do not necessarily mean to refer to the same embodiment or example. In the absence of contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples.

[0227] The exemplary systems and methods of the present invention have been specifically shown and described with reference to the above embodiments, which are merely examples of the best modes for implementing the present systems and methods. It will be appreciated by those skilled in the art that various changes may be made to the embodiments of the systems and methods described herein when implementing the present systems and / or methods without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. A video stream loading method, It is characterized in that The method is applied to a server of a communication system, the communication system further comprising a client, the server being in communication connection with the client; the method comprises: Get the original video stream; Extracting the audio file and the video file of the original video stream; The video file and the audio file are segmented according to preset time periods to generate a plurality of file segments, wherein each file segment includes time information; Receiving an access request based on the original video stream sent by the client, wherein the access request carries a play mode and a timestamp, and the timestamp is used to determine a specific loaded time segment; Sending the file segments corresponding to the audio file and / or the video file to the client according to the play mode and the timestamp includes: When a jump instruction is received, determining a current time period corresponding to the timestamp, and dividing the file segment of the current time period based on the timestamp to obtain at least two file sub-segments, wherein each file sub-segment includes time information; Based on the time information of each of the file sub-segments, the file sub-segment located after the timestamp is sent to the client, and the file segment of the next time period of the second resolution is sent to the client, wherein the file sub-segment sent to the client has a first resolution, and the second resolution is higher than the first resolution.

2. The video stream loading method according to claim 1, It is characterized in that The sending of the file segments corresponding to the audio file and / or the video file to the client according to the play mode and the timestamp further includes: When the playback mode is a normal mode without mute or static screen, the file segments corresponding to the audio file and the video file are sent to the client; When the playback mode is a silent mode, sending a file segment corresponding to the video file to the client; When the playing mode is the silent mode, the file segment corresponding to the audio file is sent to the client.

3. The video stream loading method according to claim 1 or 2, It is characterized in that Sending the file segments corresponding to the audio file and / or the video file to the client according to the play mode and the timestamp also includes: When no jump instruction is received, the file segment of the current time period and / or the file segment of the next time period corresponding to the timestamp are sent to the client.

4. The video stream loading method according to claim 3, It is characterized in that The sending of the file segments corresponding to the audio file and / or the video file to the client according to the play mode and the timestamp further includes: When a jump broadcast instruction is received, determining a current time period corresponding to the timestamp; The file segments of the current period are sent to the client.

5. The video stream loading method according to claim 4, It is characterized in that The sending of the file segments of the current period to the client further includes: Sending the file segment of the current time period at the first resolution to the client; The sending of the file segments corresponding to the audio file and / or the video file to the client according to the play mode and the timestamp further includes: Sending a file segment of a next time period at a second resolution to the client; wherein the second resolution is higher than the first resolution.

6. A video stream loading method, It is characterized in that The method is applied to a client of a communication system, wherein the communication system further comprises a server, and the server is in communication connection with the client; the method comprises: In response to a user operation, an access request is generated that carries a play mode and a timestamp, where the timestamp is used to determine a specific time segment of loading; Determine the original video stream from the server according to the access request, and pull the file segments corresponding to the audio file and / or the video file in the original video stream according to the play mode and the timestamp, wherein the file segments are generated by the server by segmenting the audio file and the video file according to a preset time period and each file segment includes time information, wherein when a jump instruction is generated, the current time period corresponding to the timestamp is determined, and a file sub-segment of the current time period and a file segment of the next time period are pulled, wherein the file sub-segment of the current time period is generated by segmenting the file segment of the current time period based on the timestamp and is located after the timestamp, the file sub-segment has a first resolution, and the file segment of the next time period has a second resolution, and the second resolution is higher than the first resolution; When the file sub-segment or file segment corresponding to the audio file and the file sub-segment or file segment corresponding to the video file are pulled simultaneously, the file segment corresponding to the audio file and the file segment corresponding to the video file are synthesized.

7. The video stream loading method according to claim 6, It is characterized in that Determining the original video stream from the server according to the access request, and pulling the file segments corresponding to the audio file and / or the video file in the original video stream according to the play mode and the timestamp, includes: When the playback mode is a normal mode that is neither silent nor silent, the file segments corresponding to the audio file and the video file are pulled, and the file segments of the audio file and the video file are synthesized; When the playback mode is the silent mode, pulling a file segment corresponding to the video file; When the playing mode is the silent mode, a file segment corresponding to the audio file is pulled.

8. The video stream loading method according to claim 6, It is characterized in that Determining the original video stream from the server according to the access request, and pulling the file segments corresponding to the audio file and / or the video file in the original video stream according to the play mode and the timestamp, including: Pull the file fragment of the current time period and the file fragment of the next time period corresponding to the timestamp.

9. The video stream loading method according to claim 6, It is characterized in that Determining the original video stream from the server according to the access request, and pulling the file fragments corresponding to the audio file and / or the video file in the original video stream, further comprising: When receiving a jump operation from the user, obtaining a target timestamp corresponding to the jump operation; Pull the corresponding file segment according to the target timestamp.

10. An electronic device, It is characterized in that include: A processor and a memory storing a computer program, wherein the processor is configured to execute the method according to any one of claims 1 to 9 when running the computer program.

11. A storage medium, It is characterized in that The storage medium stores a computer program, and the computer program is configured to execute the method according to any one of claims 1 to 9 when executed.

Images