Playing processing method, device and equipment, and storage medium

Abstract

The present disclosure provides a playing processing method and device, equipment and storage medium. It relates to the field of artificial intelligence, specifically to cloud computing and network playing technology, and can be applied in the intelligent cloud scenario. The specific implementation scheme is: in response to detecting that a preset trigger event occurs, a first target playing timestamp is determined; a target frame chasing mode is determined from candidate frame chasing modes; a playing starting frame is determined based on the target frame chasing mode; and playing of the playing starting frame is performed based on the first target playing timestamp. According to the scheme of the present disclosure, the timeliness of playing can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence, specifically cloud computing and network playback technology, and can be applied in intelligent cloud scenarios. Background Technology

[0002] With the rapid development of the internet, online streaming has become increasingly popular. However, during live broadcasts or playback, issues such as playback delays or buffering are common. Therefore, the timeliness of playback still needs improvement. Summary of the Invention

[0003] This disclosure provides a playback processing method, apparatus, device, and storage medium.

[0004] According to a first aspect of this disclosure, a playback processing method is provided, comprising:

[0005] In response to the detection of a preset trigger event, the first target playback timestamp is determined;

[0006] Determine the target frame tracking mode from the candidate frame tracking modes;

[0007] The starting frame for playback is determined based on the target frame tracking mode;

[0008] Playback of the starting frame is performed based on the first target playback timestamp.

[0009] According to a second aspect of this disclosure, a playback processing apparatus is provided, comprising:

[0010] The first determining module is used to determine the first target playback timestamp in response to the detection of a preset trigger event;

[0011] The second determining module is used to determine the target frame tracking mode from the candidate frame tracking modes;

[0012] The third determining module is used to determine the playback start frame based on the target frame tracking mode;

[0013] The playback control module is used to execute playback of the playback start frame based on the first target playback timestamp.

[0014] According to a third aspect of this disclosure, an electronic device is provided, comprising:

[0015] At least one processor; and

[0016] The memory is communicatively connected to the at least one processor; wherein,

[0017] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method provided in the first aspect above.

[0018] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform the method provided in the first aspect above.

[0019] According to a fifth aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method provided in the first aspect described above.

[0020] According to the technical solution disclosed herein, the timeliness of playback can be improved.

[0021] The above overview is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features of this application will become readily apparent from the accompanying drawings and the following detailed description. Attached Figure Description

[0022] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments disclosed in this application and should not be construed as limiting the scope of this application.

[0023] Figure 1 This is a schematic flowchart of a playback processing method according to an embodiment of the present disclosure;

[0024] Figure 2 This is a schematic diagram illustrating the determination of the first target playback timestamp according to a first scenario of an embodiment of this disclosure;

[0025] Figure 3 This is a schematic diagram illustrating the determination of the first target playback timestamp in a second scenario according to an embodiment of this disclosure;

[0026] Figure 4 This is a schematic diagram illustrating the determination of the second target playback timestamp according to a first scenario of an embodiment of this disclosure;

[0027] Figure 5 This is a schematic diagram illustrating the determination of the second target playback timestamp in a second scenario according to an embodiment of this disclosure;

[0028] Figure 6 This is a schematic diagram of the playback processing apparatus according to an embodiment of the present disclosure;

[0029] Figure 7This is a schematic diagram of a playback processing scenario according to an embodiment of the present disclosure. Figure 1 ;

[0030] Figure 8 This is a schematic diagram of a playback processing scenario according to an embodiment of the present disclosure. Figure 2 ;

[0031] Figure 9 This is a block diagram of an electronic device used to implement the playback processing method of the embodiments of this disclosure. Detailed Implementation

[0032] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0033] The terms "first," "second," and "third," etc., used in the embodiments, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, such as including a series of steps or units. A method, system, product, or apparatus is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or apparatuses.

[0034] Before introducing the technical solutions of the embodiments of this disclosure, the technical terms that may be used in this disclosure will be further explained:

[0035] GOP stands for Group of Picture. In video coding sequences, GOP refers to the distance between two I-frames.

[0036] I-frame: Short for Intra Picture, it is usually the first frame of each Group of Pictures (GOP). It uses intra-frame compression, is moderately compressed, and serves as a reference point for random access. It can be decoded and displayed independently without relying on any other frame.

[0037] P-frame: Short for Predictive Frame, it's a type of coded image that compresses transmitted data by fully utilizing the temporal redundancy of previously encoded frames in the image sequence. It employs inter-frame prediction techniques for encoding.

[0038] B-frame: The full English name is Bi-directional interpolated prediction frame. A B-frame is based on the preceding I or P frame and the following P frame. It has a very high compression ratio, so decoding it consumes more Central Processing Unit (CPU) resources. B-frames are not reference frames and do not cause the spread of decoding errors.

[0039] PTS stands for Play Start Time.

[0040] This disclosure provides a playback processing method. Figure 1 This is a schematic flowchart of a playback processing method according to an embodiment of the present disclosure. This playback processing method can be applied to a playback processing device. The playback processing device is located in the player of an electronic device, which can be a web page player or a player application. The electronic device player includes, but is not limited to, fixed devices and / or mobile devices. For example, fixed devices include, but are not limited to, servers, which can be cloud servers or ordinary servers. For example, mobile devices include, but are not limited to, one or more of the following terminals: mobile phones, tablet computers, and in-vehicle terminals. In some possible implementations, the playback processing method can also be implemented by a processor calling computer-readable instructions stored in memory. Figure 1 As shown, the playback processing method includes:

[0041] S101: In response to detecting a preset trigger event, determine the first target playback timestamp;

[0042] S102: Determine the target frame tracking mode from the candidate frame tracking modes;

[0043] S103: Determine the starting frame for playback based on the target frame tracking mode;

[0044] S104: Execute playback of the starting frame based on the first target playback timestamp.

[0045] In this embodiment of the disclosure, the preset trigger event is a pre-set event that can trigger a specific playback timestamp of the first target. The preset trigger event includes, but is not limited to: starting playback, network failure during live streaming, and video stuttering during live streaming.

[0046] In this embodiment of the disclosure, the first target playback timestamp is the target timestamp referenced during frame tracking. The first target playback timestamp can be understood as the end time of this frame tracking process.

[0047] In this embodiment of the disclosure, the first target playback timestamp can be denoted as PTS1.

[0048] In this embodiment of the disclosure, the candidate frame tracking modes can be increased or decreased according to design requirements, and this disclosure does not limit the type and number of candidate frame tracking modes.

[0049] In this embodiment of the disclosure, the candidate frame tracking mode includes N frame tracking modes, where N is a positive integer greater than or equal to 1. The N frame tracking modes can be denoted as frame tracking mode 1, frame tracking mode 2, frame tracking mode 3, ..., frame tracking mode N.

[0050] For example, frame tracking mode 1 is: using playback speed adjustment to catch up. For example, 2x fast forward playback, 4x fast forward playback, Mx fast forward playback, etc., where M is a number greater than 1.

[0051] For example, frame-following mode 2 is: discard all frames before the most recent I-frame before PTS1, and start fast-forwarding playback from the current I-frame.

[0052] For another example, frame tracking mode 3 is: discard all frames before the most recent I-frame before PTS1, decode from the current I-frame, but do not display or play them until the current decoding time (which can be denoted as PTS_decode) = PTS1 and start rendering and playing.

[0053] In some implementations, determining the target frame tracking mode from the candidate frame tracking modes includes: randomly selecting a frame tracking mode from the candidate frame tracking modes as the target frame tracking mode.

[0054] In some implementations, determining the target frame tracking mode from candidate frame tracking modes includes: selecting a frame tracking mode suitable for the frame tracking scenario from the candidate frame tracking modes as the target frame tracking mode, based on the frame tracking scenario corresponding to a preset trigger event. This helps improve the frame tracking effect.

[0055] In this embodiment of the disclosure, the playback start frame is the frame at which rendering and playback begin.

[0056] Here, the specific details of how to determine the playback start frame based on the target frame tracking mode and how to execute playback of the playback start frame based on the first target playback timestamp will be described in subsequent embodiments and will not be repeated here.

[0057] The technical solution of this disclosure embodiment, in response to detecting the occurrence of a preset trigger event, determines a first target playback timestamp; determines a target frame tracking mode from candidate frame tracking modes; determines a playback start frame based on the target frame tracking mode; and executes playback of the playback start frame based on the first target playback timestamp. Thus, by first determining the first target playback timestamp and the playback start frame, and then executing playback of the playback start frame based on the first target playback timestamp, the frame tracking method can quickly catch up with the playback progress, thereby improving the timeliness of playback.

[0058] In some embodiments, S103 includes:

[0059] S103a: When the target frame tracking mode is the first frame tracking mode, the intra-coded frame (I-frame) closest to the first target playback timestamp is taken as the decoding start frame, and the intra-coded frame is determined as the playback start frame.

[0060] Correspondingly, S104 includes:

[0061] S104a: Starting from the playback start frame, play the parsed image frames at the first preset speed until the first target playback timestamp is reached, then resume playback at the default speed.

[0062] Here, the first frame-tracking mode can be: discard all frames before the most recent I-frame before PTS1, and start fast-forwarding playback from the current I-frame.

[0063] In this embodiment of the disclosure, the first preset playback speed is a speed greater than 1. Specifically, the first preset playback speed can be determined based on the time elapsed between the current time point and the first target playback timestamp. For example, when the time elapsed between the current time point and the first target playback timestamp is greater than a certain threshold, playback speed S1 can be selected; when the time elapsed between the current time point and the first target playback timestamp is less than or equal to a certain threshold, playback speed S2 can be selected, wherein the value of S2 is less than the value of S1.

[0064] For example, if two groups of GOPs are obtained, the first group of GOPs is IPPPBBPPPPPP and the second group of GOPs is IPPPPPPBBB, then the first group of GOPs is discarded, and decoding and rendering begin from the I-frame of the second group of GOPs, and the playback is fast-forwarded.

[0065] In this embodiment of the disclosure, the default speed is the normal speed of the player, such as 1x speed.

[0066] Thus, when the target frame tracking mode is the first frame tracking mode, the parsed image frames are played at the first preset speed starting from the playback start frame until the first target playback timestamp is reached, at which point the default speed playback is restored, which can achieve the effect of frame tracking and help improve the timeliness of playback.

[0067] In some embodiments, S103 includes:

[0068] S103b: When the target frame tracking mode is the second frame tracking mode, the intra-coded frame closest to the first target playback timestamp is taken as the decoding start frame, and decoding starts from the decoding start frame. The frame decoded at the first target playback timestamp is determined as the playback start frame.

[0069] Correspondingly, S104 includes:

[0070] S104b: Play the start frame at the time corresponding to the first target playback timestamp.

[0071] Here, the second frame-tracking mode can be: discard all frames before the most recent I-frame before PTS1, decode from the current I-frame, and do not display or play them until the current decoding time (which can be denoted as PTS_decode) = PTS1 and start rendering and playing.

[0072] For example, if two GOPs are obtained, the first GOP is IPPPBBPPPPPP and the second GOP is IPPPPPPBBB, then the first GOP is discarded, and decoding starts from the I-frame of the second GOP, but no rendering is performed until the current decoding time (which can be denoted as PTS_decode) = PTS1, at which point rendering and playback begin.

[0073] Here, at the time corresponding to the first target playback timestamp, the playback start frame is played, including: at the time corresponding to the first target playback timestamp, the playback start frame is played at the default speed.

[0074] Thus, when the target frame tracking mode is the second frame tracking mode, after determining the playback start frame, playing the playback start frame at the time corresponding to the first target playback timestamp can achieve the frame tracking effect, which helps to improve the timeliness of playback.

[0075] In some embodiments, S103 includes:

[0076] S103c: When the target frame tracking mode is the first frame tracking mode or the second frame tracking mode, in response to detecting that the type of the frame corresponding to the first target playback timestamp is an intra-coded frame, the frame corresponding to the first target playback timestamp is determined as the decoding start frame and the frame corresponding to the first target playback timestamp is determined as the playback start frame.

[0077] Correspondingly, S104 includes:

[0078] S104c: Play the start frame at the time corresponding to the first target playback timestamp.

[0079] Here, at the time corresponding to the first target playback timestamp, the playback start frame is played, including: at the time corresponding to the first target playback timestamp, the playback start frame is played at the default speed.

[0080] For example, if the target frame tracking mode is the first frame tracking mode or the second frame tracking mode, and the frame corresponding to the PTS1 position is an I-frame, the playback can be switched directly to this position without needing to increase the playback speed.

[0081] Thus, if the frame type corresponding to the first target playback timestamp is an intra-coded frame, directly switching to the frame corresponding to the first target playback timestamp for playback can quickly achieve the effect of frame tracking, which helps to improve the timeliness of playback.

[0082] In some embodiments, S103 includes:

[0083] S103d: When the target frame tracking mode is the third frame tracking mode, the first intra-coded frame is used as the decoding start frame, and the intra-coded frame is determined as the playback start frame.

[0084] Correspondingly, S104 includes:

[0085] S104d: Starting from the initial playback frame, render and play the parsed image frames at the second preset playback speed until the first target playback timestamp is reached, then restore the default playback speed.

[0086] Here, the third frame-tracking mode can be: using playback speed adjustment to catch up. For example, 2x fast-forward playback, 4x fast-forward playback, Mx fast-forward playback, etc., where M is a number greater than 1.

[0087] In this embodiment of the disclosure, the second preset playback speed is a speed greater than 1. Specifically, the second preset playback speed can be selected based on the time elapsed between the current time point and the first target playback timestamp. For example, when the time elapsed between the current time point and the first target playback timestamp is greater than a certain threshold, playback speed S3 can be selected; when the time elapsed between the current time point and the first target playback timestamp is less than or equal to a certain threshold, playback speed S4 can be selected, wherein the value of S3 is less than the value of S4.

[0088] Thus, when the target frame tracking mode is the third frame tracking mode, starting from the playback start frame, the parsed image frames are rendered and played at the second preset speed until the first target playback timestamp is reached, at which point the default speed playback is restored, which can achieve the effect of frame tracking and help improve the timeliness of playback.

[0089] In some embodiments, S101 includes:

[0090] S101a: Determine the playback scene corresponding to the preset trigger event;

[0091] S101b: Determine the first target playback timestamp based on the playback scenario.

[0092] In this embodiment of the disclosure, the playback scenarios include, but are not limited to, the start-up scenario and the live streaming scenario.

[0093] Here, the timeliness requirements for playback can vary depending on the playback scenario. For example, the timeliness requirement for starting a broadcast is lower than that for the live broadcast process.

[0094] This allows the determined first target playback timestamp to be adapted to the playback scenario, thereby helping to improve frame tracking and playback timeliness.

[0095] In some embodiments, S101b includes:

[0096] S101b1: When the playback scenario is the first scenario, obtain the first moment corresponding to the start of playback and the second moment corresponding to the completion of playback.

[0097] S101b2: Determine the first broadcast duration based on the first and second time points;

[0098] S101b3: Based on the first duration, determine the first target playback timestamp of the first scene.

[0099] In this embodiment of the disclosure, the first scenario can be a start-up scenario.

[0100] In some implementations, the initial broadcast duration Δt1 is determined based on a first time point (denoted as t1) and a second time point (denoted as t2). cost The first duration Δt1 cost The first target playback timestamp for the first scene is determined.

[0101] like Figure 2 As shown, after receiving the start command, the player determines that a preset trigger event has occurred, obtains the first time t1 corresponding to the start of playback and the second time t2 corresponding to the completion of playback; and determines the first target playback timestamp PTS1 = PTS_startTime + Δt1. cost Generally, in a startup scenario, PTS_startTime = 0, therefore, PTS1 = Δt1 cost .

[0102] It should be understood that Figure 2 The schematic diagrams shown are merely illustrative and not limiting, and are scalable; those skilled in the art can use them as a basis. Figure 2 Even with various obvious changes and / or substitutions to the examples, the resulting technical solutions still fall within the scope of this disclosure.

[0103] In this way, the first target playback timestamp in the first scene can be quickly determined, which helps to improve the frame tracking speed in the first scene.

[0104] In some embodiments, S101b includes:

[0105] S101b1': When the playback scene is the second scene, obtain the third moment corresponding to buffer start and the fourth moment corresponding to buffer completion;

[0106] S101b2': Determine the second duration of the buffer based on the third and fourth time points;

[0107] S101b3': Determine the first target playback timestamp of the second scene based on the occurrence time point and the second duration corresponding to the preset trigger event.

[0108] In this embodiment of the disclosure, the second scenario can be a live streaming scenario. For example, the live streaming scenario includes, but is not limited to, scenarios where the network is interrupted or where there is buffering during the live streaming process.

[0109] For example, during a live stream, the internet connection suddenly drops and then reconnects, such as... Figure 3 As shown, assuming the network outage occurs at time td, and buffering begins after reconnection at time t3, and completes at time t4, then the first target playback timestamp PTS1 = PTS_startTime + Δt2 cost Where, PTS_startTime = t3 - td, Δt2 cost = t4-t3.

[0110] In some implementations, the second duration Δt2 of the buffer is determined at the third time t3 and the fourth time t4. cost When the value is less than a certain threshold, the occurrence time (denoted as td) of the preset trigger event and the second duration Δt2 are used as the basis. cost Determine the first target playback timestamp for the second scene: PTS1 = PTS_startTime + Δt2 cost = (t3-td)+(t4-t3).

[0111] In some implementations, the second duration Δt2 of the buffer is determined at the third time t3 and the fourth time t4. cost If the time exceeds a certain threshold, ignore the occurrence time point (denoted as td) corresponding to the preset trigger event, and base the time on the second duration Δt2. cost Determine the first target playback timestamp for the second scene, PTS1 = Δt2 cost .

[0112] It should be understood that Figure 3 The schematic diagrams shown are merely illustrative and not limiting, and are scalable; those skilled in the art can use them as a basis. Figure 3 Even with various obvious changes and / or substitutions to the examples, the resulting technical solutions still fall within the scope of this disclosure.

[0113] In this way, the first target playback timestamp in the second scene can be quickly determined, which helps to improve the frame tracking speed in the second scene, thereby improving the timeliness of frame tracking.

[0114] In some embodiments, the playback processing method further includes:

[0115] Get the time elapsed from the occurrence of the preset trigger event to the moment corresponding to the first target playback timestamp;

[0116] If the duration exceeds a preset threshold, the second target playback timestamp is determined based on the duration.

[0117] The second playback start frame is determined based on the target frame tracking mode;

[0118] Playback of the second playback start frame is performed based on the second target playback timestamp.

[0119] In this embodiment of the disclosure, the preset threshold can be set or adjusted according to the frame tracking accuracy.

[0120] In this embodiment of the disclosure, if the first target playback timestamp is denoted as PTS1 and the second target playback timestamp is denoted as PTS2, and the time taken from the occurrence of the preset trigger event to the time corresponding to the first target playback timestamp is denoted as ΔT_decode, then PTS2 = PTS1 + ΔT_decode.

[0121] like Figure 4 As shown, after receiving the start-up command, the player determines that a preset trigger event has occurred and obtains the first time t1 corresponding to the start of playback and the second time t2 corresponding to the completion of playback; the first target playback timestamp PTS1 = PTS_startTime + Δt1 cost Generally, in a startup scenario, PTS_startTime = 0, PTS1 = Δt1 cost If the decoding time corresponding to PTS1 is ΔT_decode, then PTS2 = PTS1 + ΔT_decode. The second playback start frame is determined based on the target frame tracking mode; playback of the second playback start frame is performed based on PTS2.

[0122] like Figure 5 As shown, assuming the time of network disconnection or buffering is td, the time corresponding to buffering start is t3, and the time corresponding to buffering completion is t4, the first target playback timestamp PTS1 = PTS_startTime + Δt2 cost Where, PTS_startTime = t3 - td, Δt2 cost = t4 - t3. If the decoding time corresponding to PTS1 is ΔT_decode, then PTS2 = PTS1 + ΔT_decode.

[0123] In this way, the time taken by the first frame tracking can be further tracked a second time. The timeliness of the frame tracking can be further improved by the second frame tracking, so as to achieve the effect of keeping the player and the publishing end synchronized.

[0124] In some embodiments, for ΔT_decode, the decoding time corresponding to the IPB frame can be calculated during the start-up of the video to predict the decoding time required in advance.

[0125] In practical applications, for network buffering or buffering interruptions caused by network failures, the live video stream can be restarted, and the start time can be recalculated during restart.

[0126] This disclosure provides a playback processing device, such as... Figure 6 As shown, the playback processing device may include: a first determining module 601, configured to determine a first target playback timestamp in response to detecting a preset trigger event; a second determining module 602, configured to determine a target frame tracking mode from candidate frame tracking modes; a third determining module 603, configured to determine a playback start frame based on the target frame tracking mode; and a playback control module 604, configured to execute playback of the playback start frame based on the first target playback timestamp.

[0127] In some embodiments, the third determining module 603 includes: a first determining submodule, configured to, when the target frame tracking mode is the first frame tracking mode, take the intra-coded frame closest to the first target playback timestamp as the decoding start frame, and determine the intra-coded frame as the playback start frame. Correspondingly, the playback control module 604 includes: a first control submodule, configured to, starting from the playback start frame, play the parsed image frames at a first preset speed until the first target playback timestamp is reached, then resume playback at the default speed.

[0128] In some embodiments, the third determining module 603 further includes: a second determining submodule, configured to, when the target frame tracking mode is the second frame tracking mode, take the intra-coded frame closest to the first target playback timestamp as the decoding start frame, and start decoding from the decoding start frame, determining the frame decoded at the first target playback timestamp as the playback start frame. Correspondingly, the playback control module 604 further includes: a second control submodule, configured to play the playback start frame at the time corresponding to the first target playback timestamp.

[0129] In some embodiments, the playback processing apparatus may include: a fourth determining module 605 (not shown in the figure), configured to, when the target frame tracking mode is a first frame tracking mode or a second frame tracking mode, determine the frame corresponding to the first target playback timestamp as the decoding start frame and the frame corresponding to the first target playback timestamp as the playback start frame in response to detecting that the type of the frame corresponding to the first target playback timestamp is an intra-frame coded frame. The playback control module 604 includes: a third control submodule, configured to play the playback start frame at the time corresponding to the first target playback timestamp.

[0130] In some embodiments, the third determining module 603 includes a third determining submodule, configured to, when the target frame tracking mode is the third frame tracking mode, take the first intra-frame encoded frame as the decoding start frame and determine the intra-frame encoded frame as the playback start frame. Correspondingly, the playback control module 604 further includes a fourth control submodule, configured to, starting from the playback start frame, render and play the parsed image frames at a second preset speed until the first target playback timestamp is reached, and then restore the default speed playback.

[0131] In some embodiments, the first determining module 601 includes: a fourth determining submodule, configured to determine the playback scene corresponding to the preset trigger event; and a fifth determining submodule, configured to determine the first target playback timestamp based on the playback scene.

[0132] In some embodiments, the fifth determining submodule is configured to: when the playback scenario is the first scenario, obtain the first moment corresponding to the start of playback and the second moment corresponding to the completion of playback; determine the first duration of playback based on the first moment and the second moment; and determine the first target playback timestamp of the first scenario based on the first duration.

[0133] In some embodiments, the fifth determining submodule is further configured to: when the playback scenario is the second scenario, obtain the third moment corresponding to the buffer start and the fourth moment corresponding to the buffer completion; determine the second duration of buffering based on the third moment and the fourth moment; and determine the first target playback timestamp of the second scenario based on the occurrence time point corresponding to the preset trigger event and the second duration.

[0134] In some embodiments, the playback processing apparatus may further include: an acquisition module 606 (not shown in the figure), configured to acquire the time elapsed from the occurrence of a preset trigger event to the moment corresponding to the first target playback timestamp; and a fifth determination module 607 (not shown in the figure), configured to determine a second target playback timestamp based on the time elapsed if the time elapsed exceeds a preset threshold. Correspondingly, the second determination module 602 is further configured to determine a second playback start frame based on a target frame tracking mode; and the playback control module is further configured to execute playback of the second playback start frame based on the second target playback timestamp.

[0135] Those skilled in the art should understand that the functions of each processing module in the playback processing device of this disclosure embodiment can be understood with reference to the relevant description of the playback processing method described above. Each processing module in the playback processing device of this disclosure embodiment can be implemented by an analog circuit that implements the functions described in the embodiments of this disclosure, or it can be implemented by running software that performs the functions described in the embodiments of this disclosure on an electronic device.

[0136] The playback processing apparatus of this disclosure can improve the timeliness of playback.

[0137] This disclosure also provides a scenario illustration of playback processing. Figure 1 ,like Figure 7 As shown, an electronic device, such as a cloud server, receives a preset trigger event from a terminal; determines a first target playback timestamp and a target frame tracking mode for the terminal; based on the target frame tracking mode, determines the playback start frame for the terminal; and the terminal executes playback of the playback start frame based on the first target playback timestamp. This improves playback timeliness; fundamentally solves the problems of video stuttering and time delays in different scenarios; and enhances the smoothness of video playback, providing users with a comfortable and smooth viewing experience.

[0138] This disclosure also provides a scenario illustration of playback processing. Figure 2 ,like Figure 8 As shown, the terminal detects a preset trigger event, retrieves the live data stream from an electronic device such as a cloud server, and determines the first target playback timestamp and target frame tracking mode. Based on the target frame tracking mode, it determines the playback start frame. The terminal then executes playback of the playback start frame based on the first target playback timestamp. This improves playback timeliness, fundamentally solves the problems of video stuttering and time delays in different scenarios, and enhances the smoothness of video playback, providing users with a comfortable and smooth viewing experience.

[0139] This disclosure does not limit the number of terminals and electronic devices; in practical applications, it may include multiple terminals and multiple electronic devices.

[0140] It should be understood that Figure 7 and Figure 8 The scene diagrams shown are merely illustrative and not restrictive; those skilled in the art can interpret them based on... Figure 7 and Figure 8 Even with various obvious changes and / or substitutions to the examples, the resulting technical solutions still fall within the scope of this disclosure.

[0141] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0142] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0143] Figure 9 A schematic block diagram of an example electronic device 900 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0144] like Figure 9 As shown, device 900 includes a computing unit 901, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 902 or a computer program loaded from storage unit 908 into random access memory (RAM) 903. RAM 903 may also store various programs and data required for the operation of device 900. The computing unit 901, ROM 902, and RAM 903 are interconnected via bus 904. Input / output (I / O) interface 905 is also connected to bus 904.

[0145] Multiple components in device 900 are connected to I / O interface 905, including: input unit 906, such as keyboard, mouse, etc.; output unit 907, such as various types of monitors, speakers, etc.; storage unit 908, such as disk, optical disk, etc.; and communication unit 909, such as network card, modem, wireless transceiver, etc. Communication unit 909 allows device 900 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0146] The computing unit 901 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. The computing unit 901 performs the various methods and processes described above, such as playback processing methods. For example, in some embodiments, the playback processing method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on device 900 via ROM 902 and / or communication unit 909. When the computer program is loaded into RAM 903 and executed by the computing unit 901, one or more steps of the playback processing method described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform playback processing methods by any other suitable means (e.g., by means of firmware).

[0147] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0148] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0149] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory (EPROM), flash memory, optical fiber, compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0150] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tube (CRT) or liquid crystal display (LCD) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0151] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0152] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0153] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0154] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A playing processing method, comprising: determining a first target playing timestamp in response to detecting occurrence of a preset trigger event, wherein the first target playing timestamp is related to a time length of occurrence of the preset trigger event; determining a target frame chasing mode from candidate frame chasing modes; determining a playing start frame based on the target frame chasing mode; performing playing of the playing start frame based on the first target playing timestamp; wherein the determining of the playing start frame based on the target frame chasing mode comprises: in a case where the target frame chasing mode is a first frame chasing mode, taking an intra-coded frame closest to the first target playing timestamp as a decoding start frame, and determining the intra-coded frame as the playing start frame; the performing of the playing of the playing start frame based on the first target playing timestamp comprises: starting from the playing start frame, playing the parsed image frames at a first preset speed, and resuming default speed playing when the first target playing timestamp is reached; wherein the determining of the playing start frame based on the target frame chasing mode comprises: in a case where the target frame chasing mode is a second frame chasing mode, taking an intra-coded frame closest to the first target playing timestamp as a decoding start frame, and determining a frame decoded at the first target playing timestamp as the playing start frame; the performing of the playing of the playing start frame based on the first target playing timestamp comprises: playing the playing start frame at a moment corresponding to the first target playing timestamp.

2. The method of claim 1, further comprising: in response to detecting that a type of a frame corresponding to the first target playing timestamp is an intra-coded frame, taking the frame corresponding to the first target playing timestamp as the decoding start frame, and determining the frame corresponding to the first target playing timestamp as the playing start frame; the performing of the playing of the playing start frame based on the first target playing timestamp comprises: playing the playing start frame at a moment corresponding to the first target playing timestamp.

3. The method of claim 1, wherein, the determining of the playing start frame based on the target frame chasing mode comprises: in a case where the target frame chasing mode is a third frame chasing mode, taking a first intra-coded frame as a decoding start frame, and determining the intra-coded frame as the playing start frame; the performing of the playing of the playing start frame based on the first target playing timestamp comprises: starting from the playing start frame, rendering and playing the parsed image frames at a second preset speed, and resuming default speed playing when the first target playing timestamp is reached.

4. The method of claim 1, wherein, the determining of the first target playing timestamp in response to detecting occurrence of a preset trigger event comprises: determining a playing scene corresponding to the preset trigger event; determining the first target playing timestamp based on the playing scene.

5. The method of claim 4, wherein, the determining of the first target playing timestamp based on the playing scene comprises: in a case where the playing scene is a first scene, acquiring a first moment corresponding to start playing initiation and a second moment corresponding to start playing completion; determine a first time duration of playing based on the first time point and the second time point; determine the first target play time stamp of the first scene based on the first time duration.

6. The method of claim 4, wherein, The first target play time stamp is determined based on the play scene, including: In a case where the play scene is a second scene, a third time point corresponding to a buffer start and a fourth time point corresponding to a buffer completion are acquired; determine a second time duration of buffering based on the third time point and the fourth time point; determine the first target play time stamp of the second scene based on a time point of occurrence of the preset trigger event and the second time duration.

7. The method of claim 1, further comprising: acquire a time duration of consumption from occurrence of the preset trigger event to a time point corresponding to the first target play time stamp; in a case where the time duration of consumption is greater than a preset threshold, determine a second target play time stamp based on the time duration of consumption; determine a second play start frame based on the target frame chasing mode; perform playing of the second play start frame based on the second target play time stamp.

8. A playing processing apparatus, comprising: a first determination module configured to determine a first target play time stamp in response to detecting occurrence of a preset trigger event; a second determination module configured to determine a target frame chasing mode from candidate frame chasing modes; a third determination module configured to determine a play start frame based on the target frame chasing mode; a play control module configured to perform playing of the play start frame based on the first target play time stamp; The third determination module comprises: a first determination submodule configured to, in a case where the target frame chasing mode is a first frame chasing mode, take an intra-coded frame closest to the first target play time stamp as a decoding start frame, and determine the intra-coded frame as the play start frame; The play control module comprises: a first control submodule configured to, starting from the play start frame, play the parsed image frames at a first preset speed, and resume playing at a default speed when the first target play time stamp is reached; The third determination module comprises: a second determination submodule configured to, in a case where the target frame chasing mode is a second frame chasing mode, take an intra-coded frame closest to the first target play time stamp as a decoding start frame, and decode starting from the decoding start frame, and determine a frame decoded at the first target play time stamp as the play start frame; The play control module comprises: a second control submodule configured to play the play start frame at a time point corresponding to the first target play time stamp.

9. The apparatus of claim 8, further comprising: a fourth determination module configured to, in response to detecting that a type of a frame corresponding to the first target play time stamp is an intra-coded frame, take the frame corresponding to the first target play time stamp as the decoding start frame, and determine the frame corresponding to the first target play time stamp as the play start frame; The play control module comprises: a third control submodule configured to play the play start frame at a time point corresponding to the first target play time stamp.

10. The apparatus of claim 8, wherein, The third determining module comprises: The third determining sub-module is configured to, in a case where the target frame chasing mode is a third frame chasing mode, determine a first intra-coded frame as a decoding starting frame and determine the intra-coded frame as the playing starting frame; The playing control module comprises: The fourth control sub-module is configured to, starting from the playing starting frame, render and play the parsed image frames at a second preset speed until the first target playing timestamp is reached, and restore a default playing speed.

11. The apparatus of claim 8, wherein, The first determining module comprises: The fourth determining sub-module is configured to determine a playing scenario corresponding to the preset trigger event; The fifth determining sub-module is configured to determine the first target playing timestamp based on the playing scenario.

12. The apparatus of claim 11, wherein, The fifth determining sub-module is configured to: In a case where the playing scenario is a first scenario, acquire a first time corresponding to a playing start and a second time corresponding to a playing completion; determine a first time duration of the playing based on the first time and the second time; determine the first target playing timestamp of the first scenario based on the first time duration.

13. The apparatus of claim 11, wherein, The fifth determining sub-module is configured to: In a case where the playing scenario is a second scenario, acquire a third time corresponding to a buffering start and a fourth time corresponding to a buffering completion; determine a second time duration of the buffering based on the third time and the fourth time; determine the first target playing timestamp of the second scenario based on a time point of occurrence of the preset trigger event and the second time duration.

14. The apparatus of claim 8, further comprising: an acquiring module configured to acquire a time duration from a time point of occurrence of the preset trigger event to a time point corresponding to the first target playing timestamp; a fifth determining module configured to, in a case where the time duration is greater than a preset threshold, determine a second target playing timestamp based on the time duration; The second determining module is further configured to determine a second playing starting frame based on the target frame chasing mode; The playing control module is further configured to perform playing of the second playing starting frame based on the second target playing timestamp.

15. An electronic device, comprising: at least one processor; and a memory connected to the at least one processor in communication; wherein The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

16. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to enable the computer to perform the method of any one of claims 1-7.

17. A computer program product comprising a computer program which, when executed by a processor, implements the method of any one of claims 1-7.

Images