Video recording method and device based on a multiple-speed playback environment and electronic equipment

Abstract

The application provides a video recording method and device based on a multiple-speed playing environment and an electronic device. The method comprises: obtaining a data packet related to recording in the process of accelerated playing of a player, obtaining a reference recording multiple and a target frame interval, wherein the reference recording multiple is a multiple used by the player for accelerated playing; determining an observed frame interval between the newly obtained data packets, and determining a latest time scaling multiple based on the ratio of the target frame interval to the observed frame interval and the reference recording multiple; for each data packet obtained since the start of recording, stretching the relative timestamp of the data packet based on the latest time scaling multiple determined based on the data packet obtained before the data packet, and encapsulating and writing the stretched timestamp and the data packet into a recording file. The video recording is performed by using the technical solution provided by the application, so that the playing speed during playback of the recording file is normal 1 multiple playing speed.

Description

Technical Field

[0001] This application relates to the field of multimedia data processing technology, and in particular to a video recording method, apparatus and electronic device based on a speed-up playback environment. Background Technology

[0002] With the widespread use of online streaming media, users often use playback speed adjustment to quickly browse audio and video content. During playback at increased speed, users may need to record the currently playing content.

[0003] In related technologies, video recording solutions directly acquire audio and video data packets for recording when playback is accelerated (e.g., 2x speed, 4x speed, etc.). This results in the recorded files also exhibiting a speed-up effect during playback, making it impossible to guarantee normal playback of the original single speed (1x speed) and failing to meet users' needs for saving normally playable videos. Summary of the Invention

[0004] In view of this, embodiments of this application provide a video recording method, apparatus, and electronic device based on a playback speed adjustment environment.

[0005] In a first aspect, embodiments of this application provide a video recording method based on a playback speed adjustment environment, the method comprising: During the process of accelerated video playback in the player, data packets related to recording are acquired, and the baseline recording speed and target frame interval are acquired. The baseline recording speed is the speed used by the player for accelerated playback. Determine the observation frame interval between the latest acquired data packets, and determine the latest time scaling factor based on the ratio of the target frame interval to the observation frame interval and the baseline recording speed; For each data packet acquired from the start of recording, the relative timestamp of the data packet is stretched based on the latest time scaling factor determined by the data packets acquired before the data packet, and the stretched timestamp is encapsulated with the data packet and written into the recording file.

[0006] In conjunction with the first aspect, in one possible implementation, determining the latest time scaling factor based on the ratio of the target frame interval to the observation frame interval and the baseline recording speed includes: The ratio of the target frame interval to the observation frame interval is determined as the candidate time scaling factor; Determine whether the deviation between the candidate time scaling factor and the baseline recording speed is less than a preset threshold; If the value is less than a preset threshold, the candidate time scaling factor is determined as the latest time scaling factor; otherwise, the baseline recording speed is determined as the latest time scaling factor.

[0007] In conjunction with the first aspect, in one possible implementation, determining the ratio of the target frame interval to the observation frame interval as the candidate temporal scaling factor includes: Determine whether the latest acquired data packet carries a valid duration field; If carried, the value of the duration field is used as the target frame interval, and the ratio of the value of the duration field to the observation frame interval is used as the candidate time scaling factor; If not carried, the corresponding target frame interval is calculated based on at least two of the decoder's actual frame rate, the stream's nominal frame rate, and the average frame rate, and the ratio of each target frame interval to the observed frame interval is used as the candidate multiplier. From the candidate scaling factors, the one with the smallest deviation from the reference recording speed is selected as the candidate time scaling factor.

[0008] In conjunction with the first aspect, in one possible implementation, the latest time scaling factor is determined based on the ratio of the target frame interval to the observation frame interval of the video stream data packets and the baseline recording speed; For each data packet acquired from the start of recording, the relative timestamp of the data packet is stretched based on the latest time scaling factor determined by data packets acquired before the data packet, including: For each video stream data packet acquired from the start of recording, the relative timestamp of the video stream data packet is stretched based on the latest time scaling factor determined by the video stream data packets acquired before the video stream data packet. For each audio stream data packet acquired from the start of recording, the latest time scaling factor used by the video stream data packet is reused to stretch the relative timestamp of the audio stream data packet so that the video stream and audio stream in the recorded file remain synchronized during playback.

[0009] In conjunction with the first aspect, in one possible implementation, the step of stretching the relative timestamp of each audio stream data packet acquired since the start of recording, using the latest time scaling factor used by the video stream data packet, includes: For each audio stream data packet acquired from the start of recording, a target video stream data packet whose original timestamp is closest to that of the audio stream data packet is determined, and the relative timestamp of the audio stream data packet is stretched based on the latest time scaling factor used by the target video stream data packet.

[0010] In conjunction with the first aspect, in one possible implementation, stretching the relative timestamp of the data packet includes: The relative timestamp is obtained by subtracting the reference start timestamp from the original timestamp of the data packet, wherein the reference start timestamp is the original timestamp of the first data packet written to the recording file from the start of recording. Multiply the relative timestamp by the latest time scaling factor to obtain the scaled timestamp; The scaled timestamp is converted from the source stream time base to the output container time base to obtain the stretched timestamp.

[0011] In conjunction with the first aspect, in one possible implementation, prior to the step of encapsulating the stretched timestamp and the data packet into the recording file, the method further includes: Check whether the stretched timestamp of the current data packet has rolled back compared to the stretched timestamp of the previous data packet; If a rollback occurs, the stretched timestamp of the current data packet will be corrected to the stretched timestamp of the previous data packet plus a preset increment.

[0012] In conjunction with the first aspect, in one possible implementation, the data packet is a copy of the data packet obtained from the demultiplexer output of the player via a bypass method, and the audio and video compressed data in the data packet copy maintains its original encoding format during the execution of the video recording method; and / or, the first data packet written to the recording file is: the first key data packet obtained after the start of recording, or the first data packet obtained at the start of recording.

[0013] Secondly, embodiments of this application provide a video recording device based on a playback speed adjustment environment, the device comprising: The acquisition module is used to acquire data packets related to recording during the process of accelerated video playback in the player, and to acquire the baseline recording speed and the target frame interval, wherein the baseline recording speed is the speed adopted by the player for accelerated playback; The determination module is used to determine the observation frame interval between the latest acquired data packets, and to determine the latest time scaling factor based on the ratio of the target frame interval to the observation frame interval and the baseline recording speed. The stretching module is used to stretch the relative timestamp of each data packet acquired from the start of recording, based on the latest time scaling factor determined by the data packets acquired before the data packet. The writing module is used to encapsulate the stretched timestamp and the data packet into a recording file.

[0014] Thirdly, embodiments of this application provide an electronic device including a processor, the processor being configured to invoke instructions to cause the electronic device to perform the steps of the video recording method based on a speed-up playback environment as described in any of the first aspects.

[0015] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the video recording method based on a speed-up playback environment as described in any of the first aspects.

[0016] This application provides a video recording method, apparatus, and electronic device based on a speed-up playback environment. Since the process of acquiring the observation frame interval and determining the latest time scaling factor can be continuously executed throughout the accelerated playback process, the observation frame interval is dynamically refreshed as new data packets arrive, and the latest time scaling factor is updated accordingly. Through this dynamic mechanism, the system can flexibly adjust the latest time scaling factor according to the fluctuations in the actual arrival interval of data packets, ensuring that the time scaling factor used for each data packet to be written during recording is the latest time scaling factor determined based on historical data packets acquired before that data packet. Furthermore, by performing timestamp stretching and data packet encapsulation and writing operations using this latest time scaling factor, the playback speed of the recorded file can be set to the normal 1x speed playback speed, and the playback duration of the recorded file is consistent with the duration of the recorded portion of the source video at single-speed playback. Therefore, this meets the user's actual need to quickly save complete content and watch it normally during accelerated playback. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating a video recording method based on a speed-up playback environment according to one embodiment; Figure 2 yes Figure 1 The flowchart shown is a schematic diagram of the process for determining the latest time scaling factor in step S102. Figure 3 yes Figure 2 The flowchart of step S201 is shown below; Figure 4 This is a schematic diagram of a video recording device based on a speed-up playback environment, according to one embodiment. Detailed Implementation

[0018] To make the technical solution and beneficial effects of this application more apparent and understandable, a detailed description is provided below by listing specific embodiments. The accompanying drawings are not necessarily drawn to scale, and local features may be enlarged or reduced to more clearly show the details of the local features; unless otherwise defined, the technical and scientific terms used herein have the same meanings as those in the technical field to which this application pertains.

[0019] The embodiments in this application are not exhaustive, but merely illustrative of some embodiments, and are not intended to limit the scope of protection of this disclosure. Unless otherwise specified, each step in a particular embodiment can be implemented as an independent embodiment, and the steps can be arbitrarily combined. For example, a solution after removing some steps in a particular embodiment can also be implemented as an independent embodiment, and the order of the steps in a particular embodiment can be arbitrarily interchanged. Furthermore, the optional implementation methods in a particular embodiment can be arbitrarily combined; moreover, the embodiments can be arbitrarily combined, for example, some or all steps of different embodiments can be arbitrarily combined, and a particular embodiment can be arbitrarily combined with the optional implementation methods of other embodiments.

[0020] In each embodiment of this application, unless otherwise specified or in case of logical conflict, the terminology and / or descriptions of the embodiments are consistent and can be referenced by each other. Technical features in different embodiments can be combined to form new embodiments based on their inherent logical relationships.

[0021] In the description of the embodiments of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, features defined with "first" and "second" may explicitly or implicitly include one or more of the stated features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0022] For ease of understanding, a brief explanation of some of the terms used in this application is provided.

[0023] PTS (Presentation Time Stamp) is used to indicate the specific point in time when an audio or video data packet is actually displayed or played on the player's screen or in the speaker.

[0024] DTS (Decoding Time Stamp) is used to indicate the specific time point when audio and video compressed data packets are sent to the decoder for decoding. Since video frames have bidirectional prediction frames (B-frames), the decoding order may be different from the display order, so it is necessary to distinguish between PTS and DTS.

[0025] AVPacket (Audio / Video Data Packet): In multimedia processing frameworks such as FFmpeg, a structure used to store a frame of compressed video data or a segment of audio data (i.e., the undecoded raw bitstream).

[0026] Remux (reuse / repackage) refers to the process of extracting audio and video streams from one container format (such as FLV) and directly packaging and writing them into another container format (such as MP4) without changing the underlying audio and video encoding format (without decoding and re-encoding).

[0027] A timebase is the smallest unit of time (e.g., 1 / 90000 of a second) used to represent time in a multimedia file. PTS and DTS are typically integer multiples of the timebase and need to be scaled proportionally when converting between different containers.

[0028] Profile / Level: A specification used in video coding standards (such as H.264 and H.265) to define coding complexity, maximum resolution, frame rate, and bitrate. Lowering or standardizing Profile / Level can help improve file compatibility on lower-end devices.

[0029] In scenarios where playback is accelerated on the player side, if a user plays a video at a faster speed (such as 2x, 4x, etc.) and directly obtains the audio and video data packets output by the player's underlying layer for recording, the recorded file will also exhibit a faster playback speed when it is saved because the player is pulling data packets at a faster pace than normal playback speed.

[0030] Therefore, embodiments of this application provide a video recording method based on a playback speed adjustment environment. For example... Figure 1 As shown, the method includes the following steps: S101: During the process of accelerated video playback in the player, acquire data packets related to recording, and acquire the baseline recording speed and target frame interval; S102: Determine the observation frame interval between the latest acquired data packets, and determine the latest time scaling factor based on the ratio of the target frame interval to the observation frame interval and the baseline recording speed. S103: For each data packet acquired since the start of recording, based on the latest time scaling factor determined from the data packets acquired before that data packet, the relative timestamp of the data packet is stretched, and the stretched timestamp and data packet are encapsulated and written into the recording file.

[0031] In this embodiment, the video recording method can be applied to terminal devices or player software that support playback speed adjustment and recording functions. Terminal devices include, but are not limited to, electronic devices with audio and video playback capabilities such as smartphones, tablets, laptops, desktop computers, smart TVs, and in-vehicle entertainment terminals.

[0032] The data packets related to recording refer to the data packets used for video recording obtained from the output of the player demultiplexer during the accelerated playback of the video. These data packets may include those that arrived before recording started, those that arrived at the start of recording, and those that arrived after recording started.

[0033] In step S101 above, during the accelerated video playback process, the system can acquire data packets output from the player's demultiplexer that have not yet been sent to the decoder, such as video stream data packets and audio stream data packets. These data packets are typically in a compressed encoding format. The video stream data packets and audio stream data packets are output interleaved in time. For example, the system can continuously acquire these data packets from the start of playback, or at least begin acquiring them within a preset time period before recording begins.

[0034] In some examples, the acquired data packets are obtained by bypassing and copying them in the player's data path. For example, a copy of the data packets is obtained from the player's demultiplexer output via a bypass method, wherein the original transmission path of the data packets is used for the player's normal decoding and rendering, and the bypass method is independent of the normal decoding and rendering.

[0035] In this way, the recording process does not interfere with the normal rendering of the player, and the data packet copy obtained by the recording module can be used for independent timestamp stretching without affecting the timestamp information carried by the original data packet during normal playback.

[0036] It is understandable that data packet acquisition is a continuous process. Data packets acquired before the start of recording are not written to the final recording file, but their arrival time information can provide historical data for the calculation of the observation frame interval; data packets acquired at or after the start of recording will be written to the recording file.

[0037] The target frame interval refers to the theoretical time interval between adjacent frames when played at normal 1x speed. For example, for a video with a frame rate of 25fps, the target frame interval is 40ms; for a video with a frame rate of 30fps, the target frame interval is approximately 33.3ms.

[0038] The target frame interval can be pre-obtained from the parsed bitstream information before recording begins. For example, before the recording function is activated during playback at double speed, the system can obtain the target frame interval based on the duration field and / or the frame rate metadata of the video stream carried in the acquired data packets.

[0039] The baseline recording speed is the playback speed used by the player for accelerated playback. For example, the user selects the accelerated playback speed (e.g., 2x, 4x, etc.) through the player interface before recording begins, and the player then accelerates playback according to that specified speed. This specified speed is recorded by the system as the baseline recording speed. It should be noted that the player's actual instantaneous playback speed may fluctuate slightly around the baseline recording speed due to factors such as network jitter and buffering status; the baseline recording speed is not directly used as the final time scaling factor, but rather as a reference benchmark used to dynamically infer the latest time scaling factor by combining the time information of the actually received data packets.

[0040] The baseline recording speed is acquired before the recording start command is triggered. In some implementations, the player stores the adopted speed value as a global state variable when accelerating playback begins, and the system reads it directly from this global state variable when needed (e.g., when inferring the time scaling factor). In other implementations, the system queries the player core for the currently effective acceleration speed value and records it when it first receives the data packet and begins observation. Regardless of the method used, the baseline recording speed is acquired without relying on the recording start function call, thus ensuring that the system can obtain the baseline recording speed before recording begins.

[0041] In some examples, taking a player implementation based on the FFmpeg multimedia framework as an example, the startup and initialization process of the recording function may involve the following specific operations. In response to a user's recording command triggered when the player is playing at a faster speed, the system first performs a state reset operation, such as calling the `ffp_reset_record_static_state` function (which resets the static timestamp recording variable within the recording module) to ensure that the timeline references for each recording are independent. Subsequently, in the player's core `ff_ffplay.c` file, the `ffp_start_record` function (the core function for starting the recording process) is called. Inside this function, the system can record the previously recorded user-specified playback speed value as `record_playback_rate`. Additionally, the system can trigger filter reconfiguration so that filters used for playback acceleration do not affect the acquisition of data in the recording bypass. In addition, when creating an output container instance (such as the AVFormatContext structure, a context object in FFmpeg used to manage multimedia file container formats), the system can perform parameter corrections for specific encoding formats (such as H.264 and HEVC). For example, parameters such as pixel format, color space, encoding profile, and encoding level can be forced to be adjusted to more common standard configurations to improve the compatibility of the final generated recording file on different playback devices.

[0042] In some examples, in step S102 above, the observation frame interval between the most recently acquired data packets can be the interval between the actual arrival times of the two most recently acquired adjacent data packets. The system can select video stream data packets as the basis for calculating the observation frame interval. For example, the system can calculate the arrival time difference between two adjacent video stream data packets by recording the system timestamp when each video stream data packet arrives, and use this as one observation frame interval.

[0043] After obtaining the observation frame interval, the system calculates the ratio between the target frame interval and the observation frame interval to obtain the candidate time scaling factor. The latest time scaling factor can be determined by comparing the candidate time scaling factor with the baseline recording speed.

[0044] In some examples, to smooth out instantaneous fluctuations caused by network jitter or thread scheduling, the system can perform statistical processing on the observation frame intervals between each pair of the most recent m (e.g., m=5) consecutive data packets. For example, the median, the average after removing extreme values, or the weighted average of these observation frame intervals can be taken as the currently valid observation frame interval. The size of the smoothing window and the statistical method can be flexibly set according to actual needs.

[0045] In some examples, if the system has acquired a sufficient number of data packets (e.g., at least two video stream data packets) before recording begins, the system can directly calculate the observation frame interval based on the arrival time of the latest acquired data packet, and determine the latest time scaling factor based on the ratio of the target frame interval to the observation frame interval and the baseline recording speed, so that the data packet writing process can be started when recording begins.

[0046] In some examples, when the number of data packets acquired at the start of recording is insufficient to calculate the observation frame interval (e.g., only one data packet), the following processing can be performed: the timestamp stretching and encapsulation writing operation is not started temporarily, and subsequent arriving data packets are cached until the number of data packets meets the calculation condition for the observation frame interval. Then, step S102 is executed based on the calculated observation frame interval to determine the latest time compression ratio; or, the timestamp stretching and encapsulation writing operation is performed using the baseline recording speed as the latest time compression ratio, and step S102 is executed to determine the latest time compression ratio after the number of data packets meets the calculation condition.

[0047] In step S103 above, the nth data acquired during accelerated video playback is used as the current data packet to be written. Based on the latest time scaling factor determined from the m consecutive data packets preceding the nth data packet, the relative timestamp of the nth data packet is stretched to restore the time interval between the (n-1)th data packet and the nth data packet to the time interval at normal playback speed (i.e., normal 1x playback speed). Here, n is greater than m, and m is greater than or equal to 2. Preferably, m is 5 to ensure that at least 5 consecutive historical data packets are available for calculating the observation frame interval when determining the latest time scaling factor.

[0048] It is understandable that by performing statistical processing (such as taking the average value) on the observation frame interval of every two adjacent data packets in the m consecutive data packets before the nth data packet, the statistical observation frame interval is obtained, and based on the ratio of the target frame interval to the statistical observation frame interval and the reference recording speed, the latest time scaling factor applied to the nth data packet can be determined.

[0049] Because the time scaling factor inference process is continuous, data packets written at different times may use different latest time scaling factors. Subsequently, the system performs stretching processing on the relative timestamp of the data packet. After stretching, the interval between the relative timestamps of the data packets is proportionally enlarged, compensating for timeline compression caused by accelerated playback. Then, the system calls the multimedia encapsulation interface to write the stretched timestamp along with the associated data packet into the recording file container. For example, in the FFmpeg multimedia framework, the system can call the `av_interleaved_write_frame` function to complete the encapsulation and writing. The output container format can be selected as needed, such as MP4, MOV, etc.

[0050] In the player's demultiplexed output, each video data packet typically carries a complete compressed video frame.

[0051] In this embodiment, the latest time scaling factor is used as a scaling factor (greater than 1) for stretching the relative timestamp of the data packet acquired at the current moment. For example, if the relative timestamp of the data packet acquired at the current moment is T, and the latest time scaling factor is S, then the timestamp of the data packet after stretching is T×S.

[0052] The relative timestamp of a data packet can be obtained based on the difference between the packet's original timestamp (e.g., PTS) and a reference start timestamp. The reference start timestamp is the original timestamp of the first data packet written to the recording file since the start of recording.

[0053] It should be noted that accelerated playback compresses the increment of the original timestamp of the data packet (i.e., the timestamp interval between adjacent data packets becomes smaller), and the relative timestamp of the data packet is also compressed accordingly. This embodiment stretches the relative timestamp of the data packet obtained at the current moment by multiplying it by the latest time scaling factor, so that the time interval between adjacent data packets can be restored to the time interval that should exist when playing at normal 1x speed, thereby restoring the compressed timeline to the normal playback state.

[0054] In this embodiment, since the process of acquiring the observation frame interval and determining the latest time scaling factor can be continuously executed throughout the accelerated playback process, the observation frame interval is dynamically refreshed as new data packets arrive, and the latest time scaling factor is also updated accordingly. Through this dynamic mechanism, the system can flexibly adjust the latest time scaling factor according to the fluctuations in the actual arrival interval of data packets, ensuring that the time scaling factor used by each data packet to be written during recording is the latest time scaling factor determined based on the historical data packets before that data packet was acquired. Furthermore, performing timestamp stretching and data packet encapsulation and writing operations with this latest time scaling factor allows the playback speed of the recorded file to be at normal 1x speed, and the playback duration of the recorded file is consistent with the duration of the recorded portion of the source video at single-speed playback. Therefore, this meets the user's actual need to quickly save complete content and watch it normally during accelerated playback.

[0055] In some embodiments, such as Figure 2 As shown, in step S102 above, determining the latest time scaling factor based on the ratio of the target frame interval to the observation frame interval and the baseline recording speed may include the following steps: S201: The ratio of the target frame interval to the observation frame interval is determined as the candidate time scaling factor; S202: Determine whether the deviation between the candidate time scaling factor and the baseline recording speed is less than a preset threshold. If yes, proceed to step S203; otherwise, proceed to step S204. S203: Determine the candidate time scaling factor as the latest time scaling factor; S204: Set the baseline recording speed to the latest time scaling factor.

[0056] In some examples, the deviation can be the relative deviation between the candidate time scaling factor and the reference recording speed. The relative deviation can be calculated by dividing the absolute value of the difference between the candidate scaling factor and the reference recording speed by the reference recording speed.

[0057] The preset threshold can be set according to actual application needs, for example, 10% to 15%. When the relative deviation is less than this threshold, it is considered that the candidate time scaling factor is basically consistent with the baseline recording speed, and the reliability of the candidate time scaling factor is high; otherwise, it is considered that there is a significant deviation between the two, and the reliability of the candidate time scaling factor is low.

[0058] In this embodiment, when the deviation between the candidate time scaling factor and the baseline recording speed is small, the candidate time scaling factor is used as the latest time scaling factor. This allows the system to stretch the timestamp of the current data packet to be written, adapting to minor fluctuations in network transmission as the actual arrival intervals of the acquired data packets change. Compared to directly using a fixed baseline recording speed for stretching, this method improves the accuracy of timeline reconstruction. When the deviation is large, it indicates errors in the bitstream metadata, abnormal network transmission, or significant interference with the observed data. In this case, the baseline recording speed is used as the latest time scaling factor to stretch the timestamp of the current data packet to be written. Since the baseline recording speed is derived from the playback speed specified by the user at the start of recording, it directly determines the physical operating frequency of the underlying reading thread and is relatively reliable. This improves the reliability of timestamp stretching of the current data packet under abnormal conditions and helps in processing non-standard bitstreams at high playback speeds.

[0059] In different video encoding formats (such as H.264), the frame rate-related metadata written during encapsulation often exhibits inconsistencies or inflated values. In scenarios involving accelerated recording speeds, relying solely on the single frame rate information carried in the bitstream to deduce the timeline compression rate can easily lead to significant discrepancies between the inferred result and the actual situation (for example, recording at 4x speed but the system misinterpreting it as 2x speed), ultimately resulting in abnormal playback speed of the recorded file.

[0060] To improve the reliability of time scaling factor inference, in some embodiments, such as Figure 3 As shown, in step S201 above, determining the ratio of the target frame interval to the observation frame interval as the candidate time scaling factor may include the following steps: S301: Determine whether the latest acquired data packet carries a valid duration field. If yes, proceed to step S302; otherwise, proceed to step S303. S302: Use the value of the duration field as the target frame interval, and use the ratio of the value of the duration field to the observation frame interval as the candidate time scaling factor; S303: Calculate the corresponding target frame interval based on at least two of the decoder's actual frame rate, the stream's nominal frame rate, and the average frame rate, and use the ratio of each target frame interval to the observed frame interval as a candidate multiplier. S304: Select the candidate time scaling factor with the smallest deviation from the reference recording speed from the candidate scaling factor.

[0061] For the most recently acquired data packet, the system determines whether it carries a valid duration field. A valid duration field refers to the field in the data packet structure used to indicate the frame display time span (e.g., the `duration` member of `AVPacket` in FFmpeg). If the data packet carries a valid duration field, the system uses the value of that duration field as the target frame interval and directly uses the ratio of that duration field value to the observed frame interval as the candidate time scaling factor. Since the duration field is directly derived from the encoder and reflects the theoretical frame interval between two adjacent frames during normal 1x speed playback, the candidate time scaling factor calculated in this way has high accuracy.

[0062] Due to missing or abnormal information during the encoding or encapsulation process, the latest acquired data packet may not carry a valid duration field. In this case, the system will obtain the target frame interval from multiple alternative sources. For example, the multiple alternative sources may include at least two of the following frame rates: the actual frame rate recorded in the player's decoding context, the nominal frame rate of the stream recorded in the encapsulation container header (i.e., the nominal frame rate of the video stream), and the average frame rate (estimated based on the number of received data packets and the cumulative time span). For each alternative source, the system calculates the corresponding target frame interval and the ratio of the target frame interval to the observed frame interval as alternative scaling factors, thus forming a set of alternative scaling factors. The one with the smallest deviation from the baseline recording speed is selected from this set as the final candidate time scaling factor.

[0063] In some specific implementations (such as FFmpeg-based players), the above process can be executed when the underlying reading thread sends the data packet to the ffp_record_file function in ff_ffplay.c. The system preferentially uses the duration field of the data packet itself as the target frame interval; if this field is unavailable, it obtains alternative values ​​from sources such as the decoder's actual frame rate, the stream's nominal frame rate, and the average frame rate, and selects the best candidate time scaling factor. If the deviation of all alternative scaling factors from the baseline recording speed exceeds the acceptable range, it indicates that there may be errors or missing metadata in the bitstream. In this case, the system no longer relies on the candidate scaling factor inferred from the metadata, but directly uses the baseline recording speed as the final time axis stretching factor.

[0064] This embodiment prioritizes the use of the duration field carried in the data packet, and obtains the target frame interval from multiple backup sources and selects the candidate scaling factor when the duration field is missing. This enables the system to obtain a relatively reliable time scaling factor inference result under different playback environments or bitstream quality, thus improving the applicability of the solution.

[0065] In some embodiments, the latest time scaling factor in step S102 is determined based on the ratio of the target frame interval to the observation frame interval of the video stream data packet and the baseline recording speed.

[0066] For example, determining the observation frame interval between the most recently acquired data packets may include: identifying video stream data packets from the acquired data packets starting from the acquisition of the first data packet; and determining the observation frame interval based on the arrival times of the at least two most recently arrived video stream data packets.

[0067] In step S103 above, for each data packet acquired from the start of recording, the relative timestamp of the data packet is stretched based on the latest time scaling factor determined before acquiring the data packet. This may include: For each video stream data packet acquired from the start of recording, the relative timestamp of the video stream data packet is stretched based on the latest time scaling factor determined by the video stream data packets acquired before the video stream data packet. For each audio stream data packet acquired from the start of recording, the latest time scaling factor used by the video stream data packet is reused, and the relative timestamp of the audio stream data packet is stretched to ensure that the video stream and audio stream in the recorded file remain synchronized during playback.

[0068] During accelerated video playback, the video stream data packets and audio stream data packets output by the demultiplexer typically arrive interleaved in time. Since the arrival interval of audio stream data packets is short and easily affected by floating-point arithmetic precision, this embodiment limits the observation object to video stream data packets. By observing continuously arriving video stream data packets and inferring the latest time scaling factor, a more reliable scaling factor inference result can be obtained, which can more accurately reflect the true playback speed.

[0069] In this embodiment, when the player accelerates playback, the demultiplexer retrieves data packets from both the video and audio streams synchronously at the same acceleration speed, ensuring consistent timeline stretching ratios for both audio and video. For audio stream data packets, the system does not calculate or infer a separate time scaling factor. Instead, it forces the currently written audio stream to inherit the latest determined time scaling factor from the video stream and proportionally stretches the relative timestamp of the audio stream data packet. This proportional stretching involves directly multiplying the relative timestamp of the audio stream data packet by the latest time scaling factor. The relative timestamp of the audio stream data packet is obtained based on the difference between its original timestamp and a reference start timestamp, which is the original timestamp of the first data packet written to the recording file from the start of recording.

[0070] Through the above processing, both video and audio stream data packets are stretched along the time axis based on the same time scaling factor. Since the audio stream is forced to inherit the predetermined time scaling factor of the video stream, the stretching ratio of audio and video remains consistent. Therefore, it is not necessary to rely on the frame rate metadata of the audio stream itself for independent scaling factor inference, which helps to avoid the problem of audio and video desynchronization when playing back files recorded at multiple speeds.

[0071] In some embodiments, the step of stretching the relative timestamp of each audio stream data packet acquired since the start of recording, using the latest time scaling factor used by the video stream data packet, may include: For each audio stream data packet acquired from the start of recording, determine the target video stream data packet whose original timestamp is closest to that of the audio stream data packet, and stretch the relative timestamp of the audio stream data packet based on the latest time scaling factor used by the target video stream data packet.

[0072] For example, the system can record the original timestamp information of the arriving video stream data packets. For the currently pending audio stream data packet, its original timestamp is compared with the original timestamps of each arriving video stream data packet, and the video stream data packet with the smallest absolute value of the timestamp difference is selected as the target video stream data packet. If there are multiple cases where the absolute values ​​of the timestamp differences are the same, the video stream data packet that arrived first can be selected as the target video stream data packet.

[0073] Regarding the order of data packet writing, the system performs stretching processing and encapsulates the audio and video stream data packets into the recording file sequentially according to their original timestamps. That is, both audio and video stream data packets are written to the recording file in ascending order of their original timestamps to ensure that the timestamps of the data packets in the recording file monotonically increase.

[0074] In this embodiment, by aligning the stretching ratio of the audio stream data packet with the video stream data packet whose original timestamp is closest, and processing the audio and video data packets sequentially according to the original timestamp order, the stretching ratio of audio and video on the time axis is kept consistent in time sequence, avoiding cumulative synchronization deviation caused by misalignment of the stretching ratio or out-of-order writing, and further ensuring the audio and video synchronization effect when playing back the recorded file.

[0075] In some embodiments, the method further includes: when the audio stream data packet arrives before the video stream data packet and the latest time scaling factor is not determined, temporarily storing the audio stream data packet, and then reading the temporarily stored audio stream data packet and applying the latest time scaling factor to perform proportional stretching after obtaining the latest time scaling factor based on the video stream data packet.

[0076] During the actual data packet arrival process, the arrival time of video stream data packets and audio stream data packets may differ. If the system has not yet obtained a valid and up-to-date time scaling factor, for example, if the first arriving data packet is an audio stream data packet before the video stream data packet arrives, the system can temporarily store the audio stream data packet. Once the up-to-date time scaling factor is obtained based on the video stream data packet, the temporarily stored audio stream data packet can be read, scaled proportionally using that factor, and then written to the recording file.

[0077] If the system has obtained a valid and up-to-date time scaling factor through preheating observation before recording begins, the audio stream data packets can directly obtain the currently determined up-to-date time scaling factor for stretching upon arrival. For subsequent audio data packets arriving during the recording process, the system obtains the current determined up-to-date time scaling factor in real time, performs proportional stretching on its relative timestamp, and encapsulates the stretched timestamp along with the compressed data of the audio data packet into the recording file.

[0078] In some embodiments, stretching the relative timestamp of the data packet in step S104 above may include: Subtract the reference start timestamp from the original timestamp of the data packet to obtain the relative timestamp. The reference start timestamp is the original timestamp of the first data packet written to the recording file since the start of recording. Multiply the relative timestamp by the latest time scaling factor to obtain the scaled timestamp; The scaled timestamp is converted from the source stream time base to the output container time base to obtain the stretched timestamp.

[0079] In this embodiment, for the data packet to be written, the original timestamp is subtracted from the reference start timestamp to obtain the relative timestamp, which allows the timeline of the recorded file to start from zero, making it easier for the player to parse and play the recorded file normally from the beginning.

[0080] After multiplying the relative timestamp by the latest time scaling factor to obtain the scaled timestamp, the system can call a time base conversion function (such as av_rescale_q in FFmpeg) to convert the scaled timestamp from the source stream time base to the output container time base. The source stream time base is determined by the original bitstream, such as 1 / 90000 commonly used in MPEG-TS streams or 1 / 1000 based on milliseconds; the output container time base is determined by the encapsulation format, such as 1 / 12800 or 1 / 16000 commonly used in MP4 containers.

[0081] Thus, since the timestamp interval of the data packets is proportionally amplified according to the latest time scaling factor, the timeline compression effect caused by accelerated playback is accurately compensated.

[0082] In some embodiments, the method further includes: before writing the stretched timestamp and data packet encapsulation into a recording file, detecting whether the stretched timestamp of the current data packet has rolled back relative to the stretched timestamp of the previous data packet; if a rollback occurs, correcting the stretched timestamp of the current data packet to the stretched timestamp of the previous data packet plus a preset increment. The preset increment can be set according to the time base of the output container, for example, taking one time base unit.

[0083] This effectively compensates for the tiny timestamp rollback that may occur due to floating-point operations, ensuring that the output recording file meets the monotonically increasing requirement of the encapsulation format and improving the reliability of the recording file.

[0084] In some embodiments, the first data packet written to the recording file is: the first key data packet obtained after the recording starts, or the first data packet obtained at the start of the recording.

[0085] In this embodiment, the player is typically in accelerated playback mode for a period of time before recording begins. The system can use the data packets acquired between the start of playback and the start of recording to observe, calculate the observation frame interval, and determine the latest time scaling factor. Since there is sufficient time to complete the scaling factor inference during this warm-up period, the system already has a usable time scaling factor at the start of recording, which can be directly used for writing data packets at the start of recording. For example, if the first data packet acquired at the start of recording is used as the writing starting point, the screen content at the moment of recording start can be completely preserved. This strategy is suitable for scenarios with high requirements for recording immediacy and content integrity. If the first key data packet (usually an I-frame) after the start of recording is used as the writing starting point, since key frames can be decoded independently, the recorded file can be rendered normally from the beginning position, avoiding the black screen problem that may occur if the starting data packet is not a key frame. This strategy is suitable for scenarios with high requirements for recorded file compatibility and playback smoothness.

[0086] It should be noted that the key data packet typically refers to the keyframe data packet (i.e., I-frame) in the video stream. The first data packet can be a video data packet or an audio data packet. When the first data packet obtained at the start of recording is used as the starting point for writing, if the data packet is an audio data packet, the system can scale it proportionally based on a pre-determined scaling factor (e.g., obtained before recording starts), or temporarily store the audio data packet until the video data packet arrives and the latest time scaling factor is determined before processing. In the initial state where the latest time scaling factor has not yet been determined, the system can also choose to wait for the first video data packet to arrive before starting the writing process to ensure the accuracy of the scaling process.

[0087] In some embodiments, the data packet is a copy of the data packet obtained from the demultiplexer output of the player via a bypass method, and the audio and video compressed data in the data packet copy maintains its original encoding format during the execution of the video recording method.

[0088] The bypass method refers to establishing an independent copy path outside the player's normal decoding and rendering data path. Through this method, the system can obtain a copy of the data packets from the demultiplexer output without affecting the transmission and processing of the original data packets in the main path.

[0089] In this embodiment, since the entire recording process is performed on the data transmission path from the demultiplexer output (i.e., decapsulation) to the recording file encapsulation (i.e., recapsulation), the timestamp information of the data packets is processed in a lightweight manner, without the need to perform secondary decoding and re-encoding operations on the audio and video compressed data. By avoiding high-energy-consuming encoding and decoding processing, this method can effectively reduce the performance overhead and power consumption of the CPU and GPU when running on mobile or PC devices.

[0090] Next, the technical solution provided in the embodiments of this application will be further explained with reference to a specific example.

[0091] Suppose a user is watching an H.264 encoded network video stream on a mobile device. The actual physical frame rate of the video is 25fps (i.e., the interval between adjacent frames is approximately 40ms when playing at 1x speed). However, the nominal frame rate carried in the stream encapsulation is incorrectly marked as 50fps. The user sets the player to 4.0x speedup playback and triggers the recording function at some point.

[0092] In 4.0x speed-up playback mode, the player's underlying demultiplexer pulls data packets at a faster pace than normal, and the actual physical arrival interval of video data packets is compressed to about 10ms.

[0093] The following steps illustrate the processing procedure for this example.

[0094] S1: Infer the latest time scaling factor.

[0095] Before the user triggers the recording command, the player is already in 4.0x speedup playback mode, and the demultiplexer continuously outputs data packets. The system obtains copies of the data packets via bypass and records the arrival time of the video data packets. After accumulating at least two video data packets, the observation frame interval is calculated to be approximately 10ms.

[0096] Since the data packets do not carry a valid duration field, the system obtains the target frame interval from multiple alternative sources and calculates the alternative multiplier. Assume the decoder's actual frame rate is 25fps, the target frame interval is 40ms, and the alternative multiplier is 4.0; the stream's nominal frame rate is 50fps (error value), the target frame interval is 20ms, and the alternative multiplier is 2.0; the average frame rate is 25fps, the target frame interval is 40ms, and the alternative multiplier is 4.0.

[0097] The system compares the deviation of each candidate scaling factor with the baseline recording speed of 4.0. Candidate scaling factor 2.0 is discarded if its deviation exceeds a preset threshold; candidate scaling factor 4.0 is selected as the latest time scaling factor if its deviation is within an acceptable range. The baseline recording speed of 4.0 is obtained and recorded by the system from the player core when it receives a recording command.

[0098] S2: Perform timestamp stretching and wrapping write.

[0099] In response to a recording command, the system creates an output wrapper and determines a reference start timestamp. Recording begins with the first keyframe; prior to this, the system has already performed scaling inference using non-keyframe data packets, so the latest available time scaling is available when the write begins.

[0100] If the underlying thread obtains video data packets, the system determines it as a video stream, subtracts the reference start timestamp from its original timestamp to obtain a relative timestamp, and then multiplies it by the latest time scaling factor of 4.0 for stretching. If the underlying thread obtains audio data packets, the system determines it as an audio stream, skips the scaling factor inference, and forcibly inherits the latest time scaling factor of 4.0 from the video stream for proportional stretching, without requiring independent inference. The output bypass of the audio acceleration filter (e.g., atempo=4.0) is separated from the recording bypass to ensure that the recording obtains the undecoded raw data packets.

[0101] Taking three consecutive data packets in a video stream as an example, their original timestamps are 1000, 1010, and 1020 (in milliseconds). For each data packet to be written, the system subtracts the reference starting timestamp of 1000 from its original timestamp to obtain relative timestamps of 0, 10, and 20, and then multiplies them by the latest time scaling factor corresponding to the data packet being processed for stretching. For ease of description, this example assumes that the latest time scaling factor for each of the three data packets is 4.0 (in reality, it may fluctuate around 4.0), resulting in stretched timestamps of 0, 40, and 80. Next, the system calls a time base conversion function to convert the scaled timestamps from the source stream time base to the output container time base. After the verification timestamp monotonically increases, the compensated timestamps are encapsulated together with the compressed data of the data packets and written to the recording file.

[0102] After the above processing, the data packet timestamps, originally compressed to a 10ms interval, are restored to a 40ms interval, corresponding to the frame interval that should exist at 25fps during normal 1x playback. The stretched relative timestamps are converted to the time base scale corresponding to the output container using a time base conversion function. Before calling the encapsulation and write interface (e.g., av_interleaved_write_frame in FFmpeg), the system verifies whether the timestamp of the current data packet satisfies the monotonically increasing condition (e.g., 80>40) and the constraint that PTS≥DTS. After passing the verification, the compensated timestamps are encapsulated and written to the recording file together with the compressed data packet data.

[0103] When the user stops recording, the generated recording file, when played back at normal 1x speed, will have the same playback duration as the recorded content at normal playback speed. For example, if a user records 10 seconds of physical time at 4.0x speed, the video content in the recording file will be 40 seconds long at normal playback speed. When playing back this file, the video will play at normal 1x speed, and the audio and video will remain synchronized.

[0104] The various embodiments or implementation methods described in this specification are presented in a progressive manner. Each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other.

[0105] It is worth noting that, without contradiction, different embodiments can be arbitrarily combined. For example, some or all of the steps of different embodiments can be arbitrarily combined, and one embodiment can be arbitrarily combined with the optional implementations of other embodiments.

[0106] Figure 4 This is a schematic diagram illustrating the structure of a video recording device based on a speed-up playback environment, according to one embodiment. For example... Figure 4 As shown, the video recording device 100 based on a speed-up playback environment includes: The acquisition module 101 is used to acquire data packets related to recording during the process of accelerated video playback in the player, and to acquire the baseline recording speed and the target frame interval, wherein the baseline recording speed is the speed adopted by the player for accelerated playback. The determination module 102 is used to determine the observation frame interval between the latest acquired data packets, and to determine the latest time scaling factor based on the ratio of the target frame interval to the observation frame interval and the reference recording speed. The stretching module 103 is used to stretch the relative timestamp of each data packet acquired from the start of recording, based on the latest time scaling factor determined by the data packets acquired before the data packet. The writing module 104 is used to encapsulate the stretched timestamp and the data packet into a recording file.

[0107] In some embodiments, the determining module 102 includes: The first determining unit is used to determine the ratio of the target frame interval to the observation frame interval as the candidate time scaling factor; The judgment unit is used to determine whether the deviation between the candidate time scaling factor and the reference recording speed is less than a preset threshold. The second determining unit is configured to determine the candidate time scaling factor as the latest time scaling factor if the determination result of the determining unit is less than a preset threshold; otherwise, it determines the reference recording speed as the latest time scaling factor.

[0108] In some embodiments, the first determining unit is configured to: Determine whether the latest acquired data packet carries a valid duration field; If carried, the value of the duration field is used as the target frame interval, and the ratio of the value of the duration field to the observation frame interval is used as the candidate time scaling factor; If not carried, the corresponding target frame interval is calculated based on at least two of the decoder's actual frame rate, the stream's nominal frame rate, and the average frame rate, and the ratio of each target frame interval to the observed frame interval is used as the candidate multiplier. From the candidate scaling factors, the one with the smallest deviation from the reference recording speed is selected as the candidate time scaling factor.

[0109] In some embodiments, the latest time scaling factor is determined based on the ratio of the target frame interval to the observation frame interval of the video stream data packets and the baseline recording speed; The stretching module 103 is used for: For each video stream data packet acquired from the start of recording, the relative timestamp of the video stream data packet is stretched based on the latest time scaling factor determined by the video stream data packets acquired before the video stream data packet. For each audio stream data packet acquired from the start of recording, the latest time scaling factor used by the video stream data packet is reused to stretch the relative timestamp of the audio stream data packet so that the video stream and audio stream in the recorded file remain synchronized during playback.

[0110] In some embodiments, the stretching module 103 is used for: For each audio stream data packet acquired from the start of recording, a target video stream data packet whose original timestamp is closest to that of the audio stream data packet is determined, and the relative timestamp of the audio stream data packet is stretched based on the latest time scaling factor used by the target video stream data packet.

[0111] In some embodiments, the stretching module 103 is used for: The relative timestamp is obtained by subtracting the reference start timestamp from the original timestamp of the data packet, wherein the reference start timestamp is the original timestamp of the first data packet written to the recording file from the start of recording. Multiply the relative timestamp by the latest time scaling factor to obtain the scaled timestamp; The scaled timestamp is converted from the source stream time base to the output container time base to obtain the stretched timestamp.

[0112] In some embodiments, the stretching module 103 is further configured to: Before encapsulating the stretched timestamp with the data packet and writing it into the recording file, it is detected whether the stretched timestamp of the current data packet has rolled back relative to the stretched timestamp of the previous data packet. If a rollback occurs, the stretched timestamp of the current data packet will be corrected to the stretched timestamp of the previous data packet plus a preset increment.

[0113] In some embodiments, the data packet is a copy of the data packet obtained from the demultiplexer output of the player via a bypass method, and the audio and video compressed data in the data packet copy maintains its original encoding format during the execution of the video recording method; and / or, the first data packet written to the recording file is: the first key data packet obtained after the start of recording, or the first data packet obtained at the start of recording.

[0114] The video recording apparatus based on a speed-up playback environment provided in this application embodiment belongs to the same concept as the video recording method based on a speed-up playback environment provided in the above embodiments of this application. It can execute the video recording method based on a speed-up playback environment provided in any of the above embodiments of this application, and possesses the corresponding functional modules and beneficial effects for executing the video recording method based on a speed-up playback environment. Technical details not described in detail in this embodiment can be found in the specific processing content of the video recording method based on a speed-up playback environment provided in the above embodiments of this application, and will not be repeated here.

[0115] It should be understood that the various modules in a video recording device based on a speed-up playback environment can be implemented in the form of a processor calling software, or the various modules can be implemented in the form of hardware circuits. The functions of some or all modules can be realized through the design of the hardware circuits, which can be understood as one or more processors.

[0116] This application also provides an electronic device, including a processor, which is configured to invoke instructions to cause the electronic device to perform the steps of the video recording method based on a speed-up playback environment as provided in any of the foregoing embodiments.

[0117] This application also provides a computer-readable storage medium storing an executable program thereon, which, when executed by a processor, implements the steps of the video recording method based on a speed-up playback environment provided in any of the foregoing embodiments.

[0118] For ease of understanding, the following focuses on explaining the terminology used in this embodiment: In this application embodiment, the processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction read and execute capabilities, such as a Central Processing Unit (CPU), a microprocessor, a Graphics Processing Unit (GPU) (which can be understood as a type of microprocessor), or a Digital Signal Processor (DSP). In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. The logical relationships of the aforementioned hardware circuits are fixed or reconfigurable. For example, the processor is a hardware circuit implemented using an Application-Specific Integrated Circuit (ASIC) or a Programmable Logic Device (PLD), such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units or modules. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a Neural Network Processing Unit (NPU), a Tensor Processing Unit (TPU), a Deep Learning Processing Unit (DPU), etc.

[0119] The computer-readable storage medium provided in this embodiment can execute the video recording method based on the speed-up playback environment of the above embodiment. Its implementation principle and technical effect are similar to those of the above embodiment, and will not be repeated here.

[0120] The aforementioned computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0121] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in application-specific integrated circuits (ASICs). Alternatively, the processor and the readable storage medium can exist as discrete components in an electronic device or a host device.

[0122] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0123] The various embodiments or implementation methods described in this specification are presented in a progressive manner. Each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other.

[0124] In the description of this specification, references to "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0125] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A video recording method based on a playback speed adjustment environment, characterized in that, The method includes: During the process of accelerated video playback in the player, data packets related to recording are acquired, and the baseline recording speed and target frame interval are acquired. The baseline recording speed is the speed used by the player for accelerated playback. Determine the observation frame interval between the latest acquired data packets, and determine the latest time scaling factor based on the ratio of the target frame interval to the observation frame interval and the baseline recording speed; For each data packet acquired from the start of recording, the relative timestamp of the data packet is stretched based on the latest time scaling factor determined by the data packets acquired before the data packet, and the stretched timestamp is encapsulated with the data packet and written into the recording file.

2. The video recording method according to claim 1, characterized in that, Based on the ratio of the target frame interval to the observed frame interval and the baseline recording speed, the latest time scaling factor is determined, including: The ratio of the target frame interval to the observation frame interval is determined as the candidate time scaling factor; Determine whether the deviation between the candidate time scaling factor and the baseline recording speed is less than a preset threshold; If the value is less than a preset threshold, the candidate time scaling factor is determined as the latest time scaling factor; otherwise, the baseline recording speed is determined as the latest time scaling factor.

3. The video recording method according to claim 2, characterized in that, Determining the ratio of the target frame interval to the observation frame interval as the candidate temporal scaling factor includes: Determine whether the latest acquired data packet carries a valid duration field; If carried, the value of the duration field is used as the target frame interval, and the ratio of the value of the duration field to the observation frame interval is used as the candidate time scaling factor; If not carried, the corresponding target frame interval is calculated based on at least two of the decoder's actual frame rate, the stream's nominal frame rate, and the average frame rate, and the ratio of each target frame interval to the observed frame interval is used as the candidate multiplier. From the candidate scaling factors, the one with the smallest deviation from the reference recording speed is selected as the candidate time scaling factor.

4. The video recording method according to claim 1, characterized in that, The latest time scaling factor is determined based on the ratio of the target frame interval to the observation frame interval of the video stream data packets and the baseline recording speed. For each data packet acquired from the start of recording, the relative timestamp of the data packet is stretched based on the latest time scaling factor determined by data packets acquired before the data packet, including: For each video stream data packet acquired from the start of recording, the relative timestamp of the video stream data packet is stretched based on the latest time scaling factor determined by the video stream data packets acquired before the video stream data packet. For each audio stream data packet acquired from the start of recording, the latest time scaling factor used by the video stream data packet is reused to stretch the relative timestamp of the audio stream data packet so that the video stream and audio stream in the recorded file remain synchronized during playback.

5. The video recording method according to claim 4, characterized in that, The step of stretching the relative timestamp of each audio stream data packet acquired from the start of recording, using the latest time scaling factor used by the video stream data packet, includes: For each audio stream data packet acquired from the start of recording, a target video stream data packet whose original timestamp is closest to that of the audio stream data packet is determined, and the relative timestamp of the audio stream data packet is stretched based on the latest time scaling factor used by the target video stream data packet.

6. The video recording method according to claim 1, characterized in that, Stretching the relative timestamp of the data packet includes: The relative timestamp is obtained by subtracting the reference start timestamp from the original timestamp of the data packet, wherein the reference start timestamp is the original timestamp of the first data packet written to the recording file from the start of recording. Multiply the relative timestamp by the latest time scaling factor to obtain the scaled timestamp; The scaled timestamp is converted from the source stream time base to the output container time base to obtain the stretched timestamp.

7. The video recording method according to claim 1, characterized in that, Before the step of encapsulating and writing the stretched timestamp and the data packet into the recording file, the method further includes: Check whether the stretched timestamp of the current data packet has rolled back compared to the stretched timestamp of the previous data packet; If a rollback occurs, the stretched timestamp of the current data packet will be corrected to the stretched timestamp of the previous data packet plus a preset increment.

8. The video recording method according to any one of claims 1 to 7, characterized in that, The data packet is a copy of the data packet obtained from the demultiplexer output of the player through a bypass method, and the audio and video compressed data in the data packet copy maintains the original encoding format during the execution of the video recording method; And / or, the first data packet written to the recording file is: the first key data packet obtained after the recording starts, or the first data packet obtained at the start of the recording.

9. A video recording device based on a speed-up playback environment, characterized in that, The device includes: The acquisition module is used to acquire data packets related to recording during the process of accelerated video playback in the player, and to acquire the baseline recording speed and the target frame interval, wherein the baseline recording speed is the speed adopted by the player for accelerated playback; The determination module is used to determine the observation frame interval between the latest acquired data packets, and to determine the latest time scaling factor based on the ratio of the target frame interval to the observation frame interval and the baseline recording speed. The stretching module is used to stretch the relative timestamp of each data packet acquired from the start of recording, based on the latest time scaling factor determined by the data packets acquired before the data packet. The writing module is used to encapsulate the stretched timestamp and the data packet into a recording file.

10. An electronic device, characterized in that, Includes a processor, the processor being configured to invoke instructions to cause the electronic device to perform the steps of the video recording method based on a speed-up playback environment as described in any one of claims 1 to 8.

Images