Audio and video processing method, device and related equipment

Abstract

The present application provides an audio and video processing method, apparatus and related equipment for reverse playback of audio and video media, wherein the audio and video processing method includes: determining the media stream information and transmission stream data of the target audio and video media, determining the key frames for reverse playback in the target audio and video media based on the determined target reverse playback time point, the media stream information and the transmission stream data, determining the reverse playback audio and video frame range according to the key frame image, adjusting the audio and video data playback stream order corresponding to the audio and video frame range, determining the reverse playback audio and video frame sequence, and reverse playback of the audio and video data corresponding to the target reverse playback time point according to the reverse audio and video sequence. This audio and video processing method can adapt to the needs of reverse playback of audio and video in various formats, without the need for manual progress bar dragging control, and the reverse playback process is smooth and natural, with good playback stability.

Description

Technical Field

[0001] The present application relates to the field of multimedia technology, and in particular to an audio and video processing method, device and related equipment. Background Art

[0002] With the development of audio and video playback technology, the use of shooting and monitoring applications and equipment is becoming more and more widespread. In the applications related to playback technology commonly used in society, most of them provide sequential playback according to the original audio and video playback stream, but with the monitoring video such as the dash cam installed on the vehicle or other playback scenarios, it is often necessary to play back the audio and video media in reverse so that users can find the audio and video data at the corresponding time point. However, when the audio and video players currently provided in the industry are adapted to the scene application of reverse playback of audio and video, most of them require users to manually drag the player to reverse the progress bar to make the audio and video data play in reverse, and do not provide the function of automatically reversing the audio and video media directly. This method of manually dragging the player playback progress bar is very inconvenient to use, and it is difficult to help users quickly find the corresponding key frame audio and video data, and the user experience is poor. Summary of the invention

[0003] In view of this, an embodiment of the present application provides an audio and video processing method to at least partially solve the above problems.

[0004] In a first aspect, an embodiment of the present application provides an audio and video processing method for reversely playing audio and video media, including:

[0005] Determine media stream information and transport stream data of target audio and video media;

[0006] Determining a key frame for performing reverse playback in the target audio and video media based on the determined target reverse playback time point, the media stream information, and the transport stream data;

[0007] Determine the reverse audio and video frame range according to the key frame image;

[0008] Adjusting the order of the audio and video data playback stream corresponding to the audio and video frame range to determine the reverse audio and video frame sequence;

[0009] According to the reverse audio and video sequence, the audio and video data corresponding to the target reverse time point is reversed.

[0010] Optionally, in an embodiment of the present application, the step of reversing the audio and video data corresponding to the target reversal time point according to the reversal audio and video sequence includes:

[0011] According to the playback order of the target audio and video media stream, the reverse audio and video sequence is stored in a reverse storage stack to generate a corresponding reverse audio and video media stream;

[0012] According to the first-in-last-out principle, the reverse broadcast audio and video media stream is popped from the reverse broadcast storage stack for decoding and popping, so as to play the corresponding reverse broadcast audio and video media stream.

[0013] Optionally, in an embodiment of the present application, determining a key frame for reverse playback in the target audio and video media based on the determined target reverse playback time point, the media stream information, and the transport stream data includes:

[0014] Determining corresponding transport stream data based on the target reverse broadcast time point and the media stream information;

[0015] Locating audio and video frame information close to the time point of the target reverse playback in the transport stream data;

[0016] Based on the audio and video frame information, a key frame for reverse playback is determined in the target audio and video media.

[0017] Optionally, in an embodiment of the present application, determining a key frame for reverse playback in the target audio and video media based on the audio and video frame information includes:

[0018] Determine timestamp information of the audio and video frame information;

[0019] Based on the timestamp information, searching the audio and video frames preceding the position of the audio and video frame as the base position to obtain at least one set of video frames for representing complete image information;

[0020] The at least one group of video frames obtained to represent complete image information is determined as key frames for performing the reverse playback.

[0021] Optionally, in an embodiment of the present application, based on the timestamp information, searching for a previous audio or video frame based on the position of the audio or video frame to obtain at least one key frame includes:

[0022] When the target audio and video is an audio and video media of a non-fixed GOP, the range Qn to be searched from the previous audio and video frame based on the audio and video frame is determined by the following formula:

[0023]

[0024] Among them, n is used to represent the number of times to search in the previous audio and video frames, x 1 、x 2 , …, x n It is used to indicate the time range or the number of frames to be searched in the previous audio or video frames corresponding to the number of times;

[0025] Based on the determined time range or number of frames, the at least one key frame is iteratively retrieved.

[0026] Optionally, in an embodiment of the present application, the iterative retrieval of the at least one key frame based on the determined time range or number of frames includes:

[0027] updating the basic position according to the retrieved timestamp information of at least one key frame;

[0028] The at least one key frame is iteratively retrieved according to the result of the basic position update.

[0029] Optionally, in an embodiment of the present application, determining the reverse audio and video frame range according to the key frame image includes:

[0030] The reverse audio and video range is determined according to the determined timestamp information corresponding to the key frame image.

[0031] Optionally, in an embodiment of the present application, based on the timestamp information, searching for audio and video frames preceding the position of the audio and video frame as a base position to obtain at least one group of video frames for representing complete image information includes:

[0032] Using a static or dynamic step size and taking the position of the audio and video frame as a base position, a search is performed in the preceding audio and video to obtain at least one video frame for representing complete image information.

[0033] In a second aspect, based on the audio and video processing method provided in the first aspect of the present application, an embodiment of the present application further provides an audio and video processing device for reverse playback of audio and video media, comprising:

[0034] A determination module, used to determine the media stream information and transport stream data of the target audio and video media;

[0035] A retrieval module, configured to determine a key frame for reverse playback in the target audio and video media based on the determined target reverse playback time point, the media stream information and the transport stream data;

[0036] A positioning module, used to determine the range of the reverse audio and video frames according to the key frame image;

[0037] An adjustment module, used for adjusting the order of the audio and video data playback stream corresponding to the audio and video frame range, and determining the reverse audio and video frame sequence;

[0038] The playback module is used to reverse the audio and video data corresponding to the target reverse playback time point according to the reverse audio and video sequence.

[0039] In a third aspect, an embodiment of the present application further provides a storage medium on which a computer program is stored, and when the program is executed by a processor, any one of the audio and video processing methods described in the first aspect of the present application is implemented.

[0040] The present application provides an audio and video processing method, apparatus and related equipment for reverse playback of audio and video media, wherein the audio and video processing method includes: determining the media stream information and transmission stream data of the target audio and video media, determining the key frames for reverse playback in the target audio and video media based on the determined target reverse playback time point, the media stream information and the transmission stream data, determining the reverse playback audio and video frame range according to the key frame image, adjusting the audio and video data playback stream order corresponding to the audio and video frame range, determining the reverse playback audio and video frame sequence, and reverse playback of the audio and video data corresponding to the target reverse playback time point according to the reverse audio and video sequence. This audio and video processing method can adapt to the needs of reverse playback of audio and video in various formats, without the need for manual progress bar dragging control, and the reverse playback process is smooth and natural, with good playback stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0042] Figure 1 A schematic diagram of a workflow of an audio and video processing method provided in an embodiment of the present application;

[0043] Figure 2 A schematic diagram of the structure of an audio and video processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the embodiments of the present application should fall within the scope of protection of the embodiments of the present application.

[0045] It should be understood that the various steps described in the method implementation of the present application can be performed in different orders and / or performed in parallel. In addition, the method implementation may include additional steps and / or omit the steps shown. The scope of the present application is not limited in this respect.

[0046] Embodiment 1

[0047] The present application embodiment provides an audio and video processing method, such as Figure 1 As shown, Figure 1 A schematic diagram of the structure of an audio and video processing method provided in an embodiment of the present application, the audio and video processing method is used to reversely play audio and video media, including:

[0048] S101: Determine media stream information and transport stream data of target audio and video media.

[0049] In the implementation process of the embodiment of the present application, the audio and video stream is a complete video file; for example, the media information included may be the relevant information contained in the m3u8 file, such as duration, resolution, and encoding method, and the transport stream data is a corresponding number of .ts files, which completely determine the overall duration of the target audio and video media. Of course, in the embodiment of the present application, the target audio and video media may also be an audio and video file in MP4 format or other formats, and the embodiment of the present application does not limit this.

[0050] Specifically, in one implementation of the embodiment of the present application, the time length of the target audio and video media can be determined from the media stream information and the transport stream data, which can accurately meet the user's need for reverse playback according to the specified time.

[0051] S102: Based on the determined target rewind time point, the media stream information and the transport stream data, a key frame (I frame, intra picture) for rewinding is determined in the target audio and video media.

[0052] In an embodiment of the present application, the determined target rewind time point can be determined based on the analysis of the time point information input by the user, or it can be determined by other operations such as the user dragging the player playback progress bar, so as to better meet the rewind needs of different users under different playback simulations, provide users with one or more rewind methods, and improve the user experience.

[0053] Optionally, in an embodiment of the present application, when decoding and playing a video, decoding must be performed starting from the key frame, i.e., the I frame, so that the subsequent P frames and B frames can be decoded and displayed according to the forward reference frame. However, P frames (Predictive Picture), B frames (Bidirectionally predicted picture) or other audio and video image frames that do not carry any audio and video content information, such as IDR frames, cannot fully represent a set of complete video frames (GOP, Group of Pictures). The implementation process of this embodiment can effectively prevent the occurrence of decoding failures during the reverse playback of the target audio and video media, resulting in the interruption of the reverse playback, so as to ensure the integrity of the reverse playback and the stability of the playback.

[0054] S103: Determine a reverse audio and video frame range according to the key frame image.

[0055] In an embodiment of the present application, key frame images are used to identify audio and video frame media stream images that carry audio and video related information in the target audio and video to be reversed, rather than non-key frames such as P frames, B frames, etc. that cannot be decoded sequentially or may cause the reverse playback to stop, affecting the stable progress of the reverse playback.

[0056] Optionally, in an embodiment of the present application, the determining of the reverse audio and video frame range according to the key frame image includes: determining the reverse audio and video range according to the timestamp information corresponding to the determined key frame image. In the embodiment of the present application, the timestamp information is the time information corresponding to each frame of audio and video frame image corresponding to the total duration of the complete audio and video, which is used to characterize the time point, that is, the position, of the frame of audio and video image, such as a frame of video frame image corresponding to the 3rd minute and 2nd second when a complete video is played sequentially. In other implementations of the embodiment of the present application, the timestamp information can also be determined based on the number of all image frames contained in the complete video. For example, a complete video has 20,000 frames of images, and the timestamp information can also be used to characterize the position of a certain frame of image, such as the 1000th frame or the 2000th frame of video frame image and other related sequence parameters. Of course, this embodiment is only an exemplary description of the time error information here, and does not mean that the present application is limited to this. Through the time error information, the range of the target reverse audio and video frame can be quickly determined, and the process is efficient and accurate.

[0057] Preferably, in an implementation method of an embodiment of the present application, all retrieved key frame images are used to determine the reverse audio and video frame range, that is, image frames such as gray frames, blue frames or blank frames in the original audio and video media stream that do not carry any audio or video content information are removed.

[0058] S104: Adjust the order of the audio and video data playback stream corresponding to the audio and video frame range to determine the reverse audio and video frame sequence.

[0059] Specifically, in one implementation method of the embodiment of the present application, the order of the audio and video data playback stream corresponding to the audio and video frame range is adjusted to determine the reverse audio and video frame sequence, including: based on the video frame image decoding method of the target audio and video media, the audio and video data corresponding to the determined audio and video frame range is sequentially decoded to obtain a decoded audio and video frame sequence, and the decoded audio and video sequence is reversely arranged to determine the reverse audio and video frame sequence to ensure that the reverse audio and video frame sequence can be stably decoded and played.

[0060] S105 . Reverse the audio and video data corresponding to the target reverse time point according to the reverse audio and video sequence.

[0061] Optionally, in one embodiment of the present application, the audio and video data corresponding to the target reverse time point is reversed according to the reverse audio and video sequence, including: storing the reverse audio and video sequence in a reverse storage stack in the playback order of the target audio and video media stream to generate a corresponding reverse audio and video media stream, and according to the first-in-last-out principle, popping the reverse audio and video media stream from the reverse storage stack for decoding and popping it to play the corresponding reverse audio and video media stream.

[0062] The embodiment of the present application stores the audio and video media streams to be reversed by establishing a reverse storage stack, so as to facilitate the orderliness and continuity of the playback of the reverse audio and video streaming media streams, thereby ensuring the stable progress of the reverse work without the need for unnecessary manual operations, thereby improving the efficiency of reversing the determined video and the user experience.

[0063] Optionally, in an embodiment of the present application, the key frames for reverse playback are determined in the target audio and video media based on the determined target reverse playback time point, the media stream information and the transport stream data, including: determining the corresponding transport stream data based on the target reverse playback time point and the media stream information, locating the audio and video frame information close to the target reverse playback time point in the transport stream data, and determining the key frames for reverse playback in the target audio and video media based on the audio and video frame information. In the embodiment of the present application, since the reverse playback is implemented based on the target reverse playback time point, what needs to be obtained must be the audio and video frames in the sequence before the corresponding audio and video image frame determined based on the time point, so as to ensure the accuracy of the reverse playback work and meet the needs of users.

[0064] Optionally, in one embodiment of the present application, determining the key frames for reverse playback in the target audio and video media based on the audio and video frame information includes: determining the timestamp information of the audio and video frame information, and based on the timestamp information, searching the audio and video frames preceding the position of the audio and video frame as the base position to obtain at least one group of video frames for representing complete image information, and determining the at least one group of video frames for representing complete image information obtained as the key frames for the reverse playback. The accuracy of the determination is guaranteed. The system does not need to consume too much system processing resources during the retrieval process, so as to further improve the efficiency of determining the key frames.

[0065] Optionally, in one embodiment of the present application, based on the timestamp information, searching the previous audio and video frames based on the position of the audio and video frame to obtain at least one group of video frames for representing complete image information includes: using a static or dynamic step size, taking the position of the audio and video frame as the base position, searching the previous audio and video to obtain at least one video frame for representing complete image information.

[0066] Specifically, the static step size is applied to the key frame retrieval of audio and video media with fixed GOP, while the dynamic step size is applied to the key frame retrieval process of audio and video media with non-fixed GOP.

[0067] Preferably, for audio and video media with a fixed GOP, the static step size used to retrieve key frames can be the time length or number of frames corresponding to a GOP, that is, the number of frames between two adjacent I frames; while for audio and video media with a non-fixed GOP, a dynamic step size is used. For example, the retrieval step size used for the next time is adjusted according to the step size when the key frame was retrieved the previous time or multiple times, and then the key frame retrieval is performed, which can improve the efficiency of retrieving key frames to a certain extent.

[0068] Optionally, in an embodiment of the present application, based on the timestamp information, searching for an audio or video frame that is a previous audio or video frame based on the position of the audio or video frame to obtain at least one key frame includes: when the target audio or video is an audio or video media of a non-fixed GOP, determining a range Qn to be searched from an audio or video frame that is a previous audio or video frame based on the audio or video frame by the following formula:

[0069]

[0070] Among them, n is used to represent the number of times to search in the previous audio and video frames, x 1 、x 2 , …, x nIt is used to indicate the time range or frame number for searching the previous audio or video frames corresponding to the number of times, and iteratively search the at least one key frame based on the determined time range or frame number.

[0071] In the implementation scenario of the embodiment of the present application, the media stream of the target audio and video media is generally divided into two types, namely, a media stream with a fixed GOP (Group of Pictures) and a media stream with a non-fixed GOP. In the process of searching for key frames for audio and video with a fixed GOP, a GOP frame range, time range or frame number is used as a search unit, and other key frames are searched forward. During the search process, no P frames and B frames need to be analyzed, and all key frames can be determined efficiently and accurately. However, for media streams with non-fixed GOP, the time range or frame range of each search can be adjusted by calculating the arithmetic square root of the above formula, so that key frames can be retrieved more efficiently and the search hit rate can be improved.

[0072] Optionally, in an embodiment of the present application, the iterative retrieval of the at least one key frame based on the determined time range or number of frames includes: updating the basic position according to the timestamp information of the at least one key frame retrieved, and iteratively retrieving the at least one key frame according to the result of the basic position update. In the embodiment of the present application, by updating the basic position, all key frames can be more completely determined, and the amount of data processing in the retrieval process can be reduced, thereby improving the efficiency and accuracy of retrieving key frames.

[0073] The present application provides an audio and video processing method for reverse playback of audio and video media, wherein the audio and video processing method includes: determining the media stream information and transmission stream data of the target audio and video media, determining the key frames for reverse playback in the target audio and video media based on the determined target reverse playback time point, the media stream information and the transmission stream data, determining the reverse playback audio and video frame range according to the key frame image, adjusting the audio and video data playback stream order corresponding to the audio and video frame range, determining the reverse playback audio and video frame sequence, and reverse playback of the audio and video data corresponding to the target reverse playback time point according to the reverse playback audio and video sequence. This audio and video processing method can adapt to the needs of reverse playback of audio and video in various formats, without the need for manual progress bar dragging control, and the reverse playback process is smooth and natural, with good playback stability.

[0074] Embodiment 2

[0075] Based on the audio and video processing method described in the first embodiment of the present application, the present application also provides an audio and video processing device 20 for reverse playback of audio and video media, including:

[0076] Determination module 201, used to determine the media stream information and transport stream data of the target audio and video media;

[0077] A retrieval module 202, configured to determine a key frame for reverse playback in the target audio and video media based on the determined target reverse playback time point, the media stream information and the transport stream data;

[0078] A positioning module 203, used to determine the reverse audio and video frame range according to the key frame image;

[0079] An adjustment module 204 is used to adjust the order of the audio and video data playback stream corresponding to the audio and video frame range to determine the reverse audio and video frame sequence;

[0080] The playback module 205 is used to reverse the audio and video data corresponding to the target reverse playback time point according to the reverse audio and video sequence.

[0081] Optionally, in an implementation of this embodiment, the playing module 205 is further configured to:

[0082] According to the playback order of the target audio and video media stream, the reverse audio and video sequence is stored in a reverse storage stack to generate a corresponding reverse audio and video media stream;

[0083] According to the first-in-last-out principle, the reverse broadcast audio and video media stream is popped from the reverse broadcast storage stack for decoding and popping, so as to play the corresponding reverse broadcast audio and video media stream.

[0084] Optionally, in an implementation of this embodiment, the retrieval module 202 is further configured to:

[0085] Determining corresponding transport stream data based on the target reverse broadcast time point and the media stream information;

[0086] Locating audio and video frame information close to the time point of the target reverse playback in the transport stream data;

[0087] Based on the audio and video frame information, a key frame for reverse playback is determined in the target audio and video media.

[0088] Optionally, in an implementation of this embodiment, the retrieval module 202 is further configured to:

[0089] Determine timestamp information of the audio and video frame information;

[0090] Based on the timestamp information, searching the audio and video frames preceding the position of the audio and video frame as the base position to obtain at least one set of video frames for representing complete image information;

[0091] The at least one group of video frames obtained to represent complete image information is determined as key frames for performing the reverse playback.

[0092] Optionally, in an implementation of this embodiment, the retrieval module 202 is further configured to:

[0093] Using a static or dynamic step size and taking the position of the audio and video frame as a base position, a search is performed in the preceding audio and video to obtain at least one set of video frames for representing complete image information.

[0094] Optionally, in an implementation of this embodiment, the retrieval module 202 is further configured to:

[0095] When the target audio and video is an audio and video media of a non-fixed GOP, the range Qn to be searched from the previous audio and video frame based on the audio and video frame is determined by the following formula:

[0096]

[0097] Among them, n is used to represent the number of times to search in the previous audio and video frames, x 1 、x 2 , …, x n It is used to indicate the time range or the number of frames to be searched in the previous audio or video frames corresponding to the number of times;

[0098] Based on the determined time range or number of frames, the at least one key frame is iteratively retrieved.

[0099] Optionally, in an implementation of this embodiment, the retrieval module 202 is further configured to:

[0100] updating the basic position according to the retrieved timestamp information of at least one key frame;

[0101] The at least one key frame is iteratively retrieved according to the result of the basic position update.

[0102] Optionally, in an implementation of this embodiment, determining the reverse audio and video frame range according to the key frame image includes:

[0103] The reverse audio and video range is determined according to the determined timestamp information corresponding to the key frame image.

[0104] The present application provides an audio and video processing device for reverse playback of audio and video media, wherein the audio and video processing method includes: determining the media stream information and transmission stream data of the target audio and video media, determining the key frames for reverse playback in the target audio and video media based on the determined target reverse playback time point, the media stream information and the transmission stream data, determining the reverse playback audio and video frame range according to the key frame image, adjusting the audio and video data playback stream order corresponding to the audio and video frame range, determining the reverse playback audio and video frame sequence, and reverse playback of the audio and video data corresponding to the target reverse playback time point according to the reverse playback audio and video sequence. This audio and video processing method can adapt to the needs of reverse playback of audio and video in various formats, without the need for manual dragging of the progress bar to control, and the reverse playback process is smooth and natural, with good playback stability.

[0105] Embodiment 3

[0106] The present application also provides a storage medium on which a computer program is stored. When the program is executed by a processor, any one of the audio and video processing methods described in the first embodiment of the present application is implemented. The audio and video processing method includes but is not limited to:

[0107] Determine media stream information and transport stream data of target audio and video media;

[0108] Determining a key frame for performing reverse playback in the target audio and video media based on the determined target reverse playback time point, the media stream information, and the transport stream data;

[0109] Determine the reverse audio and video frame range according to the key frame image;

[0110] Adjusting the order of the audio and video data playback stream corresponding to the audio and video frame range to determine the reverse audio and video frame sequence;

[0111] According to the reverse audio and video sequence, the audio and video data corresponding to the target reverse time point is reversed.

[0112] So far, the application has described specific embodiments of the subject matter. Other embodiments are within the scope of the appended claims. In some cases, the actions recorded in the claims can be performed in a different order and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing can be advantageous.

[0113] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0114] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0115] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0116] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0117] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0118] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0119] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0120] The present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific transactions or implement specific abstract data types. The present application may also be practiced in distributed computing environments where transactions are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0121] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0122] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. An audio and video processing method, characterized in that: Used to reverse the audio and video media, including: Determine media stream information and transport stream data of target audio and video media; Determining a key frame for performing reverse playback in the target audio and video media based on the determined target reverse playback time point, the media stream information, and the transport stream data; Determine the reverse audio and video frame range according to the key frame image; Adjust the order of the audio and video data playback stream corresponding to the audio and video frame range to determine the reverse audio and video frame sequence; and reverse the audio and video data corresponding to the target reverse time point according to the reverse audio and video frame sequence; The step of determining a key frame for reverse playback in the target audio and video media based on the determined target reverse playback time point, the media stream information, and the transport stream data includes: Based on the time point of the target reverse broadcast and the media stream information, the corresponding transport stream data is determined; and the audio and video frame information close to the time point of the target reverse broadcast is located in the transport stream data; Based on the audio and video frame information, determining a key frame for reverse playback in the target audio and video media; The step of determining a key frame for reverse playback in the target audio and video media based on the audio and video frame information includes: Determine timestamp information of the audio and video frame information; Based on the timestamp information, searching the audio and video frames preceding the position of the audio and video frame as the base position to obtain at least one set of video frames for representing complete image information; Determining the acquired at least one group of video frames for representing complete image information as key frames for performing the reverse playback; The step of retrieving the audio and video frames preceding the position of the audio and video frame based on the timestamp information to obtain at least one key frame includes: When the target audio and video is an audio and video media of a non-fixed GOP, the range Qn to be searched from the previous audio and video frame based on the audio and video frame is determined by the following formula: Wherein, n is used to represent the number of times to search in the previous audio and video frames, and x1, x2, ..., xn are used to represent the time range or the number of frames to search in the previous audio and video frames corresponding to the number of times; Based on the determined time range or number of frames, the at least one key frame is iteratively retrieved.

2. The audio and video processing method according to claim 1, characterized in that: The step of reversing the audio and video data corresponding to the target reversal time point according to the reversal audio and video frame sequence includes: According to the playback order of the target audio and video media stream, the reversed audio and video frame sequence is stored in a reversed storage stack to generate a corresponding reversed audio and video media stream; According to the first-in-last-out principle, the reverse broadcast audio and video media stream is popped from the reverse broadcast storage stack for decoding and popping, so as to play the corresponding reverse broadcast audio and video media stream.

3. The audio and video processing method according to claim 1, characterized in that: The step of retrieving the audio and video frames preceding the position of the audio and video frame based on the timestamp information to obtain at least one set of video frames for representing complete image information comprises: Using a static or dynamic step size and taking the position of the audio and video frame as a base position, a search is performed in the preceding audio and video to obtain at least one video frame for representing complete image information.

4. The audio and video processing method according to claim 1, characterized in that: The iterative retrieval of the at least one key frame based on the determined time range or frame number comprises: The basic position is updated according to the timestamp information of the at least one key frame retrieved; and the at least one key frame is iteratively retrieved according to the result of the basic position update.

5. The audio and video processing method according to claim 1, characterized in that: The step of determining the reverse audio and video frame range according to the key frame image comprises: The reverse audio and video range is determined according to the determined timestamp information corresponding to the key frame image.

6. An audio and video processing device, characterized in that: Used to reverse the audio and video media, including: A determination module, used to determine the media stream information and transport stream data of the target audio and video media; A retrieval module, configured to determine a key frame for reverse playback in the target audio and video media based on the determined target reverse playback time point, the media stream information and the transport stream data; A positioning module, used to determine the range of the reverse audio and video frames according to the key frame images; An adjustment module, used for adjusting the order of the audio and video data playback stream corresponding to the audio and video frame range, and determining the reverse audio and video frame sequence; A playback module, configured to reversely play the audio and video data corresponding to the target reverse playback time point according to the reverse playback audio and video frame sequence; The step of determining a key frame for reverse playback in the target audio and video media based on the determined target reverse playback time point, the media stream information, and the transport stream data includes: Based on the time point of the target reverse broadcast and the media stream information, the corresponding transport stream data is determined; and the audio and video frame information close to the time point of the target reverse broadcast is located in the transport stream data; Based on the audio and video frame information, determining a key frame for reverse playback in the target audio and video media; The step of determining a key frame for reverse playback in the target audio and video media based on the audio and video frame information includes: Determine timestamp information of the audio and video frame information; Based on the timestamp information, searching the audio and video frames preceding the position of the audio and video frame as the base position to obtain at least one set of video frames for representing complete image information; Determining the acquired at least one group of video frames for representing complete image information as key frames for performing the reverse playback; The step of retrieving the audio and video frames preceding the position of the audio and video frame based on the timestamp information to obtain at least one key frame includes: When the target audio and video is an audio and video media of a non-fixed GOP, the range Qn to be searched from the previous audio and video frame based on the audio and video frame is determined by the following formula: Wherein, n is used to represent the number of times to search in the previous audio and video frames, and x1, x2, ..., xn are used to represent the time range or the number of frames to search in the previous audio and video frames corresponding to the number of times; Based on the determined time range or number of frames, the at least one key frame is iteratively retrieved.

7. A storage medium, characterized in that: A computer program is stored thereon, and when the program is executed by a processor, the audio and video processing method as described in any one of claims 1 to 5 is implemented.

Images