Audio and video processing method and device, storage medium and program product

Abstract

The embodiment of the invention provides an audio and video processing method and device, a storage medium and a program product, and the method comprises the steps: carrying out the frame-by-frame analysis of a to-be-processed audio and video if the to-be-processed audio and video do not contain a target index file or the contained target index file is abnormal, and a frame index file corresponding to the to-be-processed audio and video is not stored locally, the audio and video processing method comprises the following steps: acquiring frame information of a plurality of audio and video frames, generating a frame index file according to the frame information of the plurality of audio and video frames, and storing the frame index file which is used for processing an audio and video to be processed. And after the to-be-processed audio is imported, the to-be-processed audio can be subjected to frame seek and other processing based on the pre-stored frame index file, and the target frame does not need to be decoded frame by frame, so that the processing efficiency is improved.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the technical field of audio and video processing, and in particular, to an audio and video processing method, device, storage medium, and program product. Background Art

[0002] With the improvement of terminal device hardware performance and the continuous advancement of artificial intelligence technology, more and more applications (Application, referred to as APP) are running on terminal devices. Sometimes when using these APPs, users need to enter content in the input box to interact with the network side. Usually the network side has certain restrictions on the content entered by the user in the input box. For example, when registering an account, the network side may have requirements for the category of the user name or the composition of the password. At this time, the APP needs to process these restrictions to prompt the user with audio and video.

[0003] In the related art, seek is usually performed based on the key frame index table that comes with the audio or video file.

[0004] However, the above method is not applicable to some audio and video files that do not carry a key frame index table, or the key frame index table they carry has anomalies and cannot be used normally. It is necessary to decode the file step by step to the target frame, which takes a long time.

[0005] However, the inventors have discovered that there are at least the following technical problems in the related technology: when the user clicks on the input box, the original default prompt information disappears. If the user is interrupted by other things at this time, the user will not be able to know the prompt information corresponding to the input box when he inputs again, resulting in poor interaction efficiency between the user and the network side. Summary of the invention

[0006] The embodiments of the present disclosure provide an audio and video processing method, device, storage medium and program product to improve processing efficiency.

[0007] In a first aspect, an embodiment of the present disclosure provides an audio and video processing method, including:

[0008] Get the audio and video to be processed;

[0009] If the audio and video to be processed does not contain the target index file or the target index file contained is abnormal, and the frame index file corresponding to the audio and video to be processed is not saved locally, the audio and video to be processed is parsed frame by frame to obtain frame information of multiple audio and video frames; a frame index file is generated according to the frame information of the multiple audio and video frames, and the frame index file is saved, and the frame index file is used to process the audio and video to be processed.

[0010] In a second aspect, an embodiment of the present disclosure provides an audio and video processing device, including:

[0011] An acquisition module is used to acquire audio and video to be processed;

[0012] A processing module is used to parse the audio and video to be processed frame by frame to obtain frame information of multiple audio and video frames if the audio and video to be processed does not contain a target index file or the target index file contained is abnormal, and the frame index file corresponding to the audio and video to be processed is not saved locally; generate a frame index file according to the frame information of the multiple audio and video frames, save the frame index file, and use the frame index file to process the audio and video to be processed.

[0013] In a third aspect, an embodiment of the present disclosure provides an electronic device, including: a processor and a memory;

[0014] The memory stores computer-executable instructions;

[0015] The processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the audio and video processing method described in the first aspect and various possible designs of the first aspect.

[0016] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the audio and video processing method described in the first aspect and various possible designs of the first aspect is implemented.

[0017] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the audio and video processing method described in the first aspect and various possible designs of the first aspect.

[0018] The audio and video processing method, device, storage medium and program product provided in the present embodiment include obtaining the audio and video to be processed, and if the audio and video to be processed does not contain the target index file or the target index file contained is abnormal, and the frame index file corresponding to the audio and video to be processed is not saved locally, the audio and video to be processed is parsed frame by frame to obtain the frame information of multiple audio and video frames, a frame index file is generated according to the frame information of the multiple audio and video frames, and the frame index file is saved. The frame index file is used to process the audio and video to be processed. The method provided in the present embodiment of the disclosure saves the frame index file corresponding to the audio and video to be processed locally in advance, and after importing the audio to be processed, frame search and other processing can be performed on the audio to be processed based on the pre-saved frame index file, without the need to decode to the target frame frame by frame, thereby improving processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0020] Figure 1 An application scenario diagram of the audio and video processing method provided in an embodiment of the present disclosure;

[0021] Figure 2 The audio and video processing method provided in the present disclosure is shown in the following figure. Figure 1 ;

[0022] Figure 3 The audio and video processing method provided in the present disclosure is shown in the following figure. Figure 2 ;

[0023] Figure 4 A structural block diagram of an audio and video processing device provided in an embodiment of the present disclosure;

[0024] Figure 5 A schematic diagram of the hardware structure of the audio and video processing device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0026] When audio or video files are imported into an application and played or edited in the application, seek jumps are often required to jump to the target frame.

[0027] In the related art, seek is usually performed based on the key frame index table of the audio and video file to find the key frame near the target frame, and decoding is started from the key frame to achieve the purpose of quickly jumping to the target frame. However, since some special audio and video files do not have a key frame index table, or the key frame index table is abnormal and cannot be used, it is impossible to quickly seek in the player or audio and video editing software, and the only way is to decode from the file header step by step to the target frame.

[0028] In order to solve the above technical problems, the inventors of the present disclosure have found that a frame index file corresponding to the audio to be processed can be generated locally and saved for subsequent use. Based on the pre-stored frame index file, fast seek and other processing of the audio and video to be processed can be implemented. Based on this, the embodiment of the present disclosure provides an audio and video processing method.

[0029] refer to Figure 1 , Figure 1 The application scenario diagram of the audio and video processing method provided in the embodiment of the present disclosure is shown in FIG. The terminal device 100 is installed with an application for processing audio and video. The terminal device 100 may be a computer, a laptop, a tablet computer, a mobile phone, or other device.

[0030] In the specific implementation process, the terminal device starts the application, imports the audio and video to be processed into the application, and determines whether the frame index file corresponding to the audio and video to be processed is saved locally. If the audio and video to be processed does not contain the target index file or the target index file contained is abnormal, and the frame index file corresponding to the audio and video to be processed is not saved locally, the audio and video to be processed is parsed frame by frame to obtain the frame information of multiple audio and video frames, and a frame index file is generated according to the frame information of the multiple audio and video frames. The frame index file is saved so that when the audio and video to be processed is obtained again, the audio and video to be processed can be sought and played based on the pre-saved frame index file. The audio and video processing method provided by the embodiment of the present disclosure can perform frame search and seek on the audio to be processed based on the pre-saved frame index file after importing the audio to be processed, without decoding to the target frame frame by frame, thereby improving processing efficiency.

[0031] It should be noted that Figure 1 The scene diagram shown is only an example. The audio and video processing method and scene described in the embodiment of the present application are intended to more clearly illustrate the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided in the embodiment of the present application. Ordinary technicians in this field can know that with the evolution of the system and the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is also applicable to similar technical problems.

[0032] refer to Figure 2 , Figure 2 The audio and video processing method provided in the present disclosure is shown in the following figure. Figure 1 The method of this embodiment can be applied in a terminal device or a server. The audio and video processing method includes:

[0033] 201. Obtain audio and video to be processed.

[0034] In this embodiment, the audio and video to be processed may be separate audio data, separate video data, or may include both audio data and video data.

[0035] Specifically, in response to the user's touch operation, the terminal device can start an audio and video processing application and import the audio and video to be processed into the application.

[0036] 202. If the audio and video to be processed does not contain the target index file or the target index file contained is abnormal, and the frame index file corresponding to the audio and video to be processed is not saved locally, the audio and video to be processed is parsed frame by frame to obtain frame information of multiple audio and video frames, and a frame index file is generated according to the frame information of the multiple audio and video frames, and the frame index file is saved. The frame index file is used to process the audio and video to be processed.

[0037] The format of the frame index file corresponding to the audio and video to be processed may be in xml, json or other formats.

[0038] Specifically, after importing the audio and video to be processed, you can search locally to find out whether there is a frame index file corresponding to the audio and video to be processed. Of course, if the audio and video to be processed is imported for the first time, the frame index file corresponding to the audio and video to be processed does not exist locally. At this time, you can generate the corresponding frame index file based on the audio and video to be processed, so that it can be directly used when the video to be processed is imported again later.

[0039] In one embodiment of the present disclosure, before determining whether the frame index file corresponding to the audio and video to be processed is saved locally, the audio and video to be processed can be detected first to determine whether the audio and video to be processed carries the target index file (such as a key frame index file or a frame index file). If the audio and video to be processed carries the target index file, then the carried target index file can be directly used to perform true seek and other processing on the audio and video to be processed. If the audio and video to be processed does not carry the target index file, or the carried target index file is abnormal (for example, when seeking the audio and video to be processed based on the target index file, it is impossible to seek to the correct audio and video frame), it can be searched locally whether there is a frame index file generated and saved when the audio and video to be processed was imported before. If it exists, the locally saved frame index file can be used to perform seek and other processing on the audio and video to be processed.

[0040] From the above description, it can be seen that by pre-saving the frame index file corresponding to the audio and video to be processed locally, and after importing the audio to be processed, frame search and other processing can be performed on the audio to be processed based on the pre-saved frame index file, without the need to decode frame by frame to the target frame, thereby improving processing efficiency.

[0041] refer to Figure 3 , Figure 3 The audio and video processing method provided in the present disclosure is shown in the following figure. Figure 2 In this embodiment, a method for generating a frame index file corresponding to the audio to be processed is described in detail. The audio and video processing method includes:

[0042] 301. Obtain audio and video to be processed.

[0043] Step 301 in this embodiment is similar to step 202 in the above embodiment and will not be described again here.

[0044] 302. Determine whether the audio and video to be processed contains a target index file or whether the target index file has an abnormality; if the audio and video to be processed contains a target index file and the target index file has no abnormality, process the audio and video to be processed according to the target index file. If the audio and video to be processed does not contain a target index file, or the target index file contained has an abnormality, execute step 303.

[0045] Specifically, audio and video files usually carry corresponding target index files (keyframe index files or frame index files) for seek and other processing. However, there are some special audio and video files (such as live video) that do not contain target index files. And sometimes, although the target index file is included, it may be damaged during the storage or transmission process, causing the target index file to be abnormal and cannot be used normally. If the target index file exists and there is no abnormality in the target index file, the target index file can be directly used to seek and other processing on the audio and video to be processed. If there is no target index file, or although the target index file exists but the target index file has an abnormality and cannot be used normally, a frame index file corresponding to the audio and video to be processed can be generated and saved for subsequent use.

[0046] 303. Determine whether a frame index file corresponding to the audio and video to be processed is stored locally. If so, execute step 305; if not, execute step 304.

[0047] Specifically, if the audio and video to be processed has been imported before, it is very likely that the frame index file corresponding to the audio and video to be processed has been generated during the previous import. Therefore, if it is determined that the audio and video to be processed does not contain the target index file or the target index file exists but is abnormal and cannot be used normally, a local search can be performed to determine whether there is a corresponding frame index file generated when the audio and video to be processed was previously imported.

[0048] 304. Analyze the audio and video to be processed frame by frame to obtain frame information of multiple audio and video frames, generate a frame index file according to the frame information of the multiple audio and video frames, and save the frame index file.

[0049] Specifically, if the audio and video to be processed is imported for the first time, or although the audio and video to be processed has been imported before and the corresponding frame index file has been generated, but the frame index file has been deleted or damaged, then the corresponding frame index file can be generated according to the audio and video to be processed. Among them, the audio and video to be processed can be parsed frame by frame to obtain the frame information of each frame, and then the frame index file can be generated according to the frame information of each frame and saved.

[0050] It can be understood that the frame index file generated based on the frame information of multiple audio and video frames may contain only the frame index file of key frames, or a file containing the frame information of all frames, or two files, one of which is the frame index file of key frames and the other is the frame index file of non-key frames.

[0051] In some embodiments, the audio and video to be processed are parsed frame by frame to obtain frame information of multiple audio and video frames, which may include: for each audio and video frame in the audio and video to be processed, the audio and video frame is parsed to obtain at least one of the following information of the audio and video frame: decoding timestamp, display timestamp, frame type, frame size, and frame location.

[0052] Specifically, considering that when seeking an audio or video file, it is usually necessary to determine at least one of the decoding timestamp pts, display timestamp dts, frame type, frame size, frame location and other information of the relevant frame, when parsing each frame, at least one of the above information can be parsed.

[0053] In one embodiment of the present disclosure, before generating a frame index file based on the frame information of multiple audio and video frames, it may also include: generating a file version number based on the information type in the frame information; generating a frame index file based on the frame information of multiple audio and video frames may include: generating a frame index file based on the file version number and the frame information of multiple audio and video frames.

[0054] Specifically, considering that the generated frame index file can have different uses and the content contained in the frame information for different uses may be different, different versions of frame index files can be generated for ease of use. In order to distinguish different versions of frame index files, the information type in the frame information in the frame index file can be associated with the version number. Different version numbers indicate that the frame information contains different content.

[0055] In one embodiment of the present disclosure, before generating a frame index file based on the frame information of multiple audio and video frames, it may also include: determining verification information corresponding to the audio and video to be processed; generating a frame index file based on the frame information of multiple audio and video frames may include: generating a frame index file based on the verification information corresponding to the audio and video to be processed and the frame information of multiple audio and video frames.

[0056] Specifically, considering that there may be multiple frame index files saved locally, each of which is generated according to the corresponding audio and video to be processed. Different audio and video to be processed require different frame index files. For example, the frame index file a of the audio and video A to be processed must be generated according to A, and the frame index file b of the audio and video B to be processed must be generated according to B. If the audio and video to be processed and the frame index file do not correspond, the audio and video to be processed cannot be correctly sought or processed. For example, if the audio and video to be processed A uses the frame index file b generated according to the audio and video to be processed B, then A cannot be sought normally. Therefore, in order to ensure that after importing an audio and video to be processed, the frame index file corresponding to the audio and video to be processed is accurately matched from the multiple frame index files saved locally, the verification information corresponding to the audio and video to be processed can be written into the frame index file.

[0057] The verification information corresponding to the audio and video to be processed may be verification information obtained by verifying part or all of the content of the audio and video to be processed.

[0058] In one embodiment of the present disclosure, determining the verification information corresponding to the audio and video to be processed may include: performing a data integrity check on the audio and video to be processed to obtain the verification information corresponding to the audio and video to be processed. Exemplarily, a verification algorithm such as CRC or MD5 may be used.

[0059] In one embodiment of the present disclosure, before saving the frame index file, it may also include: performing a data integrity check on the frame index file to obtain second verification information, and adding the second verification information to the frame index file to obtain a new frame index file; saving the frame index file may include: saving the new frame index file.

[0060] Specifically, considering that the frame index file may be incomplete during the saving process, for example, when the frame index file is saved to the preset storage space (such as a disk), the application suddenly crashes, then the frame index file may only save half of the content. It is impossible to process the audio and video to be processed normally using an incomplete frame index file. In order to avoid this situation, the frame index file can be integrity checked, the checksum of the frame index file can be obtained, and the checksum can be written to the beginning of the frame index file. After the audio to be processed is imported and the corresponding frame index file is obtained, the frame index file can be integrity checked to compare the newly generated checksum with the checksum stored at the beginning of the file. If they are inconsistent, it can be determined that the frame index file is incomplete. If they are consistent, the audio and video to be processed can be sought and processed according to the frame index file.

[0061] 305. Process the audio and video to be processed according to the frame index file; the frame index file is generated when the audio and video to be processed is obtained any time before the audio and video to be processed is obtained this time.

[0062] Step 305 in this embodiment is similar to step 202 in the above embodiment and will not be described again here.

[0063] From the above description, it can be seen that by pre-saving the frame index file corresponding to the audio and video to be processed locally, and after importing the audio to be processed, frame search and other processing can be performed on the audio to be processed based on the pre-saved frame index file, without the need to decode frame by frame to the target frame, thereby improving processing efficiency.

[0064] Corresponding to the audio and video processing method of the above embodiment, Figure 4 This is a structural block diagram of an audio and video processing device provided by an embodiment of the present disclosure. For ease of explanation, only the parts related to the embodiment of the present disclosure are shown. Figure 4 The device 40 includes: an acquisition module 401 and a processing module 402.

[0065] The acquisition module 401 is used to acquire the audio and video to be processed.

[0066] The processing module 402 is used to parse the audio and video to be processed frame by frame to obtain frame information of multiple audio and video frames if the audio and video to be processed does not contain the target index file or the target index file contained is abnormal, and the frame index file corresponding to the audio and video to be processed is not saved locally; generate a frame index file according to the frame information of the multiple audio and video frames, save the frame index file, and the frame index file is used to process the audio and video to be processed.

[0067] In one embodiment of the present disclosure, the processing module 402 is also used to: if a frame index file corresponding to the audio and video to be processed is stored locally, then the audio and video to be processed is processed according to the frame index file; the frame index file is generated when the audio and video to be processed is acquired at any time before the audio and video to be processed is acquired this time.

[0068] In one embodiment of the present disclosure, the processing module 402 is specifically used to: for each audio and video frame in the audio and video to be processed, parse the audio and video frame to obtain at least one of the following information of the audio and video frame: decoding timestamp, display timestamp, frame type, frame size, and frame location.

[0069] In one embodiment of the present disclosure, the processing module 402 is further used to: generate a file version number according to the information type in the frame information; and generate a frame index file according to the file version number and the frame information of the plurality of audio and video frames.

[0070] In one embodiment of the present disclosure, the processing module 402 is further used to: determine verification information corresponding to the audio and video to be processed; and generate a frame index file according to the verification information corresponding to the audio and video to be processed and frame information of multiple audio and video frames.

[0071] In one embodiment of the present disclosure, the processing module 402 is specifically used to: perform data integrity check on the audio and video to be processed, obtain first verification information, and determine the first verification information as the verification information corresponding to the audio and video to be processed.

[0072] In one embodiment of the present disclosure, the processing module 402 is specifically used to: perform data integrity check on the frame index file to obtain second verification information, and add the second verification information to the frame index file to obtain a new frame index file; and save the new frame index file.

[0073] The device provided in this embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effect are similar, and this embodiment will not be repeated here.

[0074] In order to implement the above embodiment, the embodiment of the present disclosure also provides an electronic device.

[0075] refer to Figure 5, which shows a schematic diagram of the structure of an electronic device 500 suitable for implementing the embodiment of the present disclosure, and the electronic device 500 may be a terminal device or a server. The terminal device may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (PMPs), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0076] like Figure 5 As shown, the electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 to a random access memory (RAM) 503. Various programs and data required for the operation of the electronic device 500 are also stored in the RAM 503. The processing device 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0077] Typically, the following devices may be connected to the I / O interface 505: input devices 506 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 508 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 509. The communication devices 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 5 The electronic device 500 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0078] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0079] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0080] The computer-readable medium may be included in the electronic device, or may exist independently without being installed in the electronic device.

[0081] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0082] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0083] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0084] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a unit does not limit the unit itself in some cases. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses".

[0085] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0086] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0087] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0088] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0089] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. An audio and video processing method, characterized in that: include: Get the audio and video to be processed; If the audio and video to be processed does not contain the target index file or the target index file contained is abnormal, and the frame index file corresponding to the audio and video to be processed is not saved locally, the audio and video to be processed is parsed frame by frame to obtain frame information of multiple audio and video frames; a frame index file is generated according to the frame information of the multiple audio and video frames, and the frame index file is saved, and the frame index file is used to process the audio and video to be processed.

2. The method according to claim 1, characterized in that After obtaining the audio and video to be processed, the method further includes: If a frame index file corresponding to the audio and video to be processed is stored locally, the audio and video to be processed is processed according to the frame index file; the frame index file is generated when the audio and video to be processed is obtained any time before the audio and video to be processed is obtained this time.

3. The method according to claim 1, characterized in that The step of parsing the audio and video to be processed frame by frame to obtain frame information of multiple audio and video frames includes: For each audio and video frame in the audio and video to be processed, the audio and video frame is parsed to obtain at least one of the following information of the audio and video frame: decoding timestamp, display timestamp, frame type, frame size, and frame location.

4. The method according to claim 1, characterized in that: Before generating a frame index file according to the frame information of the plurality of audio and video frames, the method further includes: Generate a file version number according to the information type in the frame information; The step of generating a frame index file according to the frame information of the plurality of audio and video frames comprises: A frame index file is generated according to the file version number and frame information of the plurality of audio and video frames.

5. The method according to any one of claims 1 to 4, characterized in that: Before generating a frame index file according to the frame information of the plurality of audio and video frames, the method further includes: Determine verification information corresponding to the audio and video to be processed; The step of generating a frame index file according to the frame information of the plurality of audio and video frames comprises: A frame index file is generated according to the verification information corresponding to the audio and video to be processed and the frame information of the multiple audio and video frames.

6. The method according to claim 5, characterized in that The determining verification information corresponding to the audio and video to be processed includes: Perform data integrity check on the audio and video to be processed to obtain verification information corresponding to the audio and video to be processed.

7. The method according to any one of claims 1 to 4, characterized in that: Before saving the frame index file, the method further includes: Performing a data integrity check on the frame index file to obtain second check information, and adding the second check information to the frame index file to obtain a new frame index file; The step of saving the frame index file comprises: The new frame index file is saved.

8. An audio and video processing device, characterized in that: include: An acquisition module is used to acquire audio and video to be processed; A processing module is used to parse the audio and video to be processed frame by frame to obtain frame information of multiple audio and video frames if the audio and video to be processed does not contain a target index file or the target index file contained is abnormal, and the frame index file corresponding to the audio and video to be processed is not saved locally; generate a frame index file according to the frame information of the multiple audio and video frames, save the frame index file, and use the frame index file to process the audio and video to be processed.

9. An electronic device, characterized in that: include: Processor and memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor executes the audio and video processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions. When the processor executes the computer-executable instructions, the audio and video processing method according to any one of claims 1 to 7 is implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for audio and video processing according to any one of claims 1 to 7 is implemented.

Images