Method and system for improving video frame processing speed by using sequence technology

Abstract

The invention discloses a method and system for improving video frame processing speed by using a sequence technology, and belongs to the technical field of video processing, and the method comprises the following steps: S1, extracting each frame of a video; s2, creating a playing sequence, putting a buffer frame, and determining buffer time according to the final total processing time; s3, putting each frame extracted from the video into a playing sequence frame by frame; s4, firstly calculating a frame extraction interval; extracting corresponding frames according to the frame extraction interval and putting the corresponding frames into a to-be-processed sequence; s5, firstly extracting a frame from the to-be-processed sequence, running a frame processing algorithm to obtain a processing result, and then putting the frame and the processing result into a processed sequence; and S6, extracting a frame and a processing result from the processed sequence, and feeding back the frame and the processing result to the playing sequence. On one hand, the video can be smoothly played without being limited and influenced by resources; and on the other hand, by optimizing the resource configuration, the number of the detected frames is remarkably increased, so that the occurrence of frame missing phenomena is effectively reduced.

Description

Technical Field

[0001] The invention belongs to the technical field of video processing, and in particular relates to a method and a system for improving video frame processing speed by using sequence technology. Background Art

[0002] With the rapid advancement of Internet technology and the continuous upgrading of related hardware equipment, the application scenarios of cameras collecting live videos are becoming more and more extensive. The video data collected by these cameras are usually uploaded to cloud servers, and users can obtain the required video content from the cloud through various client devices. This model greatly simplifies the user's dependence on server sites and hardware resources when deploying a large number of cameras, allowing users to perform video monitoring and data collection more flexibly.

[0003] However, with the explosive growth of video streaming data, cloud-based video processing has also exposed some problems. First, this method has very high requirements for network bandwidth, and because data needs to be transmitted between the cloud and the client, the real-time performance is relatively poor. In order to reduce the delay in the transmission process and improve the real-time processing capability of the video stream, more and more scenarios require local pre-processing or real-time processing of the video.

[0004] In scenarios where local video processing is required, local devices must be able to run various video processing algorithms, such as monitoring and tracking, content review, etc. These application scenarios have very high requirements for local resources, because the video stream itself and the video processing algorithm will compete for limited local computing resources at the same time. In the traditional processing mode, the system needs to wait for the previous task to be fully processed before starting the next task. This serial processing method has extremely high performance requirements for algorithms and resources. In order to maintain smooth video playback, the system often has to skip a large number of video frames, resulting in the inability to process each frame of video data in real time. Summary of the invention

[0005] The present invention provides a method and system for improving the processing speed of video frames by using sequence technology. In the face of limited local resources, this solution aims to effectively balance the resource allocation required for video processing and algorithm operation. Specifically, on the one hand, it ensures that the video can be played smoothly without being affected by resource limitations; on the other hand, by optimizing resource allocation, the number of detected frames is significantly increased, thereby effectively reducing the occurrence of frame leakage.

[0006] To achieve the above-mentioned object of the invention, the first object of the present invention is to provide a method for improving the video frame processing speed by using sequence technology, comprising:

[0007] S1, extract each frame of the video;

[0008] S2, create a play sequence, put in buffer frames, and determine the buffer time according to the final total processing time;

[0009] S3, putting each frame extracted from the video into the playback sequence frame by frame;

[0010] S4, first calculate the frame extraction interval T, T = T1 / T2, where T1 is the algorithm duration, T2 is the interval duration of each frame, T1>T2; then extract the corresponding frame according to the frame extraction interval T and put it into the sequence to be processed;

[0011] S5, first extracting a frame from the sequence to be processed, running the frame processing algorithm to obtain a processing result, and then putting the frame and the processing result into the processed sequence;

[0012] S6. Extract frames and processing results from the processed sequence and feed them back to the playback sequence.

[0013] Preferably, the buffer time is determined according to the final total processing time as follows:

[0014] T4 = T3 / (T1 / T2)*buffer*X;

[0015] Buffer = T2 * 0.01;

[0016] Among them, T3 is the video frame rate, T4 is the buffering time, and X is the total processing time.

[0017] A second object of the present invention is to provide a system for improving video frame processing speed by using sequence technology, comprising:

[0018] Frame-by-frame extraction module, extracting each frame of the video;

[0019] Play sequence creation module, creates a play sequence, puts in buffer frames, and determines the buffer time according to the final total processing time;

[0020] The frame-by-frame module puts each frame extracted from the video into the playback sequence frame by frame;

[0021] The interval frame extraction module first calculates the frame extraction interval T, T = T1 / T2, where T1 is the algorithm duration, T2 is the interval duration of each frame, and T1>T2; then extracts the corresponding frame according to the frame extraction interval T and puts it into the sequence to be processed;

[0022] The frame processing module first extracts frames from the sequence to be processed, runs the frame processing algorithm to obtain the processing results, and then puts the frames and the processing results into the processed sequence;

[0023] The feedback module extracts frames and processing results from the processed sequence and feeds them back to the playback sequence.

[0024] Preferably, the buffer time is determined according to the final total processing time as follows:

[0025] T4 = T3 / (T1 / T2)*buffer*X;

[0026] Buffer = T2 * 0.01;

[0027] Among them, T3 is the video frame rate, T4 is the buffering time, and X is the total processing time.

[0028] A third object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned method of improving the video frame processing speed by using sequence technology.

[0029] A fourth object of the present invention is to provide a computer program product, comprising a computer program, which, when executed by a processor, implements the above-mentioned method of improving the video frame processing speed by using sequence technology.

[0030] The advantages and positive effects of this application are:

[0031] The present invention subdivides the entire video processing task into multiple sequence steps by adopting sequence technology. Such a processing method can not only ensure the smooth playback of video content, but also significantly increase the number of frames detected. In this way, the phenomenon of frame leakage can be effectively reduced, thereby improving the accuracy and reliability of video processing.

[0032] Specifically, in the scenario of using edge computing for video processing, the present invention achieves the coordination of resources required for video acquisition, decoding, and video algorithm processing through sequence technology. This method not only ensures the smoothness of the video processing process, but also greatly increases the number of detected frames. In this way, the phenomenon of missing frames is significantly reduced, so that the video processing solution based on edge computing can be applied to more different usage scenarios, further expanding its scope of application and practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0034] Figure 1 It is a schematic diagram of a traditional technical solution;

[0035] Figure 2 A schematic diagram of a preferred embodiment of the present invention;

[0036] Figure 3Flow chart of a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] See also Figure 1 In traditional video frame processing solutions, due to the actual network environment, the cloud cannot realize real-time processing of video frames, so edge computing is often used, that is, the acquisition equipment is equipped with edge computing modules or units. However, due to the constraints of the installation environment, the size of these edge computing modules and units is strictly limited, which directly leads to their relatively low computing power. In this case, the solution needs to complete the acquisition and decoding of the video first, and then the video frame processing algorithm performs the calculation. After the calculation is completed, the acquisition and decoding processes are restarted. This solution has extremely high performance requirements for each link of video acquisition, decoding, and algorithm processing. With the continuous improvement of video processing content requirements, video processing algorithms are becoming increasingly complex, and processing time is also extended. Traditional implementation plans have gradually become difficult to meet the needs of actual application scenarios.

[0039] See also Figure 2 and Figure 3 ; A first embodiment, a method for improving video frame processing speed by using sequence technology, comprising:

[0040] First, a frame extraction step needs to be performed, which means that each frame image needs to be extracted from the original video frame by frame. This process provides basic data for subsequent processing steps.

[0041] Next, a playback sequence will be created and buffer frames will be placed in it. The purpose of buffer frames is to ensure smooth playback. At this point, the buffer time needs to be determined based on the final total processing time, so that there will be no lag or delay during playback.

[0042] The buffer time is determined based on the final total processing time:

[0043] T4 = T3 / (T1 / T2)*buffer*X; T1, T2 and T4 are in seconds, and T3 can be 24 frames per second or 30 frames per second;

[0044] Buffer = T2*0.01; 0.01 is the empirical value

[0045] Among them, T3 is the video frame rate, T4 is the buffering time, and X is the total processing time.

[0046] Each frame extracted from the video is then put into the playback sequence one by one. This process requires careful processing to ensure that each frame is correctly arranged in time order to ensure the coherence of the video playback.

[0047] After frame extraction, the frame extraction interval T needs to be calculated. The calculation formula is T = T1 / T2, where T1 represents the total duration of the algorithm and T2 represents the duration of each frame interval. It should be noted that T1 must be greater than T2. ​​In this way, the corresponding frames can be extracted according to the calculated frame extraction interval T and put into the sequence to be processed.

[0048] In the sequence to be processed, first extract the frame, and then run the corresponding frame processing algorithm. These algorithms are selected and adjusted according to different application scenarios. By running these algorithms, we can get the processed results. Subsequently, the processed frames and results are put into the processed sequence together.

[0049] Finally, frames and processing results are extracted from the processed sequence and fed back into the playback sequence. The purpose of this feedback mechanism is to control the playback behavior and ensure that the final played video can achieve the expected effect and quality. In this way, precise control of the video playback process can be achieved, providing a smoother and higher-quality viewing experience.

[0050] A second embodiment, a system for improving video frame processing speed by using sequence technology, comprises:

[0051] The frame extraction module is responsible for performing frame extraction operations. Specifically, it extracts each frame of the video file one by one. This process ensures the integrity of the video content and the feasibility of subsequent processing.

[0052] The play sequence creation module is responsible for creating the play sequence. It puts the buffered frames into the sequence so that the frames can be switched smoothly during the play process. In addition, the module also determines the buffering time based on the final total processing time to ensure smooth and stable play.

[0053] The buffer time is determined based on the final total processing time:

[0054] T4 = T3 / (T1 / T2)*buffer*X; T1, T2 and T4 are in seconds, and T3 can be 24 frames per second or 30 frames per second;

[0055] Buffer = T2*0.01; 0.01 is the empirical value

[0056] Among them, T3 is the video frame rate, T4 is the buffering time, and X is the total processing time.

[0057] The frame-by-frame module puts each frame extracted from the video by the frame-by-frame extraction module into the playback sequence one by one in order. This process ensures the continuity and integrity of the frames in the playback sequence.

[0058] The interval frame extraction module first needs to calculate the frame extraction interval T, the calculation formula is T = T1 / T2, where T1 represents the total duration of the algorithm, and T2 represents the interval duration between each frame. In order to ensure the feasibility of extraction, the condition T1>T2 must be met. After calculating the frame extraction interval T, the module will extract the corresponding frames according to this interval and put these frames into the sequence to be processed.

[0059] The frame processing module extracts frames from the sequence to be processed and then runs the corresponding frame processing algorithms. These algorithms are selected and adjusted according to different application scenarios to achieve the best processing effect. After the processing is completed, the module will put the processed frames and the corresponding processing results into the processed sequence to provide data support for subsequent operations.

[0060] The feedback module extracts frames and processing results from the processed sequence and feeds them back into the playback sequence. This process helps adjust the playback behavior in real time, ensuring the accuracy and controllability of the playback process. In this way, the feedback module plays a key regulatory role in the playback process.

[0061] In summary, the present invention aims to process video and algorithm tasks by using sequence technology, thereby effectively avoiding time conflicts. For detailed diagrams of the operation scheme, please refer to the attached Figure 2 .

[0062] The present invention further proposes a technical means of using cache frames before video processing. Specifically, by introducing cache frames, a temporal buffer is provided for subsequent operations, which is particularly significant for improving the performance of algorithms with relatively slow running speeds.

[0063] In addition, the present invention also proposes a strategy for managing the to-be-processed frame sequence and the processed frame sequence separately. This method can significantly reduce the phenomenon of queuing and waiting, thereby effectively alleviating the problem of concurrent contention for resources.

[0064] A third embodiment is a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned method of improving the video frame processing speed by using the sequence technology.

[0065] A fourth embodiment is a computer program product, comprising a computer program, which, when executed by a processor, implements the above-mentioned method of improving the video frame processing speed by using the sequence technology.

[0066] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When the use is implemented in whole or in part in the form of a computer program product, the computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL) or wireless (e.g., infrared, wireless, microwave, etc.) mode) to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk SolidState Disk (SSD)), etc.

[0067] The above is only a preferred embodiment of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for improving video frame processing speed using sequence technology, characterized in that: include: S1, extract each frame of the video; S2, create a play sequence, put in buffer frames, and determine the buffer time according to the final total processing time; S3, putting each frame extracted from the video into the playback sequence frame by frame; S4, first calculate the frame extraction interval T, T = T1 / T2, where T1 is the algorithm duration, T2 is the interval duration of each frame, T1>T2; then extract the corresponding frame according to the frame extraction interval T and put it into the sequence to be processed; S5, first extracting a frame from the sequence to be processed, running the frame processing algorithm to obtain a processing result, and then putting the frame and the processing result into the processed sequence; S6. Extract frames and processing results from the processed sequence and feed them back to the playback sequence.

2. The method for improving video frame processing speed by using sequence technology according to claim 1, characterized in that: The buffer time is determined according to the final total processing time: T4 = T3 / (T1 / T2)*buffer*X; Buffer = T2 * 0.01; Among them, T3 is the video frame rate, T4 is the buffering time, and X is the total processing time.

3. A system for improving video frame processing speed using sequence technology, characterized in that: include: Frame-by-frame extraction module, extracting each frame of the video; Play sequence creation module, creates a play sequence, puts in buffer frames, and determines the buffer time according to the final total processing time; The frame-by-frame module puts each frame extracted from the video into the playback sequence frame by frame; The interval frame extraction module first calculates the frame extraction interval T, T = T1 / T2, where T1 is the algorithm duration, T2 is the interval duration of each frame, and T1>T2; then extracts the corresponding frame according to the frame extraction interval T and puts it into the sequence to be processed; The frame processing module first extracts frames from the sequence to be processed, runs the frame processing algorithm to obtain the processing results, and then puts the frames and the processing results into the processed sequence; The feedback module extracts frames and processing results from the processed sequence and feeds them back to the playback sequence.

4. The system for improving video frame processing speed by using sequence technology according to claim 3, characterized in that: The buffer time is determined according to the final total processing time: T4 = T3 / (T1 / T2)*buffer*X; Buffer = T2 * 0.01; Among them, T3 is the video frame rate, T4 is the buffering time, and X is the total processing time.

5. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for improving the video frame processing speed by using sequence technology as described in claim 1 or 2 is implemented.

6. A computer-readable storage medium storing a computer program, characterized in that: When the program is executed by a processor, the method for improving the video frame processing speed by using sequence technology as claimed in claim 1 or 2 is implemented.

Images