A video frame delay detection method

Abstract

The present invention discloses a method for detecting a video frame delay, which specifically includes: firstly, preprocessing a target video to obtain a target image, then extracting text from the target image, converting the text into text form and outputting it to obtain a first text; subjecting the first text to a first process to obtain a target list; subjecting the target list to a second process to obtain a frame difference and a time difference; finally, subjecting the frame difference and the time difference to a third process to obtain a video frame delay detection result; and data reports and data logs can also be generated by the video frame delay detection process. The present invention solves the problems of slow speed, low efficiency and high error rate during manual processing. Based on Python, batch image processing and detection are performed through a video frame delay detection method, which improves the detection speed and detection accuracy, and can be widely used in the field of video analysis technology.

Description

Technical Field

[0001] The present invention relates to the technical field of video analysis, and in particular to a method for detecting video frame delay. Background Art

[0002] With the development of modern science and technology, video data has a large amount of interactive transmission. The development of communications has made the speed of network transmission faster and faster. For the market and users, the problem of network video transmission delay needs to be solved urgently. In this case, the delay detection method of video frames becomes very important.

[0003] The existing technology mainly captures pictures from videos, manually screens the pictures, compares the timestamps in the pictures with the counted frames, calculates the delay time difference and frame difference in the pictures to obtain the delay results of the video frames. Manual screening is slow, and each picture needs to be observed to select the right one; manual processing of pictures is inefficient, and the data in the pictures needs to be manually calculated, which makes it impossible to quickly batch process more pictures in a short period of time; the error rate of manual data processing is high. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a video frame delay detection method with high speed and high accuracy.

[0005] On the one hand, an embodiment of the present invention provides a video frame delay detection method, including:

[0006] Preprocess the target video to obtain the target image;

[0007] Extracting text from the target image, converting the text into text form and outputting it to obtain a first text;

[0008] Obtain a target list by subjecting the first text to a first process;

[0009] The target list is processed by a second process to obtain a frame difference and a time difference;

[0010] The frame difference and the time difference are subjected to a third process to obtain a video frame delay detection result.

[0011] Optionally, the preprocessing the target video to obtain the target image includes:

[0012] Decomposing the target video into frames to obtain an initial picture;

[0013] The initial picture is segmented to obtain the target picture.

[0014] Optionally, the obtaining a target list by performing a first process on the first text includes:

[0015] Recognize the first text to obtain a first list;

[0016] Performing regular expression extraction on the first list to obtain a second list;

[0017] Integrate the timestamps and frame numbers in the second list to generate a third list;

[0018] The target list is obtained by performing a first data cleaning on the third list.

[0019] Optionally, the obtaining a frame difference and a time difference by performing a second processing on the target list includes:

[0020] Outputting the target list in text form to obtain a second text;

[0021] Retain the numbers in the second text to obtain target data;

[0022] Determine the sending end and the receiving end through the target data;

[0023] The target data of the transmitting end and the target data of the receiving end are calculated to obtain the frame difference and the time difference.

[0024] Optionally, performing a third process on the frame difference and the time difference to obtain a video frame delay detection result includes:

[0025] Performing a second data cleaning on the frame difference and the time difference to obtain a difference list;

[0026] The video frame delay detection result is obtained by performing mean calculation on the difference list.

[0027] Optionally, the method further comprises:

[0028] Data reports and data logs are generated by the video frame delay detection process.

[0029] On the other hand, an embodiment of the present invention further provides a video frame delay detection system, including:

[0030] The first module is used to preprocess the target video to obtain the target image;

[0031] The second module is used to extract the text in the target image and convert the text into a text form to output a first text;

[0032] A third module is used to obtain a target list by performing a first process on the first text;

[0033] A fourth module is used to obtain a frame difference and a time difference by performing a second process on the target list;

[0034] The fifth module is used to perform a third process on the frame difference and the time difference to obtain a video frame delay detection result.

[0035] Optionally, the system further comprises:

[0036] The sixth module is used to generate data reports and data logs from the video frame delay detection process.

[0037] On the other hand, an embodiment of the present invention further provides an electronic device, including a processor and a memory;

[0038] The memory is used to store programs;

[0039] The processor executes the program to implement the above-mentioned video frame delay detection method.

[0040] On the other hand, an embodiment of the present invention further provides a computer-readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement the above-mentioned video frame delay detection method.

[0041] The embodiment of the present invention also discloses a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device can read the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the above method.

[0042] The embodiments of the present invention include at least the following beneficial results: the embodiments of the present invention obtain a target image by preprocessing the target video, and then extract the text in the image, so as to perform batch image data processing and improve the speed of video frame delay analysis; the embodiments of the present invention are a video frame delay detection method based on Python, which is faster and more accurate than manual detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0044] Figure 1 is a step diagram of a video frame delay detection method provided by an embodiment of the present invention;

[0045] Figure 2 It is a module diagram of a video frame delay detection system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0047] In view of the problems existing in the prior art, an embodiment of the present invention provides a video frame delay detection method, such as Figure 1 As shown, the method includes steps 100 to 600.

[0048] Step 100: Preprocess the target video to obtain a target image.

[0049] Specifically, when the sending end sends the video, a timestamp of the current sending time is inserted into each frame, which is recorded as the sending timestamp. When the receiving end receives the video, a timestamp of the current receiving time is also inserted into each frame, which is recorded as the receiving timestamp. The received video with the sending timestamp and the receiving timestamp inserted is the target video, and the insertion position of the timestamp can be set by itself during insertion.

[0050] Decompose the target video into frames to obtain an initial picture; segment the initial picture to obtain the target picture; first configure the video processing related modules, then determine the target video, decompose the target video into frames to obtain the initial picture, and finally segment the extracted initial picture. For example, the multimedia video processing tool ffmpeg can be used to preprocess the target video. ffmpeg is the most commonly used open source software for video processing, which has multiple functions such as video acquisition, video format conversion, video capture, and video watermarking. First, configure the module parameters related to ffmpeg in Python, then use ffmpeg to decompose the video into frames to extract the initial picture, and finally segment the extracted initial picture, and intercept the picture fragments including the timestamp to obtain the target picture, wherein the timestamp includes the sending timestamp and the receiving timestamp. The size of the target picture can be intercepted by setting parameters in ffmpeg according to the actual situation.

[0051] Step 200: extract the text in the target image, convert the text into text form and output it to obtain a first text.

[0052] Specifically, the text in the target image is extracted by software, and then the text is converted into text form and output to obtain the first text. For example, the text in the image is extracted by using Umi-Ocr software, which is a free, offline text recognition tool that supports batch import, clipboard reading, watermark reading and other functions; the task is started after the batch images are imported into Umi-Ocr, and then the text extraction results of the images are obtained, and the extracted text is output in text form to obtain the first text, which contains the timestamp in the target image and other text content in the image, and other text content may include numbers, Chinese characters, letters or other characters.

[0053] Step 300: Obtain a target list by subjecting the first text to a first process.

[0054] The step of obtaining a target list by performing a first processing on the first text specifically includes:

[0055] Step 310: Recognize the first text to obtain a first list.

[0056] Specifically, the timestamp in the target text and other text content in the image are identified, and the content in the target text is converted into a first list, wherein the first list includes the timestamp and other text content extracted from the image in the first text. For example, the text in ".txt" format is converted into an Excel table using Python language, and the classification of the first list is classified according to the character type.

[0057] Step 320: Perform regular expression extraction on the first list to obtain a second list.

[0058] Specifically, regular extraction uses regular expressions to perform extraction on the table content of the first list based on a python script to extract the timestamp data in the first list. Each target image has two timestamps, namely, a sending timestamp and a receiving timestamp, to obtain a second list.

[0059] Step 330: Integrate the timestamps and frame numbers in the second list to generate a third list.

[0060] Specifically, the timestamp data in the second list is used, and the frame number of the target picture corresponding to the timestamp is integrated to generate the third list.

[0061] Step 340: Perform a first data cleaning on the third list to obtain the target list.

[0062] Specifically, it is determined whether there is abnormal data in the data in the third list, and if there is abnormal data, the first data cleaning is performed. For example, the time of the timestamp is not within the time range of the target video, which is abnormal data, wherein the time range of sending the video stamp is from the time of sending the first frame of the picture to the time of sending the last frame of the picture, and the time range of receiving the video stamp is from the time of receiving the first frame of the picture to the time of receiving the last frame of the picture; if the extracted multiple timestamps are the same, these timestamp data can be determined as abnormal data, and the data determined as abnormal data will be cleared, and the remaining data within the normal range will be integrated to generate a target list, and the target list includes the frame number and the timestamp inserted on the frame number picture.

[0063] Step 400: Obtain frame difference and time difference from the target list through a second process.

[0064] The step of obtaining the frame difference and the time difference by performing the second processing on the target list specifically includes:

[0065] Step 410: Output the target list in text form to obtain a second text.

[0066] Specifically, the target list is output as a text to obtain the second text, and all the data in the second list is also output as a text to obtain a full data text as a data reference.

[0067] Step 420: retain the numbers in the second text to obtain target data.

[0068] Specifically, based on the python script, it is determined again whether there is any content other than the timestamp and the frame number in the second text. If so, the Chinese characters, letters, special characters and other text contents other than the timestamp and the frame number in the second text are removed, and the timestamp and the frame number are retained.

[0069] Step 430: Determine the sending end and the receiving end through the target data.

[0070] Specifically, two timestamps corresponding to the same frame number in the target data are compared. The one with a larger time, that is, a later time, is the receiving timestamp of the receiving end, and the one with a smaller time, that is, an earlier time, is the sending timestamp of the sending end.

[0071] Step 440: Calculate the target data of the transmitting end and the receiving end to obtain the frame difference and the time difference.

[0072] Specifically, it can be obtained from the target data: on pictures with the same frame number, the receiving timestamp of the receiving end is subtracted from the sending timestamp of the sending end to obtain the time difference; when the time on the receiving timestamp and the sending timestamp is the same, the picture frame number where the receiving timestamp is located is subtracted from the picture frame number where the sending timestamp is located to obtain the frame difference.

[0073] Step 500: Perform a third process on the frame difference and the time difference to obtain a video frame delay detection result.

[0074] Specifically, the frame difference and the time difference are subjected to a second data cleaning to obtain a difference list; and the difference list is subjected to a mean value calculation to obtain a video frame delay detection result.

[0075] First, the frame difference and the time difference are subjected to a second data cleaning to obtain a difference list, wherein the second data cleaning is to retain the frame difference and the time difference within a normal range, wherein the range of the retained frame difference is greater than 0 frame and less than 20 frames, and the range of the retained time difference is greater than 0 milliseconds and less than 800 milliseconds, and the frame difference and the time difference that are not within the range are removed to obtain a difference list; then, the difference list is subjected to a mean calculation to obtain the mean of the remaining frame difference and time difference within the range, and the time difference mean and the frame difference mean are obtained, which are the video frame delay detection results; the larger the time difference, the higher the video delay; the larger the frame difference, the higher the video delay.

[0076] Step 600: Generate data reports and data logs from the video frame delay detection process.

[0077] Specifically, the data report includes data visualization of frame difference and time difference within the range, and generates bar charts, line charts, pie charts and other icons for data visualization; the data log is automatically generated by the system during the implementation of a video frame delay detection method based on Python. Each target image generates a corresponding data log, including system initialization parameters in the video frame delay detection process, errors in the method execution process, warnings in the method execution process, data before and after data extraction in the process, some important state changes and other data log contents. Abnormal data is replaced by error.

[0078] The following example illustrates the execution and application of a video frame delay detection method.

[0079] 1. First, configure the module parameters related to ffmpeg in python, then import the target video that needs delay detection into ffmpeg, decompose the target video into frames to get the initial picture, and then segment the extracted initial picture to get the target picture;

[0080] 2. Then, the Umi-Ocr software is used to extract the text in the target image to obtain a text extraction result, and then the extracted text is output in the form of text to obtain a first text;

[0081] 3. Recognize the first text to obtain a first list, perform regular extraction on the first list to obtain a second list, integrate the timestamps and frame numbers in the second list to generate a third list, and finally perform the first data cleaning on the third list to obtain the target list;

[0082] 4. Output the target list to obtain a second text, retain the numbers in the second text to obtain target data, determine the sending end and the receiving end by the target data, calculate the target data of the sending end and the target data of the receiving end to obtain the frame difference and the time difference, perform second data cleaning on the frame difference and the time difference to obtain a difference list, and finally perform mean calculation on the difference list to obtain a video frame delay detection result;

[0083] 5. Generate data reports and data logs from the video frame delay detection process.

[0084] In summary, the video frame delay detection method of the embodiment of the present invention has the following advantages:

[0085] 1. The embodiment of the present invention obtains a target image by preprocessing the target video and then extracts the text in the image, so as to perform batch image data processing and improve the speed of video frame delay analysis.

[0086] 2. The embodiment of the present invention is a video frame delay detection method based on Python language, which is faster and more accurate than manual detection of video frame delay;

[0087] 3. The video frame delay measurement of the embodiment of the present invention can be measured remotely, and does not require the transmitter and the receiver to be in the same physical location, which is more convenient.

[0088] Reference Figure 2 The embodiment of the present invention further provides a video frame delay detection system, comprising:

[0089] The first module 201 is used to pre-process the target video to obtain a target image.

[0090] The second module 202 is used to extract the text in the target image, convert the text into text form and output it to obtain a first text.

[0091] The third module 203 is used to obtain a target list by performing a first process on the first text.

[0092] The fourth module 204 is used to obtain the frame difference and the time difference by performing the second processing on the target list.

[0093] The fifth module 205 is used to perform a third process on the frame difference and the time difference to obtain a video frame delay detection result.

[0094] The sixth module 206 is used to generate data reports and data logs from the video frame delay detection process.

[0095] The embodiment of the present invention also provides an electronic device, which can complete the above-mentioned video frame delay detection method, first pre-process the target video to obtain a target image, then extract the text in the target image, convert the text into text form and output it to obtain a first text; the first text is subjected to a first process to obtain a target list; the target list is subjected to a second process to obtain a frame difference and a time difference; finally, the frame difference and the time difference are subjected to a third process to obtain a video frame delay detection result; and data reports and data logs can also be generated from the video frame delay detection result. The present invention solves the problems of slow speed, low efficiency and high error rate of manual detection of video frame delays. Based on python, the video frame delay detection method of the present invention is used to perform batch image processing and detection, thereby improving the detection speed and detection accuracy of video frame delay detection.

[0096] An embodiment of the present invention further provides a computer-readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement the above-mentioned video frame delay detection method.

[0097] The embodiment of the present invention also discloses a computer program product or a computer program, wherein the computer program product or the computer program includes a computer instruction, and the computer instruction is stored in a computer-readable storage medium. A processor of a computer device can read the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes Figure 1 The method shown.

[0098] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part for which is described as a larger operation is performed independently.

[0099] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified, one or more of the functions and / or features described may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions, and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional skills of the engineer. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0100] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0101] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0102] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0103] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0104] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0105] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

[0106] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the described embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A video frame delay detection method, characterized in that: include: Preprocess the target video to obtain the target image; Extracting text from the target image, converting the text into text form and outputting it to obtain a first text; Obtain a target list by subjecting the first text to a first process; The target list is processed by a second process to obtain a frame difference and a time difference; Performing a third process on the frame difference and the time difference to obtain a video frame delay detection result; The step of obtaining a target list by performing a first process on the first text includes: Recognize the first text to obtain a first list; Performing regular expression extraction on the first list to obtain a second list; Integrate the timestamps and frame numbers in the second list to generate a third list; Performing a first data cleaning on the third list to obtain the target list; The step of obtaining a frame difference and a time difference by performing a second process on the target list includes: Outputting the target list in text form to obtain a second text; Retain the numbers in the second text to obtain target data; Determine the sending end and the receiving end through the target data; Calculating the target data of the transmitting end and the target data of the receiving end to obtain the frame difference and the time difference; The performing a third process on the frame difference and the time difference to obtain a video frame delay detection result includes: Performing a second data cleaning on the frame difference and the time difference to obtain a difference list; The video frame delay detection result is obtained by performing mean calculation on the difference list.

2. A video frame delay detection method according to claim 1, characterized in that: The preprocessing of the target video to obtain the target image includes: Decomposing the target video into frames to obtain an initial picture; The initial picture is segmented to obtain the target picture.

3. A video frame delay detection method according to claim 1, characterized in that: The method further includes: generating a data report and a data log by the video frame delay detection process.

4. A system for implementing the video frame delay detection method according to any one of claims 1 to 3, characterized in that: include: The first module is used to preprocess the target video to obtain the target image; The second module is used to extract the text in the target image and convert the text into a text form to output a first text; A third module is used to obtain a target list by performing a first process on the first text; A fourth module is used to obtain a frame difference and a time difference by performing a second process on the target list; The fifth module is used to perform a third process on the frame difference and the time difference to obtain a video frame delay detection result.

5. A video frame delay detection system according to claim 4, characterized in that: The system further comprises: The sixth module is used to generate data reports and data logs from the video frame delay detection process.

6. An electronic device, characterized in that: including a processor and a memory; The memory is used to store programs; The processor executes the program to implement the method according to any one of claims 1 to 3.

7. A computer-readable storage medium, characterized in that: The storage medium stores a program, and the program is executed by a processor to implement the method according to any one of claims 1 to 3.

Images