Testing method, device and readable storage medium based on video stream cutout time

By adding local timestamps to the video stream and using OCR recognition technology to calculate the cutout time, the problem of not being able to correctly feedback cutout time in the existing technology is solved, and the time-consuming test cutout at the black box level is realized, and the Test cost.

CN112580644BActive Publication Date: 2024-11-26PINGAN INT SMART CITY TECH CO LTD

Patent Information

Application Number
CN202011442562.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-12-08
Publication Date
2024-11-26
Estimated Expiration
2040-12-08

AI Technical Summary

Technical Problem

The existing technology cannot correctly feedback the time of the cutout image storage, and the test environment is inconsistent with the production environment source code, resulting in the inability to reasonably evaluate the cutout image storage time.

Method used

By adding local timestamps to the video stream and identifying timestamps using OCR, we calculate the cutout time, so as to test the cutout of the algorithm layer at the black box level, ensuring the consistency of the test environment and the production environment source code.

Benefits of technology

It realizes the cutting time test without inserting time piles at the operation level, which reduces the testing cost, provides a simple and convenient method to obtain the cutting time, and solves the problem that users cannot reasonably evaluate the algorithm to generate the cutting time.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112580644B_ABST
    Figure CN112580644B_ABST
Patent Text Reader

Abstract

The present invention relates to an artificial intelligence, and provides a testing method, device and computer-readable storage medium based on the cutout time of a video stream, wherein the method comprises: adding a local timestamp to the video stream and pushing it to an RTSP server; performing image processing on the video stream with the local timestamp added, obtaining the first frame of the picture, the optimal picture set and the original picture set with the local timestamp; recording the end time of the algorithm model processing the video stream; obtaining the time set of the corresponding optimal picture set; obtaining the initial time of the first frame of the picture and the initial time of the original picture set through OCR; determining the cutout time of the optimal picture according to the initial time of the original picture set and the time set of the optimal picture set; determining the total time of processing the video stream according to the initial time of the first frame of the picture and the end time of the video stream. The present invention provides a simple and convenient algorithm for calculating the cutout time, which solves the problem that users cannot reasonably evaluate the cutout time.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a testing method, device, electronic device and computer-readable storage medium based on video stream cutout time. Background Art

[0002] Cutting time is a performance indicator of the visual cutout algorithm. A shorter cutout time means higher efficiency, while a longer cutout time means lower efficiency. How to obtain the cutout time has always been an important problem that needs to be solved.

[0003] In visual engineering, the RTSP stream is pushed to the RTSP server through the camera gimbal, and the video stream is pulled in real time and parsed frame by frame through the RTSP server. The video frames are pushed into the algorithm model to finally obtain the optimal cutout image, and the image is returned to the upper-level application interface for storage. Finally, the front-end business obtains the final generated image from the upper-level application interface and displays it to the user. The cutout time is the time required from the input of the video frame stream to the generation of the cutout result.

[0004] Currently, the method of obtaining the cutout time generally uses time stubs inserted in the algorithm layer. However, this method of obtaining the cutout time cannot correctly reflect the time when the cutout image is saved. It not only requires inserting time calculation code stubs in the source code, but also cannot guarantee the consistency of the source code between the test environment and the production environment.

[0005] To solve the above problems, the present invention provides a new testing method based on video stream cutout time. Summary of the invention

[0006] The present invention provides a testing method, device, electronic device and computer-readable storage medium based on video stream cutout time, the main purpose of which is to provide a simple and convenient algorithm for calculating the cutout time to solve the problem that users cannot reasonably evaluate the cutout time.

[0007] To achieve the above object, the present invention provides a testing method based on video stream cutout time, the method comprising:

[0008] Add local timestamp to the video stream and push the video stream with local timestamp to the RTSP server;

[0009] Performing image processing on the video stream with the local timestamp added by the RTSP server and the preset algorithm model to obtain the first frame of the video stream with the local timestamp, the optimal image set with the time attribute value, and the original image set with the local timestamp corresponding to the optimal image set;

[0010] At the same time, the end time of the preset algorithm model processing the video stream is recorded;

[0011] Acquire a time set of a corresponding optimal graph set according to the time attribute value of the optimal graph set;

[0012] and, respectively identifying the timestamp of the first frame of the picture and the timestamp of the original picture set through OCR to obtain the initial time of the first frame of the picture and the initial time of the original picture set;

[0013] Determine the cutout time of each optimal picture according to the initial time of the original picture set and the time set of the optimal picture set; and

[0014] Determine the total time for processing the video stream according to the initial time of the first frame of the picture and the end time of the video stream;

[0015] The performance of processing the cutout of the video stream is tested based on the cutout time and the total time.

[0016] Optionally, the local timestamp is added to the video stream through an ffmpeg instruction, and the video stream with the local timestamp added is pushed to the RTSP server.

[0017] Optionally, the processing of the video stream with a local timestamp added thereto by the RTSP server and a preset algorithm model to obtain a first frame of the video stream with a local timestamp, an optimal picture set with a time attribute value, and an original picture set with the local timestamp corresponding to the optimal picture set comprises the following steps:

[0018] Performing image processing on the video stream with a local timestamp added thereto through the RTSP server to obtain a first frame of the video stream image with a local timestamp and each frame of the video stream;

[0019] Image processing is performed on each frame of the video stream through the preset algorithm model to obtain an optimal image set with time attribute values ​​and an original image set with the local timestamp corresponding to the optimal image set.

[0020] Optionally, the identifying the local timestamp of the first frame of the image by OCR to obtain the initial time of the first frame of the image includes the following steps:

[0021] Performing binarization preprocessing on the original image of the first frame of image;

[0022] Extract features from the local timestamp of the original image of the first frame after binarization preprocessing;

[0023] The extracted features are input into a trained classifier, and classification and matching processing are performed in the classifier to obtain the initial time of the first frame of the image.

[0024] Optionally, the binarization preprocessing of the original image of the first frame of image comprises the following steps:

[0025] Cropping a specified area of ​​the original image, wherein the specified area is an area estimated and set in the ffmpeg instruction;

[0026] Binarize the cropped image.

[0027] Optionally, the feature extraction of the local timestamp of the original image after binarization preprocessing comprises the following steps:

[0028] Performing dimensional analysis on the preprocessed original image, and determining a dimensional template corresponding to the preprocessed original image according to the analysis result;

[0029] Feature extraction is performed according to the dimensional template.

[0030] Optionally, the step of inputting the extracted features into a trained classifier, performing classification and matching processing in the classifier, and obtaining the initial time of the first frame of the image comprises the following steps:

[0031] According to the timestamp text learned in the classifier, a corresponding character feature template is produced;

[0032] Segmenting the characters in the timestamp text into sub-pictures containing single characters;

[0033] The sub-image and the character feature template are matched to realize recognition of the character content in the sub-image, so as to obtain the initial time of the first frame image.

[0034] In order to solve the above problem, the present invention also provides a testing device based on video stream cutout time, the device comprising:

[0035] The timestamp adding module is used to add a local timestamp to the video stream and push the video stream with the local timestamp to the RTSP server;

[0036] An image acquisition module is used to perform image processing on a video stream with a local timestamp added thereto through the RTSP server and a preset algorithm model, and obtain a first frame of an image of the video stream with a local timestamp, an optimal image set with a time attribute value, and an original image set with the local timestamp corresponding to the optimal image set;

[0037] The video stream end time acquisition module is used to simultaneously record the end time when the preset algorithm model completes processing of the video stream;

[0038] A time set acquisition module, used for acquiring a time set of a corresponding optimal graph set according to a time attribute value of the optimal graph set;

[0039] An initial time acquisition module, used to respectively identify the timestamp of the first frame of the picture and the timestamp of the original picture set through OCR, and acquire the initial time of the first frame of the picture and the initial time of the original picture set;

[0040] The optimal picture cutout time acquisition module is used to determine the cutout time of each optimal picture according to the initial time of the original picture set and the time set of the optimal picture set;

[0041] The total time acquisition module for processing the video stream is used to determine the total time for processing the video stream according to the initial time of the first frame image and the end time of the video stream; and test the cutout performance of processing the video stream according to the cutout time and the total time.

[0042] In order to solve the above problem, the present invention further provides an electronic device, the electronic device comprising:

[0043] at least one processor; and,

[0044] a memory communicatively connected to the at least one processor; wherein,

[0045] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the steps of the above-mentioned test method based on video stream cutout time.

[0046] In order to solve the above problem, the present invention also provides a computer-readable storage medium, in which at least one instruction is stored. The at least one instruction is executed by a processor in an electronic device to implement the above-mentioned test method based on video stream cutout time.

[0047] The embodiment of the present invention records the video processing time by printing a local timestamp in the video stream, and recognizes the local timestamp printed in the video stream picture by using the OCR picture text recognition technology, thereby realizing the time consumption of cutting out of the algorithm layer at the black box level. At the same time, at the operation level, there is no need to insert time piles in the source code, thereby ensuring the consistency of the source code of the test environment and the production environment; the testing cost and the test execution cost can be reduced; the testing method based on the video stream cutout time provided by the present invention can solve the problem that the user cannot reasonably evaluate the time for the algorithm to generate the cutout at the application layer, and provide a simple and convenient testing method to obtain the cutout time. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A schematic diagram of a flow chart of a testing method based on video stream cutout time provided by an embodiment of the present invention;

[0049] Figure 2 A schematic diagram of a module of a testing device based on video stream cutout time provided by an embodiment of the present invention;

[0050] Figure 3 A schematic diagram of the internal structure of an electronic device for implementing a test method based on video stream cutout time provided by an embodiment of the present invention;

[0051] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0052] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0053] The present invention provides a testing method based on video stream cutout time. Figure 1 FIG. 1 is a flow chart of a method for testing video stream cutout time according to an embodiment of the present invention. The method can be performed by a device, which can be implemented by software and / or hardware.

[0054] In this embodiment, the testing method based on the video stream cutout time includes:

[0055] S1: Add a local timestamp to the video stream and push the video stream with the local timestamp to the RTSP server;

[0056] S2: performing image processing on the video stream with a local timestamp through the RTSP server and the preset algorithm model, obtaining the first frame of the video stream with the local timestamp, the optimal image set with the time attribute value, and the original image set with the local timestamp corresponding to the optimal image set, and recording the end time of the preset algorithm model processing the video stream;

[0057] S3: acquiring a time set of a corresponding optimal graph set according to the time attribute value of the optimal graph set; and,

[0058] S4: identifying the timestamp of the first frame of the picture and the timestamp of the original picture set respectively through OCR, and obtaining the initial time of the first frame of the picture and the initial time of the original picture set;

[0059] S5: determining the cutout time of each optimal picture according to the initial time of the original picture set and the time set of the optimal picture set; and,

[0060] Determine the total time for processing the video stream according to the initial time of the first frame of the picture and the end time of the video stream;

[0061] S6: Testing the performance of processing the video stream's cutout according to the cutout time and the total time.

[0062] The above is a test method for the artificial intelligence of the present invention based on the video stream cutout time. The cutout time is a performance indicator of the visual cutout algorithm. The shorter the cutout time, the higher the efficiency, and the longer the cutout time, the lower the efficiency. The method for obtaining the cutout time will be introduced in detail below.

[0063] In step S1, the local timestamp is added to the video stream through the ffmpeg instruction. The local timestamp is added to determine the time when the processing of the first frame of the picture starts, and the time when the processing of the original picture set starts.

[0064] In the embodiment of the present invention, the ffmpeg instruction pushes a live video stream, with the video compressed using h264 and the audio compressed using faac, to another live server (RTSP server), and prints the local time of the RTSP server in the video stream during the push process. That is, the local time of the RTSP server is printed in each frame of the video through the above ffmpeg instruction.

[0065] In step S2, the video stream with a local timestamp is processed by the RTSP server and a preset algorithm model to obtain the first frame of the video stream with a local timestamp, an optimal picture set with a time attribute value, and an original picture set with the local timestamp corresponding to the optimal picture set, including the following steps:

[0066] S21: performing image processing on the video stream with the local timestamp added thereto through the RTSP server to obtain a first frame of the video stream image with the local timestamp and each frame of the video stream;

[0067] S22: Perform image processing on each frame of the video stream through an algorithm model to obtain an optimal image set with time attribute values ​​and an original image set with the local timestamp corresponding to the optimal image set.

[0068] In an embodiment of the present invention, the algorithm program accesses the RTSP server to pull down the video frame stream, pushes each frame of the read video stream to the algorithm model frame by frame, and finally obtains the optimal picture and the original picture corresponding to the picture, wherein the optimal picture has a time attribute and the original picture has a local timestamp. According to the time attribute value of the optimal picture set, the time set of the corresponding optimal picture set is obtained.

[0069] In step S4, the steps of respectively identifying the timestamp of the first frame of the picture and the timestamp of the original picture set by OCR to obtain the initial time of the first frame of the picture and the initial time of the original picture set include the following steps:

[0070] S41: Identify the timestamp of the first frame of image through OCR to obtain the initial time of the first frame of image; and

[0071] S42: Identify the timestamp of the original picture set through OCR to obtain the initial time of the original picture set.

[0072] The step of identifying the timestamp of the first frame of image by OCR to obtain the initial time of the first frame of image includes the following steps:

[0073] S411: performing binarization preprocessing on the original image of the first frame of image;

[0074] S412: extracting features from the timestamp of the original image after binarization preprocessing;

[0075] S413: Input the extracted features into the trained classifier, perform classification and matching processing in the classifier, and obtain the initial time of the first frame of the image.

[0076] The binarization preprocessing of the original first frame image includes the following steps:

[0077] S4111: Crop a designated area of ​​the original image, wherein the designated area is an area estimated and set in the ffmpeg instruction;

[0078] S4112: Binarization processing is performed on the cropped image;

[0079] S14113: Scan the binarized image to obtain characters.

[0080] The feature extraction of the timestamp of the original image after the binary preprocessing comprises the following steps:

[0081] S4121: Performing dimensional analysis on the preprocessed original image, and determining a dimensional template corresponding to the preprocessed original image according to the analysis result;

[0082] S4122: Perform feature extraction according to the corresponding dimension template.

[0083] The step of inputting the extracted features into a trained classifier, performing classification and matching processing in the classifier, and obtaining the initial time of the first frame of the image comprises the following steps:

[0084] S4131: Producing a corresponding character feature template according to the timestamp text learned in the classifier;

[0085] S4132: Segment the characters in the timestamp text into sub-pictures containing single characters;

[0086] S14133: Match the segmented sub-image with the character feature template to recognize the character content and obtain the initial time of the first frame image.

[0087] In the embodiment of the present invention, after the initial time of the first frame of the picture is obtained, the initial time of the first frame of the picture is verified, and the verification process is as follows:

[0088] Verify whether the initial time of the first frame of the image is a 13-bit millisecond timestamp;

[0089] If the initial time of the first frame of the image is not a 13-bit millisecond timestamp as a result of the check, repeat steps S411 and S412, that is, perform cropping area adjustment and image binarization adjustment;

[0090] If the result of the check shows that the initial time of the first frame of the image is a 13-bit millisecond timestamp, the initial time of the first frame of the image is format-converted to obtain a specific time.

[0091] In addition, it should be noted that, in the embodiment of the present invention, the timestamp of the original picture set is identified by the above method to obtain the initial time of the original picture set.

[0092] In step S5, the cutout time of each optimal picture is determined according to the initial time of the original picture set and the time set of the optimal picture set, that is, the cutout time of each optimal picture is obtained by subtracting the initial time of the original picture set from the time set of the optimal picture set.

[0093] The total time for processing the video stream is determined according to the initial time of the first frame image and the end time of the video stream, that is, the total time for processing the video stream is obtained by subtracting the initial time of the first frame image from the end time of the video stream.

[0094] In an embodiment of the present invention, the cutout performance of processing the video stream is tested based on the cutout time and the total time. The cutout time is a performance indicator of the visual cutout algorithm. The shorter the cutout time, the higher the efficiency, while the longer the cutout time, the lower the efficiency. Therefore, obtaining the cutout time has always been an important issue that needs to be solved.

[0095] In the first scenario of the embodiment of the present invention, the frame-by-frame cutout of the video can be simplified to reading a frame of the picture and generating the cutout result of the frame. The specific process is as follows:

[0096] Step 1: Use ffmpeg command to add local millisecond timestamp to video_i and push the stream to RTSP server;

[0097] Step 2: Read and parse the video through the RTSP server, obtain the j-th frame (j>=1) of the video stream image image_o_j (j>=1), and generate the image image_j through the algorithm model (where image_j is a part of image_o_j);

[0098] Step 3: Get T1 by obtaining the time attribute value of image_j;

[0099] Step 4: Use OCR technology to identify the timestamp of the original image image_o_j of the jth frame to obtain T0;

[0100] Step 5: Obtain the optimal total time consumption T = T1-T0 for processing consecutive frames.

[0101] In the second scenario of the embodiment of the present invention, the smallest unit constituting a video is a video frame, and the smallest unit for optimal cutout of an object or a person is the number of consecutive frames of the object or the person within a certain period of time. The specific process is as follows:

[0102] Step 1: Use ffmpeg command to add local millisecond timestamp to the video Video_s composed of the continuous frames in the period of time and push the stream to the RTSP server;

[0103] Step 2: Read and parse the video through the RTSP server to obtain the first frame of the video stream image_1;

[0104] Step 3: Pass each frame of the read video frame stream through the algorithm model, and finally generate the optimal image_res;

[0105] Step 4: Obtain T1 by obtaining the time attribute value of the optimal image image_res;

[0106] Step 5: Use OCR technology to identify the timestamp of the first frame image image_1 to obtain T0;

[0107] Step 6: Get the optimal total time taken by the algorithm to cut out images in processing consecutive frames, T = T1 – T0.

[0108] In the third scenario of the embodiment of the present invention: the conventional test video is composed of countless videos in scenario 2. Therefore, when testing the cutout algorithm, it is necessary to introduce the single cutout time Ti (generation time of generating the cutout image_i - reading time of reading the original image image_i_o of image_i), the total time of all cutouts, and the average time of cutouts (total time of all cutouts / number of cutouts). The specific process is as follows:

[0109] Step 1: Use ffmpeg command to add local millisecond timestamp to the video video_a composed of the continuous frames in the period of time and push the stream to the RTSP server;

[0110] Step 2: Read and parse the video through the RTSP server to obtain the first frame of the video stream image image1 read by the algorithm;

[0111] Step 3: Calculate each frame of the read video frame stream through the algorithm model, and finally generate the optimal image set {image_i};

[0112] Step 4: Record the end time T_end when the algorithm finishes processing the video

[0113] Step 4: Obtain the corresponding time set {Ti} by obtaining the time attribute value of the optimal image set {image_i};

[0114] Step 5: Use OCR technology to identify the timestamp of the first frame image image1 to obtain T0, identify the original image set {image_i_o} corresponding to the optimal image set {image_i} to obtain the corresponding original image reading time set {Ti_o};

[0115] Step 6: The time consumption set of the algorithm in processing and generating each optimal cutout is obtained as {T} = {Ti-Ti_o};

[0116] Step 7: According to the end time T_end–T0, the total video processing time T_all and the algorithm processing time T algorithm That is, the total time consumed by all cutouts, so the following is obtained:

[0117] 1.T algorithm =T all -T video

[0118]

[0119]

[0120] Formula 1 represents the algorithm processing time: It reflects the time from the start of processing Example 1 or Example 2 of the class by the algorithm to the time when the algorithm finally generates the matte. This intermediate process includes the time for the algorithm to read and process the frame pictures, analyze the picture features, process and determine the optimal solution, and generate the pictures.

[0121] Formula 2 represents the average time taken for all generated matte pictures on the basis of Formula 1.

[0122] Formula 3 represents the time from when the optimal solution is determined to when the generated matte is saved by the algorithm. Among them, it is assumed that the algorithm generated a total of n mattes during the T2 time period, T i c is the generation time of the i-th matte, T i o is the original image reading time of the i-th matte, T i is the total time taken for the i-th picture, and Tv is the average time taken to generate n mattes during the T2 time period.

[0123] In the embodiments of the present invention, for two different visual matte algorithms, when processing the same video, through the average time algorithm in Point 2, the average time TA for Algorithm A to generate each picture can be obtained; the average time for Algorithm B to generate each picture is TB. If TA < TB, it can be shown that Algorithm A is superior to Algorithm B in terms of timeliness. This solution can be applied to the matte timeliness analysis of different visual matte algorithms, and through it, the timeliness advantages and disadvantages comparison between algorithms can be objectively obtained.

[0124] In the embodiments of the present invention, during the streaming process, the local time of the rtsp server is printed to realize the time monitoring of streaming and pulling; the ocr technology is used to identify the time string in the original picture to obtain the streaming time of the original picture; the timeliness of the matte algorithm and the corresponding average time consumption are tested at the black box level; a convenient test method is provided to calculate the time index of the algorithm performance; and it is applied to the analysis of the advantages and disadvantages of different matte algorithms.

[0125] The embodiments of the present invention record the video processing time by printing the local time stamp in the video stream, and identify the local time stamp printed in the video stream picture through the OCR picture text recognition technology, so as to realize the matte time consumption test of the algorithm layer at the black box level. At the same time, at the operation level, there is no need to insert time stakes in the source code, ensuring the consistency of the source code between the test environment and the production environment; it can reduce the test cost and the test execution cost; by using the test method based on the matte time of the video stream provided by the present invention, the problem that users cannot reasonably evaluate the time for the algorithm to generate the matte at the application layer can be solved, and a simple and convenient test method is provided to obtain the matte time.

[0126] Such as Figure 2The figure shows a functional module diagram of a testing device based on video stream cutout time of the present invention.

[0127] The test device 100 based on the video stream cutout time of the present invention can be installed in an electronic device. According to the functions to be implemented, the test device based on the video stream cutout time may include: a timestamp adding module 101, a picture acquisition module 102, a video stream end time acquisition module 103, a time set acquisition module 104, an initial time acquisition module 105, an optimal picture cutout time acquisition module 106 and a total time acquisition module 107 for processing the video stream. The module of the present invention can also be referred to as a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0128] In this embodiment, the functions of each module / unit are as follows:

[0129] The timestamp adding module 101 is used to add a local timestamp to the video stream and push the video stream with the local timestamp to the RTSP server;

[0130] The picture acquisition module 102 is used to perform image processing on the video stream with the local timestamp through the RTSP server and the algorithm model, and obtain the first frame of the video stream with the local timestamp, the optimal picture set with the time attribute value, and the original picture set with the local timestamp corresponding to the optimal picture set;

[0131] The video stream end time acquisition module 103 is used to simultaneously record the end time when the preset algorithm model finishes processing the video stream;

[0132] A time set acquisition module 104, configured to acquire a time set of a corresponding optimal graph set according to a time attribute value of the optimal graph set;

[0133] An initial time acquisition module 105 is used to respectively identify the timestamp of the first frame of the picture and the timestamp of the original picture set through OCR to acquire the initial time of the first frame of the picture and the initial time of the original picture set;

[0134] The optimal picture cutout time acquisition module 106 is used to determine the cutout time of each optimal picture according to the initial time of the original picture set and the time set of the optimal picture set;

[0135] The total time acquisition module 107 for processing the video stream is used to determine the total time for processing the video stream according to the initial time of the first frame image and the end time of the video stream.

[0136] In an embodiment of the present invention, the video processing time is recorded by printing a local timestamp in the video stream, and the local timestamp printed in the video stream picture is identified by using OCR picture text recognition technology, thereby realizing the time consumption of cutting out of the algorithm layer at the black box level. At the same time, at the operational level, there is no need to insert time piles in the source code, thereby ensuring the consistency of the source code of the test environment and the production environment; the testing cost and the test execution cost can be reduced; the testing method based on the video stream cutout time provided by the present invention can solve the problem that the user cannot reasonably evaluate the time for the algorithm to generate the cutout at the application layer, and provide a simple and convenient testing method to obtain the cutout time.

[0137] like Figure 3 FIG. 1 is a schematic diagram of the structure of an electronic device for implementing a testing method based on video stream cutout time according to the present invention.

[0138] The electronic device 1 may include a processor 10, a memory 11 and a bus, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as a test program 12 based on video stream cutout time.

[0139] The memory 11 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. The memory 11 may be an internal storage unit of the electronic device 1 in some embodiments, such as a mobile hard disk of the electronic device 1. The memory 11 may also be an external storage device of the electronic device 1 in other embodiments, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 1. Further, the memory 11 may also include both an internal storage unit of the electronic device 1 and an external storage device. The memory 11 may not only be used to store application software and various types of data installed in the electronic device 1, such as the code of the data audit program, etc., but also be used to temporarily store data that has been output or is to be output.

[0140] The processor 10 may be composed of an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips, etc. The processor 10 is the control core (Control Unit) of the electronic device, and uses various interfaces and lines to connect various components of the entire electronic device, and executes or executes programs or modules (such as data auditing programs, etc.) stored in the memory 11, and calls data stored in the memory 11, so as to execute various functions of the electronic device 1 and process data.

[0141] The bus may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize connection and communication between the memory 11 and at least one processor 10, etc.

[0142] Figure 3 Only an electronic device with components is shown, and those skilled in the art will understand that Figure 2 The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0143] For example, although not shown, the electronic device 1 may also include a power source (such as a battery) for supplying power to each component. Preferably, the power source may be logically connected to the at least one processor 10 through a power management device, so that the power management device can realize functions such as charging management, discharging management, and power consumption management. The power source may also include any components such as one or more DC or AC power sources, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, etc. The electronic device 1 may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.

[0144] Furthermore, the electronic device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device 1 and other electronic devices.

[0145] Optionally, the electronic device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), or a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device 1 and to display a visual user interface.

[0146] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0147] The test program 12 based on the video stream cutout time stored in the memory 11 in the electronic device 1 is a combination of multiple instructions. When running in the processor 10, it can achieve:

[0148] Add local timestamp to the video stream and push the video stream with local timestamp to the RTSP server;

[0149] Performing image processing on the video stream with the local timestamp added by the RTSP server and the algorithm model to obtain the first frame of the video stream with the local timestamp, the optimal image set with the time attribute value, and the original image set with the local timestamp corresponding to the optimal image set;

[0150] At the same time, the end time of the preset algorithm model processing the video stream is recorded;

[0151] Acquire a time set of a corresponding optimal graph set according to the time attribute value of the optimal graph set;

[0152] and, respectively identifying the timestamp of the first frame of the picture and the timestamp of the original picture set through OCR to obtain the initial time of the first frame of the picture and the initial time of the original picture set;

[0153] Determine the cutout time of each optimal picture according to the initial time of the original picture set and the time set of the optimal picture set; and

[0154] Determine the total time for processing the video stream according to the initial time of the first frame of the picture and the end time of the video stream;

[0155] The performance of processing the cutout of the video stream is tested based on the cutout time and the total time.

[0156] Specifically, the specific implementation method of the processor 10 for the above instructions can refer to Figure 1 The description of the relevant steps in the corresponding embodiments will not be repeated here.

[0157] Furthermore, if the module / unit integrated in the electronic device 1 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0158] In an embodiment of the present invention, a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of a test method based on video stream cutout time are implemented. The specific method is as follows:

[0159] Add local timestamp to the video stream and push the video stream with local timestamp to the RTSP server;

[0160] Performing image processing on the video stream with the local timestamp added by the RTSP server and the preset algorithm model to obtain the first frame of the video stream with the local timestamp, the optimal image set with the time attribute value, and the original image set with the local timestamp corresponding to the optimal image set;

[0161] At the same time, the end time of the preset algorithm model processing the video stream is recorded;

[0162] Acquire a time set of a corresponding optimal graph set according to the time attribute value of the optimal graph set;

[0163] and, respectively identifying the timestamp of the first frame of the picture and the timestamp of the original picture set through OCR to obtain the initial time of the first frame of the picture and the initial time of the original picture set;

[0164] Determine the cutout time of each optimal picture according to the initial time of the original picture set and the time set of the optimal picture set; and

[0165] Determine the total time for processing the video stream according to the initial time of the first frame of the picture and the end time of the video stream;

[0166] The performance of processing the cutout of the video stream is tested based on the cutout time and the total time.

[0167] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0168] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0169] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0170] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0171] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0172] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A test method based on video stream cutout time, It is characterized in that The method comprises: Add local timestamp to the video stream and push the video stream with local timestamp to the RTSP server; Performing image processing on the video stream with a local timestamp through the RTSP server and the preset algorithm model, obtaining the first frame of the video stream with the local timestamp, the optimal image set with the time attribute value, and the original image set with the local timestamp corresponding to the optimal image set; and recording the end time of the preset algorithm model processing the video stream; Obtaining a time set of the corresponding optimal image set according to the time attribute value of the optimal image set; and identifying the timestamp of the first frame image and the timestamp of the original image set respectively through OCR to obtain the initial time of the first frame image and the initial time of the original image set; Determine the cutout time of each optimal picture according to the initial time of the original picture set and the time set of the optimal picture set; and determine the total time for processing the video stream according to the initial time of the first frame of the picture and the end time of the video stream; The performance of processing the cutout of the video stream is tested based on the cutout time and the total time.

2. The test method based on video stream cutout time as claimed in claim 1, It is characterized in that The local timestamp is added to the video stream through the ffmpeg instruction, and the video stream with the local timestamp added is pushed to the RTSP server.

3. The testing method based on video stream cutout time as claimed in claim 1, It is characterized in that The method of processing the video stream with the local timestamp added thereto by the RTSP server and the preset algorithm model to obtain the first frame of the video stream with the local timestamp, the optimal picture set with the time attribute value, and the original picture set with the local timestamp corresponding to the optimal picture set comprises the following steps: Performing image processing on the video stream with a local timestamp added thereto through the RTSP server to obtain a first frame of the video stream image with a local timestamp and each frame of the video stream; Image processing is performed on each frame of the video stream through the preset algorithm model to obtain an optimal image set with time attribute values ​​and an original image set with the local timestamp corresponding to the optimal image set.

4. The testing method based on video stream cutout time as claimed in claim 1, It is characterized in that The step of identifying the local timestamp of the first frame of picture by OCR to obtain the initial time of the first frame of picture includes the following steps: Performing binarization preprocessing on the original image of the first frame of image; Extract features from the local timestamp of the original image of the first frame after binarization preprocessing; The extracted features are input into a trained classifier, and classification and matching processing are performed in the classifier to obtain the initial time of the first frame of the image.

5. The testing method based on video stream cutout time as claimed in claim 4, It is characterized in that The binarization preprocessing of the original picture of the first frame of picture comprises the following steps: Cropping a specified area of ​​the original image, wherein the specified area is an area estimated and set in the ffmpeg command; Binarize the cropped image.

6. The testing method based on video stream cutout time as claimed in claim 4, It is characterized in that The feature extraction of the local timestamp of the original picture of the first frame of the picture after the binarization preprocessing comprises the following steps: Performing dimensional analysis on the preprocessed original image, and determining a dimensional template corresponding to the preprocessed original image according to the analysis result; Feature extraction is performed according to the dimensional template.

7. The testing method based on video stream cutout time as claimed in claim 4, It is characterized in that The step of inputting the extracted features into a trained classifier, performing classification and matching processing in the classifier, and obtaining the initial time of the first frame of the image comprises the following steps: According to the timestamp text learned in the classifier, a corresponding character feature template is produced; Segmenting the characters in the timestamp text into sub-pictures containing single characters; The sub-image and the character feature template are matched to realize recognition of the character content in the sub-image, so as to obtain the initial time of the first frame image.

8. A test device based on video stream cutout time, It is characterized in that The device comprises: The timestamp adding module is used to add a local timestamp to the video stream and push the video stream with the local timestamp to the RTSP server; An image acquisition module is used to perform image processing on a video stream with a local timestamp added thereto through the RTSP server and a preset algorithm model, and obtain a first frame of an image of the video stream with a local timestamp, an optimal image set with a time attribute value, and an original image set with the local timestamp corresponding to the optimal image set; The video stream end time acquisition module is used to simultaneously record the end time when the preset algorithm model completes processing of the video stream; A time set acquisition module, used for acquiring a time set of a corresponding optimal graph set according to a time attribute value of the optimal graph set; An initial time acquisition module, used to respectively identify the timestamp of the first frame of the picture and the timestamp of the original picture set through OCR, and acquire the initial time of the first frame of the picture and the initial time of the original picture set; The optimal picture cutout time acquisition module is used to determine the cutout time of each optimal picture according to the initial time of the original picture set and the time set of the optimal picture set; The total time acquisition module for processing the video stream is used to determine the total time for processing the video stream according to the initial time of the first frame image and the end time of the video stream; and test the cutout performance of processing the video stream according to the cutout time and the total time.

9. An electronic device, It is characterized in that The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the steps of the testing method based on video stream cutout time as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, It is characterized in that When the computer program is executed by a processor, the testing method based on video stream cutout time as described in any one of claims 1 to 7 is implemented.

Citation Information

Patent Citations

  • Processing time and recognition precision self-adaption plate number recognition method

    CN103164697A

  • Video recording method and device, and electronic device

    CN106851385A

  • Video portrait cutout method and system based on 3D somatosensory camera

    CN107610041A

  • Application program start-up time consumption test method and device, and storage medium

    CN108984395A

  • Application program performance test method and device, equipment and storage medium

    CN110221979A

Cited By

  • Power monitoring video timestamp identification method and system based on deep learning

    CN122116230A