A snapshot system and method for video key frames

Abstract

The present invention discloses a snapshot system and method of a video key frame, and relates to the technical field of video streaming media development. The system comprises a video acquisition module, a video processing module, a video frame analysis module, an extraction module, a decoding module and a control center. The video acquisition module is used to acquire video-related data, and the video processing module is used to process and evaluate whether the video quality is qualified, and then the extraction module is used to extract the key frame of the qualified video stream, and then the video frame analysis module and the control center are used to perform feature judgment on the key frame image, and the qualified key frame is obtained by dividing it, and then the decoding module is used to decode the key frame, so that the snapshot converted by the key frame after decoding is obtained, and the snapshot is displayed; the application can pre-evaluate the video quality, and then extract the key frame, and then judge the quality of the key frame, and decode the snapshot of the qualified key frame.

Description

Technical Field

[0001] The invention relates to the technical field of video streaming media development, in particular to a snapshot system and method of a video key frame. Background Art

[0002] The key frame snapshot technologies commonly used in the market currently have complex processes and high performance consumption. For example, developers generally need three to four steps to access key frame snapshots, accompanied by multiple transcoding and decoding. In addition, the server consumes a lot of performance when a large number of channels are connected. In addition, the existing technology does not pre-evaluate the quality of the video before extracting the video key frames, which may result in low-quality video indentation and the extracted video key frames not meeting the requirements. Summary of the invention

[0003] In order to solve the deficiencies mentioned in the above background technology, the purpose of the present invention is to provide a video key frame snapshot system and method, which can pre-evaluate the video quality, then extract the key frames, and then judge the quality of the key frames, and decode the qualified key frames.

[0004] In a first aspect, the purpose of the present invention can be achieved by the following technical solution: a snapshot system of a video key frame, comprising:

[0005] Video acquisition module: used to acquire video-related data and send the acquired video-related data to the video processing module for processing, wherein the video-related data includes: video resolution, video frame rate and video bit rate;

[0006] Video processing module: used to mark video-related data, perform comprehensive calculation of video quality based on the marked video-related data, obtain a comprehensive video quality coefficient, set a video quality coefficient threshold, compare the comprehensive video quality coefficient with the video quality coefficient threshold, evaluate whether the video meets the quality requirements based on the comparison result, and send a qualified signal to the extraction module for the video stream that meets the quality requirements;

[0007] Extraction module: used to extract the key frames of the video stream after the video processing module passes through the video processing module as video key frames, and send the video key frames to the video frame analysis module;

[0008] Video frame analysis module: used to extract relevant features of video key frame images as image key features, wherein the image key features include color features, texture features, shape features and spatial relationship features, use image key features to calculate feature quality, obtain feature quality judgment coefficients, assign key frame features to feature quality judgment coefficients, obtain key frame quality judgment coefficients, obtain key frame quality coefficient judgment sets stored in the control center, perform data matching between the key frame quality judgment coefficients and the key frame quality coefficient judgment sets, mark the corresponding video key frames that are successfully matched as qualified key frames, mark the corresponding video key frames that fail to match as redundant frames, count the total number of qualified key frames, if the total number of qualified key frames is less than half of the total number of key frames, mark the video stream as a non-key video stream, if the total number of qualified key frames is greater than or equal to half of the total number of key frames, mark the video stream as a key video stream, and encode the key video stream;

[0009] Control center: used to collect and store key frame quality coefficient determination sets;

[0010] Decoding module: used to decode the key frames of the key video stream, so as to obtain snapshots of the key frames after decoding and conversion, and display the snapshots.

[0011] In combination with the first aspect, in some implementations of the first aspect, the system further includes: the calculation process of the video processing module includes the following steps:

[0012] Mark the video related data, wherein the video resolution is marked as Fi, the video frame rate is marked as Zi, and the video bit rate is marked as Bi, wherein i is the number of acquisition times of the video acquisition module, and i=1, 2, 3, ..., n, and n is the total number of acquisition times of the video acquisition module;

[0013] The video quality is comprehensively calculated based on the marked video related data to obtain the video comprehensive quality coefficient. Specifically, the formula for comprehensive calculation of video quality is as follows:

[0014]

[0015] Where Zhi is the comprehensive coefficient of video quality, F0 is the preset video resolution, Z0 is the preset video frame rate, B0 is the preset video bit rate, and a 1 is the video resolution influence coefficient, a 2 is the video frame rate influence coefficient, a 3 is the video bit rate influence coefficient, and k is the preset video quality correlation coefficient.

[0016] In combination with the first aspect, in some implementations of the first aspect, the system further includes: a determination process of the video processing module is as follows:

[0017] Set the video quality coefficient threshold Zh0; use the calculated video comprehensive quality coefficient Zhi and the video quality coefficient threshold Zh0 to compare the quality:

[0018] If Zhi ≥ Zh0, it means that the video quality at this time meets the requirements, and a qualified signal is sent to the extraction module;

[0019] If Zhi<Zh0, it means that the video quality at this time does not meet the requirements, and a re-capture signal is sent to the video capture module. The video capture module re-captures the video-related data corresponding to the video stream until the video quality meets the requirements.

[0020] In combination with the first aspect, in some implementations of the first aspect, the system further includes: a calculation process of the video frame analysis module is as follows:

[0021] The key features of the image are marked, wherein the color feature is marked as Yj, the texture feature is marked as Wj, the shape feature is marked as Xj, and the spatial relationship feature is marked as Gj; wherein j is the number of the extracted video key frame, j=1, 2, 3, ..., m, and m is the total number of extracted video key frames;

[0022] The key features of the image are used to calculate the feature quality. The calculation process is as follows:

[0023]

[0024] Among them, Lzj is the characteristic quality evaluation coefficient; t 1 is the color feature influence coefficient, t 2 is the texture feature influence coefficient, t 3 is the shape feature influence coefficient, t 4 is the influence coefficient of spatial relationship feature, Y0 is the standard color feature, W0 is the standard texture feature, X0 is the standard shape feature, G0 is the standard spatial relationship feature, ln() is the logarithmic function, and N is the preset proportional coefficient.

[0025] In combination with the first aspect, in some implementations of the first aspect, the system further includes: the video frame analysis module uses the calculated feature quality judgment coefficient Lzj to perform key frame feature assignment to obtain the key frame quality judgment coefficient, and the key frame feature assignment process is as follows:

[0026]

[0027] Where Zfj is the key frame quality determination coefficient, and pf is the key frame assignment preset coefficient.

[0028] In combination with the first aspect, in some implementations of the first aspect, the system further includes: an analysis process of the video frame analysis module:

[0029] Obtain the key frame quality coefficient determination set {Zf} stored in the control center;

[0030] The key frame quality determination coefficient Zfj is matched with the key frame quality coefficient determination set {Zf}:

[0031] If Zfj∈{Zf}, the video key frame corresponding to the key frame quality determination coefficient Zfj is marked as a qualified key frame;

[0032] like The video key frame corresponding to the key frame quality determination coefficient Zfj is marked as a redundant frame;

[0033] Count the total number of qualified key frames, marked as o, if It is marked as a key video stream and the key video stream is encoded;

[0034] like It is marked as a non-critical video stream, and a video re-capture signal is sent to the video capture module to re-capture the video stream. until.

[0035] In combination with the first aspect, in some implementations of the first aspect, the system further includes: a key frame quality coefficient determination set acquisition process of the control center is as follows:

[0036] Using a data acquisition unit in a control center to collect image feature data in the video, wherein the image feature data in the video includes color feature data, texture feature data, shape feature data, and spatial relationship feature data;

[0037] The key frame determination mapping model is combined with the image feature data in the video to obtain the key frame quality determination coefficient, and all the obtained key frame quality determination coefficients are integrated to generate a key frame quality coefficient determination set, wherein the key frame determination mapping model is based on artificial intelligence model training.

[0038] In conjunction with the second aspect, in some implementations of the second aspect, the method further includes: a snapshot method of a video key frame, characterized in that the method includes the following steps:

[0039] Acquire video-related data, mark the video-related data, and use the marked video-related data to perform comprehensive calculation of video quality to obtain a comprehensive video quality coefficient, wherein the video-related data includes: video resolution, video frame rate, and video bit rate;

[0040] Set a video quality coefficient threshold, compare the video comprehensive quality coefficient with the video quality coefficient threshold, and evaluate whether the video meets the quality requirements based on the comparison result. If the video quality does not meet the requirements, re-capture until the video quality meets the requirements;

[0041] Extract key frames of the video stream that meet the requirements as video key frames, extract key image features of the video key frame images, calculate feature quality using the image key features, and obtain feature quality evaluation coefficients, wherein the image key features include color features, texture features, shape features, and spatial relationship features;

[0042] Assign key frame features to the feature quality judgment coefficients to obtain key frame quality judgment coefficients, perform data matching on the key frame quality judgment coefficients and the stored key frame quality coefficient judgment sets, mark the successfully matched video key frames as qualified key frames, and mark the unmatched key frames as redundant frames;

[0043] The total number of qualified key frames is counted. If the total number of qualified key frames is less than half of the total number of key frames, the video stream is marked as a non-critical video stream. If the total number of qualified key frames is greater than or equal to half of the total number of key frames, the video stream is marked as a critical video stream, and the key frames of the critical video stream are encoded and decoded to obtain snapshots of the key frames after decoding and conversion, and the snapshots are displayed.

[0044] Beneficial effects of the present invention:

[0045] The present invention collects video-related data through a video acquisition module, uses a video processing module to process and evaluate whether the video quality is qualified, then uses an extraction module to extract key frames of a qualified video stream, then uses a video frame analysis module and a control center to perform feature judgment on key frame images, divides to obtain qualified key frames, and then encodes and decodes the key frames, so as to obtain snapshots of the key frames after decoding and conversion, and display the snapshots, thereby realizing the function of pre-evaluating the video quality, then extracting key frames, judging the quality of key frames, and decoding snapshots of qualified key frames. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0047] Figure 1 It is a schematic diagram of the system structure of the present invention;

[0048] Figure 2It is a schematic flow chart of the method of the present invention;

[0049] Figure 3 Schematic diagram of the transcoding process of this embodiment;

[0050] Figure 4 is a key frame extraction flow chart of this embodiment;

[0051] Figure 5 Schematic diagram of key frame types in this embodiment. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] Embodiment 1:

[0054] The following is an introduction to the relevant terms involved in the embodiments of the present application:

[0055] Video keyframe: A term in computer animation, which refers to the frame where the key action of a character or object is located, which is equivalent to the original drawing in a two-dimensional animation. The animation between keyframes can be created and added by software, which is called transition frame or in-between frame. Frame - the smallest unit of a single image in animation, which is equivalent to each frame on a film. On the timeline of animation software, a frame is represented as a frame or a mark.

[0056] Video resolution refers to the number of pixels contained in a certain area of ​​a video. The more pixels, the higher the resolution, and the lower the number, the lower the resolution. When played on the same device, the higher the resolution, the better the quality. (If the video itself has a high resolution, and the hardware device that plays the video has a low resolution, the resolution of the playback device itself shall prevail.) Frame rate: The number of frames transmitted per second. The more frames per second, the smoother the picture will look and the better the viewing experience. Coding format: Different coding formats also affect video quality, such as H.264, H.265, and MPEG-4. Different coding compression ratios, different transmission methods, and different supported platforms, and the most common one is H.264. Bit rate: refers to the amount of data encoded per second. The higher the bit rate, the larger the data transmitted and the better the video quality.

[0057] The concepts of I frame, P frame and B frame were first introduced in the MPEG-1 standard developed by MPEG (Moving Picture Experts Group). The four types of coded frames are I frame, P frame, B frame and D frame. Among them, D frame is a frame containing only the DC component of DCT, which can provide a simple picture quality with a certain quality, such as for fast forward mode.

[0058] WebAssembly: (Wasm for short) is a low-level bytecode format that can run in a web browser. WebAssembly defines a binary format that can be executed directly in the browser without going through an interpreter or just-in-time compiler. It currently provides an efficient and safe way to compile high-performance programming languages ​​​​(such as C++, Rust, Go, etc.) into a bytecode format that is recognizable, portable, and high-performance for the Web platform, and can run on different browsers, different devices, and operating systems.

[0059] Convolutional Neural Networks: Convolutional Neural Networks (CNN) is a type of feedforward neural network that includes convolution calculations and has a deep structure. It is one of the representative algorithms of deep learning. Convolutional neural networks have representation learning capabilities and can perform shift-invariant classification on input information according to their hierarchical structure. Therefore, they are also called "Shift-Invariant Artificial Neural Networks (SIANN)".

[0060] The research on convolutional neural networks began in the 1980s and 1990s. Time delay networks and LeNet-5 were the earliest convolutional neural networks. After the 21st century, with the introduction of deep learning theory and the improvement of numerical computing equipment, convolutional neural networks have developed rapidly and have been applied to computer vision, natural language processing and other fields.

[0061] Convolutional neural networks are constructed by imitating the visual perception mechanism of organisms and can perform supervised and unsupervised learning. The sharing of convolution kernel parameters within the hidden layer and the sparsity of inter-layer connections enable convolutional neural networks to learn grid-like topology features such as pixels and audio with a small amount of computation, with stable results and no additional feature engineering requirements for the data.

[0062] like Figure 1 As shown, a video key frame snapshot system includes:

[0063] Video acquisition module, video processing module, video frame analysis module, extraction module, decoding module and control center;

[0064] The video acquisition module is used to acquire video-related data, and send the acquired video-related data to the video processing module for processing, wherein the video-related data includes: video resolution, video frame rate and video bit rate. Furthermore, each video-related data corresponds to a video stream, and the video quality of the video stream is determined by corresponding the video-related data to the video stream.

[0065] Specifically, the present invention is further described below by way of embodiments:

[0066] When the video acquisition module acquires video-related data, it obtains a video signal through a device that outputs video, such as an analog camera, a video recorder, an LD video disc player, a television, etc.; then performs operations such as sampling and quantization on the video signal to convert it into digital data, wherein the video signal is used as a video stream, and the digital data generated by the conversion is used as video-related data;

[0067] The video processing module is used to process the received video-related data. Specifically, the processing process of the video processing module includes the following steps:

[0068] Mark the video related data, wherein the video resolution is marked as Fi, the video frame rate is marked as Zi, and the video bit rate is marked as Bi, wherein i is the number of acquisition times of the video acquisition module, and i=1, 2, 3, ..., n, and n is the total number of acquisition times of the video acquisition module;

[0069] The video quality is comprehensively calculated based on the marked video related data to obtain the video comprehensive quality coefficient. Specifically, the formula for comprehensive calculation of video quality is as follows:

[0070]

[0071] Where Zhi is the comprehensive coefficient of video quality, F0 is the preset video resolution, Z0 is the preset video frame rate, B0 is the preset video bit rate, and a 1 is the video resolution influence coefficient, a 2 is the video frame rate influence coefficient, a 3 is the video bit rate influence coefficient, k is the preset video quality correlation coefficient;

[0072] Set the video quality coefficient threshold Zh0; use the calculated video comprehensive quality coefficient Zhi and the video quality coefficient threshold Zh0 to compare the quality, and evaluate whether the video meets the quality requirements based on the comparison result. Specifically, the evaluation process of the video processing module is as follows:

[0073] If Zhi ≥ Zh0, it means that the video quality at this time meets the requirements, and the video processing module sends a qualified signal to the extraction module;

[0074] If Zhi<Zh0, it means that the video quality at this time does not meet the requirements, and the video processing module sends a re-collection signal to the video collection module, and the video collection module re-collects the video-related data corresponding to the video stream until the video quality meets the requirements;

[0075] The extraction module is used to extract key frames of the video stream after the video processing module passes through;

[0076] Among them, in the existing method technology, the video stream data is first decoded to obtain YUV raw data, the YUV raw data is transcoded to obtain RGB image data, the RGB image data is converted into a picture format to obtain a picture, and then the obtained picture is stored;

[0077] Taking the scenario where the total number of channels connected to the platform is 1000 and a snapshot is taken every 5 minutes as an example, we can calculate the frame rate that needs to be transcoded per second as: 1000 / (5*60)=3.33fps. That is to say, after the platform is connected to the device, it does nothing but perform video decoding + image encoding at 3 frames per second. When this level reaches 10k, the consumption of video decoding + video encoding reaches 33.33fps, which is a huge consumption for service performance.

[0078] Specifically, the present invention is further described below by way of embodiments:

[0079] In this embodiment, during the video stream transmission process, we need the I frame as a key frame snapshot. So we directly take this I frame from the video stream, for example, one method is to call Ffmpeg to execute:

[0080] ffmpeg-i video_name.mp4-vfselect='eq(pict_type\,I)'-vsync 2-s 1920*1080-fimage2 core-%02d.jpeg

[0081] Command analysis:

[0082] -i: input file, here it is actually the video, replaced with the stream address is the video stream.

[0083] -vf: is a command line that represents the description of the filter graph. Selecting the filter selects the frames for output: including the filter constants.

[0084] pict_type and the corresponding type: PICT_TYPE_I indicates an I frame, i.e. a key frame.

[0085] -vsync 2: Prevents redundant copies of each keyframe.

[0086] -fimage2 name_%02d.jpeg: Write the video frame to the image. The format of the style is generally:

[0087] "%d" or "%0Nd"

[0088] -s: resolution, 1920*1080;

[0089] The naming order of the key frames saved in this way starts from 1, and the number indicates the key frame number. The position of the key frame in the original video frame needs to be saved.

[0090] After taking the I-frame, it is directly saved to the local disk of the server. Since the video stream also has a time sequence, this step is equivalent to storing the I-frame on the server by time. And after storing the I-frame, there is no need to decapsulate, decode and transcode the media file.

[0091] The extracted video key frames are used as video key frames and sent to the video frame analysis module;

[0092] The video frame analysis module performs video analysis based on the video key frames extracted by the extraction module. Specifically, the analysis process of the video frame analysis module includes the following steps:

[0093] Extract relevant features of the video key frame image as image key features, wherein the image key features include color features, texture features, shape features and spatial relationship features;

[0094] The key features of the image are marked, wherein the color feature is marked as Yj, the texture feature is marked as Wj, the shape feature is marked as Xj, and the spatial relationship feature is marked as Gj; wherein j is the number of the extracted video key frame, j=1, 2, 3, ..., m, and m is the total number of extracted video key frames;

[0095] The feature quality is calculated using the key features of the image. Specifically, the calculation process is as follows:

[0096]

[0097] Among them, Lzj is the characteristic quality evaluation coefficient; t 1 is the color feature influence coefficient, t 2 is the texture feature influence coefficient, t 3 is the shape feature influence coefficient, t 4 is the spatial relationship feature influence coefficient, Y0 is the standard color feature, W0 is the standard texture feature, X0 is the standard shape feature, G0 is the standard spatial relationship feature, ln() is the logarithmic function, and N is the preset proportional coefficient;

[0098] Obtain the key frame quality coefficient determination set {Zf} stored in the control center;

[0099] The key frame feature assignment is performed using the calculated feature quality judgment coefficient Lzj to obtain the key frame quality judgment coefficient. The key frame feature assignment process is as follows:

[0100]

[0101] Where Zfj is the key frame quality determination coefficient, and pf is the key frame value preset coefficient;

[0102] The key frame quality determination coefficient Zfj is matched with the key frame quality coefficient determination set {Zf}:

[0103] If Zfj∈{Zf}, the video key frame corresponding to the key frame quality determination coefficient Zfj is marked as a qualified key frame;

[0104] like The video key frame corresponding to the key frame quality determination coefficient Zfj is marked as a redundant frame;

[0105] Count the total number of qualified key frames, marked as o, if The video analysis module then identifies the video stream as a key video stream and encodes the key video stream;

[0106] like The video analysis module determines that this video stream is a non-critical video stream and sends a video re-collection signal to the video acquisition module. The video acquisition module re-collects the video stream. until;

[0107] In this embodiment, a region of interest is defined for a key frame image of a video so as to extract special features in the region of interest and process them as key features of the image;

[0108] In this embodiment, the delineation of the region of interest is to define the portion of interest in the image by using a rectangular region or a mask as the region of interest. By extracting the region of interest, the scope of interest can be limited, the amount of image processing calculations can be reduced, and the processing efficiency can be improved. At the same time, the region of interest can also help extract the target from the complex background, and then extract the characteristics of the target for subsequent analysis and processing.

[0109] The video frame analysis module encodes the key frames of the required key video stream according to the client requirements and sends them to the decoding module for decoding; the specific operations in this embodiment are as follows:

[0110] When the client requests the server to view the video stream of a certain channel of a certain device through the HTTP interface (that is, when the user clicks to view the device channel video), the client needs to carry the unique identifier of this channel (such as channel ID) in the request; the server receives this request, and then encodes the corresponding key frame stored locally on the server into a suitable format (such as BASE64 encoding) based on the unique identifier of the channel in the client network request, and then sends it to the decoding module through the HTTP response.

[0111] After the client obtains the corresponding / H.264 / H.265 key frame, it needs to decode it and send the / H.264 / H.265 key frame to the decoding module for decoding.

[0112] The decoding module is used to decode the key frame extracted by the extraction module to obtain a decoded picture, and display the decoded picture; specifically, the decoding process of the decoding module includes the following steps:

[0113] The decoding process requires the use of WebAssembly technology, but because WebAssembly itself does not support direct decoding of H.264 videos, it can be used to run a decoder written in C / C++, such as libvpx or x264, and compile it into WebAssembly.

[0114] Step 1: Download and compile an H.264 decoder, such as x264, to support H.264 decoding.

[0115] Step 2: Use the Emscripten toolchain to compile the compiled decoder into WebAssembly.

[0116] Step 3: Embed the generated WebAssembly module in the web page and provide the corresponding interface to call this WebAssembly module for H.264 decoding.

[0117] Step 4: After H.264 decoding is completed, a picture is obtained. This picture is a snapshot of the key frame obtained from the server after decoding. Then display the snapshot in the appropriate position.

[0118] The control center is used to collect and store the key frame quality coefficient determination set {Zf}. Specifically, the process of collecting the key frame quality coefficient by the control center is as follows:

[0119] Using a data acquisition unit in a control center to collect image feature data in the video, wherein the image feature data in the video includes color feature data, texture feature data, shape feature data, and spatial relationship feature data;

[0120] The key frame quality determination coefficient is obtained by combining the key frame determination mapping model with the image feature data in the video, and all the obtained key frame quality determination coefficients are integrated to generate a key frame quality coefficient determination set, wherein the key frame determination mapping model is trained based on an artificial intelligence model;

[0121] The key frame determination mapping model is trained based on the artificial intelligence model. The process is as follows:

[0122] Integrate and obtain standard image feature determination data through a server, wherein the standard image feature determination data includes standard color feature data, standard texture feature data, standard shape feature data, and standard spatial relationship feature data;

[0123] The artificial intelligence model is trained by standard image feature determination data to obtain and store a key frame determination mapping model; wherein the artificial intelligence model includes a deep convolutional neural network model and an RBF neural network model.

[0124] Embodiment 2: A method for taking a snapshot of a key frame of a video, the method comprising the following steps:

[0125] Acquire video-related data, mark the video-related data, and use the marked video-related data to perform comprehensive calculation of video quality to obtain a comprehensive video quality coefficient, wherein the video-related data includes: video resolution, video frame rate, and video bit rate;

[0126] Set a video quality coefficient threshold, compare the video comprehensive quality coefficient with the video quality coefficient threshold, and evaluate whether the video meets the quality requirements based on the comparison result. If the video quality does not meet the requirements, re-capture until the video quality meets the requirements;

[0127] Extract key frames of the video stream that meet the requirements as video key frames, extract key image features of the video key frame images, calculate feature quality using the image key features, and obtain feature quality evaluation coefficients, wherein the image key features include color features, texture features, shape features, and spatial relationship features;

[0128] Assign key frame features to the feature quality judgment coefficients to obtain key frame quality judgment coefficients, perform data matching on the key frame quality judgment coefficients and the stored key frame quality coefficient judgment sets, mark the successfully matched video key frames as qualified key frames, and mark the unmatched key frames as redundant frames;

[0129] The total number of qualified key frames is counted. If the total number of qualified key frames is less than half of the total number of key frames, the video stream is marked as a non-critical video stream. If the total number of qualified key frames is greater than or equal to half of the total number of key frames, the video stream is marked as a critical video stream, and the key frames of the critical video stream are encoded and decoded to obtain snapshots of the key frames after decoding and conversion, and the snapshots are displayed.

[0130] Based on the same inventive concept, the present invention also provides a computer device, which includes: one or more processors, and a memory for storing one or more computer programs; the program includes program instructions, and the processor is used to execute the program instructions stored in the memory. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is used to implement one or more instructions, specifically for loading and executing one or more instructions in a computer storage medium to implement the above method.

[0131] It needs to be further explained that, based on the same inventive concept, the present invention also provides a computer storage medium, on which a computer program is stored, and the computer program is executed by a processor to execute the above method. The storage medium can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electrical, magnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0132] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0133] The above shows and describes the basic principles, main features and advantages of the present disclosure. Those skilled in the art should understand that the present disclosure is not limited by the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present disclosure. Without departing from the spirit and scope of the present disclosure, the present disclosure may have various changes and improvements, and these changes and improvements fall within the scope of the present disclosure to be protected.

Claims

1. A video key frame snapshot system, characterized in that: include: Video acquisition module: used to acquire video-related data and send the acquired video-related data to the video processing module for processing, wherein the video-related data includes: video resolution, video frame rate and video bit rate; Video processing module: used to mark video-related data, perform comprehensive calculation of video quality based on the marked video-related data, obtain a comprehensive video quality coefficient, set a video quality coefficient threshold, compare the comprehensive video quality coefficient with the video quality coefficient threshold, evaluate whether the video meets the quality requirements based on the comparison result, and send a qualified signal to the extraction module for the video stream that meets the quality requirements; Extraction module: used to extract the key frames of the video stream after the video processing module passes through the video processing module as video key frames, and send the video key frames to the video frame analysis module; Video frame analysis module: used to extract relevant features of video key frame images as image key features, wherein the image key features include color features, texture features, shape features and spatial relationship features, use image key features to calculate feature quality, obtain feature quality judgment coefficients, assign key frame features to feature quality judgment coefficients, obtain key frame quality judgment coefficients, obtain key frame quality coefficient judgment sets stored in the control center, perform data matching between the key frame quality judgment coefficients and the key frame quality coefficient judgment sets, mark the corresponding video key frames that are successfully matched as qualified key frames, mark the corresponding video key frames that fail to match as redundant frames, count the total number of qualified key frames, if the total number of qualified key frames is less than half of the total number of key frames, mark the video stream as a non-key video stream, if the total number of qualified key frames is greater than or equal to half of the total number of key frames, mark the video stream as a key video stream, and encode the key video stream; Control center: used to collect and store key frame quality coefficient determination sets; Decoding module: used to decode the key frames of the key video stream, so as to obtain snapshots of the key frames after decoding and conversion, and display the snapshots.

2. A video key frame snapshot system according to claim 1, characterized in that: The calculation process of the video processing module includes the following steps: Mark the video related data, wherein the video resolution is marked as Fi, the video frame rate is marked as Zi, and the video bit rate is marked as Bi, wherein i is the number of acquisition times of the video acquisition module, and i=1, 2, 3, ..., n, and n is the total number of acquisition times of the video acquisition module; The video quality is comprehensively calculated based on the marked video related data to obtain the video comprehensive quality coefficient. Specifically, the formula for comprehensive calculation of video quality is as follows: Wherein, Zhi is the comprehensive coefficient of video quality, F0 is the preset video resolution, Z0 is the preset video frame rate, B0 is the preset video bit rate, a1 is the video resolution influence coefficient, a2 is the video frame rate influence coefficient, a3 is the video bit rate influence coefficient, and k is the preset video quality correlation coefficient.

3. A video key frame snapshot system according to claim 2, characterized in that: The determination process of the video processing module is as follows: Set the video quality coefficient threshold Zh0; use the calculated video comprehensive quality coefficient Zhi and the video quality coefficient threshold Zh0 to compare the quality: If Zhi ≥ Zh0, it means that the video quality at this time meets the requirements, and a qualified signal is sent to the extraction module; If Zhi<Zh0, it means that the video quality at this time does not meet the requirements, and a re-capture signal is sent to the video capture module. The video capture module re-captures the video-related data corresponding to the video stream until the video quality meets the requirements.

4. The video key frame snapshot system according to claim 1, characterized in that: The calculation process of the video frame analysis module is as follows: The key features of the image are marked, wherein the color feature is marked as Yj, the texture feature is marked as Wj, the shape feature is marked as Xj, and the spatial relationship feature is marked as Gj; wherein j is the number of the extracted video key frame, j=1, 2, 3, ..., m, and m is the total number of extracted video key frames; The key features of the image are used to calculate the feature quality. The calculation process is as follows: Among them, Lzj is the feature quality judgment coefficient; t1 is the color feature influence coefficient, t2 is the texture feature influence coefficient, t3 is the shape feature influence coefficient, t4 is the spatial relationship feature influence coefficient, Y0 is the standard color feature, W0 is the standard texture feature, X0 is the standard shape feature, G0 is the standard spatial relationship feature, ln() is the logarithmic function, and N is the preset proportional coefficient.

5. A video key frame snapshot system according to claim 4, characterized in that: The video frame analysis module uses the calculated feature quality judgment coefficient Lzj to assign key frame features to obtain the key frame quality judgment coefficient. The key frame feature assignment process is as follows: Where Zfj is the key frame quality determination coefficient, and pf is the key frame assignment preset coefficient.

6. A video key frame snapshot system according to claim 5, characterized in that: The analysis process of the video frame analysis module: Obtain the key frame quality coefficient determination set {Zf} stored in the control center; The key frame quality determination coefficient Zfj is matched with the key frame quality coefficient determination set {Zf}: If Zfj∈{Zf}, the video key frame corresponding to the key frame quality determination coefficient Zfj is marked as a qualified key frame; like The video key frame corresponding to the key frame quality determination coefficient Zfj is marked as a redundant frame; Count the total number of qualified key frames, marked as o, if It is marked as a key video stream and the key video stream is encoded; like It is marked as a non-critical video stream, and a video re-capture signal is sent to the video capture module to re-capture the video stream. until.

7. The video key frame snapshot system according to claim 1, characterized in that: The key frame quality coefficient determination set acquisition process of the control center is as follows: Using a data acquisition unit in a control center to collect image feature data in the video, wherein the image feature data in the video includes color feature data, texture feature data, shape feature data, and spatial relationship feature data; The key frame determination mapping model is combined with the image feature data in the video to obtain the key frame quality determination coefficient, and all the obtained key frame quality determination coefficients are integrated to generate a key frame quality coefficient determination set, wherein the key frame determination mapping model is based on artificial intelligence model training.

8. A method for taking a snapshot of a video key frame, characterized in that: The method comprises the following steps: Acquire video-related data, mark the video-related data, and use the marked video-related data to perform comprehensive calculation of video quality to obtain a comprehensive video quality coefficient, wherein the video-related data includes: video resolution, video frame rate, and video bit rate; Set a video quality coefficient threshold, compare the video comprehensive quality coefficient with the video quality coefficient threshold, and evaluate whether the video meets the quality requirements based on the comparison result. If the video quality does not meet the requirements, re-capture until the video quality meets the requirements; Extract key frames of the video stream that meet the requirements as video key frames, extract key image features of the video key frame images, calculate feature quality using the image key features, and obtain feature quality evaluation coefficients, wherein the image key features include color features, texture features, shape features, and spatial relationship features; Assign key frame features to the feature quality judgment coefficients to obtain key frame quality judgment coefficients, perform data matching on the key frame quality judgment coefficients and the stored key frame quality coefficient judgment sets, mark the successfully matched video key frames as qualified key frames, and mark the unmatched key frames as redundant frames; The total number of qualified key frames is counted. If the total number of qualified key frames is less than half of the total number of key frames, the video stream is marked as a non-critical video stream. If the total number of qualified key frames is greater than or equal to half of the total number of key frames, the video stream is marked as a critical video stream, and the key frames of the critical video stream are encoded and decoded to obtain snapshots of the key frames after decoding and conversion, and the snapshots are displayed.

Images